After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 738724 - [IMAP] Message flag changes not always saved
[IMAP] Message flag changes not always saved
Status: RESOLVED FIXED
Product: evolution-data-server
Classification: Platform
Component: Mailer
3.12.x (obsolete)
Other Linux
: Normal normal
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
: 721286 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2014-10-17 21:20 UTC by Julien Olivier
Modified: 2014-11-03 13:03 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Julien Olivier 2014-10-17 21:20:50 UTC
Whenever I suspend my laptop, then resume, I see that my emails are back to the state they were in after the last sync.

For example, if I have 3 unread emails, then press the send/receive button and receive 2 new unread emails, I have 5 unread emails. If, then, I suspend and resume my laptopn, I'm back to 3 unread emails (and the 2 latest emails are not there at all). However, if, just before suspending the laptop, I press the send/receive button again, the changes are saved and I get 5 unread emails when I resume.

This bug applies for the downloaded / not downloaded state of the emails, the read / unread state, as well as the "important" state on a GMail IMAP server configured through gnome-online-account.
Comment 1 Milan Crha 2014-10-20 19:14:51 UTC
Thanks for a bug report. It sounds odd to have the "claimed" messages (those, which are shown in message list) hidden after resume. it looks like a local summary changes were completely lost. Do the messages reappear when you click Send/Receive? Could you run evolution from console and check what is shown there around the suspend/resume, please? You can get the log of the printed things when you run evolution like this:
   $ evolution &>log.txt
Do not forget to close evolution gracefully. As this is about IMAP, you can even turn on IMAP debugging (which will produce significantly more information and also will show raw communication between the server and the client, including content of downloaded messages, thus not much to be shared in public. You can get such log when you run evolution from a terminal like:
   $ CAMEL_DEBUG=imapx:io evolution &>log.txt
It's good to check for passwords and such in it.

By the way, what is the exact version of the evolution-data-server and evolution you are using, please?
Comment 2 Julien Olivier 2014-10-21 06:42:54 UTC
(In reply to comment #1)
> Thanks for a bug report. It sounds odd to have the "claimed" messages (those,
> which are shown in message list) hidden after resume. it looks like a local

Well, actually, I think I was over-pessimistic in my bug description: only the read/unread and deleted states are lost after resume, not the "claimed" messages. And I forgot to say that the lost changes only appear after the next send/receive operation. What I mean is that, just after resume, things are exactly like before suspend. But if I press send/resume or wait long enough, some emails change read status and some deleted emails come back to life.

> summary changes were completely lost. Do the messages reappear when you click
> Send/Receive? Could you run evolution from console and check what is shown
> there around the suspend/resume, please? You can get the log of the printed
> things when you run evolution like this:
>    $ evolution &>log.txt

Here's what I get:

** (evolution:23521): CRITICAL **: categories_icon_theme_hack: assertion 'filename != NULL && *filename != '\0'' failed

(evolution:23521): camel-WARNING **: Failed to initialize NSS SQL database in sql:/etc/pki/nssdb: NSS error -8126
java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.2) (7u65-2.5.2-4)
OpenJDK Server VM (build 24.65-b04, mixed mode)
evolution-shell-Message: Network disconnected.  Forced offline.
evolution-shell-Message: Connection established.  Going online.

(evolution:23521): GLib-GIO-CRITICAL **: g_network_address_set_addresses: assertion 'addresses != NULL && addr->priv->sockaddrs == NULL' failed

(evolution:23521): GLib-GObject-CRITICAL **: g_closure_unref: assertion 'closure->ref_count > 0' failed
evolution-shell-Message: Network disconnected.  Forced offline.
evolution-shell-Message: Connection established.  Going online.

> By the way, what is the exact version of the evolution-data-server and
> evolution you are using, please?

evolution 3.12.7
e-d-s 3.12.7.1
Comment 3 Julien Olivier 2014-10-21 06:44:57 UTC
> Well, actually, I think I was over-pessimistic in my bug description: only the
> read/unread and deleted states are lost after resume, not the "claimed"
> messages.

Oh, and the "important" state too (the "!" symbol).
Comment 4 Milan Crha 2014-10-21 13:05:16 UTC
I see, all these things are message flags and the suspend/resume doesn't seem to be the only way to reproduce it. I looked more carefully on this and the reason are my changes from bug #735191. I'll work on a better solution.
Comment 5 Julien Olivier 2014-10-21 13:09:50 UTC
(In reply to comment #4)
> I see, all these things are message flags and the suspend/resume doesn't seem
> to be the only way to reproduce it. I looked more carefully on this and the
> reason are my changes from bug #735191. I'll work on a better solution.

Thanks Milan for taking the time to tackle this bug. If you need me to test anything, don't hesitate!
Comment 6 Milan Crha 2014-10-21 17:08:28 UTC
Thanks for the offer, I will appreciate any testing of the below change.

-----------------------------------------------------------------------------

The root cause is that the PERMANENTFLAGS are per-folder settings, while the IMAPx implementation was using a global "server" variable, which reflected only currently selected folder. It doesn't sound that bad on the first look, but the selected folder changes only before the queued command is executed, which is too long after the list of changes to be saved is created.

That might mean for example this:
a) run evolution in offline
b) select one IMAP folder
c) switch evolution to online
d) change a flag on a message in that folder
e) move to another folder (to save the changes)

The problem with this is that the server had no folder selected, thus its PERMANENTFLAGS were 0, which looked for the code (changes from bug #735191) as "the server doesn't allow saving anything", and thus no change was stored on the server.

I changed all this, the PERMANENTFLAGS are stored on respective mailboxes and if the operation reads them, then it ensures at the beginning that the PERMANENTFLAGS are read first, if necessary. The way how changes are recognized also changed, all uids are used, but the set of "to be saved" is filtered later, when the PERMANENTFLAGS are known (the first checking happens like all flags could be saved).

Another issue, though quite rare, could be when there was another pending sync_changes job in the queue, while the user changed another set of messages. The newly changed messages could be simply skipped.

Created commit e4661b0 in eds master (3.13.7+) [1]
Created commit 6d44d52 in eds evolution-data-server-3-12 (3.12.8+)

[1] https://git.gnome.org/browse/evolution-data-server/commit/?id=e4661b0
Comment 7 Julien Olivier 2014-10-22 08:43:17 UTC
(In reply to comment #6)
> Thanks for the offer, I will appreciate any testing of the below change.
> 

I've just compiled and installed it, and, so far, it works perfectly.

If it breaks again, I'll keep you informed.

Thanks!
Comment 8 Milan Crha 2014-11-03 10:38:21 UTC
*** Bug 721286 has been marked as a duplicate of this bug. ***
Comment 9 Milan Crha 2014-11-03 13:03:01 UTC
I found one more possibility when the flag save could break, it was when the application was closing. The first problem was that the changes in the folder's summary were not saved to the disk, thus the next start was no indication of the change. The second problem was with IMAPx exclusively, which rejected to open a new connection when the application was exiting. It sort of makes sense, but breaks things here. Anyway, with these two changes, even if the save fails on the exit, the saved summary makes sure that the local changes are saved the next start of the evolution.

Created commit c46addf in eds master (3.13.8+) [1]
Created commit c02bfef in eds evolution-data-server-3-12 (3.12.8+)

[1] https://git.gnome.org/browse/evolution-data-server/commit/?id=c46addf