After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 659517 - Calendar reminds of old events following upgrade
Calendar reminds of old events following upgrade
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Calendar
3.2.x (obsolete)
Other Linux
: Normal normal
: ---
Assigned To: Milan Crha
Evolution QA team
Depends on:
Blocks:
 
 
Reported: 2011-09-19 20:25 UTC by Ian B. MacDonald
Modified: 2013-09-13 01:05 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
gconf output (997 bytes, application/x-gzip)
2011-09-21 13:17 UTC, Ian B. MacDonald
  Details
proposed evo patch (1.22 KB, patch)
2011-09-23 07:44 UTC, Milan Crha
committed Details | Review

Description Ian B. MacDonald 2011-09-19 20:25:41 UTC
Upon each reboot I see what seems to be an endless stream of reminders for my Google calendars, some going back to 2009.  Following an update a few hours ago I now have EDS 3.1.92 packages and this problem still persists. 

The EDS 3.1.92 update did correct the Contacts calendar issue. I knew this right away without loading Evolution because the reminder spam on reboot following that upgrade began with many many Birthday reminders.  

In my .xession-errors I see a continuous stream of two errors from evolution-alarm-notify:

(evolution-alarm-notify:2442): evolution-alarm-notify-WARNING **: Could not send notification to daemon
(evolution-alarm-notify:2442): evolution-alarm-notify-WARNING **: alarm.c:260: Requested removal of nonexistent alarm!

Per the output below I have about 272 of these in 18 minutes of uptime, regardless of whether I launch evolution.

imac@n8-laptop:~$ cat .xsession-errors | grep evolution-alarm-notify | wc -l
272

imac@n8-laptop:~$ uptime
 16:18:07 up 18 min,  1 user,  load average: 1.05, 1.23, 0.99

I also see dbus-daemon taking up 70% cpu (shown below). Killing the evo processes seems to eliminate the dbus resource consumption, but the evo processes themselves re-spawn and the reminder spam continues.

 2325 imac      20   0 22124 4032  820 R   72  0.1  13:13.64 dbus-daemon                                                                                                                                                 
 2526 imac      20   0  479m  42m 7992 R   28  1.2   3:00.07 e-calendar-fact                                                                                                                                             
 2442 imac      20   0  420m  40m  13m S   22  1.1   5:14.69 evolution-alarm 

Some package version info is below

imac@n8-laptop:~$ dpkg -l | grep evolution
ii  evolution                              3.1.91-0ubuntu1                            groupware suite with mail client and organizer
ii  evolution-common                       3.1.91-0ubuntu1                            architecture independent files for Evolution
ii  evolution-data-server                  3.1.92-0ubuntu1                            evolution database backend server
ii  evolution-data-server-common           3.1.92-0ubuntu1                            architecture independent files for Evolution Data Server
ii  evolution-data-server-dbg              3.1.92-0ubuntu1                            evolution database backend server with debugging symbols
ii  evolution-dbg                          3.1.91-0ubuntu1                            debugging symbols for Evolution
ii  evolution-indicator                    0.2.20-0ubuntu2                            GNOME panel indicator applet for Evolution
ii  evolution-plugins                      3.1.91-0ubuntu1                            standard plugins for Evolution
ii  evolution-webcal                       2.32.0-1build1                             webcal: URL handler for GNOME and Evolution
ii  libebackend-1.2-1                      3.1.92-0ubuntu1                            Utility library for evolution data servers
ii  libebook1.2-12                         3.1.92-0ubuntu1                            Client library for evolution address books
ii  libecal1.2-10                          3.1.92-0ubuntu1                            Client library for evolution calendars
ii  libedata-book-1.2-11                   3.1.92-0ubuntu1                            Backend library for evolution address books
ii  libedata-cal-1.2-13                    3.1.92-0ubuntu1                            Backend library for evolution calendars
ii  libedataserver1.2-15                   3.1.92-0ubuntu1                            Utility library for evolution data servers
ii  libedataserverui-3.0-1                 3.1.92-0ubuntu1                            GUI utility library for evolution data servers
ii  libevolution                           3.1.91-0ubuntu1                            evolution libraries

The downstream bug is captured here:
https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/848968
Comment 1 Milan Crha 2011-09-20 16:44:11 UTC
Alarm notifications are saved per calendar, and only if the calendar doesn't have set the last-notification notice on the ESource, then the global last-notified value is used. As soon as the first notification occurs the calendar's ESource is updated with the last-notification time. I suppose this has something to do with your bug #659488, the ESource is probably overwritten or something like that.
Comment 2 Milan Crha 2011-09-20 17:00:05 UTC
I tried to reproduce this and the last-notification is correctly stored on the ESource and is not forgotten for me. I'm not re-notified the next login.
Comment 3 Ian B. MacDonald 2011-09-20 18:05:45 UTC
How can I check my last-notification value to see if it is changing. Is it an easily browseable gconf value?

Are there cases where it is normal to see all the old reminders (in the past) when adding new calendars or upgrading old ones?
Comment 4 Milan Crha 2011-09-21 07:50:00 UTC
(In reply to comment #3)
> How can I check my last-notification value to see if it is changing. Is it an
> easily browseable gconf value?

you can use this command:
   $ gconftool-2 --get /apps/evolution/calendar/sources | grep -i google | \
     grep last-notified

the "value" attribute after the "last-notified" should change as soon as a new notification from that particular calendar is received.

> Are there cases where it is normal to see all the old reminders (in the past)
> when adding new calendars or upgrading old ones?

The rationale behind this is to notify users about "lost" appointments when he/she didn't run evolution for some time, thus it shows him/her what he missed while having evolution off. It should be different for newly added calendars, though, as they should inherit the global last notification time. I briefly checked the code and I realized that the global last-notification time is not updated when the time in the ESource is changed, thus it is kept unchanged for ever. It means that the newly added calendars are notified with this old day, showing you all the old stuff, instead of only those since the last notification shown in evolution. For cases where the global last notification time is not filled at all is used beginning of the day as the starting point for searching for appointments.

Thus, to summarize:
a) the ESource should be updated with the last-notified time after each notification, if it doesn't for you, then something goes wrong, but it does work for me

b) newly added calendars are notified with time of the global last notification time, which is not updated properly, which is a bug, which I can fix. Before I do so, could you confirm the a) is working for you too, please?
Comment 5 Ian B. MacDonald 2011-09-21 13:17:25 UTC
Created attachment 197151 [details]
gconf output
Comment 6 Ian B. MacDonald 2011-09-21 13:17:57 UTC
Here are my observations from my session this morning starting from cold boot to peek at this setting.  It took me a while to get to a consistent state, so I just captured what I did until I saw consistent results.

1) Cold boot, no notifications on startup, one last-notified entry present
<property name="last-notified" value="2011-09-19T13:45:00Z"/>. out1.txt

2) Enabled an already-configured calendar in Online Accounts to try and stimulate the calendar factory in some way. It doesn't except I see the new calendar in the gconftool output. out2.txt

3) Load evolution client for first time, the reminder spam starts. The spam is not from the calendar I enabled.. but one of the ones that was already configured prior to boot.  A few minutes pass, there have been dozens of calendar messages from "2010" now and the gconf value remains the same with the client loaded.  <property name="last-notified" value="2011-09-19T13:45:00Z"/> out3.txt

4a) Closed the client normally, no change. out4.txt
4b) --force-shutdown followed by killing processes, no change
4c) Re-started the client, no change.

5) Rebooted the system, and even before I enable my network I see the notification spam - obviously there is a cache of undisplayed notifications and this explains the inconsistent spam at startup.  

Prior to starting the client, there is an extra value at the end of the properties.  It is the same last-notification property again with the same value. (out5.txt)
<property name="last-notified" value="2011-09-19T13:45:00Z"/>
<property name="last-notified" value="2011-09-19T13:45:00Z"/>

After 12 minutes of uptime, the reminder notification stops (cache cleared I assume).  So I fire up the evolution client one more time and there are no notifications.

With the client loaded, I now have four last-notified entries in the gconf output. (out6.txt)
<property name="last-notified" value="2011-09-19T13:45:00Z"/>
<property name="last-notified" value="2010-09-20T14:50:00Z"/>
<property name="last-notified" value="2011-01-06T23:50:00Z"/>
<property name="last-notified" value="2011-09-19T13:45:00Z"/>

And one of them, is set to a date in 2010. No notifications.

I closed the client, no change after closing the client. (out7.txt)

I rebooted and no change to the output prior to loading the client. (out8.txt)

Loaded the client, and no change (out9.txt)

So the results seem consistent, wrapping up my session.
Comment 7 Ian B. MacDonald 2011-09-21 13:19:20 UTC
the last comment's attachment comes first as the UI didn't carry my original comment when I tried to added the attachment .. back back commit worked.
Comment 8 Ian B. MacDonald 2011-09-21 14:55:53 UTC
One final observation. At 10:45EST today with no changes in the gconf last-notification dates and about 1:51 of uptime with the client running, the notification spam just started back up on it's own.  It is streaming in from the account that has last-notification date as follows:

<property name="last-notified" value="2010-09-20T14:50:00Z"/>

I didn't visually catch any events from other calendars this time, but I have seen them before (possibly I assume with other last-notified values in the past).
Comment 9 Ian B. MacDonald 2011-09-22 18:29:07 UTC
after letting it run its course a few times.. it finally got two calendars to current dates (<24hrs) but one (with the most events) with notification time shown here and in comment 8 still lags. Although it appears to have stopped I suspect it will start spamming soon.

<property name="last-notified" value="2011-08-05T03:50:00Z"/>

I think what is happening to move the date forwards is:

a) Loads and finds old last notification date A and starts the stream of reminders
b) I kill the process (dbus and factory consume nearly all of two cores)
c) On next boot it runs its course of cached notifications to Date B
d) At this point it dumps Date B into last-notification

This is the only way I can assume the slow progress, if it only stores the date when it runs to the end of the cache.  Otherwise it would make more progress.
Comment 10 Milan Crha 2011-09-23 07:44:11 UTC
Created attachment 197320 [details] [review]
proposed evo patch

for evolution;

I think this may fix your issue. If you could give it a try then it'll be great. The patch is fixing two things:
a) the "last-notified" property is changed only if it's in a future from the previously stored value
b) the global last notification time is always updated, regardless whether the ESource was used or not.

The a) is necessary to not downgrade the last-notified time, thus the events may not ever repeat. The b) is needed for new calendars, they will use current last notification time. To have b) in effect you should receive a notification with this patch and create a new addressbook only after that. The a) was implemented for the global last notification time since the very beginning, thus it's sort of regression from the previous state (before storing last notification time per calendar).
Comment 11 Ian B. MacDonald 2011-09-26 02:21:45 UTC
Okay, now that my calendars have "caught up" I am going to try the following, 

1) restore my previous 2.32 file structures
2) upgrade to current 3.1.92 (without patch) by loading evolution
3) validate the reminders come back with inappropriate last-notification date
4) apply the patch, dpkg-rebuild and then  
5) restore my previous 2.32 file structures 
6) upgrade to current 3.1.92 (with patch) by loading evolution
7) confirm the issues is resolved
Comment 12 Milan Crha 2011-09-26 09:16:08 UTC
Only make sure that before 5) is stored correct time in a GConf key /apps/evolution/calendar/notify/last_notification_time
because otherwise you'll be notified with too old events again. The only difference would be that there should not be any repeated notifications for already notified events.
Comment 13 Ian B. MacDonald 2011-09-26 16:10:29 UTC
The 2.32 backup tar.gz did not grab my .gconf config, so I added it to the tarball myself. The last-notification dates in 2.32 appeared current (within days).

I passed step 3) with a bunch of birthday calendar reminders.  I didn't wait to see if more calendar events followed. 

I restarted once after purging the configuration at 3a) as i was worried about cached notifications.

On step 7) I am seeing all these birthday reminders too.  If successful, your patch appears to have missed the Contact Birthday last-notification. 

I will take a while for these reminders to disappear, as I have a lot of contacts. 

I am still not 100% positive that I am not seeing cached notifications from step 3) and I am wondering if my reboot step would have gotten rid of any cache, or there is a better way to do it.
Comment 14 Milan Crha 2011-09-26 18:10:50 UTC
Does it mean that you restored from the backup with GConf keys (which should be there always, no idea why not for you)? It overwrites old values, thus also notification times. Let it run with the patch for some time, please. You had been re-reminded quite quickly last time, thus if you'll not see same reminders say till Friday, then we have it :)
Comment 15 Ian B. MacDonald 2011-09-26 18:37:49 UTC
I purged the 3.1.92 .gconf/.local/.config/.cache data for evolution and restored my 2.32 backup created with the evolution client.  It only contained .config, .local and .camel_certs directories, so I manually copied the .gconf, which means I did restore with GConf keys I believe. 

You are saying the .gconf data (keys) should be in the backup (re: always there).  If so, I'll file a separate bug on that. All I have are .config, .local and .camel_certs directories in the 2.32 backup.tar.gz 

I didn't want to start evolution processes prior to restoring data, so I just extracted the .tar.gz backups myself which appeared to contain the content of .local, .config and .camel_certs. I also extracted the .gconf data I prepared manually.  Once the evolution data was restored, I installed the patched packages and rebooted to load gconf normally with the alarm-notify process on login. 

Are the birthday calendar reminders not part of this bug?
Comment 16 Milan Crha 2011-09-27 10:54:36 UTC
(In reply to comment #15)
> You are saying the .gconf data (keys) should be in the backup (re: always
> there).  If so, I'll file a separate bug on that. All I have are .config,
> .local and .camel_certs directories in the 2.32 backup.tar.gz 

Might be related to bug #659742

> Are the birthday calendar reminders not part of this bug?

I consider them being part of this, the patch works independently from the calendar, thus even there you shouldn't see repeated birthday notifications once you saw them in the patched version of evo.
Comment 17 Ian B. MacDonald 2011-09-27 13:35:43 UTC
No duplicates so far, and no continuous notifications once the birthdays stopped.  Looks good so far.   

It is safe to assume that my google contact birthdays were not functioning in 2.32, therefore I saw all the notifications once after I upgraded.  Once that is corrected in 2.32, the upgrade would be as expected with only current/recent notifications appearing on first load.
Comment 18 Ian B. MacDonald 2011-09-29 00:21:00 UTC
I tested the patch again with 3.2.0 (current Ubuntu 11.10 B2).  i did not see duplicates and I did not see a stream of old events from Google calendars.

I did still see a stream of birthday reminders over the last year from my Contacts Calendar that persisted over a reboot (lots of contacts) but did not appear to have duplicates.  

I think this is related to https://bugzilla.gnome.org/show_bug.cgi?id=659184. The EDS I created my backup to test with is 2.32.2.
Comment 19 Milan Crha 2011-09-29 12:26:36 UTC
Thanks for the update and such extensive testing. I'm committing it right now.
Comment 20 Milan Crha 2011-09-29 12:31:15 UTC
Created commit 14f1520 in evo master (3.3.1+)
Created commit 08c9a75 in evo gnome-3-2 (3.2.1+)