After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 604029 - Modified recurring events are shown as single instance
Modified recurring events are shown as single instance
Status: RESOLVED FIXED
Product: evolution-mapi
Classification: Applications
Component: Calendar
0.28.x
Other Linux
: Normal major
: ---
Assigned To: evolution-mapi-maint
evolution-mapi-maint
: 604598 613846 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2009-12-07 22:21 UTC by Paul Smith
Modified: 2011-02-10 12:46 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Screen shot of appointment appering in outlook (84.40 KB, image/png)
2010-02-18 20:56 UTC, Jeff Rubin
  Details
Screen shot of appointment details in outlook (121.86 KB, image/png)
2010-02-18 20:59 UTC, Jeff Rubin
  Details
ema patch (1.10 KB, patch)
2011-01-28 12:13 UTC, Milan Crha
committed Details | Review
Fix for RRULE bin parsing (2.09 KB, patch)
2011-02-10 08:04 UTC, sean finney
committed Details | Review

Description Paul Smith 2009-12-07 22:21:35 UTC
When I create an Evolution MAPI account connecting to my Exchange 2007 server, then select the "Calendars" section, I do see meetings there from my Exchange calendar.

However, I only see singleton meetings; no recurring meetings are shown at all.  Actually, what MIGHT be true is that only the first recurring meeting is shown (but not marked as recurring) and none of the subsequent recurrences are shown.  Note that these meetings were created by other people, from Outlook, and I accepted them using Outlook as well so they are already in my Calendar.  They are shown fine in Outlook.  Then I create a MAPI account and it thinks for a long time downloading my calendar info when I click "Calendars", but no recurrences of meetings show up.
Comment 1 Paul Smith 2009-12-07 22:40:51 UTC
If I create a meeting in Evolution that is recurring, then that meeting shows up on both my Evo and Outlook calendars as recurring.
Comment 2 Paul Smith 2009-12-07 22:44:06 UTC
If I create a new meeting in Outlook, after my Evo account is created, it never seems to show up in my Evo calendar at all.  Is there something I have to do to make them "sync" again?
Comment 3 Paul Smith 2009-12-07 23:06:16 UTC
OK, I waited longer and the meeting I created in outlook did eventually show up in my Evo calendar.  However, even though I created it as a recurring meeting in Outlook and it does show up as recurring there, the meeting in Evolution is a singleton meeting (only the first occurrence) and has no recurring meetings.  It is not marked as recurring in Evo.
Comment 4 Paul Smith 2009-12-08 02:50:07 UTC
If you think the Importance is just "Major" then please bump the Priority to "Urgent".
Comment 5 Akhil Laddha 2009-12-08 04:08:58 UTC
Hi Paul, it could be due to bug 600560 because i see outlook created recurrence events in evolution without any issue.
Comment 6 Paul Smith 2009-12-08 05:31:37 UTC
I built my version of Evo from gnome-2.28 GIT branch, updated as of this morning.  I just checked and there are no updates and I definitely have this change (I checked my source tree).

Also, my symptoms are different; I definitely do get calendar entries for meetings, etc. in the future not just in the past.

The problem is very specifically with recurring meetings: no recurring meeting created by outlook shows up as recurring in Evolution: I get the first occurrance of the meeting only, and the meeting is not marked recurring.

This should be pretty straightforward to reproduce, I'd think... If you need help debugging please let me know.

Without recurring meeting support the calendar definitely has a major functionality hole.
Comment 7 Paul Smith 2009-12-08 05:33:18 UTC
BTW, comment 5 is exactly backwards: if I make the recurring meeting in evolution it shows up fine everywhere (both outlook and evo).  If I make the meeting in outlook, it shows up as recurring in outlook but as a singleton meeting in evo.
Comment 8 Etienne Lequeux 2009-12-11 14:32:22 UTC
This to confirm Paul's report in #6. (also build evo-mapi from gnome-2.28 git).
Comment 9 Akhil Laddha 2009-12-15 08:20:45 UTC
*** Bug 604598 has been marked as a duplicate of this bug. ***
Comment 10 Jeff Rubin 2010-01-13 01:16:47 UTC
Another confirmation of this report.  Evolution 2.28.2
Any ideas on when it will be fixed?  I agree that it is a huge
functionality hole.  I can't resume using Evolution until this
is fixed.
Comment 11 Paul Smith 2010-01-15 19:47:54 UTC
I can verify I still see this with the latest git code as of late yesterday.  This bug makes Exchange calendaring in Evo useless, which means Evolution cannot be used with Exchange if you need calendaring support (which is almost always).  Without this you might as well use IMAP to connect to your Exchange server.
Comment 12 Milan Crha 2010-02-18 19:34:49 UTC
I created three types of events in OWA, a) never ends, b) ends after X occurrences, c) ends by [date]. All three seems to be shown as expected. I tried this with Exchange 2007, same as yours, it seems, the actual master.

Could you try to attach here a recurrence setup screen shot, please? Maybe your Outlook is offering yet another type of recurrence?
Comment 13 Jeff Rubin 2010-02-18 20:56:59 UTC
Created attachment 154175 [details]
Screen shot of appointment appering in outlook

First of two screen shots: this one show appointment
appearing in outlook calendar.  Doesn't appear in
evolution calendar.
Comment 14 Jeff Rubin 2010-02-18 20:59:04 UTC
Created attachment 154176 [details]
Screen shot of appointment details in outlook

Second of two screen shots: shows details of recurring appointment.

Note: I have other recurring appointments that do show up in evolution.
One, in particular, has no end date, just like this one.
Comment 15 Milan Crha 2010-02-19 10:51:14 UTC
Thanks for the update. I tried to create exactly the same event in outlook, and it is shown with mine evolution 2.29.91 and evolution-mapi 0.29.91. More than that, I created the event in Outlook, then gone to Evolution, and it appeared in a minute.

Two possibilities I can think of:
a) your outlook is configured to work in a "cached mode", which means it doesn't write your changes immediately to the server, but some time later
b) I noticed you created the event on Feb 19, but give it a start date on Feb 12, thus in the past. If I recall correctly there were some issues with retrieving past events, but it seems to work fine for me now.
Comment 16 Jeff Rubin 2010-02-19 17:22:10 UTC
I saw the appointment in firefox which was connected to the exchange
server through ova.  So it wasn't that it was cached and not written
to the server.

I will wait until the stable release (in Fedora 12) is version >= 2.29.2
and try it again then.
Comment 17 Milan Crha 2010-02-22 10:35:36 UTC
(In reply to comment #16)
> I will wait until the stable release (in Fedora 12) is version >= 2.29.2
> and try it again then.

There (in Fedora 12) will be no 2.29.x, it's in Fedora 13. There will be an upstream release of 2.28.3 on March 1st, thus some time later after that will be this propagated to Fedora 12 as well. But I do not know how this is working there, because it worked fine for me on 2.29.91, which is a development version of 2.30.x (Fedora 13 material).
Comment 18 Akhil Laddha 2010-03-25 03:33:57 UTC
*** Bug 613846 has been marked as a duplicate of this bug. ***
Comment 19 Akhil Laddha 2010-06-01 04:11:40 UTC
Can anybody confirm the bug on evolution-mapi 0.30.1 ?
Comment 20 Philippe Gauthier 2010-06-01 20:01:32 UTC
Tested 0.30.1 with the Fedora 13 Live CD. Some recurring appointments are now showing (compared to 0.28.3), but not all. Based on the previous comments, it may be related to the meetings having an end date or not. One recurrent meeting with an end date is now visible, but two meetings without an end date still don't show.
Comment 21 Dane 2010-06-03 05:43:15 UTC
Hello,

Fedora 13 with evolution-2.30.1-6.fc13.x86_64 & evolution-mapi-0.30.1-1.fc13.x86_64 running against a exchange 2007 installation.

I can confirm this is still an issue. Please provide a list of steps/information you'd like me to try/supply.

Thanks
Comment 22 Dane 2010-06-03 06:35:48 UTC
This is not a lack of collecting event history. I've been about to find the first event in reoccuring meetings dating back several months.

I would say it seems to be missing logic around handling different ways of specifying reoccuring calendar events -- such as you can say weekly, starting Tuesday June 1st and stopping June 29th or you can say daily, every 7 days, starting June 1st, with 5 occurances. It seems the way of handling this information is not consistant.

I was able to workaround this by using google calendar sync to sync my outlook calendar to google and then adding a google calendar. Sadly this won't work for new/incoming calendar events.
Comment 23 Milan Crha 2010-06-03 09:04:48 UTC
Thanks for the update. Could you try these steps, please:
a) close evolution, and e-calendar-factory, e-addressbook-factory
b) remove the offending event from
   ~/.evolution/cache/calendar/<mapi-account>/cache.xml
   file, or just change its UID, so it'll appear as a different event when
   synchronizing with the cache (search file by subject, for example)
c) run on the console:
   $ MAPI_DEBUG=10 /usr/libexec/e-calendar-factory &>ecf.log
d) run evolution on another console:
   $ evolution -c calendar
e) let it finish synchronization of events (check you see the event in
   the past) and then close evolution
f) after ~ 10 seconds should the e-calendar-factory stop itself, or kill
   it same as in a)

Then the ecf file will contain detailed communication between evolution-mapi and the server. Also, please make sure you'll not include any private information in the log, like server addresses, user names or passwords, but if you'll not be sure, then feel free to send the log directly to me, with a bug number in the subject. Please give me the event subject, so I'll be able to search for it in the file.

Note, you can erase whole file in b), but then the initial fetch will take too long, and will be awfully long, as the protocol is quite chatty.

That's true that there are some comments in the code about not supporting recurrences on reading, but as it seemed to work fine for me whatever I set in outlook/OWA, then it seemed as a lost comment to me.
Comment 24 gyhor 2010-06-04 09:30:52 UTC
thanks Milan, i tried it and it is really chatty :)

But what i found is, that i get this message, when i try to search for a recurrent event.

libecal-Message: e_cal_recur_generate_instances_of_rule(): bogus component, does not have DTSTART.  Skipping...
Comment 25 Randy Goddard 2010-06-04 16:23:59 UTC
I'm seeing this same behavior in the following environment. I'll be happy to provide application dumps and/or debugging info, too.

Lucid Lynx
Evolution 2.28.3
Evolution-mapi 0.28.3-0ubuntu1
Exchange Server 2007
Comment 26 Dane 2010-06-28 23:57:18 UTC
I supplied the logs requested to Milan directly. Because it's an actual outlook calendar event I have not posted the log here.

I'm waiting for feedback from Milan
Comment 27 Milan Crha 2010-06-29 07:56:58 UTC
Thanks. I guess I found the most suspecting bits. I guess this is a recurring blob from which is decoded recurrence rule:
> rpc reply data:
> [0000] 00 00 00 00 BB FC 68 19   84 4C AD 46 8B 72 71 07   ......h. .L.F.rq.
> [0010] CA BA A5 02 FF 7F 00 00   00 00 00 00 C6 00 00 00   ........ ........
> [0020] 67 A5 89 A5 A5 A5 A5 A5   1D A5 A7 A4 95 A5 A7 A5   g....... ........
> [0030] B0 A5 F5 A5 C4 A5 C6 A5   CC A5 C3 A5 CC A5 C6 A5   ........ ........
> [0040] 85 A5 F6 A5 D1 A5 C4 A5   CB A5 C1 A5 C4 A5 D7 A5   ........ ........
> [0050] C1 A5 85 A5 F1 A5 CC A5   C8 A5 C0 A5 A7 A5 A7 A4   ........ ........
> [0060] 9B A5 A5 A5 73 A2 A5 A5   A5 A5 A5 A5 A5 A5 A5 A5   ....s... ........
> [0070] A5 A5 A5 A5 45 A4 A5 A5   A5 A5 A5 A5 61 5A 5A 5A   ....E... ....aZZZ
> [0080] A5 A5 AF A5 A5 A5 A0 A5   A7 A5 A5 A5 A5 A5 A5 A5   ........ ........
> [0090] A5 A5 A1 A5 A5 A5 A4 A5   A7 A5 A5 A5 A5 A5 A5 A5   ........ ........
> [00A0] A7 A4 9B A5 A7 A5 72 A2   A5 A5 A5 A5 A5 A5 A5 A5   ......r. ........
> [00B0] A5 A5 A5 A5 A5 A5 45 A4   A5 A5 A5 A5 A5 A5 61 5A   ......E. ......aZ
> [00C0] 5A 5A A5 A5 AE A5 A5 A5   A4 A5 A7 A5 A5 A5 A5 A5   ZZ...... ........
> [00D0] A5 A5 A5 A5 A6 A5 A5 A5   A7 A5 A7 A5 A5 A5 A5 A5   ........ ........
> [00E0] A5 A5 54 A6 A5 A5 C6 00   00 00 00 00              ..T..... ....

The recurrence blob should have size of 184 bytes from the above. I'll try to "decode" it, maybe evolution-mapi didn't understand the blob properly. I'll update when I'll find something. Thanks for your help with this.
Comment 28 Milan Crha 2010-06-29 17:09:44 UTC
Ah, I'm wrong, this is the beginning of the recurring blob, it should begin with 0x3004 0x3004 in little endian byte order:
> [0090] __ __ __ __ __ __ __ __   04 30 04 30 0B 20 01 00   ________ .0.0. ..
> [00A0] 00 00 20 49 00 00 04 00   00 00 00 00 00 00 02 00   .. I.... ........
> [00B0] 00 00 23 20 00 00 0A 00   00 00 00 00 00 00 01 00   ..# .... ........
> [00C0] 00 00 40 0B D5 0C 01 00   00 00 E0 10 D5 0C C0 0B   ..@..... ........
> [00D0] CD 0C DF 80 E9 5A 06 30   00 00 08 30 00 00 D0 02   .....Z.0 ...0....
> [00E0] 00 00 0C 03 00 00 01 00   B0 13 D5 0C EC 13 D5 0C   ........ ........
> [00F0] 10 0E D5 0C 21 02 17 00   16 00 __ __ __ __ __ __   ....!... ..______

It basically says that the event is weekly recurring and one instance was modified. When I try to do this (create a recurring event in OWA with no end date, say on June 30th at 5PM and I modify the second instance with changed subject and time shifted to 3PM), then the event isn't shown properly in Evolution, it shows only the first instance, most likely because it was unable to decode the RecurrencePattern structure. What I was trying were simple recurring events without modification, which worked fine for me.

I heard Bharath talking about rewriting calendar backend to use openchange's exchange2ical code, so I'm adding this to his bug list for more investigation and inspiration why would be good to do that, and also as a test.
Comment 29 Akhil Laddha 2010-06-30 03:35:32 UTC
other recurrence problems bug 586252, bug 586199
Comment 30 Dane 2010-06-30 03:54:47 UTC
Milan Crha,

I supplied a bad example log entry. You're right, I did send you a log for an updated/modified event, my apologies. It also doesn't handle simple reccuring events. I will find a better example and re-send you the log.

Apologies for the inconvience,
Dane
Comment 31 Milan Crha 2010-06-30 06:34:50 UTC
(In reply to comment #30)
> I supplied a bad example log entry. You're right, I did send you a log for an
> updated/modified event, my apologies. It also doesn't handle simple reccuring
> events. I will find a better example and re-send you the log.

No no no, please do not. This is a good example that evolution-mapi doesn't handle recurrences well, I'm fine with it.
Comment 32 Bharath Acharya 2010-07-16 04:09:07 UTC
I confirm the issue. Yes does happen with the current code in MAPI. 

Checked the scenario with exchange2ical too. Awesomeness :) A minor bug exists, but yes recurrence works neatly. So we have 2 choices,

1) Wait till we move to exchange2ical
2) Check the exchange2ical code and backport the fix into evolution-mapi

Even if we choose option 2 now, we know this particular bug might get fixed, but there still may be more loopholes and missing cases.

Ok let me target exchange2ical for 2.31.90 release. If it isn't done by then, I'll backport the fix from openchange for recurring event handling. Milan, does this sound good, or do you have better ideas for this one?
Comment 33 Milan Crha 2010-07-16 08:12:00 UTC
It's fine by me, unless 2.31.90 is too late (it shouldn't belong to feature freeze, neither add new strings or anything like that, just you'll be forced to fork anyway, because it'll be too late for an external module (openchange) version bump (I suppose)).
Comment 34 Paul Smith 2010-09-28 20:34:32 UTC
Hi all, any news on what happened/is happening with this for 2.32?
Comment 35 Paul Graham 2010-11-08 16:14:31 UTC
(In reply to comment #34)
> Hi all, any news on what happened/is happening with this for 2.32?

I am seeing this problem still with the Evolution 2.32.0 package (evolution-2.32.0-2.fc14.i686) and the Evolution MAPI connector package (evolution-mapi-0.32.0-1.fc14.i686) that come with Fedora 14, among other issues.
Comment 36 Milan Crha 2011-01-28 12:13:10 UTC
Created attachment 179506 [details] [review]
ema patch

for evolution-mapi;

I was finally brought to an example on our server with which I was able to test this and I realized that the ema is too restrictive with version checking, where the specification only says that the value 'should be', but it means it "must not", which ema wasn't respecting much.

This patch fixes the issue for me. Note that ema knew about the event, it only thrown away the recurrence information as not conforming what it was expecting.
Comment 37 Milan Crha 2011-01-28 12:17:57 UTC
Created commit fc48cc0 in ema master (2.91.6+)
Created commit abceecc in ema gnome-2-32 (0.32.2+)
Comment 38 sean finney 2011-02-08 14:34:51 UTC
FYI, modified recurring events are still not showing in 0.32.2 with abceecc applied, and this issue should be re-opened.

From what I can tell, later on near the bottom of the same function:

exchange_mapi_cal_util_bin_to_rrule (exchange-mapi-cal-recur-utils.c) 

    /* modified exceptions */
    flag16 = *((guint16 *)ptr);
    ptr += sizeof (guint16);
    if (flag16 != 0x0)
        return FALSE;

    /* reserved block1 size - has to be 0 */
    flag32 = *((guint32 *)ptr);
    ptr += sizeof (guint32);
    if (flag32 != 0x0)
        return FALSE;

    /* reserved block2 size - has to be 0 */
    flag32 = *((guint32 *)ptr);
    ptr += sizeof (guint32);
    if (flag32 != 0x0)
        return FALSE;

in my simple test case (create a daily recurrence in OWA, then modify a single instance of the recurrence with a different time and perhaps comments), i had to comment out all three if/return FALSE's.

This does not fix the entire problem, but allows the original recurrence to be parsed and displayed in evolution.  The modified recurrences still do not show, but the comments around line 635 seem to suggest that this is yet to be implemented.
Comment 39 Milan Crha 2011-02-08 14:38:14 UTC
Reopening per the previous comment, another case found.
Comment 40 sean finney 2011-02-10 08:03:40 UTC
Hi everyone,

Just to confirm my suspicions earlier, there was indeed a problem with the parsing.  namely there is a block of modified instances which is never parsed, and the code that is examining the two final "constant" Reader/WriterVersion[Size] is looking at the wrong data offset. 
from the patch comments:

 This patch fixes some incorrect parsing that would cause
 exchange_mapi_cal_util_bin_to_rrule() to incorrectly discard RRULE
 information when processing a recurrence containing modified
 instances.

 Note that this does *not* include a fix to actually parse the modified
 instances, which after some analysis will be fairly complicated to do and
 thus saved for a later point in time.

 The resulting behavior from this patch is that the recurrence will show
 in the calendar, with exception dates for the modified instances, but no
 corresponding events (i.e. the modified instances will be missing).

It's still a pretty big problem that the modified instances do not show up, but
after looking at the code and talking with Milan I think this will be
considerably more difficult (I will file a seperate issue about that), and this
patch is worth having in the meantime so that the recurrence at least shows up
in some form.
Comment 41 sean finney 2011-02-10 08:04:31 UTC
Created attachment 180557 [details] [review]
Fix for RRULE bin parsing
Comment 42 Milan Crha 2011-02-10 10:09:56 UTC
Thanks patch looks good. I'm keeping the new bug filling to you, please make here a note which it is for future reference. Thanks in advance.

Created commit f36cca5 in ema master (2.91.90+)
Comment 43 sean finney 2011-02-10 12:46:56 UTC
Okay: Bug #642022 :)