GNOME Bugzilla – Bug 793691
Implement Direct Read Access for calendar backends
Last modified: 2021-05-19 11:03:22 UTC
Currently a lot of calendar data goes through the bus on a normal desktop session that wants to display all events for the day/week/month in the shell or in an applet. Implementing Direct Read Access (DRA) for calendar backends would make it possible for EDS clients to just use the Bus for writes and signaling.
As we spoke either on the mentioned bug #793031 or on IRC, DRA is currently possible with address books only. It's done semi-transparently there, the books which do not support DRA (there are some expectations on the backend store to tbe able to read from it) are opened the old way, even when asked to open it with DRA. With DRA, only write operations go through D-Bus, read is done directly from the disk/SQLite table. While there had been discussed a possibility to open books with DRA by default, it was not done yet. What this bug asks for is to provide similar functionality to calendar backends (including tasks and memos, which is done for free). Having this made on top of ECalMetaBackend will provide the functionality to any backend which derives from it, similar to offline work support.
(In reply to Milan Crha from comment #1) > As we spoke either on the mentioned bug #793031 or on IRC, DRA is currently > possible with address books only. It's done semi-transparently there, the > books which do not support DRA (there are some expectations on the backend > store to tbe able to read from it) are opened the old way, even when asked > to open it with DRA. With DRA, only write operations go through D-Bus, read > is done directly from the disk/SQLite table. While there had been discussed > a possibility to open books with DRA by default, it was not done yet. > > What this bug asks for is to provide similar functionality to calendar > backends (including tasks and memos, which is done for free). Having this > made on top of ECalMetaBackend will provide the functionality to any backend > which derives from it, similar to offline work support. Which backends are not derived from ECalMetaBackend?
(In reply to Alberto Ruiz from comment #2) > Which backends are not derived from ECalMetaBackend? On This Computer (aka local), to avoid double-saving, once on the disk (in a single .ics file) and in the database, Birthdays & Anniversaries (which has no real content, it's generated on the fly from requested address books), the Weather backend (that's also generated on the fly). The other which are in eds, evo-ews and evo-mapi do derive from the ECalMetaBackend. I cannot speak of any other 3rd-party backends, of course.
(In reply to Milan Crha from comment #3) > (In reply to Alberto Ruiz from comment #2) > > Which backends are not derived from ECalMetaBackend? > > On This Computer (aka local), to avoid double-saving, once on the disk (in a > single .ics file) and in the database, Birthdays & Anniversaries (which has > no real content, it's generated on the fly from requested address books), > the Weather backend (that's also generated on the fly). The other which are > in eds, evo-ews and evo-mapi do derive from the ECalMetaBackend. I cannot > speak of any other 3rd-party backends, of course. Can clients locate the .ics file? A client could choose to use libical to read contents of the .ics file in that case right?
Yes, it can locate it, but when you "open it with libical", then it is read completely into memory. Imagine years of events, literally thousands saved in the file, then you read it into memory and then use only few of them from the recent time. That's wasting resources from my point of view. Furthermore, it would mean two different implementations of DRA and I strongly disagree to have such complex code to maintain.
(In reply to Milan Crha from comment #5) > Yes, it can locate it, but when you "open it with libical", then it is read > completely into memory. Imagine years of events, literally thousands saved > in the file, then you read it into memory and then use only few of them from > the recent time. That's wasting resources from my point of view. > Furthermore, it would mean two different implementations of DRA and I > strongly disagree to have such complex code to maintain. That makes sense. So perhaps my question then is, what's the real downside of caching the .ics file to something that it's a lot faster to read/parse such as sqlite? I undertand that duplication is an issue, but at the end of the day we could use the mtime of the .ics file to synchronize and save a lot of parsing and memory that way.
The main downside is the on-disk duplication, yes. It can be understandable with custom local calendars, but with those having their .ics files in the evolution private directory (~/.local/share/evolution/{calendar|memos|tasks}/) it's rather wasting. The other part I can think of is when saving changes. In that case you read the whole cache into memory, flush it to the disk as an .ics calendar and then free the memory again. Such thing would have pretty bad impact on performance. As a compromise for the On This Computer calendars/memos/tasks, I think those with non-custom .ics file (which might or might not be a majority) could avoid saving into the .ics file completely, which will save time. Only those with custom .ics file would save twice and suffer of the performance hit on save.
(In reply to Milan Crha from comment #7) > The main downside is the on-disk duplication, yes. It can be understandable > with custom local calendars, but with those having their .ics files in the > evolution private directory > (~/.local/share/evolution/{calendar|memos|tasks}/) it's rather wasting. The > other part I can think of is when saving changes. In that case you read the > whole cache into memory, flush it to the disk as an .ics calendar and then > free the memory again. Such thing would have pretty bad impact on > performance. My understanding is that .ics files are sequential right? There shouldn't be any need to put it all in memory, you could do it sequentially event by event (think SAX vs DOM in XML land). > As a compromise for the On This Computer calendars/memos/tasks, I think > those with non-custom .ics file (which might or might not be a majority) > could avoid saving into the .ics file completely, which will save time. Only > those with custom .ics file would save twice and suffer of the performance > hit on save. I really like this approach. Let the default backend for "On This Computer" be sometihng based on ECalMetaBackend that doesn't pull data remotly and performs writes directly onto the cache... This would solve our problem and would limit the non DRA backend problem to local .ics sources and third party backends right?
(In reply to Alberto Ruiz from comment #8) > My understanding is that .ics files are sequential right? Yes, just like (similar to) mbox format for messages. > There shouldn't be any need to put it all in memory, you could do it > sequentially event by event (think SAX vs DOM in XML land). Well, libical is used to parse those files. I didn't look whether there's any API to read them one-by-one while free the component as soon as possible, but I doubt there is anything like that. There are some rules I'm not going to explain in lengthy here (also because of the below) which are kind of mandated by libical itself (as I understand it). > This would solve our problem and would limit the non DRA backend problem to > local .ics sources and third party backends right? As far as I know, yes. The problem with custom local .ics files is that it might mean to maintain two different backends for the same source kind, which is not ideal.
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines and create a new ticket at https://gitlab.gnome.org/GNOME/evolution-data-server/-/issues/ Thank you for your understanding and your help.