GNOME Bugzilla – Bug 631034
Message cache keeps zero-sized mails on cancel
Last modified: 2010-10-18 22:05:48 UTC
I semi-regularly get emails that appear in the message list fine (From, Subject, Date fields populated ), but that 'appear' to be empty ( preview pane is blank and opening the email results in an empty 'viewer' ). The emails are not empty -- viewing the email in another client provides the full text of said emails. I'm not sure if the issue is that in some cases evo is not downloading/storing the body of the email (i'm using OWA ) or if evo is failing on the rendering of the email. Sorta hoping it's the former and that there's a quick/easy way to get evo to download said email again. I saved an example message that exhibits this issue for me using evo's 'save as mbox' functionality. It's attached -- the body of the email is missing. Here is the email source taken from mutt: $ cat cur/1285764408.11663_116.raker\:2\,S Received: from mailblocker.ateb.com ([172.16.48.222]) by sr002-2k3exc.ateb.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 29 Sep 2010 05:53:22 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvcBAOOookzILtBqkWdsb2JhbACDHZ5+FQEBAQEJCwoHEQMftCU8ghcBBZANAQSBIoMudI09 X-IronPort-AV: E=Sophos;i="4.57,252,1283745600"; d="scan'208";a="6990640" Received: from mx1.hub.org ([200.46.208.106]) by mailblocker.ateb.com with ESMTP; 29 Sep 2010 05:53:21 -0400 Received: from postgresql.org (mail.postgresql.org [200.46.204.86]) by mx1.hub.org (Postfix) with ESMTP id 5C671326760A; Wed, 29 Sep 2010 09:53:19 +0000 (UTC) Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 29B0F1336FA7 for <pgsql-general-postgresql.org@mail.postgresql.org>; Wed, 29 Sep 2010 06:52:54 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 48091-09 for <pgsql-general-postgresql.org@mail.postgresql.org>; Wed, 29 Sep 2010 09:52:48 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from web29017.mail.ird.yahoo.com (web29017.mail.ird.yahoo.com [212.82.110.163]) by mail.postgresql.org (Postfix) with SMTP id 7669C1336971 for <pgsql-general@postgresql.org>; Wed, 29 Sep 2010 06:52:45 -0300 (ADT) Received: (qmail 4544 invoked by uid 60001); 29 Sep 2010 09:52:45 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.it; s=s1024; t=1285753965; bh=bt6qyE2Sa5Nzv6I5P72mA5Xc/1nA/KgLISPI7Z7/Gig=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=EgzfR+3lPMjRNOh6OqDhRBe7lGv3fKX4vOysxeSEtWXcDKNXcbubwbA2ZPTo1qJmtYXPzt2LvgbaJ+rtCG2Udcqcw8gmy6/oEDQNi5Ad+4qXEzzu6S7BfXQGMGWSCeuLHfwSHxB9RGulVh9yM7FakYHGEVkyR0FebXOuI1cpGrc= DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.it; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=YToSHlZPEDO/NPc6vepZNNF2xH2bpBKMZ4oC48U3Oa1twt7vf+E+HfFFbkzc/Y4KwBbEBfayKS4jB+i0EsqdXF4y4FdImvJRIUhzPkVOBkp0d+Zjnb+1lkenCjb+LMKXUqqGE2B/szkBWfhNqSw58uNPbgV6KsC6t+sy/kUk9yg=; Message-ID: <165530.4179.qm@web29017.mail.ird.yahoo.com> X-YMail-OSG: jqLQYicVM1n6QRk28r1cZtuZxhHAphHh.qsoEZjHoWxktMU PgbzKHICRZtXILtg5tWba0Jc_Pqx9gJjAaMyuqufxAWMwGuI59MHOa5X6Dvt usAxkf3U9E6EMYd1mnIO7Qn5daArKY9cmSEdIHCao92D3b4Wfp5M47Jb5kOG pp69mrFF3I8mMH7oxpcBE6zMwnV9z1N5MmRB_hUSJ40BIpttoAosgfTU- Received: from [85.33.62.89] by web29017.mail.ird.yahoo.com via HTTP; Wed, 29 Sep 2010 09:52:45 GMT X-Mailer: YahooMailRC/497 YahooMailWebService/0.8.105.279950 Date: Wed, 29 Sep 2010 09:52:45 +0000 (GMT) From: Leonardo Francalanci <m_lists@yahoo.it> Subject: [GENERAL] Postgresql for a CEP app To: pgsql-general <pgsql-general@postgresql.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Mailing-List: pgsql-general List-Archive: <http://archives.postgresql.org/pgsql-general> List-Help: <mailto:majordomo@postgresql.org?body=help> List-ID: <pgsql-general.postgresql.org> List-Owner: <mailto:pgsql-general-owner@postgresql.org> List-Post: <mailto:pgsql-general@postgresql.org> List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-general> List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-general> Precedence: bulk Sender: pgsql-general-owner@postgresql.org Return-Path: pgsql-general-owner+M167801@postgresql.org X-OriginalArrivalTime: 29 Sep 2010 09:53:22.0219 (UTC) FILETIME=[26F3BFB0:01CB5FBC] Content-Length: 1493 Lines: 43 Hi, I need to generate aggregates of data coming from a stream. I could easily doing it inserting data coming from the stream into a table, and then query it using something like: select <my aggregation function> from atable group by <a column list> The problem with this approach is that I would have to wait for the whole stream to be finished before making the above query; since we're talking about 20M+ rows, it would take some time for the query to finish. What if I do something like: select <my aggregation function> from my_fifo_function([...])=20 group by <a column list> where my_fifo_function reads data from the stream and returns "rows" as soon as they are available on the stream? This way I would get the reply as soon as the stream has finished (assuming postgresql can keep up with that). In other words, the query would be made even before the stream has "started", and would last at least as long as the stream. (of course, I don't need data from the stream to be saved in any way, that's why I don't need to store it in any table). Does postgresql read data from a function returning a SETOF row by row? Or it waits the whole function to be finished (caching the whole resultset) before starting to use the returned data? If it reads row by row I think it could work... Would that make sense?=20 --=20 Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Milan's response... Hi, I saw something similar quite long time ago, in IMAP, and it was enough to select a different message and then back this one, and the body was shown. But it might not be your case, and I saw it very rarely, in time of 2.27 or the most 2.29. Look at ~/.evolution/mail/exchange/<account>, there is stored your local cache of emails. You can try to find the exact body of the message and see whether it was really downloaded "empty" from the server. You can remove the file, when evo is closed, and force it redownload. If you see this consistently with some messages, then we may investigate further, the best through bugzilla. Bye, Milan _________________________
I cannot find any reference to this email (and a second one exhibiting the same issue) under ~/.evolution/mail/exchange/<account>. When selecting these two emails, the window briefly displays "Retrieving Message..." where the ids for the two current issue messages are N leading zeros 742a83c and 742a83cd (this flashes really fast, so i'm not exactly sure of the count of leading 0's -- ). Where does evo store the information that it's displaying in the message list (Subject, From, etc)? View message source for the above noted email only contains: X-Evolution-Source: exchange://reid.thompson@sr002-2k3exc.ateb.com/ From: Date: Thu, 30 Sep 2010 14:55:46 -0400 Subject: No Subject Message-ID: <1285872946.14396.1.camel@raker.ateb.com> Mime-Version: 1.0
So is this with evolution-exchange with today's (or yesterday's) git master of it? If so, then the files are stored in ~/.local/share/evolution/mail/exchange/<account>/... and there should not be any ~/.evolution (it's migrated to new directory structure on the first start and if everything goes fine then it removed the ~/.evolution after that (when there doesn't left any file after migration). Folder summary, the values shown in the message-list, are stored in folders.db file, one table (inside the file) per folder (plus two additional tables, but that's not interesting now). The folders.db file is stored in .../mail/exchange/<account>/folders.db Depending on your version and place of local mail cache I would also try to run evolution with E2K_DEBUG=5 and redownload the message, if it's consistently reproducible with it. It may show whether proper body was received and only evolution-exchange mangled it to empty email.
Well, that makes a difference ;). I found the cached files 742a83c and 742a83cd under .local/. They were empty. I rm'd them, and then selected them in evo (no restart). Evo re-downloaded and displayed them fine. My guess is that I manage to interrupt Evo during the initial cache download after the cache file is created, but before it is populated. The folders.db notes a size column, any possibility of auto-forcing a reload if cached file size is 0 and folders size column != 0;
or adding a menu option to force a re-download of selected email(s)?
ok -- i can repeat this issue. If I select an email that has not been cached yet and quickly select another email it appears that the cache file gets created with 0 size but the download for the cache file is cancelled. This results in a cached empty file for the message. Going back to the message results in 'loading' of the empty cache file - i.e. an 'empty' email. If I rm the empty cache file and select the message again, and wait for the message retrieval to finish, then everything is fine. rm'ing the fully cached file, selecting the message again and interrupting the cache download again by quickly selecting another message will again exhibit the creation of a 0 size cache file.
Nice investigative work! I guess we need to check for and possibly even delete empty cache files before claiming that we have a cached copy.
Thanks Reid, good job with investigation :) evolution-exchange shouldn't create an empty cache files for sure, and checking on message select for the cache file size is also a good thing to do.
Created attachment 171498 [details] [review] proposed eex patch for evolution-exchange; This should make it, I guess, though I didn't test it. The issue is when the operation thread is cancelled, as discovered David Woodhouse in another bug, that stream rejects writing to itself and thus the result is an empty file, event there were some data to be written initially. It'll be great if you could test this change.
patch invocations are missing a cancellable parameter it seems? What should cancellable be for CC libcamelexchange_la-camel-exchange-store.lo ../../../evolution-exchange/camel/camel-exchange-folder.c: In function ‘exchange_folder_append_message_data’: ../../../evolution-exchange/camel/camel-exchange-folder.c:87: error: too few arguments to function ‘camel_stream_write’ ../../../evolution-exchange/camel/camel-exchange-folder.c:87: error: too few arguments to function ‘camel_stream_flush’ ../../../evolution-exchange/camel/camel-exchange-folder.c: In function ‘exchange_folder_get_message_data’: ../../../evolution-exchange/camel/camel-exchange-folder.c:124: error: too few arguments to function ‘camel_stream_write_to_stream’ ../../../evolution-exchange/camel/camel-exchange-folder.c:161: error: too few arguments to function ‘camel_stream_write’ ../../../evolution-exchange/camel/camel-exchange-folder.c:162: error: too few arguments to function ‘camel_stream_flush’
I presume ? the passed in cancellable parameter ?
patched as: myeex.patch attachment (using passed in cancellable) results: no 0 size files are created, fixes original issue side effect: if I select a cached email it displays in the preview pane fine. if I select a non-cached email and quickly click back to a cached email, the cached email does not display in the preview pane It appears that cancelling the download of a non-cached email by selecting a cached email does not invoke loading of the cached email for viewing in the preview pane. Clicking on another cached email displays in the preview pane, then if I click back to the 'does not invoke loading' email, it displays fine.
Created attachment 171523 [details] [review] shows the cancellable addition i put in just to show the change i did in the patch, in case what I did was incorrect.
(In reply to comment #11) > I presume ? the passed in cancellable parameter ? Yes, that's the one to use.
Thanks for testing. My initial patch is for gnome-2-32, as I had a feeling you are using the stable branch now, but you obviously do not :) (In reply to comment #12) > side effect: > if I select a cached email it displays in the preview pane fine. > if I select a non-cached email and quickly click back to a cached email, the > cached email does not display in the preview pane > It appears that cancelling the download of a non-cached email by selecting a > cached email does not invoke loading of the cached email for viewing in the > preview pane. Clicking on another cached email displays in the preview pane, > then if I click back to the 'does not invoke loading' email, it displays fine. If I got it right, these all are describing good behaviour, right?
Created commit 1309bc5 in eex master (2.91.2+) Created commit 026e05b in eex gnome-2-32 (2.32.1+)
I was ill over the weekend, so am just seeing the latest comments. (In reply to comment #15) > Thanks for testing. My initial patch is for gnome-2-32, as I had a feeling you > are using the stable branch now, but you obviously do not :) > > (In reply to comment #12) > > side effect: > > if I select a cached email it displays in the preview pane fine. > > if I select a non-cached email and quickly click back to a cached email, the > > cached email does not display in the preview pane > > It appears that cancelling the download of a non-cached email by selecting a > > cached email does not invoke loading of the cached email for viewing in the > > preview pane. Clicking on another cached email displays in the preview pane, > > then if I click back to the 'does not invoke loading' email, it displays fine. > > If I got it right, these all are describing good behaviour, right? Not quite, IMO. I would expect: if I select a cached email it displays in the preview pane fine. if I select a non-cached email and quickly click back to a cached email, I would expect the cached email to load and display in the preview pane. I would not expect to have to click on another cached email, then back to the non-loading cached email to get it to load
Aah, I see, I should learn reading more carefully. I think I saw something similar on IMAP, when "quickly moving in a folder". I end up with the preview pane filled "Receiving message 'xxx'", but it actually does nothing. It would worth fixing it/filling a new bug report for that, if not here yet.
I've seen a very similar thing with Exchange: https://bugzilla.redhat.com/show_bug.cgi?id=595613 Just for posterity. Can this be backported to 2.32.x please?
Ah, never mind - I see it's already been committed to .32 branch.