After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 631034 - Message cache keeps zero-sized mails on cancel
Message cache keeps zero-sized mails on cancel
Status: RESOLVED FIXED
Product: Evolution Exchange
Classification: Deprecated
Component: Connector
2.30.x
Other Linux
: Normal normal
: ---
Assigned To: Connector Maintainer
Ximian Connector QA
Depends on:
Blocks:
 
 
Reported: 2010-09-30 18:49 UTC by Reid Thompson
Modified: 2010-10-18 22:05 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
proposed eex patch (1.73 KB, patch)
2010-10-01 15:12 UTC, Milan Crha
committed Details | Review
shows the cancellable addition i put in (2.42 KB, patch)
2010-10-01 17:30 UTC, Reid Thompson
committed Details | Review

Description Reid Thompson 2010-09-30 18:49:29 UTC
I semi-regularly get emails that appear in the message list fine (From,
Subject, Date fields populated ), but that 'appear' to be empty
( preview pane is blank and opening the email results in an empty
'viewer' ).  The emails are not empty -- viewing the email in another
client provides the full text of said emails.  I'm not sure if the issue
is that in some cases evo is not downloading/storing the body of the
email (i'm using OWA ) or if evo is failing on the rendering of the
email.  Sorta hoping it's the former and that there's a quick/easy way
to get evo to download said email again.

I saved an example message that exhibits this issue for me using evo's
'save as mbox' functionality.  It's attached -- the body of the email is
missing.

Here is the email source taken from mutt:

$ cat cur/1285764408.11663_116.raker\:2\,S 
Received: from mailblocker.ateb.com ([172.16.48.222]) by sr002-2k3exc.ateb.com with Microsoft SMTPSVC(6.0.3790.4675);
         Wed, 29 Sep 2010 05:53:22 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AvcBAOOookzILtBqkWdsb2JhbACDHZ5+FQEBAQEJCwoHEQMftCU8ghcBBZANAQSBIoMudI09
X-IronPort-AV: E=Sophos;i="4.57,252,1283745600"; 
   d="scan'208";a="6990640"
Received: from mx1.hub.org ([200.46.208.106])
  by mailblocker.ateb.com with ESMTP; 29 Sep 2010 05:53:21 -0400
Received: from postgresql.org (mail.postgresql.org [200.46.204.86])
        by mx1.hub.org (Postfix) with ESMTP id 5C671326760A;
        Wed, 29 Sep 2010 09:53:19 +0000 (UTC)
Received: from maia.hub.org (maia-5.hub.org [200.46.204.29])
        by mail.postgresql.org (Postfix) with ESMTP id 29B0F1336FA7
        for <pgsql-general-postgresql.org@mail.postgresql.org>; Wed, 29 Sep 2010 06:52:54 -0300 (ADT)
Received: from mail.postgresql.org ([200.46.204.86])
 by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024)
 with ESMTP id 48091-09
 for <pgsql-general-postgresql.org@mail.postgresql.org>;
 Wed, 29 Sep 2010 09:52:48 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.6
Received: from web29017.mail.ird.yahoo.com (web29017.mail.ird.yahoo.com [212.82.110.163])
        by mail.postgresql.org (Postfix) with SMTP id 7669C1336971
        for <pgsql-general@postgresql.org>; Wed, 29 Sep 2010 06:52:45 -0300 (ADT)
Received: (qmail 4544 invoked by uid 60001); 29 Sep 2010 09:52:45 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.it; s=s1024; t=1285753965; bh=bt6qyE2Sa5Nzv6I5P72mA5Xc/1nA/KgLISPI7Z7/Gig=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=EgzfR+3lPMjRNOh6OqDhRBe7lGv3fKX4vOysxeSEtWXcDKNXcbubwbA2ZPTo1qJmtYXPzt2LvgbaJ+rtCG2Udcqcw8gmy6/oEDQNi5Ad+4qXEzzu6S7BfXQGMGWSCeuLHfwSHxB9RGulVh9yM7FakYHGEVkyR0FebXOuI1cpGrc=
DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.it;
  h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=YToSHlZPEDO/NPc6vepZNNF2xH2bpBKMZ4oC48U3Oa1twt7vf+E+HfFFbkzc/Y4KwBbEBfayKS4jB+i0EsqdXF4y4FdImvJRIUhzPkVOBkp0d+Zjnb+1lkenCjb+LMKXUqqGE2B/szkBWfhNqSw58uNPbgV6KsC6t+sy/kUk9yg=;
Message-ID: <165530.4179.qm@web29017.mail.ird.yahoo.com>
X-YMail-OSG: jqLQYicVM1n6QRk28r1cZtuZxhHAphHh.qsoEZjHoWxktMU
 PgbzKHICRZtXILtg5tWba0Jc_Pqx9gJjAaMyuqufxAWMwGuI59MHOa5X6Dvt
 usAxkf3U9E6EMYd1mnIO7Qn5daArKY9cmSEdIHCao92D3b4Wfp5M47Jb5kOG
 pp69mrFF3I8mMH7oxpcBE6zMwnV9z1N5MmRB_hUSJ40BIpttoAosgfTU-
Received: from [85.33.62.89] by web29017.mail.ird.yahoo.com via HTTP; Wed, 29 Sep 2010 09:52:45 GMT
X-Mailer: YahooMailRC/497 YahooMailWebService/0.8.105.279950
Date: Wed, 29 Sep 2010 09:52:45 +0000 (GMT)
From: Leonardo Francalanci <m_lists@yahoo.it>
Subject: [GENERAL] Postgresql for a CEP app
To: pgsql-general <pgsql-general@postgresql.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9
X-Spam-Level: 
X-Mailing-List: pgsql-general
List-Archive: <http://archives.postgresql.org/pgsql-general>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-general.postgresql.org>
List-Owner: <mailto:pgsql-general-owner@postgresql.org>
List-Post: <mailto:pgsql-general@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-general>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-general>
Precedence: bulk
Sender: pgsql-general-owner@postgresql.org
Return-Path: pgsql-general-owner+M167801@postgresql.org
X-OriginalArrivalTime: 29 Sep 2010 09:53:22.0219 (UTC) FILETIME=[26F3BFB0:01CB5FBC]
Content-Length: 1493
Lines: 43

Hi,


I need to generate aggregates of data coming from a stream.

I could easily doing it inserting data coming from the stream into a table,
and then query it using something like:

select <my aggregation function> from atable group by <a column list>

The problem with this approach is that I would have to wait for the whole
stream to be finished before making the above query; since we're talking
about 20M+ rows, it would take some time for the query to finish.

What if I do something like:

select <my aggregation function> from my_fifo_function([...])=20
group by <a column list>

where my_fifo_function reads data from the stream and returns "rows" as
soon as they are available on the stream? This way I would get the reply
as soon as the stream has finished (assuming postgresql can keep up
with that). In other words, the query would be made even before the
stream has "started", and would last at least as long as the stream.

(of course, I don't need data from the stream to be saved in any way,
that's why I don't need to store it in any table).

Does postgresql read data from a function returning a SETOF row by row?
Or it waits the whole function to be finished (caching the whole resultset)
before starting to use the returned data? If it reads row by row I think it
could work...


Would that make sense?=20




--=20
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Comment 1 Reid Thompson 2010-09-30 18:50:03 UTC
Milan's response...
        Hi,
I saw something similar quite long time ago, in IMAP, and it was enough
to select a different message and then back this one, and the body was
shown. But it might not be your case, and I saw it very rarely, in time
of 2.27 or the most 2.29.

Look at ~/.evolution/mail/exchange/<account>, there is stored your local
cache of emails. You can try to find the exact body of the message and
see whether it was really downloaded "empty" from the server. You can
remove the file, when evo is closed, and force it redownload.

If you see this consistently with some messages, then we may investigate
further, the best through bugzilla.
        Bye,
        Milan

_________________________
Comment 2 Reid Thompson 2010-09-30 18:56:41 UTC
I cannot find any reference to this email (and a second one exhibiting the same issue) under  ~/.evolution/mail/exchange/<account>.

When selecting these two emails, the window briefly displays "Retrieving Message..."  where the ids for the two current issue messages are
N leading zeros  742a83c and 742a83cd (this flashes really fast, so i'm not exactly sure of the count of leading 0's -- ).  Where does evo store the information that it's displaying in the message list (Subject, From, etc)?

View message source for the above noted email only contains:

X-Evolution-Source: exchange://reid.thompson@sr002-2k3exc.ateb.com/
From: 
Date: Thu, 30 Sep 2010 14:55:46 -0400
Subject: No Subject
Message-ID: <1285872946.14396.1.camel@raker.ateb.com>
Mime-Version: 1.0
Comment 3 Milan Crha 2010-10-01 07:33:06 UTC
So is this with evolution-exchange with today's (or yesterday's) git master of it? If so, then the files are stored in 
   ~/.local/share/evolution/mail/exchange/<account>/...
and there should not be any ~/.evolution (it's migrated to new directory structure on the first start and if everything goes fine then it removed the ~/.evolution after that (when there doesn't left any file after migration).

Folder summary, the values shown in the message-list, are stored in folders.db file, one table (inside the file) per folder (plus two additional tables, but that's not interesting now). The folders.db file is stored in

   .../mail/exchange/<account>/folders.db

Depending on your version and place of local mail cache I would also try to run evolution with E2K_DEBUG=5 and redownload the message, if it's consistently reproducible with it. It may show whether proper body was received and only evolution-exchange mangled it to empty email.
Comment 4 Reid Thompson 2010-10-01 13:14:51 UTC
Well, that makes a difference ;).   I found the cached files 742a83c and 742a83cd under .local/.  They were empty.  I rm'd them, and then selected them in evo (no restart).  Evo re-downloaded and displayed them fine.  My guess is that I manage to interrupt Evo during the initial cache download after the cache file is created, but before it is populated.

The folders.db notes a size column, any possibility of auto-forcing a reload if cached file size is 0 and folders size column != 0;
Comment 5 Reid Thompson 2010-10-01 13:17:18 UTC
or adding a menu option to force a re-download of selected email(s)?
Comment 6 Reid Thompson 2010-10-01 13:57:31 UTC
ok -- i can repeat this issue.  If I select an email that has not been cached yet and quickly select another email it appears that the cache file gets created with 0 size but the download for the cache file is cancelled.  This results in a cached empty file for the message.  Going back to the message results in 'loading' of the empty cache file - i.e. an 'empty' email.  If I rm the empty cache file and select the message again, and wait for the message retrieval to finish, then everything is fine.   rm'ing the fully cached file, selecting the message again and interrupting the cache download again by quickly selecting another message will again exhibit the creation of a 0 size cache file.
Comment 7 Matthew Barnes 2010-10-01 14:21:28 UTC
Nice investigative work!  I guess we need to check for and possibly even delete empty cache files before claiming that we have a cached copy.
Comment 8 Milan Crha 2010-10-01 14:47:31 UTC
Thanks Reid, good job with investigation :)

evolution-exchange shouldn't create an empty cache files for sure, and checking on message select for the cache file size is also a good thing to do.
Comment 9 Milan Crha 2010-10-01 15:12:49 UTC
Created attachment 171498 [details] [review]
proposed eex patch

for evolution-exchange;

This should make it, I guess, though I didn't test it. The issue is when the operation thread is cancelled, as discovered David Woodhouse in another bug, that stream rejects writing to itself and thus the result is an empty file, event there were some data to be written initially.

It'll be great if you could test this change.
Comment 10 Reid Thompson 2010-10-01 16:54:08 UTC
patch invocations are missing a cancellable parameter it seems?  What should cancellable be for 

  CC     libcamelexchange_la-camel-exchange-store.lo
../../../evolution-exchange/camel/camel-exchange-folder.c: In function ‘exchange_folder_append_message_data’:
../../../evolution-exchange/camel/camel-exchange-folder.c:87: error: too few arguments to function ‘camel_stream_write’
../../../evolution-exchange/camel/camel-exchange-folder.c:87: error: too few arguments to function ‘camel_stream_flush’
../../../evolution-exchange/camel/camel-exchange-folder.c: In function ‘exchange_folder_get_message_data’:
../../../evolution-exchange/camel/camel-exchange-folder.c:124: error: too few arguments to function ‘camel_stream_write_to_stream’
../../../evolution-exchange/camel/camel-exchange-folder.c:161: error: too few arguments to function ‘camel_stream_write’
../../../evolution-exchange/camel/camel-exchange-folder.c:162: error: too few arguments to function ‘camel_stream_flush’
Comment 11 Reid Thompson 2010-10-01 16:58:25 UTC
I presume ? the passed in cancellable parameter ?
Comment 12 Reid Thompson 2010-10-01 17:29:44 UTC
patched as: myeex.patch attachment (using passed in cancellable)

results:
  no 0 size files are created, fixes original issue

side effect:
  if I select a cached email it displays in the preview pane fine.
  if I select a non-cached email and quickly click back to a cached email, the cached email does not display in the preview pane
  It appears that cancelling the download of a non-cached email by selecting a cached email does not invoke loading of the cached email for viewing in the preview pane.  Clicking on another cached email displays in the preview pane, then if I click back to the 'does not invoke loading' email, it displays fine.
Comment 13 Reid Thompson 2010-10-01 17:30:57 UTC
Created attachment 171523 [details] [review]
shows the cancellable addition i put in

just to show the change i did in the patch, in case what I did was incorrect.
Comment 14 Matthew Barnes 2010-10-01 17:41:22 UTC
(In reply to comment #11)
> I presume ? the passed in cancellable parameter ?

Yes, that's the one to use.
Comment 15 Milan Crha 2010-10-04 07:46:26 UTC
Thanks for testing. My initial patch is for gnome-2-32, as I had a feeling you are using the stable branch now, but you obviously do not :)

(In reply to comment #12)
> side effect:
>   if I select a cached email it displays in the preview pane fine.
>   if I select a non-cached email and quickly click back to a cached email, the
> cached email does not display in the preview pane
>   It appears that cancelling the download of a non-cached email by selecting a
> cached email does not invoke loading of the cached email for viewing in the
> preview pane.  Clicking on another cached email displays in the preview pane,
> then if I click back to the 'does not invoke loading' email, it displays fine.

If I got it right, these all are describing good behaviour, right?
Comment 16 Milan Crha 2010-10-04 08:45:21 UTC
Created commit 1309bc5 in eex master (2.91.2+)
Created commit 026e05b in eex gnome-2-32 (2.32.1+)
Comment 17 Reid Thompson 2010-10-05 12:51:30 UTC
I was ill over the weekend, so am just seeing the latest comments.


(In reply to comment #15)
> Thanks for testing. My initial patch is for gnome-2-32, as I had a feeling you
> are using the stable branch now, but you obviously do not :)
> 
> (In reply to comment #12)
> > side effect:
> >   if I select a cached email it displays in the preview pane fine.
> >   if I select a non-cached email and quickly click back to a cached email, the
> > cached email does not display in the preview pane
> >   It appears that cancelling the download of a non-cached email by selecting a
> > cached email does not invoke loading of the cached email for viewing in the
> > preview pane.  Clicking on another cached email displays in the preview pane,
> > then if I click back to the 'does not invoke loading' email, it displays fine.
> 
> If I got it right, these all are describing good behaviour, right?

Not quite, IMO.
I would expect:
if I select a cached email it displays in the preview pane fine.
if I select a non-cached email and quickly click back to a cached email, I would expect the cached email to load and display in the preview pane.

I would not expect to have to click on another cached email, then back to the non-loading cached email to get it to load
Comment 18 Milan Crha 2010-10-05 13:01:38 UTC
Aah, I see, I should learn reading more carefully.

I think I saw something similar on IMAP, when "quickly moving in a folder". I end up with the preview pane filled "Receiving message 'xxx'", but it actually does nothing. It would worth fixing it/filling a new bug report for that, if not here yet.
Comment 19 Bojan Smojver 2010-10-18 22:04:13 UTC
I've seen a very similar thing with Exchange:

https://bugzilla.redhat.com/show_bug.cgi?id=595613

Just for posterity.

Can this be backported to 2.32.x please?
Comment 20 Bojan Smojver 2010-10-18 22:05:48 UTC
Ah, never mind - I see it's already been committed to .32 branch.