I replied to your email on the mailing list and then discovered this bug, so I'm kind of re-answering.

If you type "from:bug" in the Search box, do you see the message you're expecting to see?

Is it possible the message is located down the list?  Search results are sorted in descending order.

Server search is filed at bug #714742.

Created attachment 281056 [details]
The message when browsing the message list

Created attachment 281057 [details]
The message does no"t appear when searching "from:bug"

Created attachment 281058 [details]
Geary window when searching for "bug"

I myself have seen this problem from time to time.  And in fact I'm seeing it now.  I was just looking for a past conversation with subject "signed lease attached", so I searched for [lease].  To my surprise it didn't appear.  I then went to All Mail and scrolled down until I saw that conversation: its last message was on June 27.  That confirmed that the message was actually in Geary, but I tried searching again and the message still wasn't found.

I then exited Geary, moved my database directory (~/.local/share/geary/adam@medovina.org) to another folder, restarted Geary and created a fresh account with my email address (adam@medovina.org).  In that account, I scrolled down in All Mail until I saw the signed lease message.  I then searched.  In this instance, the search found the message.

I then exited Geary, restored my database directory to the version I originally had and searched again.  Once again, the message was not found.

So it appears that my database (or the SQLite state?) is somehow corrupted, preventing Geary from finding this one message at least.

I think investigating this should be a pretty high priority since local search is a key feature for Geary.  Let me know how I can help.  Jim, I'd be willing to tar up my database directory and email it to you if you think it would help you investigate.

Adam, if you could tar up both databases (or at least the one with the problem, if you wiped the other) that would be great.  I have looked into this problem a few times but can't find a repro case on my own database nor looking through the search table can I find anything wrong.  Some hard data would be very helpful here.

Sure - I should be able to do that.  One question: is my email password stored in the database directory itself?

No, passwords are stored in your keyring.  I don't need the attachments or the .ini file, just geary.db.

Hi Jim, I just sent you my database.
Note that it is reproductible in my case: if I remove the database and rebuild it with geary, the message still does not appear in the search.

Frédéric's database helped me immensely.  It turns out his problem is due to a comparison method not falling back on a stable sort when two dates were the same.  That's why the one message wasn't showing up, there was another message in the search results with the same date/time.  Coincidence, but not improbable when dealing in machine-generated emails delivered in batches.

Adam, I don't know that this solves your problem, but it could very well.  I've posted a branch (wip/733271-search) with the fix.  If you want to try it out and let me know, that would be great.

I've pushed this fix to master, commit 9933f2.  Adam, if you pull from master you can see if this fixes your problem.  (At this point, I suspect it doesn't, as this is not something that will change if you re-build your database; the sort collision was due to date/times stored on the server.)  However, this is an important fix and I wanted to get it in.

I'll leave this ticket open until I hear from you, Adam.

Jim, I'm glad that Frédéric's database helped you find a bug.  Unfortunately even after rebuilding from master I'm still seeing this: when I search for [lease] Geary doesn't find a message with subject ""signed lease attached" that is visible in All Mail.

I'm currently attempting to upload a compressed copy of my geary.db to Google Drive so I can share it with you.  I'm on vacation in Maine where my Internet connection is over a cell phone and only works sporadically, so I'm not sure whether the upload will succeed.  If so, I'll let you know; if not I'll upload it when I get back to Massachusetts next week.

I'm now back where I have a decent Internet connection and would be able to upload my database, but unfortunately at this moment I can no longer reproduce this: a search for 'lease' does find the message I was unable to find before.  Drat.

I'm pretty sure I've seen this bug from time to time in the past, and will be sure to post again here the next time I see this.

The search stabilization fix has been released w/ 0.6.2.

Recently installed 0.8.0, and now I have this bug.  Search for the email address of a close friend and I only find 5 conversations (we have *far* more email threads than that).  Upgrade to 0.8.1: same problem.

Tried installing 0.6.3, and I still have this problem.  Tried reinstalling 0.8.1, removing that email account, and adding it again.  Still same problem.  Geary is finding a tiny fraction of the search results it should. 

Linux Mint 17.

Elizabeth, a couple of things.  First, this bug is still open, hence I wouldn't expect going back and forth between versions to change what you see.  The fix applied to 0.6.2 solved a certain subset of search problems, but not all of them.

Regarding your friends' email, there's a few things to know.  First, Geary by default only stores some of your entire mail database in the local cache.  Geary can only search locally (searching on the server is a separate problem and is not implemented today).  By default Geary will only download the last two weeks' of your mail locally.  That affects your search results.

Second, even if you set Geary to download more mail (in the Accounts dialog), it can take a while for Geary to pull it all down.  Please make sure you're giving it some time before expecting full results.

Third, if you're still having problems, I'd be curious if putting quotes around your friends email changes anything, i.e.

"jim@example.com"

instead of

jim@example.com

Ah, that was very helpful!  I *had* forgotten that the Geary default is only to download the last 2 weeks' of mail locally.  My bad.  Wierdly, one of the 5 conversations from my friend that it was able to find before I fixed the settings was from 2007!

(I've just upgraded my operating system ... should have mentioned that...  and so had wiped all my settings, and completely forgotten it was necessary to re-set that.)  

Have set it to download more mail and it is now functioning as it was before.

(Before making that change, I tried the quotes, which made no difference).

Ok, good.  That clears things up.

The time-limit setting isn't an exact science; for efficiency Geary downloads mail in chunks, and if the tail end of the chunk is beyond the two-week limit (or whatever), Geary doesn't trim it off.  Duration of time is not directly correlated to disk space, but it's a more useful approximation for more users.

Hey, late to the party but I'm on Geary 0.10.0 and I still have this problem, i.e. I'm missing emails.  Also, I do have the email account set to download everything, but clearly it's not doing that.   It only fetches emails when I scroll to the end of what I have downloaded already.

+1 using 0.10.0 compiled from git.... I've got the same email account in Thunderbird, Evolution, and Geary and am comparing features.  Geary's search seems very unusable.  It only finds things that the other two find occasionally and so far I have not been able to notice a pattern on hits versus misses, but would love to help contribute to improving.

(In reply to Antony from comment #20)
> It only fetches
> emails when I scroll to the end of what I have downloaded already.

I assume you're referring to the loading of new conversations when you scroll do the bottom of the conversation list.  This has nothing to do with what has been downloaded.  We load only a small number of emails initially and pull in more from the local database as you scroll backwards in time.

(In reply to mt from comment #21)
> +1 using 0.10.0 compiled from git.... I've got the same email account in
> Thunderbird, Evolution, and Geary and am comparing features.  Geary's search
> seems very unusable.  It only finds things that the other two find
> occasionally and so far I have not been able to notice a pattern on hits
> versus misses, but would love to help contribute to improving.

If you can show us some searches that fail, it may shed some light on what's going wrong.

(In reply to Robert Schroll from comment #22)
> (In reply to Antony from comment #20)
> > It only fetches
> > emails when I scroll to the end of what I have downloaded already.
> 
> I assume you're referring to the loading of new conversations when you
> scroll do the bottom of the conversation list.  This has nothing to do with
> what has been downloaded.  We load only a small number of emails initially
> and pull in more from the local database as you scroll backwards in time.

Interesting that you responded to both of these.  If I scroll backward in time to the emails I was searching for (first), then search works.  I'm guessing for mail in an imap server that was not originally ingested on delivery by geary, its actually doing the downloading as I'm scrolling?  

> 
> (In reply to mt from comment #21)
> > +1 using 0.10.0 compiled from git.... I've got the same email account in
> > Thunderbird, Evolution, and Geary and am comparing features.  Geary's search
> > seems very unusable.  It only finds things that the other two find
> > occasionally and so far I have not been able to notice a pattern on hits
> > versus misses, but would love to help contribute to improving.
> 
> If you can show us some searches that fail, it may shed some light on what's
> going wrong.

(In reply to mt from comment #23)
> Interesting that you responded to both of these.  If I scroll backward in
> time to the emails I was searching for (first), then search works.  I'm
> guessing for mail in an imap server that was not originally ingested on
> delivery by geary, its actually doing the downloading as I'm scrolling?  

Yes, that's true.  Scrolling back can trigger further downloads, and the email needs to be downloaded before it'll show up in search results.  But loading new conversations does not necessarily indicate more downloads are occurring.  (I have several years of email in my local database, so I never trigger more downloads this way.   I forget that's not the case for everyone.)

Once it's loaded, it should always show up in the search results, even if it's not in the currently loaded conversations.  But apparently it doesn't; hence this bug.

Hi all, a couple of fixes have been made recently that may have helped/fixed this issue:

1. Search was in general unreliable due to the database being corrupted - Bug 772522.

2. Regardless of how many weeks/months/etc you had configured Geary to prefetch in your account settings, it may never have reliably prefetched messages your Inbox - Bug 776029

Can you please try building Geary from git (either master or the LTS "geary-0.11" branch) or when 0.11.3 is released at the end of the week, and let me know if this is still a problem for you?

Ta!

Closing this bug report as no further information has been provided. Please feel free to reopen this bug report if you can provide the information that was asked for in a previous comment.
Thanks!