It looks like Geary is leaking memory when going online - probably at startup and whenever the network does down then combes back up again. To repro, just turn the network off, then turn it on again, and watch the memory go up.

Memory use seems to increase when the following debug is getting printed:

 [deb] 11:02:54 1.083205 geary: util-connectivity-manager.vala:163: Network changed: some available
 [deb] 11:02:54 0.000052 geary: util-connectivity-manager.vala:103: Checking if mail.quuxo.net:143 reachable...
 [deb] 11:02:55 0.998631 geary: smtp-outbox-folder.vala:116: Starting outbox postman with 0 messages queued
 [deb] 11:02:55 0.004462 geary: imap-engine-minimal-folder.vala:1472: account_01:INBOX (open_count=2 remote_opened=true): fetching 20 flags
 [deb] 11:02:55 0.002998 geary: imap-engine-minimal-folder.vala:269: account_01:INBOX (open_count=2 remote_opened=true): Claiming folder session

So a good place to start is to look at those classes around the code printing that debug output.