ah, this may be relevant...

* OK [HIGHESTMODSEQ 29395] Highest
* VANISHED (EARLIER) 9405:90896,90898:151314,151324,151328,151330,151333,151340:151342,151345:151348,151350,151352:151354,151358:151362,151364:151370,151372:151380,151382:151394,151485,151488,151613:151625,151638,151640:151642,151649:151659,151662:151664,151966:151968,151970,152175:152177,152222,152374:152385,152748:152750,153823,153825,153900:153910,154476,154611:154621,154714,155144,155334,155345:155349,155352,155357,155360:155361,155364,155366:155367,155370,155372:155377,155396,155401:155403,155407,155412,155415,155418,155421:155428,155435:155436,155443:155445,155449,155451,155463:155464,155467:155469,155473,155478:155488,155491:155492,155494:155497,155500,155502:155506,155508:155511,155513,155516:155517,155519:155532,155535:155537,155541,155545:155572,155575:155583,155586:155589,155591,155599:155604,155608:155610,155612,155621,155624,155626,155630:155631,155642:155644,155648,155650,155652:155653,155666,155668:155670,155672,155679:155680,155703:155704,155710:155711,155717:155718,155724,155734,155739,155741:155742,155753,155755,155757:155758,155768,155771,155805,155807,155819,155827,155830:155832,155836,155848:155849,155854,155856,155858:155859,155861:155865,155867:155869,155871:155873,155875:155877,155880,155883:155884,155896,155900:155901,155903:155904,155907:155908,155926:155927,155942,155945,155972,155977,156001:156002,156004:156008,156027,156052,156059:156094,156098,156104,156106,156149:156151,156156,156173,156176:156177,156179:156182,156184,156186:156187,156189,156195,156213,156221,156223,156227,156229,156232:156233,156263:156267,156279,156285,156300,156302,156313,156316,156332,156334:156336,156338,156344,156356:156377,156379,156386:156392,156398:156399,156404,156422,156428,156441,156444:156445,156452:156456,156458,156460:156461,156471:156488,156490,156494:156495,156498,156500,156505:156514,156516,156518:156528,156532,156534:156539,156541,156544,156555,156557:156559,156565:156575,156609:156610,156615,156619,156631,156634:156636,156691,156706:156708,156710,156716,156722:156723,156725:156727,156730,156732:156733,156735:156738,156740,156748,156750,156753,156761,156765:156767,156770:156774,156776,156780,156783,156791:156792,156797:156800,156802:156803,156805,156807:156809,156820:156823,156825:156828,156832,156834:156837,156839:156840,156842,156854:156855,156858:156869,156874,156877:156880,156882:156890,156895:156916,156918:156922,156926,156930,156932,156939,156941,156945,156958,156963,156977,156991,157003,157013:157014,157016:157017,157019,157021,157026:157028,157032,157035:157036,157039:157041,157043,157045,157047,157049,157055,157057:157060,157062,157065,157067,157069:157075,157078,157081:157082,157084:157085,157094,157102,157105,157107,157111,157113,157115:157116,157121,157124,157136,157140:157141,157143,157150:157151,157153,157166:157168,157171:157172,157176:157178,157180,157184:157191,157195:157196,157199:157200,157208,157212:157214,157222,157226,157233,157249:157250,157253,157255:157265,157267:157281,157283:157293,157301:157302,157304,157329,157344,157365,157375:157376,157388:157393,157395,157397:157398,157403:157406,157408:157415,157417:157418,157421,157423,157425,157431,157443,157451,157465,157471:157475,157477:157478,157492,157494:157495,157497,157499,157501,157503:157511,157520:157523,157525:157530,157532:157533,157545,157547:157548,157550:157551,157556,157561,157564,157566,157569,157571,157581:157588,157591:157592,157594:157596,157611,157614:157624,157644:157651,157653:157655,157658,157660:157661,157677,157695:157703,157705:157706,157733:157734,157736:157738,157740,157742:157744,157747,157749:157753,157760,157762,157765,157767,157772:157775,157777:157781,157793:157795,157798,157801:157802,157806:157808,157814,157816,157818:157819,157821:157822,157834,157837:157844,157857,157861:157866,157870,157873,157880,157882,157884:157885,157889:157894,157899,157903,157911,157913,157915:157930,157937:157954,157956:157958,157960,157964:157966,157968:157969,157972,157975:157976,157982,157986,157988:157989,157992,157997,158009,158012:158033,158035:158048,158060,158064:158065,158069,158080,158082:158090,158114,158128,158134:158142,158144,158149,158153:158154,158160:158161,158167:158168,158171:158176,158178:158179,158187,158191,158197,158200:158201,158203:158205,158207:158209,158211,158213,158216,158218,158220:158225,158227:158228,158233,158246,158279,158284:158285,158287:158291,158294,158297:158301,158303,158307:158310,158312,158314,158320,158323,158330,158333:158335,158337:158338,159058'
Protocol error: token too long
Got protocol error

We should probably try to read the uid list from the stream one UID or range at a time (delimited by the commas), rather than pulling the whole 'token' and then using g_strsplit on it.

Created attachment 169725 [details] [review]
patch to automatically grow buffers

This takes a different approach -- just grow the buffers automatically as required. I removed a couple of checks on token size which could never trigger (since camel_imapx_stream_token() wasn't going to return a token larger than the buffer size anyway), and changed a couple more to grow the buffer instead of aborting. As you can see, I started with a buffer size of 4 bytes to exercise it.
Tempted to leave it like that for a little while, for testing.

Hah, this breaks valgrind even though I tried to be careful to use 'nice' pointer arithmetic...

Er, no -- it's just broken. S'what I get for hacking on pointer arithmetic at 2am. Need to reset e = is->end after each grow() call, at least.

Created attachment 169726 [details] [review]
less broken patch to automatically grow buffers

This one survives for more than two minutes of testing.

Would GString be easier here?  It's a text buffer that grows automatically.

http://library.gnome.org/devel/glib/stable/glib-Strings.html

I don't think GString really meets our needs here. We have special buffer management to pull in more bytes from the server when we need the tokeniser needs them, etc.

But thanks for the suggestion. As a kernel hacker straying into userspace, it's useful to have the facilities pointed out to me.

To ssh://dwmw2@git.gnome.org/git/evolution-data-server
   32ad644..de5bdc3  master -> master

Committed a variant of the above patch, with a couple of supposedly minor cleanups split into separate patches (for bisectability), and the starting buffer size still set to 4 bytes to ensure it gets tested.