GNOME Bugzilla – Bug 600322
assertion failed: (reply == msg) in cs_waitinfo()
Last modified: 2010-04-19 23:38:07 UTC
+ Trace 218719
Thread 19 (Thread 23700)
Looks like another one:
+ Trace 218734
Thread 28 (Thread 17649)
And another:
+ Trace 218735
Thread 27 (Thread 19759)
I am sure you can imagine how infuriating this is getting considering every restart of evolution takes 10 minutes before I can start using it due to the severe lack of scalability, yes?
+ Trace 218736
Thread 25 (Thread 19857)
Can I please have a patch that fixes this pretty quickly? Evolution is completely unusable in this state.
BTW, it's not lost on me that this is all stemming from a: camel_getaddrinfo (name=0xa459f68 "mail", service=0x24c2758 "imap", hints=0xa4dfef90, ex=0xa4dff1e8) at camel-net-utils.c:700 -> cs_waitinfo (worker=<value optimized out>, msg=0x9cf0e188, error=0x4a5efa "Host lookup failed", ex=0xa4dff1e8) at camel-net-utils.c:516 Where "mail" is the imap server, indeed, but a) an assertion and an abort() from a host lookup failure and b) this host lookup should not be failing. This host is perfectly resolvable: $ host mail mail.interlinx.bc.ca is an alias for linux.interlinx.bc.ca. linux.interlinx.bc.ca has address 10.75.22.3 linux.interlinx.bc.ca has IPv6 address 2001:4978:161:0:2d0:b7ff:fe3f:8b74 And at the point where this abort() is happening, at least one connection to this IMAP server has already been made.
I've even added this host "mail" to the /etc/hosts file to eliminate the possibility of network issues here resolving it. I don't believe for a minute that the host "mail" is actually not resolvable.
looks dupe of bug 578827
I've even disabled the imap account for which the server "mail" serves and it still abort()s.
(In reply to comment #6) > looks dupe of bug 578827 Yeah. Does look very dupe. :-) Suspend/hibernation has got nothing to do with it here though.
Now another one, with a different host:
+ Trace 218743
And again, I don't believe for a minute that this host was not found. Again again, what's so tragic about a host not found that evolution needs to abort()? Can I please, please, please, get some help on this one? It's making evolution *completely* useless for me. Please?!
Some further debugging yields that when this happens: reply == nil and msg = 0xa422b490 I've NOOPed that g_assert and the fallout looks like a subsequent: (evolution:28106): camel-WARNING **: Error storing 'brian@mail:Drafts': Could not connect to mail: Connection timed out (evolution:28106): camel-WARNING **: Error storing 'brian@mail:INBOX': Could not connect to mail: Connection timed out (evolution:28106): camel-WARNING **: Error storing 'brian@mail:IK': Could not connect to mail: Connection timed out ... But the world didn't end. I will dig in further to see what's going on.
I do not know this part of a code, but I think that cs_getaddrinfo recognized a cancel of the operation, thus instead of returning the msg through the msgport it silently freed it, and the later pop returned NULL, where the assert got triggered. I'm also wondering how you compile, as there support for IPv6 in configurable in compile time, and also one version of cs_getaddrinfo simply doesn't support them at all: > /* we only support ipv4 for this interface, even if it could supply ipv6 */ > if (h.h_addrtype != AF_INET) { > msg->result = EAI_FAMILY; > goto reply; > } all that depends on NEED_ADDRINFO define.
(In reply to comment #11) > I do not know this part of a code, but I think that cs_getaddrinfo recognized a > cancel of the operation, How does a cancel happen? If you mean user driven (i.e. I pressed the red circle with the white X in it) that was definitely not being done. Maybe there are other internal drivers of cancels. > thus instead of returning the msg through the msgport > it silently freed it, and the later pop returned NULL, where the assert got > triggered. Sounds like a reasonable hypothesis > I'm also wondering how you compile, as there support for IPv6 in > configurable in compile time, and also one version of cs_getaddrinfo simply > doesn't support them at all: Yes, IPv6 support is compiled in and used here. So maybe we have a smoking gun. > > /* we only support ipv4 for this interface, even if it could supply ipv6 */ > > if (h.h_addrtype != AF_INET) { > > msg->result = EAI_FAMILY; > > goto reply; > > } > all that depends on NEED_ADDRINFO define. Where is the code is that from? FWIW, this seemed to mysteriously go away last night some time. This issue seems pretty stable for the moment but we really should try to get to the bottom if it's not that complicated.
(In reply to comment #12) > (In reply to comment #11) > > I do not know this part of a code, but I think that cs_getaddrinfo recognized a > > cancel of the operation, > > How does a cancel happen? If you mean user driven (i.e. I pressed the red > circle with the white X in it) that was definitely not being done. Maybe there > are other internal drivers of cancels. Some operations can call cancel too, like when you are moving from one message to the other, then the 2.29 calls cancel for the previous message fetch operation (if any). It's not the case for 2.28, but there could be other similar places too. > > > /* we only support ipv4 for this interface, even if it could supply ipv6 */ > > > if (h.h_addrtype != AF_INET) { > > > msg->result = EAI_FAMILY; > > > goto reply; > > > } > > all that depends on NEED_ADDRINFO define. > > Where is the code is that from? it's from eds/camel/camel-net-utils.c, I have it on line 564 (but it's part of a conditional compile, thus it's a question whether your version is actually using this or the other part). When you see this again, try to add a breakpoint in both cs_getaddrinfo in that file, on the place where if the if (msg/info->cancelled), to check whether it was really cancelled (a breakpoint just after the if, not on the if statement itself, as it's called too often). I recall Matt speaking about a possibility to drop this old code and rather use a GResolver or something like that. Might be interesting to know what happened to the idea during the time.
Here's another one. In this case, I might actually believe there was a host lookup error as the name server might have been AWOL a the time of the lookup. But really? The reaction to a host lookup failure is to abort()? I wonder how such code passed peer review. abort() is supposed to be reserved for situations of "impossibility". So somehow boththe code author and review(s) in this case allowed a very possible failure to result in abort(). I don't mean to belittle or berate but these are the kinds of standards that need to be raised if evolution is to become a stable piece of software.
+ Trace 219459
Thread 16 (Thread 3144)
I thought that this cannot happen after patch from bug #600449, but it seems you have it applied there, right? Just in case, is it applied cleanly and fully? With respect of assert usage here, it is used correctly, the situation should never happen, and if it happens, then something is broken. Note the code didn't change much in this source file for years.
(In reply to comment #15) > I thought that this cannot happen after patch from bug #600449, but it seems > you have it applied there, right? Yup. > Just in case, is it applied cleanly and > fully? Here's the output from it's application during build: $ cat debian/patches/99_bz-600449.patch.level-1.log patching file camel/camel-net-utils.c And to further verify that it was applied: $ patch -p1 --dry-run -R < debian/patches/99_bz-600449.patch patching file camel/camel-net-utils.c So, I'd say, yes, it is applied there and cleanly and I have verified that I have the locally patched and built version of libcamel installed here. > With respect of assert usage here, it is used correctly, the situation should > never happen, and if it happens, then something is broken. Note the code didn't > change much in this source file for years. It just seems strange that a host lookup failure appears at the top of the stack on the way to an assert and abort().
Created attachment 149366 [details] test patch for evolution-data-server; What confuses me most is that the file wasn't much changed in ages and it shows some strange behaviour suddenly. Anyway, could you apply this test patch on the previous one and the best compile evo and eds without optimization, thus there will be no <value optimized out> in the stack trace? It's sometimes better for debugging. This test patch tries to avoid the crash. It will print on the console a message if it avoided it for the first time, with some additional information as well. Please give it a try and let me know. Just in case it'll abort, please add also output of a gdb command "p *msg" here, to see in what state the msg itself is. Thanks in advance.
(In reply to comment #17) > Created an attachment (id=149366) [details] > test patch Hrm. Many of the pthread_testcancel() calls that you replace with test_cancelled(msg) just don't exist in my source. I will attach a modified version of your patch that applies to my 2.28.1 source.
Created attachment 149398 [details] [review] version of patch that applied to my source patch with extraneous - pthread_testcancel (); + test_cancelled (info); hunks removed so that it applies to my local 2.28.1 source.
The only way how that can happen is that you do not have applied patch from bug #600449 comment #11. I do not see any other possibility of missing calls of pthread_testcancel in your sources. Could you attach here all the patches you are applying to your 2.28.x eds? Or even better, if you use GIT's gnome-2-28 branch, could you update it and apply the test patch? It should be applied cleanly, unless you apply other patches in the camel-net-utils.c file, which is unnecessary for the gnome-2-28 branch of eds, as I just tested that on a commit fcb122c, which is the actual state as of today.
(In reply to comment #20) > The only way how that can happen is that you do not have applied patch from bug > #600449 comment #11. Doh! It's a patching dependency that was not reflected in the order which I had selected patches to be applied. Once I changed the ordering, it all patched just fine.
Aha, ok, and when you changed the ordering, is this still triggering? If yes, what is the output on console from a test patch?
OK. Patch applied. I have gotten this crash a couple of times now with the patch applied. Unfortuntely I lost the first one when the second one happened and I could not report the first one because bugzilla.gnome.org was not working yesterday. I'm not sure exactly what the patch was supposed to do. If it was only to remove the (reply == msg) assertion then in this crash it seems to have achieved that, but caused a new/different problem: GThread-ERROR **: file /build/buildd/glib2.0-2.22.2/gthread/gthread-posix.c: line 171 (g_mutex_free_posix_impl): error 'Device or resource busy' during 'pthread_mutex_destroy ((pthread_mutex_t *) mutex)' aborting... I'm not sure if you want a new bug for this issue or not, so I will just paste it here:
+ Trace 219581
Thread 18 (Thread 26157)
Created attachment 149702 [details] test patch ][ for evolution-data-server; Slightly improved previous patch, not allowing cancel a thread when in the middle of an operation on the msgport (it seemed to cause the latest trace). I've a question: Is there any line on the console beginning with "cs_waitinfo:" information while this crashes? It should tell you hopefully "thread was cancelled" with two pointers.
*** Bug 601252 has been marked as a duplicate of this bug. ***
*** Bug 605371 has been marked as a duplicate of this bug. ***
Got another one of these, with attachment 149702 [details] applied:
+ Trace 219822
Thread 20 (Thread 13442)
(In reply to comment #24) > > I've a question: > Is there any line on the console beginning with "cs_waitinfo:" information > while this crashes? The only output (with cs_waitinfo) is: camel:ERROR:camel-net-utils.c:516:cs_waitinfo: assertion failed: (reply == msg) > It should tell you hopefully "thread was cancelled" with > two pointers. I'm afraid not. :-(
Just got another of these, again without the debug you were hoping for on the console:
+ Trace 219836
I wonder if this one provides any more info:
+ Trace 219934
Created attachment 150926 [details] debug patch III for evolution-data-server; Thanks for the update. I see I didn't add enough debug information to the test patch II. This version doesn't crash, but every time it would it will print a message on console about actual state of local variables, namely those which are optimized out in your traces and I think are involved in this issue. I've not much idea what it will do instead of crashing, either it'll survive, lock itself, or crash a bit later. Hard to tell.
*** Bug 601828 has been marked as a duplicate of this bug. ***
So, per bug 606813, I backed out the patch from this bug and now I am getting crashes from this bug again:
+ Trace 220526
Thread 22 (Thread 27228)
Hmm, test patch ][ from bug #606813 is removing all the thread cancelling done by pthread, so it should have pretty the same effect as this debug patch ]I[, apart of the removed assert, changed with a debug printf. So comment out the camel-net-utils.c:514 and if you can, please upload here or to the other bug yours camel-net-utils.c after all patches applied, I want to be sure you've there what I expect should be there. Thanks in advance.
Created attachment 153524 [details] camel-net-utils.c after patch from bug #606813 Per comment #34, here's my camel-net-utils.c after the patch from bug #606813 is applied and with line 514 commented out.
Thanks, it looks good. I think I'm finally getting to understand what's happening here, but I would prefer to verify my hypothesis. Could you, instead of commenting out that g_assert, replace it with these three lines, please? if (reply != msg) { fprintf (stderr, "!!!\n!!!\n!!! %s: reply:%p msg:%p differ; cancel:%d/%d\n!!!\n!!!\n", __FUNCTION__, reply, msg, msg->cancelled, cancel); } (that fprintf is one long line) and if my hypothesis is correct, then you should see this message only when "cancel:1/0". Please run evolution for example as: $ evolution 2>evo.log and just use it, and after some time, when you close it, check the evo.log, whether it contains any lines beginning with "!!!" and if so, please paste here those parts with "reply, msg and cancel" information. Thanks in advance.
(In reply to comment #36) > > and if my hypothesis is correct, then you should see this message only when > "cancel:1/0". I'm afraid not. :-( > Please run evolution for example as: > $ evolution 2>evo.log > and just use it, and after some time, when you close it, check the evo.log, > whether it contains any lines beginning with "!!!" and if so, please paste here > those parts with "reply, msg and cancel" information. Thanks in advance. I'm sure this is way more than you need, but better too much info than not enough, I always say: !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c622f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9957aed0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96d66a60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99430678 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f3e998 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a2763c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99852398 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974386e0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967fee60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9941fa68 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9691f720 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a77da78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b205ec8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b14c300 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f083a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98292798 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b166088 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x998ecb98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c765c90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f43e70 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dc6b10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f37ab0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f37ab0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4579a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9957aed0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b1c2810 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9755f040 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967fa430 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9aab4c58 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f036c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9530ed10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa615b850 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c765c90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x953a4b98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967fee60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99430678 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f3e998 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98277fa8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x975ad498 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f43e70 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a77da78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ec78a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9948a508 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98292798 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9aa0ba48 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99852398 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9928e430 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c9893e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cd4020 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99438e90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dc3cc8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f00848 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ec9a500 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dd5100 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9948bed8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3b737b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95785618 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b1c1500 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a5e6590 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a5e6590 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cd46e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98054cf0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967f5b90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ac4e6e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b22e750 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9574dcd0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97574688 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa38281e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b21ef98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cd2390 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996f3610 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967ff248 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98054c20 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa632ca50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a8696a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c765590 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c75e758 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96903a10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996c04a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996c04a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996c04a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996c04a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98275920 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996eed40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96defb88 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dc4f18 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9943a2d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99418280 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9abee8d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9f68bef8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a73e190 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb453d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95763068 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a2c4278 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x995a0610 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a72aca8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9828cb10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a5cbf80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99681ef8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9965f908 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b49bd08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c798200 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9949aa00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98262eb0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974c3900 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9757bab0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3d55b00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b1554d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95799ac8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c73a868 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ef4b788 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x998fbe10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96df0e00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97c53d50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb8a778 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b583d58 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b583d58 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cd8cf0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974df4c8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a59d6b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c703688 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f45c20 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cfc800 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3d8d6d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9740ba10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99418348 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b513b40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99424698 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967e9b00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c7bc4d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996f9268 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a4e46b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99473a98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99408050 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96915f30 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a87aa80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974d6008 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96967b38 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96921b68 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967f4968 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x957c6f70 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9755a9b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f49250 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b29afc8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x957bb2b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9f53ea00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cec680 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a026870 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a523e08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3e03c00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996fd680 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996fd680 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980c79c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9565f488 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9568f310 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b205ec8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9691f720 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996ae0d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99852398 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dd5198 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ec78a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b48a6f8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9530ed10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b1c2810 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3d748c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xb178ab98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95780a60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a7cccf0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f3e998 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96d66a60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f083a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9948a508 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cec858 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967a2a90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967fee60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996ce8e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96903a10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c765590 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa632ca50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967ff248 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cd2390 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97574688 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ac4e6e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98054cf0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b1c1500 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980d0e40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x980d0e40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cfa540 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39318 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967f1e10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x957b6460 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96df7f60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ce639f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c982000 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c710a78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96913680 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c75e758 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98054c20 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa38281e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967f5b90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95785618 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dd5100 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f00848 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99438e90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cd3c260 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x994626d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa37cc698 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c73a868 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96c55e38 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a056c00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98294210 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a5cbf80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99681ef8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4caac0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x95763068 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39a00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x94f31130 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97cfa748 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9accd4c8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x957a8698 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b4f2b90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c7d9a50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974b0fa8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97579a38 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96da4388 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99455758 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c750210 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x974a4b28 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa0bc1548 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a496818 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a72aca8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9828cb10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9828cb10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c798200 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a5ee6f8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a73e190 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb453d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa377b080 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b239d30 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b5b40d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96910f80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9568ead8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99429ad0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3e03c00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996f9610 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3c2b6c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x957c6f70 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3ded4b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3841e00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc16fc98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xcdf8db8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0a09c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9517e9b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9817ca50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9790b928 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9722dc60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0a09c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9713a360 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98cdc348 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9790b928 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3988498 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98cdc348 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x971b46a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa02e15f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984f1bc0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0a09c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x971b46a8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9574dcd0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98054cf0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c9893e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b5b3450 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cfa540 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967f5b90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96df7f60 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a32c200 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x967fec28 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a05bf68 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ec9a500 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96903a10 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x996f3610 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97483b50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb99c80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96738390 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa3b737b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9ac9d100 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9cb39b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9948bed8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96dc3cc8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x99438e90 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x96cc0590 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce16bf8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xace4988 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xcd53aa8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce58738 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc3fc4f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xcef84d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xace4988 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc12f848 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc2e91e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa4f1258 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xce55318 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc3fc4f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xcab26e0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc4eeec0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976458d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97142a00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b780a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97327c88 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x972355f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9f2f8ce0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9f2f8ce0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa39d5950 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x971f1d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b47680 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b780a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c575298 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c5aa688 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97327c88 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e240d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e240d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e240d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e240d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x972355f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97142a00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b17a00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b17a00 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa64cdb40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa39d5950 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9f2f8ce0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b871d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97260e80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97909d98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b03388 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981238b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981275a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981993e8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b81098 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x951b2280 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0c73b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x985ee738 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98858b30 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97327d50 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b46698 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa39d07f0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x981238b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97166d70 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x971773a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98c70ed8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b7c2b0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x985232b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x988c48b8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x990d7eb8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0d74d0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e91f98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a93e800 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97351738 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9886bd58 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa39782a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98cc3ed0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa1674d08 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0e1068 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98878b78 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0d1750 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9711af80 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x971ef400 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b014480 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x972695d8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b079a0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x988e7320 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a630ac0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0354c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e9e150 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97160758 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x976707c8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa011ab40 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x986ad548 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97234cc8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9a645098 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98e5c428 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b0ea970 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x98554dc8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97621b58 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97b06a98 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9722c678 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9b6b71c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9764a550 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x97675390 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c585948 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xa0242180 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9c587710 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9760dc30 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x9848ceb8 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0x984d90c0 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc70a238 differ; cancel:0/0 !!! cs_waitinfo: reply:(nil) msg:0xc60fee8 differ; cancel:0/0
Thanks for the update, the pattern here is obvious, same as my failed hypothesis. So something failed elsewhere, let me try to create a new debug patch (with added printf lines only).
Created attachment 153616 [details] debug patch IV for evolution-data-server; OK, I got an idea, hypothesis #2, from your findings. Will see whether with this patch it'll be as I suppose it (I'm quite good in making hypotheses). The attached debug patch is pretty the same as from bug #606813, but the output and its place is different. Please give it a try and add here few first lines with !!!, expecting there will be some pattern in them like in those above. Thanks in advance.
I would like to ask you for a favour, could you give a try to the patch from comment #39 and paste here few lines of its output like before (comment #37), say in a week, so if it'll report things as I expect then the final fix will be included in the upcoming release of 2.28.3, which will be done on March 1st? Thanks in advance.
(In reply to comment #40) > if it'll report things as I expect then the final fix will be > included in the upcoming release of 2.28.3, which will be done on March 1st? Ping, any update on this, please?
(In reply to comment #41) > (In reply to comment #40) > > if it'll report things as I expect then the final fix will be > > included in the upcoming release of 2.28.3, which will be done on March 1st? > > Ping, any update on this, please? We do seem to have hit some: !!! cs_waitinfo: port:0xb015698 reply:(nil) (0x9c6a50e0) msg:0x9c6a50e0 differ; cancel:0/0 !!! cs_waitinfo: port:0xa426330 reply:(nil) (0x9f58c558) msg:0x9f58c558 differ; cancel:0/0 !!! cs_waitinfo: port:0x9c5d85c8 reply:(nil) (0x9f58c558) msg:0x9f58c558 differ; cancel:0/0 !!! cs_waitinfo: port:0xaf08138 reply:(nil) (0x9f58c558) msg:0x9f58c558 differ; cancel:0/0 !!! cs_waitinfo: port:0xa2febb0 reply:(nil) (0x9f58c558) msg:0x9f58c558 differ; cancel:0/0 FWIW, I am in the midst of building the current gnome-2-28 branch from git. Doing E-D-S at the moment, but of course, Evo-proper will be next.
Good, thanks for the update. So I will change slightly the patch and use it for 2.28.3. From the above I see that eds was asking for a reply, then it was waiting for a thread end. There was nothing in the queue for some reason, but after thread finished it was (seems your machine is pretty quick and able to switch between threads also quickly). I will also remove the g_assert and replace it with g_warning, because from the above it seems like it was there "just to be sure". I will also use the remove part of pthread from the file, as the thread cancelling breaks used libraries, as shown in the other bug. Thanks for all the testing.
Created attachment 154584 [details] [review] eds patch for evolution-data-server; Final patch, doesn't break string freeze.
Created attachment 154585 [details] [review] really final eds patch Err, nonsense, it didn't break string freeze. This is the one.
Created commit 2dd2693 in eds master (2.29.92+) Created commit 6b42bea in eds gnome-2-28 (2.28.3+)
*** Bug 606813 has been marked as a duplicate of this bug. ***
*** Bug 578827 has been marked as a duplicate of this bug. ***
*** Bug 605575 has been marked as a duplicate of this bug. ***
*** Bug 616188 has been marked as a duplicate of this bug. ***