GNOME Bugzilla – Bug 429423
Copy from PDA failed for Memos and Todos
Last modified: 2007-09-14 06:56:15 UTC
Please describe the problem: I want to use the EMemos and ETodo conduits to copy each time all records from my palm to the desktop every time I do a sync. It does work if there are no records on the PC. But after the first sync, there are of course records there and if I then do the sync, which ist set to Copy from PDA, the message from gpilotd is (gnome-pilot:12389): gpilotd-WARNING **: Copy from PDA failed! I have already opened a Bug on Launchpad (73149), but it tooks me too long for an answer. Steps to reproduce: 1. set conduits EMemos and ETodo to Copy from PDA 2. sync the device 3. do it again 4. the error occurs Actual results: Expected results: Does this happen every time? Yes. Other information: I think this happens in gnome_pilot_conduit_standard_real_copy_from_pilot which calls sync_CopyFromPilot. In sync_CopyFromPilot I found this code: while (sh->ForEach(sh, &drecord) == 0 && drecord) { result = sh->DeleteRecord(sh, drecord); if (result < 0) goto cleanup; } For me it seems that this does not work if there are already records here, but I am not sure.
Created attachment 86303 [details] Output from gpilotd
Thanks for your bug report. Could this be an example of bug 319076 aka bug 363102 ? Does evolution-data-server crash? What version of e-d-s do you have? Can you update to a version later than e-d-s 1.8.4, or 1.9.7? This may fix your problem. To debug further, we'll probably need a debug trace from gpilotd and e-d-s.
No, it is not the same. Yes, I had the problem with e-d-s crashing. But it does not anymore. I have get the patch adds mutex locking to E-D-S 1.6.3 from bug 363102 and compiled e-d-s with it. It works perfectly. This bug has an other problem. There is no crashing, but it seems that it is impossible to delete the existing memo and toto records. Is this also implemented in e-d-s?
In comment #1, are you sure the DeleteRecord() call is failing? Have you run in a debugger? I'm not able to reproduce this problem, so your best bet might be to dig down into the DeleteRecord() procedure and see where the operation is failing. Sorry not to be more help. It looks like this might be a tricky one to diagnose.
No, I am not sure. I think it is possible that the return code is 0 if there are no records available. How can I debug the code to see if DeleteRecord gives a return code < 0?
Here's another possibility: could it be bug 316370? To debug: 1. killall gpilotd 2. gdb gpilotd create a breakpoint on entry to sync_CopyFromPilot: gdb> b sync_CopyFromPilot now run gpilotd: gdb> r and try to sync. you'll need to read the gdb manual if you're not familiar with its usage.
I have tried this, see the attachments. The first is without records on the desktop, the second has already some (from the first try). I cannot go on step by step after the breakpoint, so I continued. The error occurs after the message Got default component. The first time you can see the num_records, the second time the program comes not at that point. I can not understand why the timezone is set to NULL, I have configured for my location. Years ago I had a problem with gpilotd if I had not set the timezone correctly. It seems that the call to sync_CopyFromPilot is made before pre_sync in todo-conduti.c from evolution. I will try to change gpilotd to print out the returncode from sync_CopyFromPilot. Showing the code I have seen that on the signal POST_SYNC the method pre_sync is called insead. Is that right? pilot_conduit_sync_abs_signals[PRE_SYNC] = gtk_signal_new ("pre_sync", GTK_RUN_LAST, GTK_CLASS_TYPE (object_class), GTK_SIGNAL_OFFSET (GnomePilotConduitSyncAbsClass, pre_sync), gtk_marshal_INT__POINTER, GTK_TYPE_INT, 1, GTK_TYPE_POINTER); pilot_conduit_sync_abs_signals[POST_SYNC] = gtk_signal_new ("post_sync", GTK_RUN_LAST, GTK_CLASS_TYPE (object_class), GTK_SIGNAL_OFFSET (GnomePilotConduitSyncAbsClass, pre_sync), gtk_marshal_INT__POINTER, GTK_TYPE_INT, 1, GTK_TYPE_POINTER);
Created attachment 86703 [details] Debugging the first sync
Created attachment 86704 [details] Debugging the second sync
Have just managed to grab some time to look at this. I haven't been able to reproduce your problem, unfortunately. I've been trying with evolution 2.8.1-0ubuntu4.1. Looking at the evolution code, it seems that the pre_sync method in todo_conduit.c is not completing successfully on your second sync attempt. I can't tell why (but the timezone issue looks suspicious). Try stepping through todo_conduit.c -> pre_sync and see where it is failing. If you can't step through the code with gdb, you may need to install debug packages for evolution, pilot-link and gnome-pilot. If you've compiled your own evolution then try figuring out how to avoid stripping the symbols... Hope this helps.
I have installed evolution-gdb and evolution-data-server-gdb. You are right. The pre_sync code is not completing. The problem seems to be in e_cal_get_changes from evolution-data-server. With next I go through e_cal_get_changes with gdb. Suddenly the message comes from gpilotd “Copy from PDA failed”. Sometimes already after the first statement in e_cal_get_changes, sometimes after the tenth, sometimes after the fifth. If the task list is empty and I debug, all tasks from the PDA are written to evolution as soon as I type once next in e_cal_get_changes. I am confused. This is my software: pilot-link 0.12 gnome-pilot 2.0.14 evolution 2.6.1 with patches from 2.8 to work with gnome-pilot 2.0.14 I also have tested this with the unmodified evolution from ubuntu dapper 2.6.1 and with gnome-pilot 2.0.13 (also from dapper). My PDA is a Sony Clié SJ30 with Palm OS 4.1 I have tried with a newer device, a Palm Tungsten E2 with Palm OS 5.4. With this device it works! Also I can go step for step through e_cal_get_changes and the codes returns properly to pre_sync. What can I test in addition to that? Thank you for any help.
Matt asked me to take a look at this, since I'm familiar with the Evo code (especially the conduits). Could be the timezone problem. The timezone bug report also mentions that it seems to fail in e_cal_get_changes. I also realized, that the Memos conduit never had the timezone fix! I'll come up with one in the next day or two and post it. I can't recreate the problem myself, though, so there might be more fixes that were done between 2.6 and 2.10 (the version I'm using) in the timezone code. As for the ToDo conduit fix, I believe it just means moving the start_calendar_server() call to after the timezone is set. Since I don't know exactly what version of code you have (SVN is making it hard to figure out exactly what's in each version), I don't want to create a patch that might easily fail when you apply it, and then we have to diagnose that problem. You can see the difference page I've been looking at here: http://svn.gnome.org/viewcvs/evolution/trunk/calendar/conduits/todo/todo-conduit.c?r1=30729&r2=31981 Let us know if this fixes the todo conduit problem. That'll give me more confidence that it's the same problem in the memo conduit. If it doesn't fix it, can you step INTO e_cal_get_changes at all? Couldn't tell if that was possible or not. If you can, maybe you can see where it's failing. The e_cal_get_changes is in the evolution-data-server (EDS), so EDS needs to be compiled with debugging turned on as well. If you feel up to it, you might want to upgrade to 2.8 or 2.10. There've been quite a few bug fixes between 2.6 and 2.10, so it might be worth the hassle.
I will try to start_calendar_server() after the timezone set. I have looked at the code from Evolution 2.4, there was a call to e_cal_set_default_timezone() after getting the timezone in the ToDo conduit. This call is not available anymore in newer versions >= 2.4. But as the Memo counduit use e_cal_set_default_timezone() and also does not work for me, I think it does not help. Perhaps I will try to backport evolution 2.10 for me. Also I will try to convert the categories to UTF-8. But this is an other problem for which I will open a bug later.
The function e_cal_set_default_timezone() was moved to start_calendar_server(). This was done when adding the authentication code (auth_func_cb() and related code). Memos does not currently support anything except local file store (though I'm working on it), so the call to e_cal_set_default_timezone() was never moved out of the pre_sync() function.
I have applied your patch that moves the timezone code after the call of start_calendar_server(). But it helps not. By the way if evolution-data-server is not running, start_calendar_server() cannot start ist. The message Could not start evolution-data-server was printed out. I have debuged the code several times and attached the output. As you can see I stopped in e_cal_get_changes() and then go forward step by step. Each time the problem occurs on an other statement but it seems defintively that we does not return to pre_sync() from e_cal_get_changes(). I have debugged again with the newer device and attached the output. This time we returns to pre_sync() and the data transfer was done. This is reproducible. I have find out an interesting thing: deleting the file tasks.ics-pilot-sync-evolution-toto-999.db from my home/.evolution/tasks/local/system makes the function Copy from PDA working again. Deleting the other files helps not. Also if I load tasks.ics-pilot-sync-evolution-toto-999.db into Gedit and save it as a UTF-8 file, Copy from PDA works again. But I have to do one of this operations with this file every time. As I can see the file tasks.ics-pilot-sync-evolution-toto-999.db is a ISO-8859 file. Has it to be in this locale?
Created attachment 88846 [details] debug 1
Created attachment 88847 [details] debug 2
Created attachment 88848 [details] debug 3
Created attachment 88849 [details] debug 4
Created attachment 88850 [details] debugging with a newer device
I have tested more find out that the memo conduit has the same problem. The DB file ist in ISO format instead of UTF-8. If I open it with Gedit and save it with UTF-8 then the sync works. At opposite to memo and todo the calendar file calendar.ics-pilot-sync-evolution-calendar-999.db is in UTF-8. With the calendar I have no problem to sync. If I open this DB file with Gedit and save it as ISO-8859 I can create the same problem with memos and todos: the calendar does not sync, only the message Copy from PDA failed ist printed out. Conclusion: I think the problem with memos and todos is that the DB file ist saved as ISO instead as UTF-8. I can reproduce that the sync is working if the DB file ist UTF-8. As the DB file for the calendar is in UTF-8 and this works, this is the confirmation of my assumption. In addition if I change the calendars DB file to ISO-8859 this conduit does also not sync any more. Therefore the problem is in evolution-data-server. Could you correct this or give me a hint where I found the appropriate code?
I have done more tests and found that the file tasks.ics normally is saved as UTF-8 file. If I delete it and create a new task with evolution, the file is written always as UTF-8. But after the first sync with my palm the file is saved as ISO-8859 file. So from the three files pilot-map-todo-999.mml tasks.ics tasks.ics-pilot-sync-evolution-toto-999.db only the first one is written as a UTF-8 file. The other ones are ISO-8859 files. I think this is wrong, as GNOME usually works with UTF-8 files. If I can do anything more for you please let me know. Thank you.
I haven't been able to find why this would be happening yet. It seems to have been fixed in later versions, as I'm not having this problem. I'll see if I can track down the change(s) that might have fixed it (maybe there's a closed/verified bug out there that I haven't found yet). Thanks for all the work you've put into diagnosing this bug.
I have renamed my Categories on my Palm device so that they does not have any umlaut (only 7 bit ASCII characters). Then I have deleted all mentioned files above and syncronized again. Checking the files now shows that they are all UTF-8 files. Now I can syncronize again and again and again. Ther problem is gone! But this is not the solution for me and I have renamed my categories again (eg. Persönlich). I think the problem is that the category string is not converted to UTF-8. I have changed todo-conduit.c to convert the category to UTF-8 with e_pilot_utf8_from_pchar(). I will attach a diff file from evo 2.6.1. Now I have two problems solved: my categories are showed correctly in Evolution *and* I can syncronize more than once. But I cannot find to make the reverse: converting the category name from UTF-8 to Palm with e_pilot_utf8_to_pchar(). I cannot find where to place it. Can you help me, as you know the evo conduit code and it seems not to be a problem in evolution-data-server. Thank you.
Created attachment 89611 [details] [review] Changed code to convert the category to UTF-8
Yep, that's the problem. I can reproduce it myself now. When I implemented the category code, I didn't realize how important the character set was. I actually saw that in a related 2 bugs earlier today (bug 214611 and bug 385805), but didn't think it would cause the entire sync to fail. The category syncing is currently being updated in bug 201167. I'll make sure what you found gets in there.
Sounds good... Not good that this charset issue could abort the sync. It would be better if problems like this resulted in a record being skipped (and logged, ideally) rather than killing the conduit. I'd just like to echo Nathan's thanks to dbet for his perseverance and help on this one.
This should be fixed in SVN soon (and the next stable release of Evolution). A patch has been added to bug 201167 (implementation of category syncing) to fix this. Sorry it took so long. If there's still a problem after the patch is applied, please let us know.
closing this bug. Should be fixed in most recent Evolution release. Reopen if necessary.
I have backported the UTF-8 conversion code from bug 201167 to the memo and todo conduit from evo 2.6.1 an it works now. Thank you very much.