GNOME Bugzilla – Bug 567009
MBOX import does not check line endings
Last modified: 2021-05-19 11:57:17 UTC
Please describe the problem: MBOX import filters do not check for alternate file line endings. This causes problems when trying to migrate from apple entourage to evolution, which causes evolution to hang. Evolution will attempt to load the entire file contents into ram in one sweep, then it appears to try to parse it (and fails), causing evolution to stop responding. Steps to reproduce: 1.Construct a MBOX file that has \r line endings ( you can use sed, or similar) 2. Attempt to import via File -> import -> single file -> (mbox file) -> OK Actual results: Evolution will accept that it is an MBOX file, and then attempt to parse it and fail. Expected results: Should detect line endings then perform conversion on the fly prior to processing Does this happen every time? Yes. Other information:
Workaround: You can convert the line endings back to UNIX line endings via the following command: cat file.mbox | tr \\r \\n > file-fixed.mbox
2.2.x is ancient. Please check again with 2.22.x or 2.24.x and report back.
Sorry, selected wrong version: $ evolution --version GNOME evolution 2.22.3.1
Created attachment 126072 [details] mbox file with mac line endings I can somewhat reproduce this. Evolution 2.25.4 doesn't hang, but the import produces a blank email.
Created attachment 126077 [details] [review] Proposed patch Read the file into memory, normalize DOS or Mac line endings to Unix, then feed it to the MIME parser by way of a CamelMemStream.
Matt, should we really do this copy all to memory and deal with? Its common for users to have GBs of mbox files. It might be tooo expensive. How about either fixing up the parser to deal this or a pre-migration step which does it over the disk? Just thinking aloud.. any other ideas?
Valid point. I avoided changing the parser since I didn't want to add constant overhead for what would otherwise be a one-time check during import. Also don't think we should be modifying the source mbox file. If anything, copy it somewhere and modify that. But then we may run into disk space limitations. I guess the best solution would be to stream the file to the parser by reading chunks from disk and converting the line endings of each chunk in memory. Then we just have to make sure we don't split a DOS line ending in half while reading the chunks. I'll give this another shot...
*** Bug 580441 has been marked as a duplicate of this bug. ***
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines and create a new bug report ticket at https://gitlab.gnome.org/GNOME/evolution/-/issues/ Thank you for your understanding and your help.