GNOME Bugzilla – Bug 671863
Cannot open a big file
Last modified: 2012-03-14 07:18:00 UTC
when I open(writing) a big file, stop the opening by Gnumeric. Why? Can(would) you remote connect to my computer to see what happen when Gnumeric write(open) the big file?
Of which file type is your big file? You wrote "open(writing)", so do you see a problem both when you read and when you save the file?
Hi, Andreas. Thank you for quick replying. The type of this big file is ods. Follow are the errors of opening(reading) this big .ods file by Gnumeric : errors: error even name: APPCRASH App. : gnumeric.exe App. verison: 1.10.16.0 Date/time: 4dfa9942 error model's name: libglib-2.0-0.dll 錯誤模組版本: 2.26.0.0 錯誤模組時間戳記: 4ddc4d13 例外狀況代碼: 40000015 例外狀況位移: 00044fc3 作業系統版本: Win 7 :6.1.7601.2.1.0.768.3 地區設定識別碼: 1028 其他資訊 1: bd69 其他資訊 2: bd69c66909ea6759726eb4cc95e17554 其他資訊 3: 303d 其他資訊 4: 303db9a9cd9995d360de3c8903cd1126 This big .ods file come from a small .ods file, Gnumeric can opens(reads)the small file is okay, but open the big file is always crash. The big .ods file is too big to attach.. so I'd have emailed you a email with the big .ods file. pls. view and check it. Thanks.
Hi, I can open the large file just fine. It takes a while but the progress is appropriately shown. The file, when loaded in memory, requires nearly 2GB of memory. During loading not significantly more memory seems to be used. Your error information indicates that the problem occurs in glib, this lends credence to the guess that you may simply run out of memory. Note that I was using 1.11.2 and 1.10.17 while you are using 1.10.16. (There is no Windows version for 1.11.2).
Andreas, Thank you for your helping testing the trouble file and explaining the errors in detail. I'd have the point as same as you, maybe this big file eat up all my computer resource..... RAM or CPU, it's 16.5 M big file. By the way, what time will I have the latest version that can run in Windows( win7), and any big file make the same errors can be saved by using version 1.11.2 Gnumeric in Windows? Thank you again!
Gnumeric 1.11.2 depends on gtk3. As far as I know so far there is no viable port of that to MSWindows. There is no specific Gnumeric development on MSWindows, so a Windows version should become available whenever teh underlying libraries are ported. I don't think there is any significant change between these versions fo rthis kind of file. You are just running out of memory...
How many cells are there in the file? There used to be an inefficiency in glib that caused hashes to use more memory than necessary. See bugs 644197 and 644437.
Morten, there don't seem to be that many cells. I tried to send you the file but got: A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: mortenw@gnome.org SMTP error from remote mail server after MAIL FROM:<aguelzow@pyrshep.ca> SIZE=23232320: host barmail1.idig.net [64.34.111.254]: 552 Message size exceeds fixed limit ------ This is a copy of the message, including all the headers. ------ ------ The body of the message is 22932635 characters long; only the first ------ 106496 or so are included here.
Thank Andreas, Morten for checking and understanding this big and trouble file(size: 16.57M). Just for more your information: My computer system :CPU(inteli3-2310M.10GHz),RAM(4GB),Win7(32). Would you like to tell me next step how to fix the error of opening(reading) this big and trouble file by Gnumeric?
(In reply to comment #6) > How many cells are there in the file? > > There used to be an inefficiency in glib that caused hashes to use more > memory than necessary. See bugs 644197 and 644437. Morten, I've sent Andreas an email with the file. This big and trouble .ods file(均線統計Gn.ods) can be downloaded from: https://docs.google.com/open?id=0B1r2GzCRhZEhUU94Wk41RjhRNGFtNjlYdnZ3Um1Jdw Looking forward to hearing your advice for fix the errors of opening(reading) this big and trouble .ods file.
Looking closely at the loaded file I see about 1.65 Million cells. So its obviously bigger than I thought.
(In reply to comment #10) > Looking closely at the loaded file I see about 1.65 Million cells. So its > obviously bigger than I thought. Yes, the trouble file is so big that over Gnumeric's ability to open(read) it okay. Andreas, I can't open the big file now. for resizing(reduce size) all sheets, can you help delting all blank cells in the bottom of all sheets in the big file. And then sendback me the big file with all resized(reduce size) sheets.
Lee, deleting blank cells will not change the memory used: we do not allocate memory for blank cell location. I'll see what I can do to make the sheet a little less memory intensive.
What we have here is a serious flaw in the ODF format. It was rushed out by the text processing people and never vetted by spreadsheet people. Specifically the file format does not allow for shared expressions, so if we look at this file we see that it's almost 700M underneath: # unzip -v ~/Download/b671863.ods Archive: /home/welinder/Download/b671863.ods Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 46 Stored 46 0% 03-12-2012 07:08 8a396c85 mimetype 695574597 Defl:N 16971416 98% 03-12-2012 07:08 c0e1853b content.xml 5742 Defl:N 1194 79% 03-12-2012 07:13 7a710a42 styles.xml 1126 Defl:N 409 64% 03-12-2012 07:13 d8c3b944 meta.xml 3374 Defl:N 698 79% 03-12-2012 07:13 6922acd0 settings.xml 642 Defl:N 211 67% 03-12-2012 07:13 ff4c2973 META-INF/manifest.xml -------- ------- --- ------- 695585527 16973974 98% 6 files The way Gnumeric currently works is to load the spreadsheet and then go hunting for cells that can be shared. That too late. I have a patch that does the sharing also during ODF load and with that patch we store just 43 different expressions instead of 1022130.
This problem has been fixed in the development version. The fix will be available in the next major software release. Thank you for your bug report.
Just for reference. The corresponding .gnumeric file is 8M based on ~150M of xml. # ./ssconvert ~/Download/b671863.ods /tmp/b671863.gnumeric # ls -l /tmp/b671863.gnumeric -rw-r--r-- 1 welinder research 8713126 Mar 13 13:38 /tmp/b671863.gnumeric # gzip --list /tmp/b671863.gnumeric compressed uncompressed ratio uncompressed_name 8713126 155657785 94.4% /tmp/b671863.gnumeric
(In reply to comment #12) > Lee, deleting blank cells will not change the memory used: we do not allocate > memory for blank cell location. I'll see what I can do to make the sheet a > little less memory intensive. Morten, I'm sorry, it's NOT to delete blank cells, but to Resize(adjust) every sheet(operation: mouse right click every sheet to select Resize, and then adjust the number of column and row. ) and then save the resized file. Does the resizing can reduce the memory of the big file?
Lee, resizing the sheet. deleting cells etc. will have little effect. Tonight (my time) I will be able to create a .gnumeric file for you that you should be able to load to access the file content.
(In reply to comment #17) > Lee, resizing the sheet. deleting cells etc. will have little effect. Tonight > (my time) I will be able to create a .gnumeric file for you that you should be > able to load to access the file content. Andreas, thank you very much, and good work.
(In reply to comment #14) > This problem has been fixed in the development version. The fix will be > available in the next major software release. Thank you for your bug report. Morten, Really? It's a good news. I'd can't wait to see and use your newest version Gnumeric. May the Gnumeric be a world-class spreadsheet software in the world in short time.
Morten, do we need this console output:? Expressions in: 1022130 Expressions stored: 43 Expressions killed: 1022087
(In reply to comment #17) > Lee, resizing the sheet. deleting cells etc. will have little effect. Tonight > (my time) I will be able to create a .gnumeric file for you that you should be > able to load to access the file content. Andreas, Yes, your hope does come true, I've got your email, it'so great that I can download it by firefox in 3 seconds,and then can open(read)it by Gnumeric in 5 seconds. How can you do reducing 17 MB to 8.3MB! and let me open it in seconds , it's a great job. I'd believe that Gnumeric will become a world-class spreadsheet software in a short time, because your nice and good helping. Thank you for your nice and good helping. Now, I want to use Gnumeric's functions to programming my spreadsheet. You are good job! Thank you again.