GNOME Bugzilla – Bug 168875
Wishlist: increase/set the max number of rows
Last modified: 2009-12-09 20:57:38 UTC
for a professionnal usage of gnumeric, for example to analyze genetic data, we need more than the 64k rows, so it would be extremly useful either to increase the default number, or to be able to set it via the preferences. Thanks for your good job
In src/gnumeric.h edit these and recompile. #define SHEET_MAX_ROWS (16*16*16*16) /* 0, 1, ... */ #define SHEET_MAX_COLS (4*4*4*4) /* 0, 1, ... */ It would be nice to have a command-line option to set this, but right now it is compile-time only.
*** Bug 172241 has been marked as a duplicate of this bug. ***
*** Bug 309400 has been marked as a duplicate of this bug. ***
This should not be a compile OR runtime max. This should be a dynamic value with sparse arrays. GNU sort doesn't have a max, GNU sed doesn't have a max, GNU emacs doesn't have a max. It's harder but is the GNU way.
Indeed ! And this could be a GOOD feature that no other spreadsheet has.
Let me see... GNU sed has a LONG_MAX limit on number of lines. Gnu sort undoubledly has a similar limit for key numbers. GNU emacs has lots of limits. For example, file max 128MB (== UINT_MAX/32). Or try evaluating (* 65536 65536) and get a nice round zero. We will fix this sooner or later, but it really is not a supreme priority.
FYI, in http://bugs.debian.org/156374 Jody pointed out that changing these values may break Excel compatibility in a limited way: 1) My gut reaction is to keep the current limits to maintain compatibility with XL's habit of wrapping adresses. eg if you copy the expression '=A1' from B2 in B1 it will convert into IV1. Its not used alot, but something may depend on it. I'd love to see things change so the dimensions can be adjusted on the fly; the current values could remain as defaults for this compatibility.
JHM : I'm working on the next generation of the cell storage data structure (an extension of the style quadtree) that would give us runtime resizability.
Glad to hear it, Jody. Just ping me if this needs to be tested in Debian through "experimental" packages.
*** Bug 498363 has been marked as a duplicate of this bug. ***
Created attachment 109913 [details] [review] incomplete patch This patch allows larger sheets and, to some extents sheets of different sizes in the same workbook. Points that still need to be addressed, at least: * GnmRange should be replaced by GnmSheetRange * Fix ROW_COL_KEY macro in format-templates.c so that it supports more than MAXINT cells * Fix BUCKET_LAST in dependent.c * replace TILE_TOP_LEVEL in sheet-style.c by a Sheet member.
Forgotten one important point: add a user interface for the size of the inserted sheet (at the moment, it inserts a 32 cols x 1024 rows sheet, used for testing).
*** Bug 353004 has been marked as a duplicate of this bug. ***
Created attachment 109928 [details] [review] updated patch Things start to work with this one.
Created attachment 109943 [details] [review] one more version Added a simple UI (help button not managed, but do we need a help button there?). The dialog allows sheets with 1 to 4096 columns and 1 to 16777216 rows. While testing, I got a crash on exit (somewhere in dependent.c), but could not reproduce it.
Created attachment 109971 [details] [review] still a better one Only one identified issue remaining: gnm_dep_container_dump needs to know how many rows are valid. Several solutions are possible: pass a Sheet* as argument to this function or store the row number in struct _GnmDepContainer. The only call to this function is at src/wbc-gtk.c:1497 and takes sheet->deps as argument, so the Sheet is known at the caller stage.
Created attachment 109995 [details] [review] this one adds serialization
Created attachment 110075 [details] [review] Update patch This is an updated version of the patch after my committing some parts to both branches. (I wish to be able to move other, later patches between branches easily.)
Created attachment 110117 [details] [review] Updated patch I committed some more SHEET_MAX_foo things. This is the leftover. Jean: We used to have array types like int foo[SHEET_MAX_COLS]; We cannot just replace that constant with a function call.
We'll have to allocate/free them, IMHO. I did not address that?
Created attachment 110118 [details] [review] Updated patch Enhanced the new sheet dialog. If we have a dialog, we can set more than the size.
Looking good. Comments: src/xml-sax-read.c, second hunk: something is seriously weird with your intentation. plugins/openoffice/openoffice-write.c: this is one of these [SHEET_MAX_COLS] cases. plugins/excel/ms-excel-write.c: this is one of these [SHEET_MAX_COLS] cases. plugins/excel/boot.c: leaked debug code. plugins/excel/xlsx-write.c: another case: - GnmStyle *col_styles [MIN (XLSX_MAX_COLS, SHEET_MAX_COLS)]; + GnmStyle *col_styles [MIN (XLSX_MAX_COLS, gnm_sheet_get_max_cols (state->sheet))];
I committed the xml-sax-write.c part too. It broke the t5* tests -- anything that changes output does -- so I had to update those as well.
Created attachment 110143 [details] [review] Updated patch Fixed previous issues. Btw, I don't remember what the ms-chart.c changes should fix, looks like an old non-commited patch, but unrelated to sheets sizes. In xml-sax-read.c, using array_cols and array_rows for the sheet size breaks things unless they are reinitialized to -1 before leaving xml_sax_wb_sheetname. Which solution is the best? This has to be fixed ASAP.
> In xml-sax-read.c, using array_cols and array_rows for the sheet size breaks > things unless they are reinitialized to -1 before leaving xml_sax_wb_sheetname. Fixed. Now using new state variables sheet_cols and sheet_rows.
Created attachment 110164 [details] [review] Updated patch Updated accordingly. Thanks
All right, we might be getting close to having to deal with the actual contents of the patch, :-) I would like to see the ability to resize existing sheets by setting the cols and rows properties. If we had that, then things like undo-sheet-delete would work. Right now, I do not see how it can work. If we had that, insert/append/etc. sheet could take the initial size from another sheet in the workbook and there ought not be any need for a special command to insert a specially sized sheet. Changing sheet size would probably have to queue everything for recalc. (This is for discussion purposes only -- don't rush off and spend a lot of time until and unless we agree that it is the right thing to do.)
My feeling is that we'll have a few issues to address which are just not obvious, especially in ranges.c and sheet-styles.c. Jody's work about using a quadtree might hopefully make things much easier.
I am adding on to the row and column limit request. Since MS Office 2007 can now support over 1 million rows and 16,000 columns (probably 1M and 16K), and since Gnumeric can do the same after recompile, would it be hard to make a version of Gnumeric with these row and column limitations available for download so that I myself would not have to do recompile Gnumeric.
How to get more columns to be compatible to large OO-files (*.sxc, *.ods)? ** (gnumeric-1.9.6:13332): WARNING **: Ignoring column information beyond the range we can handle. the latest statements are quite a long time ago. Are there any new possibilities / patches to increase the number of columns up to 1024? Actually the last column of a gnumeric spreadsheet is "IV".
*** Bug 577477 has been marked as a duplicate of this bug. ***
Created attachment 131814 [details] [review] updated patch This patch is updated to apply against svn as of 2009/3/31.
Patch is in, except that all gui parts are commented out. Search for GNUMERIC_VARIABLE_SHEET_SIZE. We need to get rid of all uses of gnm_sheet_get_(max|last)_(rows|cols) with a NULL sheet. There are a pile of them. (Some might be hidden by using an expression that evaluates to NULL instead of an explicit NULL.) After that, we need to figure out if things like 3D deps survive having different sheet sizes within one workbook. And inter-workbook refs have just become interesting to say the least.
about comment #30: set max_cols to 1024 using gconf-editor (the actual key is apps/gnumeric/core/workbook/n_cols), and ods files will be opened with 1024 columns in the next gnumeric section using most recent svn trunk.
Things are changing rapidly. The advice from comment 34 is already out of date. Please wait for things to settle down.
text and csv files now load into sheets as big as needed.
The loading of "oversized" csv-files works now. But the columns above "IV" and the rows above 65536 cannot be addressed yet. The VARIABLE_SHEET_SIZE is a very useful and often missed feature. I wonder why anybody has ever reached to integrate this feature in a spreadsheet-program before. (free, fast, accurate and unlimited:-)
Parsing is known to be broken. It is the last user of gnm_sheet_get_max_cols (NULL). It'll be a couple of weeks before anything happens on that front.
This problem has been fixed in the development version. The fix will be available in the next major software release. Thank you for your bug report.