GNOME Bugzilla – Bug 694501
hash resize broken for large sizes
Last modified: 2018-05-24 15:03:26 UTC
_GHashTable.size is a 32 bit signed integer. The resize policy is to double the table size. This is done updating size using a left shift (1 << shift), then allocating memory using g_new0. When shift reaches 31, size becomes -2147483648, this is then interpreted as a 64 bit integer 18446744071562067968 and malloc gives the following cryptic error: (process:47597): GLib-ERROR **: /build/buildd/glib2.0-2.32.3/./glib/gmem.c:382: overflow allocating 18446744071562067968*8 bytes Trace/breakpoint trap (core dumped) This condition should be detected and a more accurate error message should be given. Ideally, in the longer term, with todays 64 bit computers and large RAM sizes, larger hashes and arrays should be allowed.
+ Trace 231556
*** Bug 789432 has been marked as a duplicate of this bug. ***
The sizes in ghash.c being ints gives me immediate pause; wouldn't gsize be semantically and practically the right type to use? or at least some type of known fixed width, if we're going to deliberately limit the number of bits Parallel to that, the check in maybe_resize() that if ((size > hash_table->nnodes * 4... seems like it should first check that hash_table->nnodes isn't too big to multiply up without overflow (at whatever fixed width is decided). And then yes, a useful diagnostic would be preferable.
(In reply to Daniel Boles from comment #2) > The sizes in ghash.c being ints gives me immediate pause; wouldn't gsize be > semantically and practically the right type to use? or at least some type of > known fixed width, if we're going to deliberately limit the number of bits > > Parallel to that, the check in maybe_resize() that if ((size > > hash_table->nnodes * 4... seems like it should first check that > hash_table->nnodes isn't too big to multiply up without overflow (at > whatever fixed width is decided). And then yes, a useful diagnostic would be > preferable. Basically: • GHashTable is from the era of writing C when everything was an int and nobody used types like size_t • It was never written to be overflow-safe Ideally both of those would be fixed (without breaking backwards compatibility).
First of all, sorry for submitting duplicate bug report 789432. I searched but could not find this one. Then, as I say in that bug report, the problem is that integer overflow occurs earlier than what someone would expect given that size is 32-bit integer. So someone would expect that at least her application would work correctly for up to 2^31, but the problem occurs at 2^29 due to the hash_table->nnodes * 4 expression.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/672.