After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 686451 - glib-2.0: make sure to null terminate the argument of string.joinv
glib-2.0: make sure to null terminate the argument of string.joinv
Status: RESOLVED FIXED
Product: vala
Classification: Core
Component: Bindings: GLib
unspecified
Other All
: Normal normal
: ---
Assigned To: Vala maintainers
Vala maintainers
Depends on:
Blocks:
 
 
Reported: 2012-10-19 09:19 UTC by Daiki Ueno
Modified: 2014-01-31 03:28 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
glib-2.0: make sure to null terminate the argument of string.joinv (1.45 KB, patch)
2012-10-19 09:19 UTC, Daiki Ueno
none Details | Review
glib-2.0: make sure to null terminate the argument of string.joinv (2.09 KB, patch)
2012-10-19 09:45 UTC, Daiki Ueno
needs-work Details | Review
glib-2.0: make sure to null terminate the argument of string.joinv (2.23 KB, patch)
2012-10-19 23:24 UTC, Daiki Ueno
none Details | Review
glib-2.0: make sure to null terminate the argument of string.joinv (2.24 KB, patch)
2012-10-22 02:40 UTC, Daiki Ueno
needs-work Details | Review
glib-2.0: port string.joinv to Vala (1.84 KB, patch)
2012-10-24 07:28 UTC, Daiki Ueno
rejected Details | Review

Description Daiki Ueno 2012-10-19 09:19:51 UTC
I'm often trapped into the following (wrong) code:

  Gee.ArrayList<string?> elements = new ArrayList<string?> ();
  elements.add ("one");
  elements.add ("two");
  elements.add ("three");
  // elements.add (null); is needed!
  string.joinv(",", elements.to_array ());

that gets buffer overrun.

How about null-terminate the argument in Vala level?
Comment 1 Daiki Ueno 2012-10-19 09:19:53 UTC
Created attachment 226802 [details] [review]
glib-2.0: make sure to null terminate the argument of string.joinv
Comment 2 Daiki Ueno 2012-10-19 09:45:31 UTC
Created attachment 226805 [details] [review]
glib-2.0: make sure to null terminate the argument of string.joinv

Test added.
Comment 3 Jürg Billeter 2012-10-19 18:51:43 UTC
Review of attachment 226805 [details] [review]:

If the patch is changed to keep the overhead at a minimum, I'm ok with merging it. However, we may want to consider handling this in a more general matter in the code generator to make it work for all functions expecting null-terminated (string) arrays.

::: vapi/glib-2.0.vapi
@@ +946,3 @@
+	public static string _joinv (string separator, [CCode (array_length = false, array_null_terminated = true)] string[] str_array);
+	public static string joinv (string separator, string[] str_array) {
+		string[] strv = str_array;

This makes an unconditional deep copy of the string array. I think we should avoid this in the case that the string array is already null-terminated.
Comment 4 Daiki Ueno 2012-10-19 23:24:30 UTC
Created attachment 226862 [details] [review]
glib-2.0: make sure to null terminate the argument of string.joinv

Don't copy the array if it is already null terminated.

--
Thanks.  I agree with that it would be better handled in codegen,
but attaching a fix fwiw.
Comment 5 Daiki Ueno 2012-10-22 02:40:23 UTC
Created attachment 226954 [details] [review]
glib-2.0: make sure to null terminate the argument of string.joinv

Oops, fixed typo in array access (str_array[-1] -> str_array[str_array.length - 1]).
Comment 6 Evan Nemerson 2012-10-24 05:34:39 UTC
Review of attachment 226954 [details] [review]:

+		if (str_array.length > 0 && str_array[str_array.length - 1] == null) {

Arrays created by valac are null-terminated by default.  If you create an n member array valac will actually allocate room for n+1 members and set the last one to NULL.  This is a pretty common case, so instead of checking str_array[str_array.length - 1] we should probably check str_array.length].

+			return _joinv (separator, str_array);
+		} else {
+			// make sure to null terminate str_array
+			string[] strv = str_array;
+			strv += null;
+			return _joinv (separator, strv);

This is still a lot of potential copying.  valac will allocate an array, *g_strdup each element*, resize the array (possibly having to allocate a whole new one), set the last element to NULL, pass it to g_strjoinv, then free each element in the array before finally freeing the array itself.
Comment 7 Evan Nemerson 2012-10-24 05:42:28 UTC
How about something like

public static string joinv (string separator, string[] str_array) {
  string res;
  if (str_array.length > -1 && str_array[str_array.length] != null) {
    unowned string[] data = (string[]) GLib.malloc (sizeof(string) * (str_array.length + 1));
    GLib.Memory.copy (data, str_array, sizeof(string) * str_array.length);
    GLib.Memory.set (&(data[str_array.length]), 0, sizeof(string));
    res = string._joinv (separator, data);
    GLib.free (data);
  } else {
    res = string._joinv (separator, str_array);
  }
  return res;
}

It's a bit ugly (and hackish) but it gets the job done with a single malloc/free.  The other option is to just re-implement g_strjoinv which would save us a malloc but IMHO that's probably going a bit overboard.

FWIW I like the idea of implementing this in codegen instead.
Comment 8 Jürg Billeter 2012-10-24 05:52:40 UTC
(In reply to comment #6)
> Arrays created by valac are null-terminated by default.  If you create an n
> member array valac will actually allocate room for n+1 members and set the last
> one to NULL.  This is a pretty common case, so instead of checking
> str_array[str_array.length - 1] we should probably check str_array.length].

You're right that we would need to check [length] instead of [length - 1] to detect already NULL-terminated arrays. However, if the array is not already NULL-terminated, accessing [length] might crash the process. I don't currently see a solution that works without the help of the code generator.
Comment 9 Evan Nemerson 2012-10-24 06:34:21 UTC
(In reply to comment #8)
> You're right that we would need to check [length] instead of [length - 1] to
> detect already NULL-terminated arrays. However, if the array is not already
> NULL-terminated, accessing [length] might crash the process. I don't currently
> see a solution that works without the help of the code generator.

Do you see a solution which works /with/ the help of the code generator?  We have no way of knowing whether it is safe to check [length] for arrays returned by libraries.

The best I can think of is checking each element of the array in order until we find a NULL or reach [length].  At least then we're not making a crash /more/ likely (if we do access [length] g_strjoinv would have done so anyways).

Also, what about cases where the method we're calling wants to change the array?  Something like qsort but without nmemb.
Comment 10 Daiki Ueno 2012-10-24 07:28:38 UTC
Created attachment 227123 [details] [review]
glib-2.0: port string.joinv to Vala

Thanks for the insightful comments.

So, for this particular case, how about implementing joinv entirely in Vala, as the actual length information is available in Vala level?
Comment 11 Evan Nemerson 2014-01-25 07:26:07 UTC
Review of attachment 227123 [details] [review]:

Unfortunately, Vala doesn't always know the array length.  Sometimes it will pass -1 and expect it to be treated as a null-terminated array...  IIRC anything which returns a no_array_length unowned array falls into this category.

Also, I would prefer to use g_stpcpy to avoid the second strlen.
Comment 12 Evan Nemerson 2014-01-25 07:27:23 UTC
commit 3c0674fafe5d11362fe3f88bb4c849c34789b5d5
Author: Evan Nemerson <evan@coeus-group.com>
Date:   Fri Jan 24 23:21:05 2014 -0800

    glib-2.0: make string.joinv handle non-null-terminated arrays
    
    Fixes bug 686451.
Comment 13 Jussi Kukkonen 2014-01-28 22:10:57 UTC
There seems to be a problem with handling empty array argument: see bug 723195.
Comment 14 Daiki Ueno 2014-01-31 03:28:32 UTC
Cool, confirmed it's working now.  Thanks.