GstMessageType above GST_MESSAGE_EXTENDED not usable with python3 on windows 

The following python3 code:

    import gi
    gi.require_version("Gst", "1.0")
    from gi.repository import Gst
    print(Gst.MessageType.get_name(Gst.MessageType.DEVICE_ADDED))

fails on windows 32 and 64 with this error:

    OverflowError: Python int too large to convert to C long

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):

+ Trace 237898

The same is true for any GstMessageType above or equal to GST_MESSAGE_EXTENDED.

My understanding of the issue is that, something, somewhere, is trying to map the Gst enum/flag to a C long.
On Windows, according to [1], a long is a signed 32 bit integer (range is –2147483648 through 2147483647).
But GST_MESSAGE_EXTENDED (and above enums) are too big to fit a Windows C long.
GST_MESSAGE_EXTENDED = (1 << 31) = 2147483648 > 2147483647.

I found a somewhat similar issue : https://bugzilla.gnome.org/show_bug.cgi?id=732633
But this issue is related to signed vs unsigned (and is fixed in 1.12.2).

Is that issue fixable ? If yes, where should I look : pyobject, gobject-introspection, ... ?

This issue appears with GStreamer 1.12.2, gobject-introspection 1.52.1 and pygobject 3.24.1 (which are all relatively recent) on a MSYS2 system.

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa383751(v=vs.85).aspx

PS : gstmessage.h has this comment : /* FIXME: 2.0: Make it NOT flags, just a regular 1,2,3,4.. enumeration */
Would be great to do ;) but would break ABI I guess...

The OverflowError emanates from PyLong_AsLong(PyObject *obj) method (https://github.com/python/cpython/blob/3.6/Objects/longobject.c#L471)

But I am having a hard time finding the call path that leads to this method being called...

I should be able to build all components involved (gstreamer, gst-python, gobject-instrospection, pyobject) from source so I can try things out given some pointers or guidance.

Some findings:

>>> from gi import _gi
>>> from gi.repository import Gst
>>> info = Gst.MessageType.__info__
>>> info.get_storage_type() == _gi.TypeTag.INT32
False
>>> info.get_storage_type() == _gi.TypeTag.UINT32
True

The above results are different from the same test done in 
https://bugzilla.gnome.org/show_bug.cgi?id=732633

So something was fixed since or, for a change, windows is better than linux.

Extract from Gst-1.0.gir:

      <member name="extended"
              value="2147483648"
              c:identifier="GST_MESSAGE_EXTENDED"
              glib:nick="extended">
        [...]
      </member>
      <member name="device_added"
              value="2147483649"
              c:identifier="GST_MESSAGE_DEVICE_ADDED"
              glib:nick="device-added">
        [...]
      </member>
        [...]
      <member name="any"
              value="4294967295"
              c:identifier="GST_MESSAGE_ANY"
              glib:nick="any">
        [...]
      </member>

Looks correct too.

Using this test program:

  import gi
  gi.require_version("Gst", "1.0")
  from gi.repository import Gst
  print(Gst.MessageType.get_name(Gst.MessageType.DEVICE_ADDED))

I captured the back trace leading to the conversion error:

Thread 1 hit Breakpoint 2, PyLong_AsLong (obj=0x280d2f8)
    at ../Python-3.6.2/Objects/longobject.c:478
478             PyErr_SetString(PyExc_OverflowError,
(gdb) bt

+ Trace 237911

Hacking the _pygi_marshal_from_py_interface_flags() method to make it use PYGLIB_PyLong_AsUnsignedLong() instead of PYGLIB_PyLong_AsLong() seems to fix the test case...

>>> import gi
>>> from gi.repository import Gst
>>> print(Gst.MessageType.get_name(Gst.MessageType.DEVICE_ADDED))
device-added

Not sure at all if this hack is acceptable though...

flags aka "bitfields" (in gobject-introspection speak) are always unsigned, enums are signed. So changing it for flags should be fine, for enums not so much.

Fortunately enums and flags are marshalled/unmarshalled through separate code pathes. 

I'll submit a bug fix to pyobject.

Yeah, reassigned this bug to pygobject :)

Created attachment 358860 [details] [review]
fix an overflow when marshalling flags from py

Tried to submit a PR (https://github.com/GNOME/pygobject/pull/1) but it got closed (by a robot?).

Having the patch here is all that is needed. GitHub is only a mirror for making life easier for people using GitHub, but patch review and bug tracking happens here.

Created attachment 362091 [details] [review]
flags: Add testcase for bug 786948

Created attachment 362092 [details] [review]
flags: Add testcase for bug 786948

Apparently out marshaling and comparisons are also broken, so reducing the test to only test the fixed case.

Review of attachment 358860 [details] [review]:

lgtm

Thanks

Thanks