After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 732633 - Gst.MessageType.ANY with Python3 leads to "Python int too large to convert to C long"
Gst.MessageType.ANY with Python3 leads to "Python int too large to convert to...
Status: RESOLVED OBSOLETE
Product: pygobject
Classification: Bindings
Component: introspection
unspecified
Other Linux
: Normal normal
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
Depends on:
Blocks:
 
 
Reported: 2014-07-02 15:18 UTC by Sebastian Dröge (slomo)
Modified: 2016-12-23 01:54 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Sebastian Dröge (slomo) 2014-07-02 15:18:22 UTC
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753505

Hi,

The following test program in python works with python2 but fails with python3.

--
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

Gst.Pipeline().get_bus().poll(Gst.MessageType.ANY, Gst.CLOCK_TIME_NONE)
--

Output:
--
$ python3 t.py 
Traceback (most recent call last):
  • File "t.py", line 5 in <module>
    Gst.Pipeline().get_bus().poll(Gst.MessageType.ANY, Gst.CLOCK_TIME_NONE)
OverflowError: Python int too large to convert to C long
--

Thanks,

Alex
Comment 1 Sebastian Dröge (slomo) 2014-07-02 16:03:27 UTC
I think this is a problem in pygobject with properly converting the value.

>>> b.poll(Gst.MessageType.ANY, 0)
Traceback (most recent call last):
  • File "<stdin>", line 1 in <module>
OverflowError: Python int too large to convert to C long
>>> Gst.MessageType.ANY
<flags GST_MESSAGE_EOS | GST_MESSAGE_ERROR | GST_MESSAGE_WARNING | GST_MESSAGE_INFO | GST_MESSAGE_TAG | GST_MESSAGE_BUFFERING | GST_MESSAGE_STATE_CHANGED | GST_MESSAGE_STATE_DIRTY | GST_MESSAGE_STEP_DONE | GST_MESSAGE_CLOCK_PROVIDE | GST_MESSAGE_CLOCK_LOST | GST_MESSAGE_NEW_CLOCK | GST_MESSAGE_STRUCTURE_CHANGE | GST_MESSAGE_STREAM_STATUS | GST_MESSAGE_APPLICATION | GST_MESSAGE_ELEMENT | GST_MESSAGE_SEGMENT_START | GST_MESSAGE_SEGMENT_DONE | GST_MESSAGE_DURATION_CHANGED | GST_MESSAGE_LATENCY | GST_MESSAGE_ASYNC_START | GST_MESSAGE_ASYNC_DONE | GST_MESSAGE_REQUEST_STATE | GST_MESSAGE_STEP_START | GST_MESSAGE_QOS | GST_MESSAGE_PROGRESS | GST_MESSAGE_TOC | GST_MESSAGE_RESET_TIME | GST_MESSAGE_STREAM_START | GST_MESSAGE_NEED_CONTEXT | GST_MESSAGE_HAVE_CONTEXT | GST_MESSAGE_EXTENDED | GST_MESSAGE_DEVICE_ADDED | GST_MESSAGE_DEVICE_REMOVED | GST_MESSAGE_ANY of type GstMessageType>
Comment 2 Simon Feltman 2014-07-02 23:58:43 UTC
First, a workaround:

>>> Gst.Pipeline().get_bus().poll(Gst.MessageType(0xffffffff), Gst.CLOCK_TIME_NONE)


Gst.MessageType.ANY seems to be a sign extended int64:

>>> hex(Gst.MessageType.ANY)
'0xffffffffffffffff'



Gjs is also has similar problems with this:

gjs> var Gst = imports.gi.Gst;
gjs> var pipe = new Gst.Pipeline();
gjs> pipe.get_bus().poll(Gst.MessageType.ANY, Gst.CLOCK_TIME_NONE);
Error: 0xffffffffffffffff is not a valid value for flags GstMessageType

gjs> Gst.MessageType.ANY
-1



The underlying type specified by the GI typelib is a signed int32 which seems wrong:

>>> from gi import _gi
>>> from gi.repository import Gst
>>> info = Gst.MessageType.__info__
>>> info.get_storage_type() == _gi.TypeTag.INT32
True



GI's interpretation of GST_MESSAGE_ANY (~0) is -1:

typedef enum
{
  ...
  GST_MESSAGE_ANY               = ~0
} GstMessageType;

GIR:

      <member name="any"
              value="-1"
              c:identifier="GST_MESSAGE_ANY"
        ...



This seems to be C's interpretation as well:

#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>

int main(void)
{
  printf("~0         == 0x%" PRIx64 "\n", (int64_t)~0);
  printf("0xffffffff == 0x%" PRIx64 "\n", (int64_t)0xffffffff);
}

---- output with gcc on x86-64  ----
~0         == 0xffffffffffffffff
0xffffffff == 0xffffffff



The documentation for "gint64 g_value_info_get_value()" [1] states:

"Returns: the enumeration value. This will always be representable as a 32-bit signed or unsigned value. The use of gint64 as the return type is to allow both."

So it seems pygobject is receiving -1 as a sign extended gint64 from GI [2] and just trying to use it. I think changing ~0 to 0xffffffff in Gst may fix the problem, but I'm not sure and it might not be the only problem in this realm...


[1] https://developer.gnome.org/gi/stable/gi-GIEnumInfo.html#g-value-info-get-value
[2] https://git.gnome.org/browse/gobject-introspection/tree/girepository/gienuminfo.c?id=1.39.0#n221
Comment 3 Sebastian Dröge (slomo) 2014-07-03 08:10:41 UTC
Thanks, let's work around that in GStreamer then by just using 0xffffffff.

This is still a problem in pygobject or g-i though, "~0" should be interpreted as an int and not as a long or guint64. enums are ints in C, and there's no implicit casting of integer literals to the bigger types. For that to work it would have to be something like "~0ULL".
Comment 4 Simon Feltman 2014-07-03 09:48:48 UTC
(In reply to comment #3)
> This is still a problem in pygobject or g-i though, "~0" should be interpreted
> as an int and not as a long or guint64. enums are ints in C, and there's no
> implicit casting of integer literals to the bigger types. For that to work it
> would have to be something like "~0ULL".

I think it is a bit more tricky than that. From the C spec:

"4 Each enumerated type shall be compatible with char, a signed integer type, or an unsigned integer type. The choice of type is implementation-defined, 110) but shall be capable of representing the values of all the members of the enumeration."

If you look at some clang ast output it reveals this:

typedef enum {
  AA = ~0,
  BB = 0xffffffff,
} EnumAB;

Dump with: clang -Xclang -ast-dump test.c

|-EnumDecl 0x2446010 <test.c:8:9, line:11:1>
| |-EnumConstantDecl 0x2446100 <line:9:3, col:9> AA 'int'
| | `-UnaryOperator 0x24460e0 <col:8, col:9> 'int' prefix '~'
| |   `-IntegerLiteral 0x24460c0 <col:9> 'int' 0
| `-EnumConstantDecl 0x2446170 <line:10:3, col:8> BB 'long'
|   `-ImplicitCastExpr 0x24461c0 <col:8> 'long' <IntegralCast>
|     `-IntegerLiteral 0x2446150 <col:8> 'unsigned int' 4294967295
|-TypedefDecl 0x2446230 <line:8:1, line:11:3> EnumAB 'enum EnumAB':'EnumAB'

AA is a *signed int* of 0 with the unary ~ applied making it an integer of -1.
BB is an *unsigned int* cast to a long, making the sizeof(EnumAB) == 8 because it needs to represent both -1 and unsigned 0xffffffff.

If you separate out the values from the previous example we get different signs for each enum type:

typedef enum {
  A = ~0,
} EnumA;

typedef enum {
  B = 0xffffffff,
} EnumB;

EnumA will be a signed int and EnumB will be an unsigned int (both 32 bits). I think GI needs to know the sign of the underlying type in order to properly serialize/deserialize the values. And I'm pretty sure the casting to int64 is used to accommodate this craziness, but will only work if the signed-ness of the enum values are what we'd expect.
Comment 5 Simon Feltman 2016-12-23 01:54:03 UTC
The specific problem here was solved in GStreamer. There isn't much we could have done in PyGI anyhow.