GNOME Bugzilla – Bug 113258
gst_caps_intersect() returns NULL for empty intersection
Last modified: 2004-12-22 21:47:04 UTC
gst_caps_intersect() returns NULL if the intersection is empty, even though NULL caps has the opposite meaning. I suggest having it return GST_CAPS_EMPTY instead, and change other caps math functions to handle GST_CAPS_EMPTY properly.
I don't think it's wrong. Basically, NULL as caps on a pad template or as a return value of _getcaps() means 'anything', but NULL as an intersection means 'nothing' - the combination of these still means 'anything'. This is somewhat insonsistent, but it makes sense.
No, it still doesn't make sense. :) It is easy to generate a case where gst_caps_intersect(a,gst_caps_intersect(b,c)) and gst_caps_intersect(gst_caps_intersect(a,b),c) return different values. Looking at it from a set theory perspective, we use NULL to indicate both the empty set (curiously also called the null set), and the union of all sets. This plays much havoc with our set operations. We also use NULL to indicate the end of chained caps. I'd like to change my recommendation to define a GST_CAPS_ANY to indicate the union of all possible caps, which might not be NULL. In any case, we should probably start using GST_CAPS_ANY and GST_CAPS_NONE instead of NULL.
Just want to voice support for Davids proposal.
So how will we define GST_CAPS_ANY? I could propose: #define GST_CAPS_NONE NULL #define GST_CAPS_ANY gst_caps_get_any() GstCaps * gst_caps_get_any (void) { static GstCaps *caps = NULL; if (!caps) { caps = GST_CAPS_NEW ("gst_caps_any", "*", NULL); } return gst_caps_ref (caps); } That'd sound ok to me.
My first thought was to use ((void *)1) and ((void *)2) for ANY and NONE, but that causes problems testing for NULL. I don't know why anyone would need to test for NULL, though. GST_CAPS_NONE probably should be NULL. It can be used as the end of chain indicator as well. I like the idea of making GST_CAPS_ANY a fixed number, since that makes it easy to test, instead of needing a GST_CAPS_IS_ANY(). Alternatively, we could special-case gst_caps_copy() so that it won't actually copy the ANY caps.
I'm starting with the migration. The first step is to define GST_CAPS_ANY and GST_CAPS_NONE (done) and identify as many places in gstreamer and gst-plugins where NULL needs to be converted to GST_CAPS_ANY. (working on it) (GST_CAPS_ANY is still defined as NULL.) After that is done, I'll add code to properly intersect caps with GST_CAPS_ANY, and add an assertion to make sure that pad caps are not set to GST_CAPS_NONE. I *think* that this can all be done so that one can flip a #define and get either the old behavior or new behavior. At least, I'm going to attempt to do it that way, to make debugging easier. One thing that hasn't been discussed: props may need similar treatment. However, it should only need to be internal. A caps with GST_PROPS_NONE would imply GST_CAPS_NONE, so it would only need to be used internally in the props intersection code.
Fixed in CAPS merge