From mail conversation with Sebastian Dröge

The resolution is usually propagated upstream to elements with the CAPS
query, and the elements are notified of something changing with the
RECONFIGURE event. Unfortunately in this case this won't work yet as
1) after hlsdemux there is another demuxer that does not forward and
   transform the CAPS query downstream (and can't sensibly). This
   is tsdemux usually.
2) after that second demuxer there is a video decoder, and video
   decoders currently don't forward and transform the CAPS query.
   The decoder could do this quite easily, and it could probably
   even be implemented in the video decoder base class.

1) is a real problem and I currently don't know how to solve that
without putting demuxers inside hlsdemux so that hlsdemux can directly
query the CAPS on the video stream.

Hi sebastian
I checked xvideosink and other video sinks
and I simply can't find any notice that videosink sent upstream information when window size change.
ximagesink send gst_event_new_reconfigure() in gst_ximagesink_xwindow_update_geometry => regenerate caps
As this isn't caps related it is just some event that will notify upstream elements  

The reconfigure event is sent upstream, and tells upstream elements that
they should reconfigure. I.e. check with downstream if there's a more
optimal configuration now, regarding caps (e.g. the resolution!) or the
used allocation mechanism. hlsdemux would get this event whenever the
resolution changes and can then do something.

But as explained in the previous mail, it currently can't do much and
you would need to find a way to proxy the resolution in the caps up to
hlsdemux.

We need something like in your patch in bug #730225 here. I think ideally we would do the following:
- Add a "reason" to the reconfigure event, which is an array of strings. This can be "caps", "allocation", ..., "window-size". Reason being an empty array means all have to be checked (default case)
- Add a query to libgstvideo to query the downstream window size

Then before first rendering and whenever we get such a reconfigure event we would run the query. I'll take a look at implementing that.

API proposals here btw

> GstEvent*       gst_event_new_reconfigure              (void) G_GNUC_MALLOC;
> void            gst_event_add_reconfigure_reason       (GstEvent * event, const gchar * reason);
> guint           gst_event_get_n_reconfigure_reasons    (GstEvent * event);
> gboolean        gst_event_parse_nth_reconfigure_reason (GstEvent * event, guint n, gchar ** reason);
> gboolean        gst_event_has_reconfigure_reason       (GstEvent * event, const gchar * reason);

> gboolean      gst_video_query_is_surface_info    (GstQuery * query);
> GstQuery *    gst_video_query_new_surface_info   (void);
> 
> void          gst_video_query_set_surface_size   (GstQuery * query, gint width, gint height);
> gboolean      gst_video_query_parse_surface_size (GstQuery * query, gint * width, gint * height);
> 
> void          gst_video_query_set_surface_framerate   (GstQuery * query, gint fps_n, gint fps_d);
> gboolean      gst_video_query_parse_surface_framerate (GstQuery * query, gint * fps_n, gint * fps_d);
> 
> void          gst_video_query_set_surface_pixel_aspect_ratio   (GstQuery * query, gint par_n, gint par_d);
> gboolean      gst_video_query_parse_surface_pixel_aspect_ratio (GstQuery * query, gint * par_n, gint * par_d);


Will implement that in a bit if nobody has any comments

And what if we have pipeline with videoscale element?
videoscale element pass query to another element?

Yes

Then we still need to add support for window size to all elements in possible pipelines.
Like mpegdemux for POSITION for example?
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/mpegdemux/gstmpegdemux.c#n1344

hlsdemux/dashdemux/.. query src element in pipeline for window size
and elements will pass this query .. to some sink that really understand this query.
Right? :)

(In reply to comment #2)
> We need something like in your patch in bug #730225 here. I think ideally we
> would do the following:
> - Add a "reason" to the reconfigure event, which is an array of strings. This
> can be "caps", "allocation", ..., "window-size". Reason being an empty array
> means all have to be checked (default case)

How do you get the reconfigure event, usually we detect that reconfigure through the reconfigure flag on the pad. Should the event become the first upstream sticky, with a new method to check reconfigure ?

(In reply to comment #6)
> Then we still need to add support for window size to all elements in possible
> pipelines.
> Like mpegdemux for POSITION for example?
> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/mpegdemux/gstmpegdemux.c#n1344
> 
> hlsdemux/dashdemux/.. query src element in pipeline for window size
> and elements will pass this query .. to some sink that really understand this
> query.
> Right? :)

Nope, that should all be handled automagically as long as you don't have multiple video sinks... in which case you just get one of the values automatically.


(In reply to comment #7)
> (In reply to comment #2)
> > We need something like in your patch in bug #730225 here. I think ideally we
> > would do the following:
> > - Add a "reason" to the reconfigure event, which is an array of strings. This
> > can be "caps", "allocation", ..., "window-size". Reason being an empty array
> > means all have to be checked (default case)
> 
> How do you get the reconfigure event, usually we detect that reconfigure
> through the reconfigure flag on the pad. Should the event become the first
> upstream sticky, with a new method to check reconfigure ?

gst_pad_check_reconfigure_full(pad, &reasons)

Created attachment 277303 [details] [review]
video: Add new query for querying video surface information

... like the effective width and height of the video.

Created attachment 280618 [details] [review]
prototype emit query from HLSdemux

Created attachment 280619 [details] [review]
route sink query in mpegtsdemux

Created attachment 280620 [details]
reconfigure event + sink query

Anybody can plz review my last 3 patches.
I did it as prototype.
If everybody agree with this prototype I will add support for more sinks and demuxers(mkv,..)

anybody can look at this?

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/146.