Created attachment 207664 [details]
info on the buffers output by ffdec_mpeg4, as provided by debugspy

Only some of the buffers output by ffdec_mpeg4 have the pixel-aspect-ratio set in the caps, but it is absent from the other buffers.
On the video with which I have tested it, this seems to be every 6 frames.

To observe that, I used the following command:

gst-launch -m filesrc location=/path/to/file.avi ! avidemux ! ffdec_mpeg4 ! debugspy ! fakesink  num-buffers=20 | grep buffer

I have attached the output of that command, where you can see that only every 6th buffer has the pixel-aspect-ratio set in its caps.

After some git bisect, it seems that this has been introduced by the following commit:

commit 99e61c5f3b8cec62091bb76cb074aa883f92a74c
Author: Edward Hervey <edward.hervey@collabora.co.uk>
Date:   Mon Oct 17 16:27:36 2011 +0200

    gstffmpegdec: Re-enable MT-decoding by default

Meaning it is likely to be due to a bug in the MT mpeg4 decoder of ffmpeg.

I noticed this because this makes gdppay/gdpdepay send a newsegment event again when the caps change. Then the following pipeline breaks:
gst-launch filesrc location=/path/to/file.avi ! avidemux ! ffdec_mpeg4 ! gdppay ! gdpdepay ! xvimagesink