After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 541130 - [avimux] full and fast recording
[avimux] full and fast recording
Status: RESOLVED NOTABUG
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other All
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2008-07-01 20:09 UTC by Julien Isorce
Modified: 2010-12-06 10:34 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Julien Isorce 2008-07-01 20:09:15 UTC
Steps to Reproduce:

linux:
gst-launch-0.10 v4l2src num-buffers=200 ! "video/x-raw-yuv, format=(fourcc)YUY2, width=640, height=480, framerate=(fraction)20/1" ! queue ! avimux name=mux alsasrc buffer-time=500000 provide-clock=1 num-buffers=10 ! "audio/x-raw-int, rate=44100, channels=2" ! queue ! mux. mux. ! filesink location=record.avi

win32: (you need to apply the dshowvideosrc patch from bugzilla in plugins-bad section, in order the choice the video size and framerate)
gst-launch-0.10 dshowvideosrc num-buffers=200 ! "video/x-raw-yuv, format=(fourcc)YUY2, width=720, height=576, framerate=(fraction)25/1" ! queue ! avimux name=mux dshowaudiosrc buffer-time=500000 provide-clock=1 num-buffers=10 ! "audio/x-raw-int, rate=44100, channels=2" ! queue ! mux. mux. ! filesink location=record.avi


Expected Results:

When playing back, the video is jerky and video/audio are not synchronized.


Additional Information:

It's works fine when replacing dshowaudiosrc (alsasrc),  by the audiotestsrc element (but provides only one channel).

You need a good hard-disk to record the big flow that the pipelines generates.
You can try first by replacing the videosrc, by the videotestsrc element to test your hard-disk.
Comment 1 Mark Nauwelaerts 2008-07-01 22:13:28 UTC
Thank you for taking the time to report this.  However:
* running the above pipeline is very likely (and does so on my system) to produce only a very small amount of audio data, with a lot more video data (because there will only be 10 small audio buffers).  It is therefore impossible to have a/v sync (due to lack in audio)
* "replacing with audiotestsrc"; wat does "it works fine" then mean (since it is hard to judge a/v sync in that case)
* there has been a bug in avimux (into the latest release) that caused a/v desync in case of uncompressed raw audio; please make sure CVS HEAD is used

In any case, a slightly adjusted pipeline (to cater for hardware and above comments) produces a fine result (that is; nothing jerky and a/v sync ok (*)):

gst-launch-0.10 v4l2src num-buffers=200 ! "video/x-raw-yuv, format=(fourcc)YUY2, width=384, height=480, framerate=(fraction)25/1" ! \
queue ! avimux name=mux \
alsasrc buffer-time=500000 provide-clock=1 num-buffers=16 blocksize=48000 ! "audio/x-raw-int, rate=48000, channels=1" ! queue ! mux. \
mux. ! filesink location=record.avi

(*) which is not necessarily the case with other hardware and/or other circumstances; please also see the use of the videorate element and/or
http://gentrans.sourceforge.net/docs/head/manual/html/howto.html#sect-recording
for a discussion of a/v sync issues (which is out of scope of avimux here).
Comment 2 Julien Isorce 2008-07-01 22:38:08 UTC
Hi,

I did a mistake in the linux pipeline above: alsasrc num_buffers=2000 or 1000.
I do not know why I have to set it to 20 on win32. (20 is about 10 sec with the dshwoaudiosrc)

About replacing alsarc by audiotestsrc, I mean that the video is not jerky, but sure it's not possible to judge a/v sync.

About width and height, It works fine with "small" video size, but not with 720*576. Maybe that's why your pipeline works. Yes it's very strange.

About videorate, I have already tried to put it in the pipeline but the result is the same. Moreover I Have a strong capture card (S-video and composite, audio stereo), so I am sure the card really provides 25 FPS.

I am using a cvs version from last week, but it seems it has been updated now.

I will update my checkout and test again.

Take in touch.

Sincerely

J.


Comment 3 Julien Isorce 2008-07-02 16:21:50 UTC
Oh I now using head cvs and the problem is still the same with big and full resolution YUY2.

I have tried to add the ffenc_huffyuv element (which is lossless), and the video is now not jerky, and everything (a/v sync) is ok. So it definitively seems that the problem comes from avimux/YUY2 with big video size.

J.
Comment 4 Mark Nauwelaerts 2008-07-04 21:46:50 UTC
If I use the pipeline I gave above, but with 720x576, I also experience problems.  However, if I adjust to the following pipeline, I do not:

gst-launch-0.10 v4l2src queue-size=15 num-buffers=200 ! "video/x-raw-yuv, format=(fourcc)YUY2, width=720, height=576,framerate=(fraction)25/1"  !  queue ! avimux name=mux \
alsasrc buffer-time=500000 provide-clock=1 num-buffers=56 blocksize=12000 ! "audio/x-raw-int, rate=48000, channels=1" ! queue ! mux. \
mux. ! filesink location=record.avi

The difference is in:
* queue-size has been set for v4l2src
* audio buffer size has been changed

and the explanation, I believe, is as follows:
* dumping a lot of (raw) data creates a lot of I/O activity, which keeps kernel and/or PCI bus (somewhat more) occupied, so having a small queue along the way (= the v4l2 capture buffer queue) could then be more painful than otherwise (i.e. drops, irregular capture, ...)
* smaller audio buffers allow for better interleaving, otherwise players might experience jerkyness, latency and/or complain "bad interleaving"

Even with the above issues addressed, there can still be the following effect that might cause minor glitches, though likely not so pronounced (based on test):
* whatever (video) does make it through is likely to have even more irregular timestamps (than they otherwise already are), specifically without videorate or so to straighten these out.  avimux then orders/muxes those accordingly (as it should), which can then also lead to bad/irregular interleaving

But, as said, the pipeline given above already does a (near) impeccable job in my case, and going to full tricks/techniques according to aforementioned URL, it is likely even more so the case.
Comment 5 Julien Isorce 2008-07-10 11:17:15 UTC
Hi, ok I am agree with your explanation.

Unfortunnaly dshowvideosrc has no queue-size for now. And changing the dshowaudiosrc's blocksize did not work (no sound).

As I said in my last post (#3), it works fine with ffenc_huffyuv.
So I tried something else to have yuy2, first record to huffyuv, then record to yuy2:

gst-launch-0.10 dshowvideosrc num-buffers=200! "video/x-raw-yuv, format=(fourcc)YUY2, framerate=(fraction)25/1, width=720, height=576" ! ffmpegcolorspace ! queue ! ffenc_huffyuv ! queue name=qv dshowaudiosrc buffer-time=500000 provide-clock=1 num_buffers=N ! "audio/x-raw-int, rate=44100, channels=2" ! queue name=qa avimux name=mux ! filesink location=huffyuv.avi qv. ! mux. qa. ! mux. 

Then:

gst-launch filesrc location=huffyuv.avi ! queue ! avidemux name=demux ! queue ! \
ffdec_huffyuv ! queue ! ffmpegcolorspace ! "video/x-raw-yuv, format=(fourcc)YUY2" ! \
queue ! avimux name=mux demux. ! queue ! mux. mux. ! queue ! \
filesink location=yuy2.avi

But I still experienced the same problems.

What can I try on the filesrc and queue properties ? 
(queue's max-size-buffers is 200 by default)

Comment 6 Mark Nauwelaerts 2008-07-12 21:20:19 UTC
So, if I understand things right, the result of the first pipeline (recording) is an avi that is (plays) just fine.  It is then transcoded (off-line) to another avi (2nd pipeline), where this pipeline runs OK, but the resulting file is not so nice, as in jerky playback, etc.

That seems very strange; the latter pipeline is a basic transcoding one, and it should end up with fine output (sync etc) given good input.

In any case, I can not reproduce a problem; I have performed a recording using similar first pipeline, and transcoded with exactly the same second one, the result is an avi with raw YUY2 that is just fine (several times).

Unfortunately, there are no properties on filesrc or queue I can imagine that can affect an off-line transcoding run.  I would only recommend not to put a queue after filesrc as that will force avidemux in push-mode (which is not so traditional, though it should work, and it does in this case).  I also tend to limit queue only in the max-size-time sense, since that is meaningful across many formats, buffer sizes, audio vs video, etc, but no hard rule here either.

So, I can only suspect something wrong with the first (live) recording ...
Comment 7 Mark Nauwelaerts 2010-12-06 10:17:34 UTC
Is there still a problem here, most notably with the "second pipeline" (i.e. off-line transcoding) as referred to above?
Comment 8 Julien Isorce 2010-12-06 10:34:01 UTC
no problem here anymore