After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 623821 - qtmux produces brocken .mov files
qtmux produces brocken .mov files
Status: RESOLVED NOTABUG
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
0.10.x
Other Linux
: Normal critical
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2010-07-08 10:33 UTC by Ivan Zoli
Modified: 2010-07-09 11:36 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
sterm out of my h264 encoder with header in caps (838.61 KB, application/octet-stream)
2010-07-08 14:42 UTC, Ivan Zoli
Details

Description Ivan Zoli 2010-07-08 10:33:30 UTC
I use an embedded hardware and I want to record an .mov.

But I am not able to produce an .mov that plays in QT or other popular software for edit movies. Only VLC can play is. I use this pipeline:

gst-launch -e my_embedded_v4lsrc preview=1 num-buffers=300 ! \
  my_embeded_h264encoder width=720 height=400 codec-type=2 \
    name=video \
  alsasrc num-buffers=10 ! \
  audio/x-raw-int,rate=16000,channels=2,width=16,depth=16,signed=true ! \
  queue name=audio \
  qtmux name=mux ! \
  filesink location=qtmux_test.mov \
  video.src ! mux.video_0 \
  audio.src ! mux.audio_0

If I use exactly the same pipeline with ffmux_mov it works flawlessly and the .mov is perfecly played by QT and any other software.
I want to use qtmux because the feature to recover unfinished .mov. 


My hardware has an H.264 compressor.
I have access to the sources of the GStreamer plugins to handle the compressor, but I can't change the H.264 firmware, so I have to live togheter with it.

In attached file the resoult of the input of my hadrware

gst-launch my_embedded_v4l2src num-buffers=50 ! my_embedded_264enc ! gdppay
! filesink location=input.gdp

I am able to made any test that can be helpfull in order to fix this.

Thank you
Ivan Zoli
Comment 1 Mark Nauwelaerts 2010-07-08 12:38:49 UTC
An attachment as mentioned seems to be missing.

Specifically, is it certain the output of the encoder is in avc sample style (as opposed to bytestream), and provides proper codec-data (in caps)?
Comment 2 Ivan Zoli 2010-07-08 14:42:14 UTC
Created attachment 165480 [details]
sterm out of my h264 encoder with header in caps
Comment 3 Ivan Zoli 2010-07-08 14:43:27 UTC
Yes, sorry, maybe the previous attached file didn't has it.
Anyway I push the codec-data in caps. Before doing this neither ffmux_mov is working. What I concern is that ffmux_mov is working great with the same pipeline. So I can't figure out why in the same condiction qtmux didn't works.

Also I made an .avi file wit my system then I extract the
media inside of it and made two .mov, one with
qtmux and one with ffmux_mov.
This is for have the same media content and only
the containter different.

You may find the two files here:

http://ivanonline.xoom.it/RB_06280941_ff.mov

http://ivanonline.xoom.it/RB_06280941_qt.mov

You may wget them :)

The one produced with ffmux is perfect.
The file with qtmux is not playable with QuickTime
and VLC is able to play it.

I've used this pipeline to do this job:

-------------------------------------------------
To record the .avi file

gst-launch -e my_embedded_v4lsrc ! \
 my_embedded_264enc width=720 height=400 send-h264-header=1 \
   name=video \
 alsasrc ! \
 audio/x-raw-int,rate=16000,channels=2,width=16,depth=16,signed=true ! \
 queue name=audio \
 avimux name=mux ! \
 filesink location=test.avi \
 video.src ! mux.video_0 \
 audio.src ! mux.audio_0

-------------------------------------------------
To made the mov with ffmux_mov,
i made an avi_to_ff.sh
Call avi_to_ff.sh <avifile>

#!/bin/sh

of=`ls "$1" | sed s/.avi/_ff.mov/`

echo "playing $1"
echo "Making $of"

gst-launch -v filesrc blocksize=131250 location=$1 \
 ! avidemux name=dmx \
 ffmux_mov name=mux ! filesink location="$of" \
 dmx.video_00 ! queue ! mux.video_0 \
 dmx.audio_00 ! queue ! mux.audio_0


-------------------------------------------------
To made the mov with qtmux,
i made an avi_to_qt.sh
Call avi_to_qt.sh <avifile>

#!/bin/sh

of=`ls "$1" | sed s/.avi/_qt.mov/`

echo "playing $1"
echo "Making $of"

gst-launch -v filesrc blocksize=131250 location=$1 \
 ! avidemux name=dmx \
 qtmux name=mux ! filesink location="$of" \
 dmx.video_00 ! queue ! mux.video_00 \
 dmx.audio_00 ! queue ! mux.audio_00



In attached file the input.gdp made with the pipeline
gst-launch my_embedded_v4l2src num-buffers=50 ! my_embedded_264enc send-h264-header=1 ! gdppay ! filesink location=input.gdp

Thank you

Ivan
Comment 4 Mark Nauwelaerts 2010-07-08 14:57:46 UTC
I had a quick fakesink look at the provided gdp stream:

----
/GstPipeline:pipeline0/GstGDPDepay:gdpdepay0.GstPad:src: caps = video/x-h264, height=(int)400, width=(int)720, framerate=(fraction)30/1, stream-format=(string)avc, codec_data=(buffer)000000016742001ea680b433900000000168ce3880
----

So, the codec-data is not in proper format, they are just SPS and PPS NALUs in byte-stream format, which is consistent with the stream itself, also in byte-stream:
----
00000000 (0x2590850): 00 00 00 01 67 42 00 1e a6 80 b4 33 90 00 00 00  ....gB.....3....
00000010 (0x2590860): 01 68 ce 38 80 00 00 00 01 65 88 82 07 7b 0e e2  .h.8.....e...{..
00000020 (0x2590870): 44 42 57 5e 24 00 04 59 58 2a 8c 61 26 44 d9 2b  DBW^$..YX*.a&D.+
----

So, the caps are basically lying to qtmux when claiming stream-format=avc.

It may happen to work with ffmux_mov if it has some hacks that convert bytestream to AVC sample format or whatever.
Comment 5 Ivan Zoli 2010-07-09 10:29:45 UTC
That's make sense, but if I remove the codec_data ffmux_mov stop working. 
I take a quick look at the code and ffmux_mov build the avcC atom from SPS and PPS.

from file avc.c of gst-ffmpeg packet:

int ff_isom_write_avcc(ByteIOContext *pb, const uint8_t *data, int len)
{
    if (len > 6) {
        /* check for h264 start code */
        if (AV_RB32(data) == 0x00000001 ||
            AV_RB24(data) == 0x000001) {
            uint8_t *buf=NULL, *end, *start;
            uint32_t sps_size=0, pps_size=0;
            uint8_t *sps=0, *pps=0;

            int ret = ff_avc_parse_nal_units_buf(data, &buf, &len);
            if (ret < 0)
                return ret;
            start = buf;
            end = buf + len;

            /* look for sps and pps */
            while (buf < end) {
                unsigned int size;
                uint8_t nal_type;
                size = AV_RB32(buf);
                nal_type = buf[4] & 0x1f;
                if (nal_type == 7) { /* SPS */
                    sps = buf + 4;
                    sps_size = size;
                } else if (nal_type == 8) { /* PPS */
                    pps = buf + 4;
                    pps_size = size;
                }
                buf += size + 4;
            }
            assert(sps);
            assert(pps);

            put_byte(pb, 1); /* version */
            put_byte(pb, sps[1]); /* profile */
            put_byte(pb, sps[2]); /* profile compat */
            put_byte(pb, sps[3]); /* level */
            put_byte(pb, 0xff); /* 6 bits reserved (111111) + 2 bits nal size length - 1 (11) */
            put_byte(pb, 0xe1); /* 3 bits reserved (111) + 5 bits number of sps (00001) */

            put_be16(pb, sps_size);
            put_buffer(pb, sps, sps_size);
            put_byte(pb, 1); /* number of pps */
            put_be16(pb, pps_size);
            put_buffer(pb, pps, pps_size);
            av_free(start);
        } else {
            put_buffer(pb, data, len);
        }
    }
    return 0;
}

while qtmux seems to copy the avcC atom provided in codec_data.
From gstqtmux.c line 2299

  } else if (strcmp (mimetype, "video/x-h264") == 0) {
    entry.fourcc = FOURCC_avc1;
    qtpad->is_out_of_order = TRUE;
    if (!codec_data)
      GST_WARNING_OBJECT (qtmux, "no codec_data in h264 caps");
    ext_atom = build_codec_data_extension (FOURCC_avcC, codec_data);
    if (ext_atom != NULL)
      ext_atom_list = g_list_prepend (ext_atom_list, ext_atom);

So it seems that the request for codec_data format is not the same. It is correct?
Is qtmux asking for a complete "avcC atom" on codec_data?

Sorry but I'm not a developer of video container, I have to face this, trying to fix the plugin provided by the maker of the processor. I spent 3 month of phone calls with their technical service without get any usefull information.
Then I discover that if I push out of caps in codec_data the header of h264 ffmux_mov is working, while qtmux didn't. I ask assitance on GStreamer mailing list and they ask me to open a bug here. So please let me know if I'm on wrong place.

Thank you

Ivan
Comment 6 Mark Nauwelaerts 2010-07-09 10:53:04 UTC
If the caps claim stream-format=avc, then codec-data should contain avc decoder configuration (as in spec, iirc MPEG-4 part 15, see also elements x264enc and h264parse for example) and the data buffers should *not* be in stream-format, but in avc sample format (i.e. with size prefixes) (see also spec) [see also bug #606662 for likely far more background on this than is healthy].

Of course, elements can go above and beyond this and have all sorts of verification and automagic transformation code, e.g. it appears ffmux_mov may do so, qtmux does not (which is not a bug).  Btw, ffmux_mov will have to do more than some bitpicking of codec-data; the data itself should probably also be transformed from byte-stream to avc-sample format for things to be really spec-wise (which QT player might want).

Btw, h264parse can do some/most of the above transformation from byte-stream to the other format, but at least the h264 encoder element should then not set wrong caps.  And one would expect the encoder can be tweaked to produce real stream-format=avc output (codec-data and actual data -wise).
Comment 7 Ivan Zoli 2010-07-09 11:36:49 UTC
Cool, Thank you.

Now anything it's clear for me. I've alreay tried h264parse without success, off course I miss to ask for output-format=0 (avc) properties. So qtmux didn't works. Now I use this (quicker and safer than trying to fix my encoder plugin) and anything works great!!!

Thank you very very much!

...real cool!

Ivan Zoli