Of course I meant mpegtsmux from gst-plugins-bad... flutsmux didn't support mpeg4 video

I think I have the same issue although it's somewhat generalized to all codecs in my case. I was able to track down the problem to the PID / PID type the muxer sets for the data streams.

If you take a look at gst_fluts_demux_parse_stream, there is a switch case for the PID type (which I believe is derived from the PID). The audio/video streams are expected to be "PID_TYPE_ELEMENTARY", and I saw that an output TS with H.264 and AAC has PID types 1 and 3 for PIDs 0x0001 and 0x0000 respectively. Therefore the demux sees those streams as PID_TYPE_CONDITIONAL_ACCESS and PID_TYPE_PRIVATE_SECTION, and simply drops them.

I'm not sure why FFMPEG demuxes the streams, maybe it doesn't rely on PID for content type.

Investigating the mux to see where these values come from...

Oups, that was an operation error. In my pipeline I specified the mux pad names (sink_0 and sink_1) and just figured these are use directly as PID in mpegtsmux_request_new_pad... Retested without pad names and it works perfectly.

Tested successfully H.264, MPEG-4, MPEG-2 video, AAC and MP3 audio, so I couldn't reproduce original problem sorry.

I can still reproduce it here as mentioned above.

When playing it with gstreamer I get millions of warnings from gst-ffmpeg:

0:00:10.671266760 12574  0x90aaaa0 ERROR               ffmpeg :0:: Error at MB: 247
0:00:10.703037216 12574  0x90aaaa0 ERROR               ffmpeg :0:: header damaged
0:00:10.703629872 12574  0x90aaaa0 ERROR               ffmpeg :0:: ac-tex damaged at 16 11

I suppose you're using the latest gst-ffmpeg ... For my specific usecase I wrote a couple of patches for the mpegtsdemux (pushing complete payloads with adjusted timestamps), and my gst-ffmpeg also has a couple timestamp hacks, but nothing which actually changes the bitstream. Nevertheless it could be one of those changes which "fixes" the problem ;-)

Can you attach a sample so I could check?

This issue is still around. You can run the pipeline that Sebastian wrote (gst-launch-0.10 videotestsrc num-buffers=1000 ! ffenc_mpeg4 !
"video/mpeg,mpegversion=4" ! mpegtsmux ! filesink location=test.ts) to generate a sample.

It works with ffdemux_mpegts and does not with mpegtsdemux.

Adding a mpeg4videoparse after mpegtsdemux makes it work. (Edward noticed this).

From IRC:

bilboed-pi: I don't know exactly which way it should be fixed though
bilboed-pi: we could either go the mp3 way... and force parsed=True on the mpeg4 decoder input caps
bilboed-pi: (so that when autoplugging it automatically adds the mpeg4videoparse)
bilboed-pi: but that would require having mpeg4videoparse in -ugly
bilboed-pi: the alternative is... doing the parsing in the demuxer

Just checked, we deactivated the parser for mpeg4 in gstffmpegdec.

But even if I reactivate it, it still fails as-is... but if one adds the width/height to the incoming caps it works fine.

Video properties such as dimensions or framerate are out of the scope of the demuxer. FFMPEG's demuxes cheat in that regard by parsing the streams. The decoder should be capable of handling a raw stream without parameters. If it can't, maybe the encoding/muxing did not provide the required codec data.

For instance, H.264 has parameter sets on IDR frames which must be kept intact when muxed in a Transport Stream. The TS format doesn't specify a way to contain codec data so it must be embedded in the bitstream.

I'm not sure what's the specification for codec data in MPEG-4 but it's probably missing in this case.

If ffdec_mpeg4 doesn't parse the stream and the demux doesn't push complete payloads there's no way it can work. Furthermore, the current implementation of the demux which pushes data as soon as it's available is not very good. This creates a lot of traffic of GstBuffers of <= 182 bytes and doesn't allow basic error correction by sequence number validation. Since most of FFMPEG's decoders are not robust to data corruption there's higher chances of crash.

One way to improve this is to accumulate data on a given PID until a new PUSI flag is found and then push a complete PES payload in a single GstBuffer. If there is a gap in sequence numbers, the whole payload should be dropped. It's way more efficient this way and each outgoing GstBuffer gets a valid timestamp.

Just tested that this bug is still here with current git versions of GStreamer.

It seem to be working properly with latest git and tsdemux (not mpegtsdemux which should be deprecated in the near future).

Works fine with tsdemux.