*** Bug 785405 has been marked as a duplicate of this bug. ***

As said in the other post, this can be easily solved, by only changing (h264parse.c) these lines:

      case GST_H264_PARSER_NO_NAL_END:
        GST_DEBUG_OBJECT (h264parse, "not a complete nal found at offset %u",
            nalu.offset);
        /* if draining, accept it as complete nal */
        if (drain) {
          nonext = TRUE;


with:

      case GST_H264_PARSER_NO_NAL_END:
        GST_DEBUG_OBJECT (h264parse, "not a complete nal found at offset %u",
            nalu.offset);
        /* if draining, accept it as complete nal */
        if (drain || aux_flag) {
          nonext = TRUE;


...where "aux_flag" is the result of some property which tells the parser that the stream is au-aligned, then a complete frame is already received.
However, before sending the patch, it's important to decide if the property must be included only into h264parse module, or in all the modules that can be output frames to h264parse too (i.e: tsdemux). This is a conceptual problem.

Let's clarify.

tsdemux has src caps for h264 with nal-alignment only. Then, I see two choices:

1) add an au-alignment src cap for tsdemux and check this alignment in h264parse before deciding to drain the data or to buffer it.

2) add a "framed-source" property to h264parse, and check it before deciding to drain the data or to buffer it.

The first choice means that multiple files have to be patched, with the result that tsdemux is coupled to h264 codec (which is bad, IMHO).

The second choice simply tells to the parser that we are parsing a complete frame, for each cycle; and it requires to patch only h264parse.c

I can't make any sense of this, please attach a proper patch (see attachements). My view on this is that we should:

1. Modify gst_h264_parse_handle_frame_packetized() to work with byte-stream too
2. Set packetized and split_packetize for byte-stream too
3. Drop the 1 fps latency when h264parse->in_align == GST_H264_PARSE_ALIGN_AU

Properties are just a bad idea, they won't work automatically.

As said in IRC-freenode, tsdemux doesn't have au-alignment caps even if it produces an au-aligned stream. Then, even if I "Drop the 1 fps latency when h264parse->in_align == GST_H264_PARSE_ALIGN_AU", this won't work for this case.

The only workaround for that, IMHO, is a property on h264parse that _reinterpret_ the input alignment. 
Something like "reinterpret-input-alignment" [NAL/AU]. And THEN the the check Nicolas said (h264parse->in_align == GST_H264_PARSE_ALIGN_AU)

It doesn't seem so bad to me, but it's a conceptual problem and before sending the patch I need a feedback on that.

(In reply to Paolo Pr from comment #4)
> As said in IRC-freenode, tsdemux doesn't have au-alignment caps even if it
> produces an au-aligned stream.

I'm not sure why you think that tsdemux can frame a whole access unit? Even using the data_alignment flag and data_stream_alignment type of access unit, all we can know is that the first byte of the PES marks the start of the a AU. We can't know where the NAL or AU ends without parsing the stream (which is what h264parse does).

Also, tsdemux currently doesn't handle any of those alignments markers.

So I think it's largely a conceptual problem with tsdemux as it requires parsing the ES stream to do the actual framing.

I also wonder if we should add a alignment=none to the video/x-h26[45] caps in tsdemux to reflect that.

(In reply to Olivier Crête from comment #5)
> (In reply to Paolo Pr from comment #4)
> > As said in IRC-freenode, tsdemux doesn't have au-alignment caps even if it
> > produces an au-aligned stream.
> 
> I'm not sure why you think that tsdemux can frame a whole access unit? Even
> using the data_alignment flag and data_stream_alignment type of access unit,
> all we can know is that the first byte of the PES marks the start of the a
> AU. We can't know where the NAL or AU ends without parsing the stream (which
> is what h264parse does).
> 
> Also, tsdemux currently doesn't handle any of those alignments markers.
> 
> So I think it's largely a conceptual problem with tsdemux as it requires
> parsing the ES stream to do the actual framing.
> 
> I also wonder if we should add a alignment=none to the video/x-h26[45] caps
> in tsdemux to reflect that.

I clarify what I meant. It's not tsdemux alone which frames a whole access unit. It's a tsdemux with other elements at its left, in the pipeline. For example, a souphttpsrc which receives one complete frame per time and pipes it to tsdemux. So, in this case, the output of tsdemux is a complete au per time (that is what I meant when I said "it produces an au-aligned stream"). I agree that it's a conceptual problem, and I agree too that video/x-h264[5] of tsdemux should not have a precise alignment cap for its output.

(In reply to Paolo Pr from comment #6)
> It's not tsdemux alone which frames a whole access
> unit. It's a tsdemux with other elements at its left, in the pipeline. For
> example, a souphttpsrc which receives one complete frame per time and pipes
> it to tsdemux.

That sounds like a very specific use-case, in that case you may want to just override the caps with "capsetter join=1 caps="video/x-h264, alignment=nal" between tsdemux and h264parse.

Then, I think that a complete solution for this issue is:

1) remove alignment=nal from tsdemux (IMHO this has the good side effect that the demuxer will be much less coupled with h264)

2) set nal alignment as default for bytestream, in h264parse

3) add an "alignment" property/filter in h264parse (so that it has nal as default when it's connected to tsdemux, but it can be set to au as well)

4) patch h264 according to what Nicolas said


This is better, IMHO, than overriding the caps with alignment=nal.

-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/578.