GNOME Bugzilla – Bug 725536
encoder: add h264 scalable encoder
Last modified: 2018-11-03 15:45:01 UTC
Created attachment 270746 [details] [review] add svc profile. h264 have many differences compare to pure h264, such as bit rate control, frame type assignment. So I created total new profile for svc. Init commit for h264 scalable.Only temporal supported.
Created attachment 270747 [details] [review] add reference flag for b frame reference svc need B frame as reference. So I add a flag for reference frame.
Created attachment 270748 [details] [review] add h264 svc encoder. 1.I do not want pollute h264 encoder, so I created a standalone svc encoder. 2. Many functions are copy/past from h264 encoder,just rename them to svc. It's easy to refactor them after we add quality and spatial.
Created attachment 270749 [details] [review] add h264 svc encode copy/paste from h264 encode too. need refactoring it after quality and spatial added.
I am going to obsolete the proposed patches for svc. Libva is not exposing any svc specific headers in master branch, we can have temporal-svc with out any new apis. Here is my development branch: https://cgit.freedesktop.org/~sree/gstreamer-vaapi/log/?h=svc @Guangxin: I kept few of your ideas in my branch with you as Author :)
Review of attachment 270746 [details] [review]: obsolete..
Review of attachment 270747 [details] [review]: Obsolete.
Review of attachment 270748 [details] [review]: Obsolete.
Review of attachment 270749 [details] [review]: Obsolete
The full-blown svc implementation is tricky, and requires many driver changes too. But we can have a restricted temporal svct-t with out disturbing the driver. At the same time, since the driver can't support svc specification as it is right now, it won't be exposing SVC profile capability. So the plan is to use the temporal-encoding inside avc. This will give many advantages: 1: Currently we only have prediction from immediate previous/future frames, the new feature will give more tuning options for the user 2: Encoded stream will be decodeable with any legacy h264 decoder since we are not exposing SVC profile 3: If needed, clients (decoder, broadcasters) can still select the required temporal levels using Scalability SEI headers. 4: Later we can easily make it as svc stream with out changing the already implemented code by inserting few packed headers + setting profile as svc. ====== If we have to advertise the encoded stream as H264-SVC ====== There is no profile called SVC-T, so we need to expose one of the profile mentioned in spec (svc-constrained-baseline,svc-high-profile etc for eg). Driver is not exposing the svc capability through svc profile. So we by-pass driver capability checking in gstreamer-vaapi, and expose the encoded stream profile as SVC, also insert PREFIX_NAL units which can be dropped by legacy h264 decoders. But keep in mind, the stream won't be decodeable with current vaapidecode or many other existing avc decoders unless it is exposing svc capability through svc-prfoile. Of course We can make them decode the base layer which should still expose avc profile (#732266). This (making stream as svc) can be done at any time with out changing hierarchical-p/b encode code block. ===== Current plan ================= -- Add hierarchical-p frame prediction model in AVC encode eg: with 3 temporal layers T3: P1 P3 P5 P7 T2: P2 P6 T1: P0 P4 P8 T1, T2, T3: Temporal Layers P1...pn: P-Frames: Frames in each Tx will reference only pictures in the lower or same layer, so depends on bandwidth clients can drop the upper layers. P0->P1 , P0->P2, P2->P3, P0->P4......repeat -- Add hierarchical-b frame prediction model in AVC encode http://www.hhi.fraunhofer.de/departments/video-coding-analytics/research-groups/image-video-coding/research-topics/svc-extension-of-h264avc/hierarchical-prediction-structures.html These are the two well known reference modeling used in industry. Both will be normal avc streams , but encoded as different temporal layers. And we will pack SEI headers too for clients to deal with temporal levels if needed, which means ability to drop higher temporal levels without affecting the decodability of lower levels. vaapih264enc will have two new properties: prediction-type : Reference Picture Selection Types flags: readable, writable Enum "GstVaapiEncoderH264PredictionType" Default: 0, "default" (0): default - Default prev/next frame as ref frames (1): hierarchical-p - Hierarchical P frame encode (2): hierarchical-b - Hierarchical B frame encode temporal-level : Number of temporal levels included in the encoded stream flags: readable, writable Unsigned Integer. Range: 1 - 4 Default: 1
(In reply to sreerenj from comment #4) > I am going to obsolete the proposed patches for svc. Libva is not exposing > any svc specific headers in master branch, we can have temporal-svc with out > any new apis. Here is my development branch: > https://cgit.freedesktop.org/~sree/gstreamer-vaapi/log/?h=svc > > @Guangxin: I kept few of your ideas in my branch with you as Author :) Hi @Sreej: Welcome, use it as you wish ;)
SVC basic implementation is here: https://cgit.freedesktop.org/~sree/gstreamer-vaapi/log/?h=svc-enc We can have bitrate control separately for each temporal layes based on the upcomming VA API changes , required patches for both libva and vaapi-intel-driver are already in the mailing list: https://lists.freedesktop.org/archives/libva/2016-September/thread.html
Hopefully, we will have it the driver portion merged this week. Sean
This problem has been fixed in our software repository. The fix will go into the next software release. Once that release is available, you may want to check for a software upgrade provided by your Linux distribution. This was fixed by Hyunjun
(In reply to Víctor Manuel Jáquez Leal from comment #13) > This problem has been fixed in our software repository. The fix will go into > the next software release. Once that release is available, you may want to > check for a software upgrade provided by your Linux distribution. > > This was fixed by Hyunjun Not yet :), The patches are only adding Hierarchical-p/b encode not SVC. See this commit log: https://cgit.freedesktop.org/~sree/gstreamer-vaapi/commit/?h=svc-enc&id=b706ae2c6f873493420e7b16b50914505aacb0b7 And the patches https://cgit.freedesktop.org/~sree/gstreamer-vaapi/log/?h=svc-enc
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/14.