GNOME Bugzilla – Bug 719952
encoder: mpeg2: fix coded buffer size
Last modified: 2013-12-06 09:08:15 UTC
VA coded buffers are GPU allocated memory and we should avoid possible buffer overflows that would hang the system. The coded buffer limits set in base GstVaapiEncoder are only valid for H.264, and the macroblock layer in particular. For reference, Table 8-6 in the MPEG-2 specification defines the maximum number of bits in a macroblock. We also need to compensate for extra packed that are to be injected in the output coded buffer.
Since only YUV 4:2:0 formats are supported for now, resolution is to use the 4608 bits limit for a macroblock and account for Sequence Header, Sequence Extension, Group of Pictures, Picture Header, and Picture Extension headers. Besides, since we use a slice per line of macroblock, we also need to account for each extra slice header size.
Fixed in git master branch actually. Bug opened to keep track of the resolution. It's incomplete but enough for the current usages and implementation. This means that, if quantization matrices are to be supported, then the VA coded buffer size limit needs to be reviewed and updated. Likewise for any additional header we did not handle yet.
commit 2940a74ea45956edb8ec3e94bb983e255fcf0dc1 Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com> Date: Thu Dec 5 18:13:54 2013 +0100 encoder: fix computation of max coded buffer size (again). The previous fix was only valid to express the maximum size of the macroblock layer, i.e. without any headers. Now, also account for the slice headers and top picture header, but also any other header we might stuff into the VA coded buffer, e.g. sequence headers. commit b864d1f71acc47bd41b8e7171c4ad4d6628849f4 Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com> Date: Wed Dec 4 19:10:13 2013 +0100 encoder: fix computation of max coded buffer size. Fix coded buffer size for each codec. A generic issue was that the number of macroblocks was incorrectly computed. The second issue was specific to MPEG-2 were the max number of bits per macroblock, and as defined by the standard, was incorrectly mapped to the (lower) H.264 requirement. i.e. 4608 bits vs. 3200 bits limit.