GNOME Bugzilla – Bug 749088
vtenc: VTCompressionSessionEncodeFrame failing with vt_status == noErr -> memory leak
Last modified: 2015-05-09 20:00:46 UTC
vtenc currently assumes that either: 1. gst_vtenc_encode_frame is called, or 2. VTCompressionSessionEncodeFrame returns failure code In fact, in some cases it's neither -- for example, when this cryptic error is print to the console: vtCompressionSessionTrackFrameExitedEncoder signalled err=-12902 (kVTParameterErr) (Unrecognised VTVideoEncoderFrame token passed in by encoder) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.235/Sources/VideoToolbox/VTCompressionSession.c line 2222 <<<< GVAVideoEncoder >>>> GVAVideoEncoder_EmitEncodedFrame: AVF encoder error: VT, GVAVideoEncoder_EmitEncodedFrame: outputFrameInfoPtr ==NULL While we certainly shouldn't deal with cryptic errors, we also shouldn't make things worse. Right now, in such cases we don't release frames at all, and memory keeps growing indefinitely. The solution is to release the data regardless. We must invoke gst_video_encoder_finish_frame, since it's the only way to make GstVideoEncoder release its reference to the frame.
Created attachment 303053 [details] [review] vtenc: ensure frame is always released This is a preliminary patch since it doesn't handle: * directly using CVPixelBuffer meta * creating CVPixelBuffer on iOS In fact, the approach used in the patch will have to be modified. Perhaps the only way is to attach a CF object with CVBufferSetAttachment to the pixbuf.
I'm having second thoughts. There's no good way to do it. VT guarantees the "outputCallback" will be called. If it would not, then if you pass a "sourceFrameRefCon", you don't ever know when you can free it. The pixbuf being released is no reason to think the "outputCallback" won't be called. Therefore, there's no nice way to do it.