GNOME Bugzilla – Bug 754504
vtdec: invalid colorimetry and generating greenish output
Last modified: 2015-09-11 18:35:47 UTC
sender raspberry pipeline: raspivid -t 0 -w 1024 -h 576 -fps 30 -b 3000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1380 ! udpsink host=192.168.0.103 port=6000 receiver iphone pipeline : udpsrc port=6000 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! vtdec ! glimagesink get greenish image: https://dl.dropboxusercontent.com/u/3808405/IMG_2155.PNG here is the log: https://dl.dropboxusercontent.com/u/3808405/gstreamer-log.txt get same result on simulator and real iPhone and iPad. and the sender is does not matter. i try with raspberry ant with rtsp link
For me, with a test H.264 file: playbin uri=rtsp://IP:8554/test -> correct colours, using vtdec rtspsrc location=rtsp://IP:8554/test ! rtph264depay ! h264parse ! vtdec ! glimagesink -> broken colours. The broken colour case looks like Cb Cr reversal to me.
In the playbin case, vtdec is outputting NV12. Attaching pipeline dumps.
Created attachment 310826 [details] vtdec outputting GL
Created attachment 310827 [details] vtdec outputting NV12
I'm currently on a prolonged vacation, but I hope to look into this next week. One thing to try is to test with a non-GL sink such as the avsamplevideosink, to determine whether it's the texture cache support that's misbehaving.
If it outputs NV12, it works with GLimagesink - it's only the RGBA/texture cache mode that's broken. I can't see any recent changes there though (not since January), so either it's been broken a while and nobody noticed because it tends to negotiate non-GL output, or something changed in iOS.
Created attachment 310944 [details] [review] Prefer openGL output if available Here's a patch to break things properly, by preferring opengl output if downstream supports it - avoids things sometimes working because it accidentally negotiates to NV12 even though downstream could take a GL texture directly.
(About being broken for a while, as a blanket statement, the 'applemedia' stuff doesn't get tested as much as most GStreamer code.)
Comment on attachment 310944 [details] [review] Prefer openGL output if available Looks good and makes things at least consistent and correct in the negotiation. We should get this in... and if we can't get the texture cache to work, just disable it from the pad template caps.
By the way, trying to use avsamplebufferlayersink crashes on iOS in gst_av_sample_video_sink_get_caps() inside CVPixelFormatDescriptionArrayCreateWithAllPixelFormatTypes() at: formats = CVPixelFormatDescriptionArrayCreateWithAllPixelFormatTypes (kCFAllocatorDefault); I don't see any obvious reason why that should crash, unless there's some CoreVideo init call that's missing.
Created attachment 311171 [details] [review] Fix colour rendering for vtdec with GL output