GNOME Bugzilla – Bug 738681
qtdemux: does not detect orientation
Last modified: 2015-08-16 13:39:38 UTC
Attached is a video file that has orientation info but gstreamer's qtdemux does not detect it properly. Running mediainfo on the file says this: Rotation : 90° but if you run gst-play-1.0 with debug level 3, GStreamer says the following: qtdemux qtdemux.c:7212:qtdemux_inspect_transformation_matrix:<qtdemux0> Unhandled transformation matrix values Because of this you can't use the videoflip element with automatic reorientation.
The video was just too big to attach so here's a link instead: https://dl.dropboxusercontent.com/u/37517477/orientationbug.mp4
Do you know what was used to encode/mux this file? The transformation matrix isn't exactly correct and is being rejected. x' = 0 * x + y + 0 y' = -1 * x + 0 * y + 0 For the corner input points (w is width, h is height): 0, 0 : x' = 0, y' = 0 w, 0 : x' = w, y' = -w 0, h : x' = h, y' = 0 w, h : x' = h, y' = -w A rotation is happening but you can see that y is now ranging from -w to 0. The correct way would be to have the matrix translate this into the 0-w range. qtdemux can be fixed to handle this kind of matrix but it would be better to fix the tool that generated this.
Isn't that just rotation plus vertical flipping? Also see bug #738914
The video was recorded with Ubuntu phone. More information can be found on the original bug here: https://bugs.launchpad.net/thumbnailer/+bug/1191934
Created attachment 305912 [details] [review] qtdemux: didn't check x, y range In tkhd box, contents has transformation matrix. [a b c] [d e f] [g h i] [g, h] matrix value that the video was recorded with android cell phone(galaxy s4, s5, s6 / G2, etc..) is zero. How didn't check [g, h] value for contents compatibility?
The matrix in the file in ubuntu's launchpad is actually doing a rotate-90 + vertical flip as Sebastian pointed out. The vertical axis now goes from 0 to -width, meaning it goes into the opposite direction. This is supposed to indicate it should be flipped. Every player I tried seems to ignore that (vlc, quicktime, ios quicktime). So either we are not interpreting the spec correctly or this seems to be ignored everywhere. Ignoring it in gstreamer would mean that it would be impossible to fully support affine transforms as they were supposed to be. Even implementing the flip would be weird as we would be showing the images in the correct dimensions but flipped, users wouldn't be happy I guess. Are there samples out there generated without ubuntu phone that have this same behavior? I generated some with an ipad and they seemed to be correctly indicating only the rotation.
Just tried with an android phone (lolipop) and it also has the same issue as the ubuntu phone.
It was decided over IRC to handle those cases to be able to play those videos properly. Please, for future patches consider including a commit message other than the title in the commit. I wrote one for this commit myself. commit 0dcd76447a5057541a69918d74a2c2d4e40f1069 Author: Gilbok Lee <gilbok.lee@samsung.com> Date: Tue Jun 23 20:15:13 2015 +0900 qtdemux: does not detect orientation Most files don't contain the values for transposing the coordinates back to the positive quadrant so qtdemux was ignoring the rotation tag. To be able to properly handle those files qtdemux will also ignore the transposing values to only detect the rotation using the values abde from the transformation matrix: [a b c] [d e f] [g h i] https://bugzilla.gnome.org/show_bug.cgi?id=738681