After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 738681 - qtdemux: does not detect orientation
qtdemux: does not detect orientation
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
1.4.3
Other Linux
: Normal normal
: 1.5.90
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-10-17 11:22 UTC by Jussi Pakkanen
Modified: 2015-08-16 13:39 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
qtdemux: didn't check x, y range (2.14 KB, patch)
2015-06-23 11:46 UTC, Gilbok Lee
committed Details | Review

Description Jussi Pakkanen 2014-10-17 11:22:13 UTC
Attached is a video file that has orientation info but gstreamer's qtdemux does not detect it properly. Running mediainfo on the file says this:

Rotation   : 90°

but if you run gst-play-1.0 with debug level 3, GStreamer says the following:

qtdemux qtdemux.c:7212:qtdemux_inspect_transformation_matrix:<qtdemux0> Unhandled transformation matrix values

Because of this you can't use the videoflip element with automatic reorientation.
Comment 1 Jussi Pakkanen 2014-10-17 11:24:16 UTC
The video was just too big to attach so here's a link instead:

https://dl.dropboxusercontent.com/u/37517477/orientationbug.mp4
Comment 2 Thiago Sousa Santos 2014-10-20 09:04:39 UTC
Do you know what was used to encode/mux this file? The transformation matrix isn't exactly correct and is being rejected.

x' = 0 * x + y + 0
y' = -1 * x + 0 * y + 0

For the corner input points (w is width, h is height):

0, 0 : x' = 0, y' = 0
w, 0 : x' = w, y' = -w
0, h : x' = h, y' = 0
w, h : x' = h, y' = -w

A rotation is happening but you can see that y is now ranging from -w to 0. The correct way would be to have the matrix translate this into the 0-w range. qtdemux can be fixed to handle this kind of matrix but it would be better to fix the tool that generated this.
Comment 3 Sebastian Dröge (slomo) 2014-10-21 08:54:18 UTC
Isn't that just rotation plus vertical flipping? Also see bug #738914
Comment 4 Jussi Pakkanen 2014-10-21 14:33:53 UTC
The video was recorded with Ubuntu phone. More information can be found on the original bug here:

https://bugs.launchpad.net/thumbnailer/+bug/1191934
Comment 5 Gilbok Lee 2015-06-23 11:46:32 UTC
Created attachment 305912 [details] [review]
qtdemux: didn't check x, y range

In tkhd box, contents has transformation matrix.

[a b c]
[d e f]
[g h i]

[g, h] matrix value that the video was recorded with android cell phone(galaxy s4, s5, s6 / G2, etc..) is zero.

How didn't check [g, h] value for contents compatibility?
Comment 6 Thiago Sousa Santos 2015-06-24 00:40:01 UTC
The matrix in the file in ubuntu's launchpad is actually doing a rotate-90 + vertical flip as Sebastian pointed out. The vertical axis now goes from 0 to -width, meaning it goes into the opposite direction. This is supposed to indicate it should be flipped.

Every player I tried seems to ignore that (vlc, quicktime, ios quicktime). So either we are not interpreting the spec correctly or this seems to be ignored everywhere. Ignoring it in gstreamer would mean that it would be impossible to fully support affine transforms as they were supposed to be.

Even implementing the flip would be weird as we would be showing the images in the correct dimensions but flipped, users wouldn't be happy I guess.

Are there samples out there generated without ubuntu phone that have this same behavior? I generated some with an ipad and they seemed to be correctly indicating only the rotation.
Comment 7 Thiago Sousa Santos 2015-06-24 01:00:14 UTC
Just tried with an android phone (lolipop) and it also has the same issue as the ubuntu phone.
Comment 8 Thiago Sousa Santos 2015-06-25 03:46:49 UTC
It was decided over IRC to handle those cases to be able to play those videos properly.

Please, for future patches consider including a commit message other than the title in the commit. I wrote one for this commit myself.

commit 0dcd76447a5057541a69918d74a2c2d4e40f1069
Author: Gilbok Lee <gilbok.lee@samsung.com>
Date:   Tue Jun 23 20:15:13 2015 +0900

    qtdemux: does not detect orientation
    
    Most files don't contain the values for transposing the coordinates
    back to the positive quadrant so qtdemux was ignoring the rotation
    tag. To be able to properly handle those files qtdemux will also ignore
    the transposing values to only detect the rotation using the values
    abde from the transformation matrix:
    
    [a b c]
    [d e f]
    [g h i]
    
    https://bugzilla.gnome.org/show_bug.cgi?id=738681