After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 768639 - Unrecognized jpg file
Unrecognized jpg file
Status: RESOLVED FIXED
Product: frogr
Classification: Other
Component: general
1.0
Other Linux
: Normal blocker
: ---
Assigned To: frogr maintainers
frogr maintainers
: 678241 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2016-07-10 16:17 UTC by Mathieu
Modified: 2020-12-01 10:34 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Mathieu 2016-07-10 16:17:00 UTC
Hi,

I just got a new phone (Moto X Play) and every time I try to upload pictures taken with this phone I get the following error message:

Unable to load picture xxx.jpg:
Error interpreting JPEG image file (Unsupported marker type 0x06)

The last hex code is never the same.
These are clearly jpg files though, every other software opens them without any problems. I even ran a "file" command on them, and it returns JPEG.

The files are huge so I uploaded one here if you want to try:
https://dl.dropboxusercontent.com/u/488552/test_frogr.jpg

Please let me know as this is quite a showstopper.

Thanks,

Mat
Comment 1 Mario Sánchez Prada 2016-07-12 12:48:35 UTC
Thanks for the report and for the test image, I confirm I can reproduce it locally on my Fedora 23 machine.

I took a quick look to this and it seems that the error could be coming from the libjpeg library (mine is libjpeg-turbo-1.4.1-2.fc23.x86_64) that GdkPixbuf uses to load the data from the picture at hand, since that's where the error is generated and GdkPixbuf & Frogr simply deal with the error that come propagater, all the way from there.

I'll try to look into why libjpeg is returning that error and see if there's something we can do here, although I suspect this looks like a bug in the libjpeg library itself.
Comment 2 Mathieu 2016-07-12 16:30:31 UTC
I am glad to hear you can reproduce the bug. Hopefully this will help fixing it faster.

I am running Ubuntu 16.04 x64 and my version of libjpeg is 1.4.2-0ubuntu3.

If you need anything else from me, please let me know.

Thanks,

Mat
Comment 3 Mathieu 2016-07-17 15:34:05 UTC
I've been looking a little more into this problem and tried a few things.

All the following commands allow me to convert my pictures into files which Frogr can read, but I am pretty sure they are lossy transformations:
convert test.jpg test01.jpg
mogrify -format jpg test02.jpg
jpegoptim test03.jpg

This command is supposed to be lossless, but strips out all the EXIF:
jpegtran -perfect test.jpg > test04.jpg

When I add the "-copy all" option, the EXIF data remains, but Frogr does not open the file:
jpegtran -perfect -copy all test.jpg > test05.jpg

I've also searched in Google for other people reporting problems with this phone's camera but could not find anything.

I hope you find a solution on your side.

Thanks,

Mat
Comment 4 Mario Sánchez Prada 2016-07-18 23:43:46 UTC
Thanks, I'll try to find a solution to this soon, but I'm pretty busy at the moment with other projects and I'm afraid this will have to wait a bit unless someone else comes up with a patch.

This said, I'll might have some time on Saturday, so I'll try to look at it then.
Comment 5 Mathieu 2016-08-20 12:45:05 UTC
I just got back from vacation (with lots of pictures) and was wondering if you had been able to find a fix for this bug.

Thanks,

Mat
Comment 6 Mario Sánchez Prada 2016-08-20 16:26:10 UTC
No, I haven't had time yet, sorry. I'm on holidays right now so I might have some time I can devote if I can sneak some time out for this, but can't promise anything on that, unfortunately.

Re-reading the comments here, however, I still think this is an issue in the libjpeg library, so perhaps you might want to report the problem in the relevant  in the meantime? I actually think that's the right thing to do but I don't have time to do that either right now, sorry. :(
Comment 7 Mathieu 2016-09-02 10:03:28 UTC
Hi,

Here is some more information I was able to obtain.
I wasn't sure how to contact the libjpeg team so I sent an email to the address I found here: https://lists.sourceforge.net/lists/listinfo/libjpeg-devel-6x

The first person who responded said this:
"I am not a JPEG developer, but I did take one of my windows applications, and opened that large (9 meg) image without any problem.  I am using a build from jpeg9-a.
I also loaded a simple GTK test program, and it displayed the image, although much larger than my monitor could handle.
I would suspect that the way GTK developers coded it to handle a large image is the problem.  The image is 48meg in RAM when uncompressed."

This made sense, so I then took 2 very simple pictures with the front and back cameras (links: https://dl.dropboxusercontent.com/u/488552/front.jpg & https://dl.dropboxusercontent.com/u/488552/back.jpg) which were only 1 and 5 MB, but the problem remained: "Error interpreting JPEG image file (Unsupported marker type 0x20)".

To which a second person replied:
"No, that sounds more like a corrupted file.  0x20 is not a legal JPEG marker byte.
(The fact that you can read it with some other library does not alter the conclusion that it's not a valid JPEG file. Some code is pretty forgiving about ignoring garbage; libjpeg is less so.)"

But the hex code is always different and so far I've seen at least: 0x49 / 0x1d / 0x18 / 0x3a / 0xbb / 0x53 / 0x69 / 0x61 / 0x63 / 0x40 / 0x43 / 0x7d / 0x6d ...

And that's the last I heard from them.

I have tried opening one of these pictures with the following programs and all of them were able to open it without any problems:
firefox / chrome / vivaldi / opera / midori / gthumb / gimp / imagemagick / gnome image viewer / shotwell / shutter / hugin / libreoffice / mpv / smplayer / vlc / openshot
so basically every other software I have can open it, but not frogr
the only thing is I don't know which of these use libjpeg
do you know any other software that uses libjpeg for sure to test if it comes from there?

Thanks,
Mat
Comment 8 Mario Sánchez Prada 2016-09-06 17:16:19 UTC
Thanks for the thorough investigation. It's very weird indeed that some other apps can open it but not frogr, as we're not doing anything special other than using the GdkPixbufLoader class to get the data.

Perhaps those other programs use other means to load the data? In any case, if the generated JPEG file is corrupted, I'm not sure whether there's going to be an easy fix for this, at least in the short term, unlese there's a simple patch we can use at the app's level to work it around (which I would be happy to integrate if that's the case).
Comment 9 karlch 2017-07-13 14:29:20 UTC
I have recently run into a similar problem. To me it looks like GdkPixbufLoader fails with some image files that work fine otherwise. It is for example possible to load them via pixbuf_new_from_file(...).

I temporarily worked around this by catching the error raised by the loader and falling back to pixbuf_new_from_file(...) in that case. It would be much nicer to consistently work with the loader though, obviously.

Not sure if this is the correct thread to continue this, maybe we should file another bug report directly concerning the loader.

Link to the issue opened at my project:
https://github.com/karlch/vimiv/issues/49
Comment 10 Mathieu 2017-07-17 04:55:27 UTC
I'm glad to see I'm not the only one with this problem! Hopefully this can get fixed.
Comment 11 Mario Sánchez Prada 2017-07-18 21:34:20 UTC
@karlch: thanks for the feedback. I actually got hit myself by this recently when trying to upload pictures from my wife's phone (a Motorola Moto G3), so this is definitely an issue.

My only problem is lack of time to work on this, but this new information you just posted might be exactly what I needed to unstuck this on a timely manner.

No promises on the time frame, though, but I will definitely look into it unless someone else beats me to it.
Comment 12 Ian 2017-10-14 20:48:31 UTC
I have a situation that's the inverse of this - JPEGs produced by GIMP being unrecognizable by some other applications. Although not all of my photos are taken with Nikon cameras, I mostly use Nikon's free ViewNX2 application to browse albums. I have found that, when I use GIMP to modify a photo and export it as a JPEG, ViewNX2 is able to display the thumbnail, but cannot display the full photo, which it reports as "Unsupported file type." It's also not able to display the EXIF data, even though I am able to show that the data is still present in the GIMP JPEG. Similarly, I use "Photo Vacuum Packer" (PVP) to reduce the extraneous space in JPEG files, by up to two-thirds; PVP simply doesn't recognize JPEG files created by GIMP, and won't compress them.
Comment 13 Mario Sánchez Prada 2017-12-26 20:24:01 UTC
I'm so sorry for the sloooow latency on this issue, but I have a heads-up on it:

I've spent some time today (on a free slot during holidays) debugging this and I'm quite sure this has to be a problem in gdk-pixbuf since it's quite clear that the problem happens when using gdk_pixbuf_loader_write(), which relies on the internal gdk_pixbuf__jpeg_image_load_increment() function to load the data... well... incrementally, while the problem doesn't happen when using gdk_pixbuf_new_from_file(), which relies on the internal gdk_pixbuf__jpeg_image_load() function instead (which is much simpler).

Since my knowledge of that code is basically NULL and I've proved myself that I could not fix it myself (at least not on short notice), I've just pushed a workaround for frogr that should "solve" the problem here while gdk-pixbuf is not fixed, and I will file a bug for gdk-pixbuf with the details learned from this bug.

So, this is fixed in master, and will be ready in the next release of frogr, which I hope will be out later this week (maybe today, even): https://git.gnome.org/browse/frogr/commit/?id=4e48c5e666205c1793abe6bf4294f03933ebe10a

Again, sorry for the absurd delay, thanks for the patience and even many more thanks for the comments in this bug that helped me a lot to debug it and find a workaround.
Comment 14 Mario Sánchez Prada 2017-12-26 20:29:08 UTC
*** Bug 678241 has been marked as a duplicate of this bug. ***
Comment 15 Mario Sánchez Prada 2017-12-27 02:22:54 UTC
(In reply to Mario Sánchez Prada from comment #13)
> [...]
> Since my knowledge of that code is basically NULL and I've proved myself
> that I could not fix it myself (at least not on short notice), I've just
> pushed a workaround for frogr that should "solve" the problem here while
> gdk-pixbuf is not fixed, and I will file a bug for gdk-pixbuf with the
> details learned from this bug.

Bug filed for gdk-pixbuf: https://bugzilla.gnome.org/show_bug.cgi?id=791971