GNOME Bugzilla – Bug 723252
testsuite failure: libs/tag - exif tag: "Conversion from character set 'utf8' to 'latin1' is not supported"
Last modified: 2015-04-03 20:08:18 UTC
I've been having some trouble with gstreamer 1.2.2 and ran the gst-plugins-base test-suite. One of the tests failed: FAIL: libs/tag ============== Running suite(s): tag support library 85%: Checks: 14, Failures: 2, Errors: 0 libs/tag.c:1354:F:general:test_exif_tags_serialization_deserialization:0: Assertion 'gst_tag_list_is_equal (taglist, taglist2)' failed libs/tag.c:1354:F:general:test_exif_multiple_tags:0: Assertion 'gst_tag_list_is_equal (taglist, taglist2)' failed
Thanks for the bug report. What architecture / OS / system is this on? Could you run: $ cd tests/check/ $ GST_DEBUG=*:9 G_DEBUG=fatal_warnings GST_CHECKS=test_exif_tags_serialization_deserialization make libs/tag.check-norepeat 2>dbg.log $ xz -9 dbg.log and attach dbg.log.xz please?
David: ping?
I'm on an i686/slackware 13.75+lots of updates/linux box. Running the check as you asked I get: make: *** No rule to make target `libs/tag.check-norepeat'. Stop. In the interim I've upgraded to 1.2.3.
> Running the check as you asked I get: > make: *** No rule to make target `libs/tag.check-norepeat'. Stop. This is from the tests/check/ directory or with make -C tests/check/ libs/tag.check-norepeat ?
It's from teh tests/check directory of gstreamer.
I get the same error. Works for stuff in elements/, not for stuff in libs/ for some reason. The common/check.mak rules don't seem to be specific to elements, so not sure why this is so.
I mean, the same error as comment 3, for the -norepeat command.
Wait - I tried in gstreamer as per comment 5, but this is from -base :D Only realized when I tried look at the Makefile.am for what built libs/tags. -norepeat works in -base. David, please try in gst-plugins-base/tests/check, not gstreamer/tests/check.
Created attachment 273602 [details] debuging output for gst-plugins-base
I went to gst-plugins-base/tests/check and ran GST_DEBUG=*:9 G_DEBUG=fatal_warnings GST_CHECKS=test_exif_tags_serialization_deserialization make libs/tag.check-norepeat 2>dbg.log I see: CC libs/libs_tag-tag.o CCLD libs/tag Running suite(s): tag support library 0%: Checks: 1, Failures: 1, Errors: 0 libs/tag.c:1354:F:general:test_exif_tags_serialization_deserialization:0: Assertion 'gst_tag_list_is_equal (taglist, taglist2)' failed I uploaded the debug output (comment 9).
Can you add GstMapInfo map; gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); in each of these tests before before the gst_tag_list_is_equal() comparison (and move the gst_buffer_unref() call after it) and then paste the output of that?
Created attachment 273743 [details] Output after modifications mentioned in 11 I ran: GST_DEBUG=*:9 G_DEBUG=fatal_warnings GST_CHECKS=test_exif_tags_serialization_deserialization make libs/tag.check-norepeat 2>dbg.log CC libs/libs_tag-tag.o CCLD libs/tag and see: Running suite(s): tag support library 0%: Checks: 1, Failures: 0, Errors: 1 libs/tag.c:1431:E:general:test_exif_tags_serialization_deserialization:0: (after this point) Received signal 5 (Trace/breakpoint trap) Just to be sure that I made the correct changes, here's what's been changed in tag.c: static void do_exif_tag_serialization_deserialization (GstTagList * taglist) { GstTagList *taglist2; GstBuffer *buf; /* LE */ buf = gst_tag_list_to_exif_buffer (taglist, G_LITTLE_ENDIAN, 0); taglist2 = gst_tag_list_from_exif_buffer (buf, G_LITTLE_ENDIAN, 0); gst_buffer_unref (buf); GstMapInfo map; gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); /* BE */ buf = gst_tag_list_to_exif_buffer (taglist, G_BIG_ENDIAN, 0); taglist2 = gst_tag_list_from_exif_buffer (buf, G_BIG_ENDIAN, 0); gst_buffer_unref (buf); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); /* APP1 */ buf = gst_tag_list_to_exif_buffer_with_tiff_header (taglist); taglist2 = gst_tag_list_from_exif_buffer_with_tiff_header (buf); gst_buffer_unref (buf); gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); }
(In reply to comment #12) > Just to be sure that I made the correct changes, here's what's been changed in > tag.c: Not completely, sorry. > static void > do_exif_tag_serialization_deserialization (GstTagList * taglist) > { > GstTagList *taglist2; > GstBuffer *buf; > > /* LE */ > buf = gst_tag_list_to_exif_buffer (taglist, G_LITTLE_ENDIAN, 0); > taglist2 = gst_tag_list_from_exif_buffer (buf, G_LITTLE_ENDIAN, 0); > gst_buffer_unref (buf); This gst_buffer_unref() should be after gst_tag_list_unref(taglist2) > GstMapInfo map; > gst_buffer_map (buf, &map, GST_MAP_READ); > gst_util_dump_mem(map.data, map.size); > gst_buffer_unmap (buf, &map); > g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); > g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); > fail_unless (gst_tag_list_is_equal (taglist, taglist2)); > gst_tag_list_unref (taglist2); > > /* BE */ > buf = gst_tag_list_to_exif_buffer (taglist, G_BIG_ENDIAN, 0); > taglist2 = gst_tag_list_from_exif_buffer (buf, G_BIG_ENDIAN, 0); Here you also want to check > gst_buffer_unref (buf); > > fail_unless (gst_tag_list_is_equal (taglist, taglist2)); > gst_tag_list_unref (taglist2); > > /* APP1 */ > buf = gst_tag_list_to_exif_buffer_with_tiff_header (taglist); > taglist2 = gst_tag_list_from_exif_buffer_with_tiff_header (buf); > gst_buffer_unref (buf); And also this one should be moved after the gst_tag_list_unref(taglist2) > gst_buffer_map (buf, &map, GST_MAP_READ); > gst_util_dump_mem(map.data, map.size); > gst_buffer_unmap (buf, &map); > g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); > g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); > fail_unless (gst_tag_list_is_equal (taglist, taglist2)); > gst_tag_list_unref (taglist2); > }
Created attachment 273933 [details] modifed test I tried again GST_DEBUG=*:9 G_DEBUG=fatal_warnings GST_CHECKS=test_exif_tags_serialization_deserialization make libs/tag.check-norepeat 2>dbg.log CC libs/libs_tag-tag.o CCLD libs/tag Running suite(s): tag support library 00000000 (0x8121f98): 00 00 00 00 00 00 ...... tags 1: taglist, copyright=(string)"my\ string"; tags 2: taglist; 0%: Checks: 1, Failures: 1, Errors: 0 libs/tag.c:1359:F:general:test_exif_tags_serialization_deserialization:0: Assertion 'gst_tag_list_is_equal (taglist, taglist2)' failed Here's the modified code: static void do_exif_tag_serialization_deserialization (GstTagList * taglist) { GstTagList *taglist2; GstBuffer *buf; /* LE */ buf = gst_tag_list_to_exif_buffer (taglist, G_LITTLE_ENDIAN, 0); taglist2 = gst_tag_list_from_exif_buffer (buf, G_LITTLE_ENDIAN, 0); GstMapInfo map; gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); gst_buffer_unref (buf); /* BE */ buf = gst_tag_list_to_exif_buffer (taglist, G_BIG_ENDIAN, 0); taglist2 = gst_tag_list_from_exif_buffer (buf, G_BIG_ENDIAN, 0); gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); gst_buffer_unref (buf); /* APP1 */ buf = gst_tag_list_to_exif_buffer_with_tiff_header (taglist); taglist2 = gst_tag_list_from_exif_buffer_with_tiff_header (buf); gst_buffer_map (buf, &map, GST_MAP_READ); gst_util_dump_mem(map.data, map.size); gst_buffer_unmap (buf, &map); g_print ("tags 1: %s\n", gst_tag_list_to_string (taglist)); g_print ("tags 2: %s\n", gst_tag_list_to_string (taglist2)); fail_unless (gst_tag_list_is_equal (taglist, taglist2)); gst_tag_list_unref (taglist2); gst_buffer_unref (buf); }
Ok, so for some reason the serialisation generated a zero filled buffer. Thanks!
The problem is with g_convert / iconv: gstexiftag.c:816:write_exif_ascii_tag: Failed to convert exif tag to ascii: 0x8298 - my string. Error: Conversion from character set 'utf8' to 'latin1' is not supported There are two issues here: a) the string is pure ascii already, there's nothing to convert at all (we can special-case this, which would at least fix the unit test) b) we should really write all our EXIF tag strings as UTF-8 instead of Latin1
This specific error would happen if iconv does not support UTF-8 to Latin1 conversion at all btw
I think this should fix it: commit 99929b7ffa6f6eb43292881599a21c580f13821e Author: Tim-Philipp Müller <tim@centricular.com> Date: Fri Apr 3 21:00:53 2015 +0100 tag: exiftag: don't try to convert utf-8 to latin1 if string is ASCII already Bypass g_convert/iconv if there's nothing to convert. That way, conversion won't fail on systems where iconv doesn't support converting utf-8 to latin1 and there's nothing to convert. https://bugzilla.gnome.org/show_bug.cgi?id=723252