GNOME Bugzilla – Bug 731029
Incorrect reading of the GST_TAG_DATE_TIME field of a MPEG-4 AAC encoded file
Last modified: 2018-05-09 13:51:01 UTC
Created attachment 277602 [details] wrong year displayed in rhythmbox - 1910 vs Nautilus seeing 2012 I originally reported this bug here - https://bugzilla.gnome.org/show_bug.cgi?id=723020 However, the maintainer requested this be refiled here since he stated "gstreamer reads that one incorrectly." Thus, a brief explanation - Rhythmbox 3.0.3 (trunk) reads and displays the year 1910. Nautilus and EasyTag sees the Year field (correctly) as 2012. Any thoughts on this that you can feedback to the Rhythmbox maintainer Jonathan Matthews? Using Rhythmbox 14.04 and python3 Gst.version() reports 1.2.4
oops - that should read Ubuntu 14.04 and gstreamer 1.2.4!
further to this, in a virtual machine of 14.04 I've tried to compile gstreamer v1.3.2 but this has not resolved this issue. I've compiled as follows: sudo apt-get build-dep gstreamer1.0-plugins-base gstreamer1.0-plugins-bad sudo apt-get install bison flex libfaac-dev then untar'd the packages: tar -Jxvf gstreamer-1.3.2.tar.xz tar -Jxvf gst-plugins-base-1.3.2.tar.xz tar -Jxvf gst-plugins-bad-1.3.2.tar.xz ./configure make sudo make install for each folder gstreamer, gst-plugins-base & gst-plugins-bad in that order I saw that the faac plugin was going to be built (I'm assuming this is the correct plugin here) and then rebooted and retested rhythmbox.
Most likely an issue in qtdemux which is in gst-plugins-good.
It seems GStreamer is correct. There are two dates in Quicktime (don't ask). One is a creation date in seconds from 1904 (or1970 it's a bit ambiguous, the format seems to use 1904 only), and this file has a value of 216850891, which is almost 7 years. So that date goes to late 1910. Alternatively, using the 1970 epoch, it could be interpreted as late 1976. Probably not what was intended. This value is 32 bit, so can go up to 138 years, which I guess explains the apparently non standard 1970 epoch, as people might want to create Quicktime files after 2042. The second date does say 2012, and only has day granularity. So I guess some software uses one of the dates, and some the other. The culprit seems to be the software and/or human that created the file (if the human was right and the software was GStreamer, please file another bug). I'll wait for comments before closing as NOTABUG.
Hi Vincent, thanks for the feedback. It appears that many of the iTunes m4a files (I cant upload here due to the size and obvious copyright issues) produced since 2011 have the same issue - a GST-TAG-DATE-TIME field and a contradictory alternative GST-TAG-DATE field. Before 2011, just the GST-TAG-DATE was filled. Basically I think Jonathan Matthew (Rhythmbox maintainer) is wanting feedback here. Jonathan's latest commit here ... https://github.com/GNOME/rhythmbox/commit/1d921b4e77c1d50d3ac8cfc63855af0b1adb0b32 He is trying to calculate the year value to be displayed by the application by first pulling the GST-TAG-DATE-TIME and if this fails, falls back to pulling the value for GST-TAG-DATE. As Jonathan has said here (https://bugzilla.gnome.org/show_bug.cgi?id=723020#c11) - this algorithm is sort of arbitrary. In your experience, should Rhythmbox be doing the flip side - pulling GST-TAG-DATE first before falling back to GST-TAG-DATE-TIME ? Note - this same section of code is used to calculate the year value for all audio file types not just m4a thanks in advance
Hi there, Just to give you a little bit of background info on this problem. It started sometimes in 2011 when iTunes started to populate the GST_TAG_DATE_TIME. This tag is supposed to represent the release date of music file not its creation date. Before then, only the GST_TAG_DATE tag was used. What makes us believe GStreamer is incorrect when reading this tag is that if we read this file in Windows with MP3Tag, we get a date along with a time stamp. Unless, it is fetching it from another tag, I don't know how it would return a correct value if that tag was incorrect. You can find another example of a recent file here: https://www.dropbox.com/s/6poxelgjku2q3hf/05%20My%20Little%20Universe.m4a.zip The year showned in MP3Tag is 2013-03-22T07:00:00Z. This is the correct release date.
GStreamer also finds '2013-03-22T07:00:00Z'. The time part is discarded when setting the tag, as this tag is a date only. Both date and datetime are documented as when the stream was created, not last modified. Maybe they should both be set to the same value. Or maybe we need another date tag for last modified date.
Hi Vincent - that's an interesting observation on Jean-Rene's latest file. I'm using Ubuntu 14.04 (latest stable gstreamer 1.2.4) and the datetime field is displaying "1986-08-02T06:48:45Z" - this is what Rhythmbox using Jonathan Matthew's latest commit is also displaying. Which tag are you displaying? N.B. - I'm displaying the datetime tag via python gi.introspection using the Gst class and the method to_iso8601_string() N.B. - 2 - any thoughts about my query to help Jonathan and the rhythmbox algorithm to calculate and display the correct Year value?
$ gst-discoverer-1.0 05\ My\ Little\ Universe.m4a | grep -i date datetime: 1986-08-02T06:48:45Z date: 2013-03-22 This exposes the two dates available. The correct value is apparently "date" here, but this just happens to be what qtdemux does, so it's possible other demuxers might do the reverse if the format has more than one date. Maybe taking the latest of both might work ? But then whatever's setting this early timestamp in the creation date might well put some random later date too, so... If that creation date is always wrong on a large corpus of files though, maybe we could just ignore it on the assumption that nothing gets a sensible value in there, and fill everything with the modification date.
Hi Vincent, I cant really comment about which dates are correct. The only online documentation I found is this - http://www.jiscdigitalmedia.ac.uk/guide/aac-audio-and-the-mp4-media-format It looks like two dates exists for itunes - year which I imagine is mapped to gst_tag_date and purchase date. This latter field in itunes I dont see in the returned fields by gstreamer I dont know what the gst_tag_datetime is mapped to - but probably gstreamer should just return the Year field for both tags ... or maybe dont return anything at all for gst_tag_date_time
I just found this: - http://www.mp4ra.org/atoms.html is the gst_tag_date_time field mapped to "date"? - this appears to be an Apple only tag - not an ISO standard "Date and time, formatted according to ISO 8601, when the clip’s recording started" - now 1986 being returned by gst-discoverer doesnt really make much sense to me - the track was indeed released in 2013. I can only assume that Apple have used this date tag with their own algorithm - not as gstreamer is reading it. Should gstreamer ignore "non-ISO standard" fields somehow ?
Is there any way with Gstreamer to get the Apple tag name that correspond to the datetime tag read? I have been using this project: http://sourceforge.net/projects/atomicparsley/?source=dlp to read the tags and none shows 1986-08-02T06:48:45Z. Atom "----" [iTunSMPB] contains: 00000000 00000840 000003B0 0000000000B1D410 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Atom "----" [iTunNORM] contains: 00000497 00000389 00002397 00001AD5 0001B360 0003C0BA 00007F3E 00007F57 000386AD 000386AD Atom "©nam" contains: My Little Universe Atom "©ART" contains: Depeche Mode Atom "aART" contains: Depeche Mode Atom "©wrt" contains: Martin L. Gore Atom "©alb" contains: Delta Machine (Deluxe Version) Atom "©gen" contains: Alternative Atom "trkn" contains: 5 of 17 Atom "disk" contains: 1 of 1 Atom "cpil" contains: false Atom "pgap" contains: 0 Atom "©day" contains: 2013-03-22T07:00:00Z Atom "apID" contains: someuser@hotmail.com Atom "cprt" contains: ℗ 2013 Venusnote Ltd., under exclusive license to Columbia Records, a Division of Sony Music Entertainment Atom "cnID" contains: 594094257 Atom "rtng" contains: Inoffensive Atom "atID" contains: 148377 Atom "cmID" contains: 996871 Atom "plID" contains: 594094251 Atom "geID" contains: 20 Atom "sfID" contains: Canada (143455) Atom "stik" contains: Normal Atom "purd" contains: 2013-03-29 15:48:08 Atom "xid " contains: SonyBMG:isrc:USQX91300029 Atom "----" [iTunMOVI] contains: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>asset-info</key> <dict> <key>file-size</key> <integer>9452154</integer> <key>flavor</key> <string>2:256</string> </dict> </dict> </plist> Atom "covr" contains: 1 piece of artwork
I figured out since then that these atom tags are different from the file tags, so please ignore my previous comments. To help you determine if this issue is valid or not I see that this it is also being reported by another user here: http://stackoverflow.com/questions/23787088/gnome-music-and-rhythmbox-dont-read-tags-properly https://bbs.archlinux.org/viewtopic.php?id=181839
I was just looking into this. The problem here is that the tags returned are different but imply to be the same: https://cgit.freedesktop.org/gstreamer/gstreamer/tree/gst/gsttaglist.c#n137 lists 2 content created related tags (DATE & DATE_TIME), their descriptions are the same and only say they return different time objects. qtdemux fills these 2 tags with different values depending on what it encounters. However it specifically states in https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/isomp4/qtdemux.c#n12596 that we should take the mhvd field value as less important than any specific tags. Now the mhvd field gets tagged in gstreamer as DATE_TIME and the real tags get tagged as DATE only. So as far as I can see the commenter in the qtdemux code above meant that tag to be bottom of the pack, it just doesn't get handled that way. As a solution I think the other fields should be converted to a full datetime to get the desired (and expected) behaviour. If there are no other tags present, only then the mvhd tag gets returned.
Created attachment 369882 [details] [review] isomp4: Use full date time if available The ©day tag contains a full date time, use it for the DATE_TIME tag instead of just the DATE tag. This overrules the unreliable qt creation time.
This patch overrides the qt create time with the ©day tag if it is a full date time. The qtdemux_tag_add_date now adds both a GST_TAG_DATE & GST_TAG_DATE_TIME if available. Not sure this is wanted, but it otherwise it changes behaviour. Not sure what is wisdom here: in my small sample size the ©day tag is a full date time, but information on the usual contents of this tag are scarce. Anyway, this does solve the problem described here as it will overwrite the bogus information in the DATE_TIME tag.
Ping. Patch available in #15 and in gnome-music we get regular bug reports because of this issue, so would like to get it fixed.
Thanks ! Attachment 369882 [details] pushed as c2cce8a - isomp4: Use full date time if available