GNOME Bugzilla – Bug 692368
mpegaudioparse: VBR mp3 files - query position not accurate
Last modified: 2018-11-03 14:48:19 UTC
Currently I'm working on support for the banshee mediaplayer for playing of audio (also vbr mp3) files using cuesheets, i.e. one large audio file with an index of the tracks. This is great for e.g. classical music or livesets. In order to support random play or playlists, I need to seek at the right moment, to another position. This can only be done accurately, when the current position is accurately reported. This is done for CBR files, but not for VBR files. However, it should be possible to report an accurate position in time for a VBR mp3 file using gst_element_query_position(). If the plugin knows at what frame it is in the mp3 file, it can be calculated easy, but the mp3 plugin doesn't work like that. I think it estimates the time using the current byte position in the file * the average bitrate, something like that. This works for CBR files, but not accurate for VBR files. It mostly underestimates the time passed. So songs are ended to quickly. However. All frames in an mp3 file are 26ms in length. See http://www.mp3-converter.com/mp3codec/frames.htm So also for a VBR file it must be possible to calculate the corrent current position, if you know the current frame that is being processed. CurrentFrameNumber*26 would be the current position in ms.
You (and banshee) should aim to move to and target GStreamer 1.x, 0.10 is not maintained any longer. I'm a bit confused about what your issue actually is. There are different components involved for mp3 playback. These days there's usually a parser in front of any mp3 decoder (mpegaudioparse from gst-plugins-good/gst/audioparsers). The parser will typically read data from the mp3 file, and parse it into mp3 frames, and timestamps these frames. It is also responsible for seeking. A position query on a playback pipeline would typically be answered by the audio sink depending on the current playback position. Perhaps you could make a small sample program that shows your issue, so we can see what it is?
To add what I wrote in the previous post. If you query the mp3 parser/decoder directly, you will typically get a different position (more advanced in the stream) than the current playback position. I'm not sure why "songs are ended too quickly" - audio samples should be played back until the last sample has been played and an EOS event has been received. So are you saying that the parser/decoder finish too early, or .. ?
Ohw. I'm not that deep into gstreamer. Banshee simply does a position request using gst_element_query_position() on a playing stream, when asking for the current position in the stream that's playing. What I notice with VBR mp3s is that the requested time position is usually not accurate. When my program 'listens' to the position that's reported back by Banshee, it knows the End offset and Begin offset (in ms) of a piece of music in the mp3 file (that contains a whole CD or concert). When the current position in the stream (in ms) reaches End offset, I seek to a new piece of music (other Begin Offset). However, the reported position mostly reaches 'End offset' several 100s of ms earlier than it should. The consequence is that Banshee seeks to fast, i.e. before the piece is over. With CBR mp3s or FLAC files, no problems occur, because the reported position by gstreamer is accurate. Seeking is accurate (given the right accurate seeking flag). Using gst_element_seek to set a play-range and waiting for EOS is not an option, because starting up the playback again and setting a new play_range, results in a noticeable gap.
You shouldn't and can't go by the reported position, how accurate it is depends on a lot of factors, most of which are outside of GStreamer's control and lower in the stack. It might also vary depending on the audio output method used and the drivers etc. A stream is finished at EOS. What you need is the "about-to-finish" signal, but with a way to tell it to start playing from position X instead of 0.
Ok, but the DEFAULT position would be accurate, wouldn't it be? So, if I can seek accurately, and somehow seek reports back the (calculated) seek positions in DEFAULT positions, that would give me enough information, because I would not query by time, but by DEFAULT. If this is true, the next question is: Is there a way to determine the resulting seek positions (maybe on forehand) of an accurate seek (if provided by the plugin of course)?
DEFAULT format and TIME format are easily convertible using the sample rate, shouldn't make a difference which one you use precision-wise. The resulting seek position of an accurate seek should always be exactly the requested seek start position (maybe I misunderstood your question?).
It is not really a problem. Position reporting is quite accurate with VBRs when one lets them play all the way. When I was testing this, I seeked songs quite fast from beginning to end. So there was no much data to base position calculation on.
I didn't actually think this through properly. I said the end is when you get EOS and that's it, but what you have is a very long song (e.g. a whole album), and you want to stop at some pre-defined position when you know song #N is finished, right? So knowing when the very end isn't particular useful. What might work is to use SEGMENT seeks instead. You'd have to wait until playbin reaches PAUSED, then you'd have to do a FLUSHING | ACCURATE seek with both start and stop-position set. You will then get a SEGMENT_DONE message on the bus when the parser has finished pushing the last mp3 frame containing the last sample, and is ready for a new (non-flushing) seek to a different position in the file. Which you can then do. That solves seamlessly jumping to different bits within the same file without a gap, but it won't help with switching to another file without a gap. Bug #619864 might be of mild interest though (but also doesn't help, because there's no API yet to tell playbin to start the next file from an offset).
Thank you for your solution, however I run into some strange problems (gstreamer ersion 0.10.36.0). Whenever I do a non-flushing seek (after the first segment setting seek (flushing or non flushing, it doesn't matter)), I get the following assertion failing: First seek: - output of banshee (including my own fprintf statements) [1 Debug 23:35:19.973] State change: Playing [1 Debug 23:35:19.973] Settting playing range to 63500 - 319100 bp_set_segment: begintime=63500,000000, endtime=319100,000000 Creating seek event (flush=1) Sending seek event Result=1 Second seek: [1 Debug 23:35:39.514] Got SegmentDone callback [1 Debug 23:35:39.515] OnEventChanged:SegmentDone Playerstate=Playing [1 Debug 23:35:39.515] Handling Segment Done event [1 Debug 23:35:39.515] OnEventChanged:RequestNextTrack Playerstate=Playing [1 Debug 23:35:39.519] Querying model for track to play in off:Next mode [1 Debug 23:35:39.522] HandleNextTrack :System of a Down - 03 - Revenga (on Mezmerize (2005)) <00:03:48.0200000> [file:///home/hans/.banshee_cuesheets/c/d/title=03_-_Revenga%3Boffset=319100%3Bend_offset=547120.mp3] [1 Debug 23:35:39.524] Handling cuesheet next [1 Debug 23:35:39.544] OnEventChanged:EndOfStream Playerstate=Playing [1 Debug 23:35:39.545] Track System of a Down - 02 - B.Y.O.B. (on Mezmerize (2005)) <00:04:15.6000000> [file:///home/hans/.banshee_cuesheets/5/9/title=02_-_B_Y_O_B_%3Boffset=63500%3Bend_offset=319100.mp3] had playtime of 82422 msec (82sec), duration 255600 msec, queued: False [1 Debug 23:35:39.546] OnEventChanged:StartOfStream Playerstate=Playing [1 Debug 23:35:39.582] Settting playing range to 319100 - 547120 bp_set_segment: begintime=319100,000000, endtime=547120,000000 Creating seek event (flush=0) Sending seek event (Banshee:31010): GStreamer-CRITICAL **: gst_event_new_new_segment_full: assertion `start <= stop' failed Result=1 [1 Debug 23:35:39.582] OnEventChanged:Range Playerstate=Playing Code of my seek routine: P_INVOKE gboolean bp_set_segment (BansheePlayer *player, guint64 time_ms, guint64 end_time_ms,gboolean accurate_seek,gboolean wait) { g_return_val_if_fail (IS_BANSHEE_PLAYER (player), FALSE); //GstSeekFlags seek_flag = GST_SEEK_FLAG_FLUSH|GST_SEEK_FLAG_SEGMENT; GstSeekFlags seek_flag = GST_SEEK_FLAG_SEGMENT; if (!wait) { //seek_flag |= GST_SEEK_FLAG_FLUSH; } if (accurate_seek) { seek_flag |= GST_SEEK_FLAG_ACCURATE; } fprintf(stderr,"bp_set_segment: begintime=%lf, endtime=%lf\n",(double) time_ms,(double) end_time_ms); /*if (player->playbin == NULL || !gst_element_seek (player->playbin, 1.0, GST_FORMAT_TIME, seek_flag, GST_SEEK_TYPE_SET, time_ms * GST_MSECOND, GST_SEEK_TYPE_NONE, GST_CLOCK_TIME_NONE)) { g_warning ("Could not seek in stream"); return FALSE; } fprintf(stderr,"seeked\n");*/ //seek_flag |= GST_SEEK_FLAG_SEGMENT; if (player->playbin == NULL) { g_warning("Could not seek in stream (playbin==null)"); return FALSE; } else { GstEvent *event; fprintf(stderr,"Creating seek event (flush=%d)\n",!wait); event=gst_event_new_seek(1.0,GST_FORMAT_TIME,seek_flag, GST_SEEK_TYPE_SET,time_ms*GST_MSECOND, GST_SEEK_TYPE_SET,end_time_ms*GST_MSECOND ); fprintf(stderr,"Sending seek event\n"); gboolean result=gst_element_send_event(player->playbin,event); fprintf(stderr,"Result=%d\n",result); if (!result) { g_warning("Could not seek in stream"); return FALSE; } else { player->range_begin=(gint64) time_ms; player->range_end=(gint64) end_time_ms; return TRUE; } } /* gint64 etm=(wait) ? -1 : end_time_ms*GST_MSECOND; if (player->playbin == NULL || !gst_element_seek (player->playbin, 1.0, GST_FORMAT_TIME, seek_flag, GST_SEEK_TYPE_SET, time_ms * GST_MSECOND, GST_SEEK_TYPE_SET, etm )) { // end_time_ms * GST_MSECOND)) { g_warning ("Could not seek in stream"); return FALSE; } fprintf(stderr,"set segment\n"); return TRUE; */ }
I'm completely stuck here. Here's some gstreamer debug info attached. After the second seek, after the playbin has been emptied, it just doesn't feed more musich to the pipeline. What am I doing wrong?
Created attachment 234350 [details] Log file of playback in banshee with gstreamer debug=3
Could you make a small stand-alone test case against GStreamer 1.0 please? 0.10 is not maintained any longer, and the last release was nearly a year ago. (*:3 log are rare useful unfortunately, *:5 is better [=*:6 in 1.0]).
I'll add a *:5 log and I will look into gstreamer 1.0. Don't know how fast I can do that. Is 1.0 backwards compatible with 0.10? That would make it easy to compile banshee against 1.0.
Created attachment 234367 [details] Log file of banshee with GST_DEBUG=5 I ran banshee with GST_DEBUG=5. I'm trying to seek to the next song (without FLUSH, to let the first song play the last fragments). The next song has indexes: 1:03 - 5:19 (or start/stop: start 63500000000, stop 319100000000). Log file starts when segment_done event is posted
I'm sorry. I haven't got time to create a sample for gstreamer 1.0. Maybe the log file gives enough information to see what's happening. the case for a gstreamer 1.0 sample would be: - Open a url with an mp3 audio file of about 5 minutes music - Set a playback segment (e.g 1:00 - 2:00) using SEEK_FLUSH and SEEK_SEGMENT - when SEGMENT_DONE occurs, set the playback segment to 2:00-3:00 without SEEK_FLUSH, but with SEEK_SEGMENT - the playback should go on without a noticable gap. All other use cases may introduce gaps.
Test program is here: commit 7b64b3e109dbbe107f652c5457faf24f365b389c Author: Tim-Philipp Müller <tim@centricular.com> Date: Fri Nov 28 10:41:55 2014 +0000 tests: add interactive test for gapless playback using SEGMENT seeks Not working too well yet, there are glitches even with WAV or FLAC. https://bugzilla.gnome.org/show_bug.cgi?id=692368 Did not check accuracy of ACCURATE mp3 seeking yet, I would have thought GstBaseParse handles that for us these days (but then I don't think it's changed much since 0.10.36). Moving this to mpegaudioparse for now.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/80.