GNOME Bugzilla – Bug 721362
[PATCH] RFC: additional toc api to represent edit lists and loops
Last modified: 2018-11-03 12:19:19 UTC
GstTocEntries can be sequence types for chapters/tracks/titles. Right now they only have start/stop times. Having a loop_type and a repeat could would allow to represent all kind of edit-lists.
Created attachment 265192 [details] [review] expand GstTocEntry with loop fields I plan to make use of this from wav and sndfile elements.
I don't know if this should go into the TOC, I see TOC more as a navigational aid for the user interface, you're overloading it a little now, though it is admittedly convenient. You're now also palming off all the work onto the app if it wants to play back certain files correctly, which doesn't really seem right. If we forced playback applications to handle edit lists in quick time files, people would make fun of us (not that you're proposing that here, just saying) :)
For the edit-list in e.g. mp4 I was wondering too how to deal with it. We want the demuxer to apply the edit list by default, but also give control to applications to request the raw-data + the edit list. I could definitely create a new subsystem for edit lists that works similar to the toc. One benefit of having it as part of the toc api is that we can do things like Media Edition1 Chapter1 0:00 - 1:00, loop=none, repeat=0 Chapter2 1:00 - 5:00, loop=forward, repeat=infinite Edition2 Chapter1 ... That is associate the loops with segments in the media. If we have a separate loop api, we would probably still relate it to the toc eventually. If you look at the AIFF spec [1]. They have markers ('MARK') and then reference them from an instrument chunk ('INST') for looping. We could have a separate loop api, that uses the uuid from GstTocEntry to lookup the loop data. If we can't think of too many additional fields, we could just add them as I do with this patch (other stuff I could think of is a "play_back_rate" field). Alternatively we could add a GstStucture with extra_data to GstTocEntry. I could then add "loop::type" and "loop::repeat_count". Other elements could add e.g. "playback::rate". While this is flexible, we would need to define the keys like in GstTaglist though. [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/Docs/AIFF-1.3.pdf
I think the TOC API is a good choice here
It's not that I think this is horrible or anything, I'm just wary of overloading the GstToc API with "wtf", and I also wonder if these fields will be sufficient in future, or if we just keep adding more and more stuff to it to accommodate more use cases. I'm just looking at this from an application writer perspective, and they tend to find GstToc already somewhat confusing in my experience, so I would not be happy to see more and more exotic/weird stuff added to it that just confuses people even more. The documentation in your patch tells users nothing about how this should be used, where they would get it, if they should worry about it or not. What should an application writer do with a loop entry?
(In reply to comment #5) > It's not that I think this is horrible or anything, I'm just wary of > overloading the GstToc API with "wtf", and I also wonder if these fields will > be sufficient in future, or if we just keep adding more and more stuff to it to > accommodate more use cases. If that is the concern we can either: a) create an extension mechanism b) or create new api (new event) that references toc-subentries I think a) is overkill and would make it complicated for apps to deal with loops. As I said earlier, I can think of too many extra fields. > I'm just looking at this from an application writer perspective, and they tend > to find GstToc already somewhat confusing in my experience, so I would not be > happy to see more and more exotic/weird stuff added to it that just confuses > people even more. The documentation in your patch tells users nothing about how > this should be used, where they would get it, if they should worry about it or > not. What should an application writer do with a loop entry? If we would send the Loop info as a separate event, how would this change anything for media-player apps? They will most likely ignore it, as they would ignore looped subentries in the toc. Maybe we just need another toc-entry type: GST_TOC_ENTRY_TYPE_SEGMENT (sequence type).
(In reply to comment #6) > (In reply to comment #5) > > It's not that I think this is horrible or anything, I'm just wary of > > overloading the GstToc API with "wtf", and I also wonder if these fields will > > be sufficient in future, or if we just keep adding more and more stuff to it to > > accommodate more use cases. > > If that is the concern we can either: > a) create an extension mechanism > b) or create new api (new event) that references toc-subentries > > I think a) is overkill and would make it complicated for apps to deal with > loops. As I said earlier, I can think of too many extra fields. > I *can't* ...
c) A new GST_LOOP_MESSAGE containing GstLoopEntries with type, start, stop, count
I added a section to http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-toc.txt with reference material and some status of what we have and ideas how to go forward.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/46.