GNOME Bugzilla – Bug 731853
New tool : Add a CodecAnalyzer
Last modified: 2014-11-08 14:14:58 UTC
Hi, I would like to add a new debug-tool "codecanalyzer: https://github.com/sreerenjb/codecanalyzer" as a part of gst-devtools. Codecanalyzer is for doing in-depth analysis on compressed media. The initial version only supports mpeg2, but that too will be a minimal one because we don't have all necessary patches in upstream yet(one eg:https://bugzilla.gnome.org/show_bug.cgi?id=704865). The codecanalyzer will have the following features (not all are implemented yet): -- It can do in-depth analysis on compressed media. -- It can unpack the elementary stream from a container -- Capable of doing packetization for the non-packetized stream -- Parse all the syntax elements from the elementary video stream -- A simple UI to navigate through all the headers of each frame separately together with the hex-dump. -- Users would be able to analyze the media files residing in the local machine and the remote streams via http or rtp The Codecanalyzer is mainly based on the codecparser libaray and videoparser plugins in gst-plugins-bad. Also it has a new plugin called "analyzersink" to generate the xml file for each video frame. Codecanalzyer has dependency to gstreamer, glib, libxml and gtk. Why we need a tool like this?: --We don't have any open source (or even free binary ) codec analzyers and the proprietary analyzers are too expensive to buy. --This will be helpful tool for debugging. --Useful for anyone to verify the specification compliance of their codec specific products. Comments?
I am adding a couple of screenshots too. But this screenshots are based on a version of codecanalyzer(which is not in GitHub yet) which needed features missing in upstream gst-plugins-bad. For eg: we don't have meta support for mpeg2 slice header, the patch is already in the bugzilla: https://bugzilla.gnome.org/show_bug.cgi?id=704865.
Created attachment 278680 [details] screenshot1
Created attachment 278681 [details] screenshot2
Created attachment 278682 [details] screenshot3
Created attachment 278683 [details] screenshot4
Cool!
I've wanted an open source version of of one these for a while too !
We can add all parsing related stuffs. But there are many legal issues to implement the decoding part of most of the specifications. Still a specification compliant syntax element parser is more than handy for debugging.
Hello sreerenj, I have not looked in deep details to your tool but from what I saw it is quite similare to the gst-validate-media-check in the end, just that you go one step deeper in the parsing of each frame (in the media-check toool we check the metadatas of the frames, ie pts, dts, etc...) but this is something we which to do too there. You can find the implementation of that tool at: http://cgit.freedesktop.org/gstreamer/gst-devtools/tree/validate/tools/gst-validate-media-check.c http://cgit.freedesktop.org/gstreamer/gst-devtools/tree/validate/gst/validate/media-descriptor-writer.c That tool also allows to check that discovering of the medias file is constantly correct through time letting us notice regressions. I am just wondering if we could not share code here?
Hi Thibault, I am aware of both tools(mediainfo and validate) in gst-devtools even though i didn't look deeper into the source code. There were some reasons for starting a new tool like this. Analyzer needs its on UI because it might grow much bigger in future, I would say beyond just a simple debugging tool :). If we can bypass some of the patent issues in future then both the back-end and UI needs to redesign completely with lots of other features (for eg: display motion vectors, dispaly DPB info, comparisons of quality parameters with other codecs etc). I would like to add decoder+renderer also so that we can display decoded frames inside the analyzer.
Nice, I always wanted to add something like this in gst-mediainfo too, but so far we're missing generic infrastructure, so that one can implement this in a tool so that it works in a codec agnostic fashion. Once we have such infrastructure, it would be nice to hook it up into the the existing tools.
Created attachment 280068 [details] [review] New Tool: Add a CodecAnalyzer
Anyone not okay with adding this to gst-devtools git for now? Otherwise I'll push it in a few days.
Review of attachment 280068 [details] [review]: Can you lower the requirements a little. I just built it and it works fine using [libxml-2.0 >= 2.7.8] [gtk+-3.0 >= 3.4.2]
Created attachment 280069 [details] [review] New Tool: Add a CodecAnalyzer Changed the required version of libxml and gtk. [libxml-2.0 >= 2.7.8] [gtk+-3.0 >= 3.4.2]
+1
Not a showstopper for submission, but it would be nice to have some top-level comment block in main classes to describe how it works, e.g. gst-analyzersink could have one. The gst-pipeline is producing intermediate files at ~/tmp/codecanalyzer/{hex,xml}/ the code is from analyzer_create_dirs(). This should use g_get_user_cache_dir() if the data is reused or g_get_tmp_dir(). On the other hand I am not super fond of the filesystem interaction. What about using nested GstStructures for the data and emitting them as GstBusMessages?
(In reply to comment #17) > Not a showstopper for submission, but it would be nice to have some top-level > comment block in main classes to describe how it works, e.g. gst-analyzersink > could have one. Thanks for the review. I will add those. > The gst-pipeline is producing intermediate files at > ~/tmp/codecanalyzer/{hex,xml}/ > the code is from analyzer_create_dirs(). This should use g_get_user_cache_dir() > if the data is reused or g_get_tmp_dir(). I will add this too. > > On the other hand I am not super fond of the filesystem interaction. What about > using nested GstStructures for the data and emitting them as GstBusMessages? Do you mean to keep pipeline alive all the time during the analysis? Right now the pipeline is active only during parsing time. After that it is all about xml data manipulation and there is no gstreamer interaction. Anyway Edward had also some ideas about emitting parser info as GstMessages. But that we can discuss later since it needs more changes in videoparser plugins too. what do you think?
The analyzersink is handling all codec-specific details. The UI generation is completely based on the format of xml files. Which means, on top of analyzersink anyone can write better/fancy UIs with their own tool-kit easily (with no interaction to the gstreamer pipeline). May be later we can even move the analyzer sink as a generic plugin under gst-plugins-bad.
(In reply to comment #18) > (In reply to comment #17) > > Not a showstopper for submission, but it would be nice to have some top-level > > comment block in main classes to describe how it works, e.g. gst-analyzersink > > could have one. > > On the other hand I am not super fond of the filesystem interaction. What about > > using nested GstStructures for the data and emitting them as GstBusMessages? > > Do you mean to keep pipeline alive all the time during the analysis? > Right now the pipeline is active only during parsing time. After that it is all > about xml data manipulation and there is no gstreamer interaction. > > Anyway Edward had also some ideas about emitting parser info as GstMessages. > But that we can discuss later since it needs more changes in videoparser > plugins too. what do you think? Not all the time, the pipeline would run with sync=false and the app would store the messages in memory (And yes I meant to put the GstStructures into a GstMessage). As it receives messages it can update the UI. Another suggestion would be to make the analyzersink a filter and end the pipeline with fakesink. A filter is simpler code wise and multiple filters can be chained. This would make it easier to re-use the analyzer. Finally a closer integration with the parser libraries might be interesting. I am wondering if the actual parser element could get a "message" property and actually emit the messages.
(In reply to comment #20) > (In reply to comment #18) > > (In reply to comment #17) > > Do you mean to keep pipeline alive all the time during the analysis? > > Right now the pipeline is active only during parsing time. After that it is all > > about xml data manipulation and there is no gstreamer interaction. > > > > Not all the time, the pipeline would run with sync=false and the app would > store the messages in memory (And yes I meant to put the GstStructures into a > GstMessage). As it receives messages it can update the UI. Then the app have to store all those messages in memory unless the pipeline is active during the full session of analysis and we have to support seeking too in this case. Because the user should be able to check the frames forth and back. Suppose the user set the number of frames to be analyzed is equal to 1000, then it needs to store a long list of structures in memory which in unnecessary (if pipeline is not active whole the time). The situation will be even worse if the user just try to analyze the whole number of frames in the media file(with out specifying the number of frames to be analyzed). Make the pipeline active and parse the stream again and again is another option.
> du -sh ~/tmp/codecanalyzer/ 282M /home/ensonic/tmp/codecanalyzer/ And that is xml + hex text. I think it is fine to keep this in memory. Otherwise what is your strategy to purge disk (unless it goes to g_get_tempdir().
Created attachment 280120 [details] [review] codecanalyzer: Use XDG_CACHE_HOME path for app specific data storage
Created attachment 280121 [details] [review] codecanalyzer: Add more documentation
(In reply to comment #22) > > du -sh ~/tmp/codecanalyzer/ > 282M /home/ensonic/tmp/codecanalyzer/ How many frames got analyzer here ? just curious:) > > And that is xml + hex text. I think it is fine to keep this in memory. > Otherwise what is your strategy to purge disk (unless it goes to > g_get_tempdir(). Honestly I didn't think about this in more detail, but yes have to fix a limit for the hard-drive usage too. It would be better to avoid system memory usage here since we have to add decoder later on which will take more memory for sure. WDT?
(In reply to comment #25) > (In reply to comment #22) > > > du -sh ~/tmp/codecanalyzer/ > > 282M /home/ensonic/tmp/codecanalyzer/ > > How many frames got analyzer here ? just curious:) 1000
Review of attachment 280120 [details] [review]: ::: codecanalyzer/src/codecanalyzer.c @@ +957,3 @@ return FALSE; + sprintf (analyzer_home, "%s/codecanalyzer", user_cache_dir); g_build_filename() to handle path on e.g. windows. Also below.
Created attachment 280288 [details] [review] New Tool: Add a CodecAnalyzer Fixed most of the issues which Stefan mentioned and squashed all the patches to a single one.
The following fix has been pushed: c73a9a3 New Tool: Add a CodecAnalyzer
Created attachment 280292 [details] [review] New Tool: Add a CodecAnalyzer