GNOME Bugzilla – Bug 751203
videostitching: Open CV video stitching plugin based on GstVideoAggregator
Last modified: 2018-11-03 13:36:16 UTC
I developed one Open CV video stitching plugin based on GstVideoAggregator. Test command: gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. videostitching name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. videostitching feturetypes=0 warptypes=2 seamfindtypes=3 bacostfuncs=1 name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152913.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152918.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152924.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152929.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152933.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152938.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152942.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152947.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152951.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. videostitching feturetypes=0 warptypes=2 seamfindtypes=3 bacostfuncs=1 name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false
And do you plan to share the code? :)
Created attachment 305643 [details] [review] Rename gstopencv.c to gstopencv.cpp
Created attachment 305644 [details] [review] Add video stitching element.
Created attachment 305645 [details] test images.
Nice, note that I'd call it cvstiching, so we can avoid using too generic names.
Created attachment 305887 [details] [review] Update to fix memory leakage.
(In reply to Nicolas Dufresne (stormer) from comment #5) > Nice, note that I'd call it cvstiching, so we can avoid using too generic > names. yes, there are many kinds of method to do video stitching. I will change the name later.
Thanks.
Created attachment 305975 [details] [review] Add video stitching element based on Open CV. Rename videostitching to cvstitching.
Created attachment 305976 [details] The output image of cvstitching.
New test command line: gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! stitcher. cvstitching name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. cvstitching feturetypes=1 warptypes=2 seamfindtypes=0 bacostfuncs=1 name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152913.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152918.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. cvstitching feturetypes=1 warptypes=2 seamfindtypes=0 bacostfuncs=1 name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152913.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152918.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152924.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152929.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152933.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152938.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152942.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152947.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152951.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. cvstitching feturetypes=1 warptypes=2 seamfindtypes=0 bacostfuncs=1 name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false
Help to review or any suggestion?
Review of attachment 305643 [details] [review]: 748377 already fix this. Will verify the cvstitching element after 748377 resolved.
Created attachment 309343 [details] [review] Add video stitching element based on Open CV. Update based on latest code.
test command line: gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegparse ! jpegdec ! videoconvert ! imagefreeze ! video/x-raw,format=RGB ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegparse ! jpegdec ! videoconvert ! imagefreeze ! video/x-raw,format=RGB ! stitcher. cvstitching feturetypes=1 name=stitcher stitcher.src ! videoconvert ! xvimagesink sync=false
The test command line should be below after https://bugzilla.gnome.org/show_bug.cgi?id=753687 gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegparse ! jpegdec ! imagefreeze ! videoconvert ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegparse ! jpegdec ! imagefreeze ! videoconvert ! stitcher. cvstitching feturetypes=1 name=stitcher stitcher.src ! videoconvert ! xvimagesink sync=false
Review of attachment 309343 [details] [review]: I had a quick look at this. The base class you create, GstOpenCVAggregator, seems like a lot of the negotiation code should be merged into GstAggregator/GstVideoAggregator directly. See https://bugzilla.gnome.org/show_bug.cgi?id=746529 for that effort. ::: ext/opencv/gstcvstitching.cpp @@ +52,3 @@ + * <title>Example launch line</title> + * |[ + * gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. cvstitching name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false Should probably split these up to avoid going over 80 chars. @@ +55,3 @@ + * ]| image stitching. + * |[ + * gst-launch-1.0 filesrc location=IMG_20150529_152901.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152907.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152913.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152918.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152924.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152929.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152933.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152938.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152942.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152947.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. filesrc location=IMG_20150529_152951.jpg ! jpegdec ! videoconvert ! imagefreeze ! stitcher. cvstitching name=stitcher stitcher.src ! videoconvert ! ximagesink sync=false and here @@ +81,3 @@ +float match_conf = 0.3f; +int blend_type = Blender::MULTI_BAND; +float blend_strength = 5; These should at least be static or added to the class if they can be changed dynamically. Most of these seem like they would be good candidates for properties. @@ +187,3 @@ + static const GEnumValue cv_stitching_feature_type[] = { + {GST_CV_STITCHING_FEATURE_TYPES_SURF, "feature type surf", "surf"}, + {GST_CV_STITCHING_FEATURE_TYPES_ORB, "feature type surf", "orb"}, s/surf/orb/ ? @@ +564,3 @@ + (*stitcher->finder)(img, features[i]); + features[i].img_idx = i; + //LOGLN("Features in image #" << i+1 << ": " << features[i].keypoints.size()); Either remove or add as a GST_DEBUG log. @@ +613,3 @@ + cameras[i].R.convertTo(R, CV_32F); + cameras[i].R = R; + //LOGLN("Initial intrinsics #" << indices[i]+1 << ":\n" << cameras[i].K()); GST logging or remove. I won't highlight the rest.
Thanks review. Will update based on review later.
Created attachment 314072 [details] [review] Add video buffer negotiation support to reuse buffer.
Created attachment 314073 [details] [review] Video stitching based on Open CV. It is based on GstVideoAggregator.
Created attachment 343852 [details] [review] videoaggregator: Add buffer pool negotiation support.
Created attachment 343853 [details] [review] cvstitching: Add open cv video stitching support.
Update to latest version.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/260.