GNOME Bugzilla – Bug 789075
opencv: new cameracalibrate and cameraundistort elements
Last modified: 2018-11-03 14:14:37 UTC
The two new elements are based on this OpenCV tutorial : Based on this tutorial: https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html In a nutshell: The cameracalibrate elements takes an operator through a camera calibration procedure. The operator presents a calibration pattern to the camera for multiple detections. See the photos at the end of the tutorial to get a glimpse of what it looks like. This procedure can be CPU intensive while it is in progress. All the processing is done inline (can stall the pipeline at times). The cameraundistort uses the results of the calibration to correct the camera images. In that process some pixels will be lost and the images will be scaled up. It is basically a transform element and CPU usage is moderate.
Created attachment 361697 [details] [review] opencv: new cameracalibrate and cameraundistort elements
would be great if this patch was reviewed and merged :)
any chance to get this reviewed and merged ?
Its in my to-do, but will have to wait till next week when I'm back (unless someone else wants to take it? Jan maybe ?)
Any news ?
Bump...
Attachment 361697 [details] pushed as f460448 - opencv: new cameracalibrate and cameraundistort elements
In file included from gstopencv.cpp:45:0: gstcameracalibrate.h:51:28: fatal error: opencv2/core.hpp: No such file or directory #include <opencv2/core.hpp> ^ compilation terminated. Makefile:980: recipe for target 'libgstopencv_la-gstopencv.lo' failed make: *** [libgstopencv_la-gstopencv.lo] Error 1 On Debian stretch.
Interesting, build worked for me, but I must admit, I only ran this in meson. Do you have a patch to fix it ?
I have no time to look at it currently, sorry. I might get back to it maybe tonight or tomorrow, but if you have time to look at it before, feel free.
Please close if CI agree. commit 1f65a3aa136821c89410a925b72464ab74a1f913 (HEAD -> master) Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Thu Aug 2 08:13:08 2018 -0400 cameracalibrtate: Fix opencv2 core.hpp path This is to support older release of OpenCV2 https://bugzilla.gnome.org/show_bug.cgi?id=789075
$ make -j4 make: *** No rule to make target '../../gst-libs/gst/opencv/libgstopencv-1.0.la', needed by 'libgstopencv.la'. Stop. make: *** Waiting for unfinished jobs.... CXX libgstopencv_la-gstcameracalibrate.lo CXX libgstopencv_la-gstcameraundistort.lo gstcameraundistort.cpp:83:31: fatal error: opencv2/calib3d.hpp: No such file or directory #include <opencv2/calib3d.hpp> ^ compilation terminated. Makefile:1162: recipe for target 'libgstopencv_la-gstcameraundistort.lo' failed make: *** [libgstopencv_la-gstcameraundistort.lo] Error 1 gstcameracalibrate.cpp:90:31: fatal error: opencv2/calib3d.hpp: No such file or directory #include <opencv2/calib3d.hpp> ^ compilation terminated. Makefile:1155: recipe for target 'libgstopencv_la-gstcameracalibrate.lo' failed make: *** [libgstopencv_la-gstcameracalibrate.lo] Error 1 [...] $ git diff diff --git a/ext/opencv/gstcameracalibrate.cpp b/ext/opencv/gstcameracalibrate.cpp index ba59a194b..6f076fcc3 100644 --- a/ext/opencv/gstcameracalibrate.cpp +++ b/ext/opencv/gstcameracalibrate.cpp @@ -87,7 +87,7 @@ #if (CV_MAJOR_VERSION >= 3) #include <opencv2/imgproc.hpp> #endif -#include <opencv2/calib3d.hpp> +#include <opencv2/calib3d/calib3d.hpp> #include <gst/opencv/gstopencvutils.h> diff --git a/ext/opencv/gstcameraundistort.cpp b/ext/opencv/gstcameraundistort.cpp index a50300851..dc731ec91 100644 --- a/ext/opencv/gstcameraundistort.cpp +++ b/ext/opencv/gstcameraundistort.cpp @@ -80,7 +80,7 @@ #if (CV_MAJOR_VERSION >= 3) #include <opencv2/imgproc.hpp> #endif -#include <opencv2/calib3d.hpp> +#include <opencv2/calib3d/calib3d.hpp> #include <gst/opencv/gstopencvutils.h> justin@walter:~/gst/master/gst-plugins-bad/ext/opencv$ make CXX libgstopencv_la-gstcameracalibrate.lo gstcameracalibrate.cpp: In function ‘void gst_camera_calibrate_init(GstCameraCalibrate*)’: gstcameracalibrate.cpp:310:23: error: ‘cv::fisheye’ has not been declared calib->flags = cv::fisheye::CALIB_FIX_SKEW | cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC | ^~~~~~~ gstcameracalibrate.cpp:310:53: error: ‘cv::fisheye’ has not been declared calib->flags = cv::fisheye::CALIB_FIX_SKEW | cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC | ^~~~~~~ gstcameracalibrate.cpp:312:12: error: ‘cv::fisheye’ has not been declared cv::fisheye::CALIB_FIX_K2 | cv::fisheye::CALIB_FIX_K3 | cv::fisheye::CALIB_FIX_K4; ^~~~~~~ gstcameracalibrate.cpp:312:40: error: ‘cv::fisheye’ has not been declared cv::fisheye::CALIB_FIX_K2 | cv::fisheye::CALIB_FIX_K3 | cv::fisheye::CALIB_FIX_K4; ^~~~~~~ gstcameracalibrate.cpp:312:68: error: ‘cv::fisheye’ has not been declared cv::fisheye::CALIB_FIX_K2 | cv::fisheye::CALIB_FIX_K3 | cv::fisheye::CALIB_FIX_K4; ^~~~~~~ gstcameracalibrate.cpp:319:25: error: ambiguous overload for ‘operator=’ (operand types are ‘cv::Mat’ and ‘int’) calib->cameraMatrix = 0; ^ In file included from /usr/include/opencv2/core/core.hpp:4856:0, from gstcameracalibrate.h:51, from gstcameracalibrate.cpp:85: /usr/include/opencv2/core/mat.hpp:281:13: note: candidate: cv::Mat& cv::Mat::operator=(const cv::Mat&) inline Mat& Mat::operator = (const Mat& m) ^~~ In file included from gstcameracalibrate.h:51:0, from gstcameracalibrate.cpp:85: /usr/include/opencv2/core/core.hpp:1775:10: note: candidate: cv::Mat& cv::Mat::operator=(const Scalar&) Mat& operator = (const Scalar& s); ^~~~~~~~ gstcameracalibrate.cpp:320:23: error: ambiguous overload for ‘operator=’ (operand types are ‘cv::Mat’ and ‘int’) calib->distCoeffs = 0; ^ In file included from /usr/include/opencv2/core/core.hpp:4856:0, from gstcameracalibrate.h:51, from gstcameracalibrate.cpp:85: /usr/include/opencv2/core/mat.hpp:281:13: note: candidate: cv::Mat& cv::Mat::operator=(const cv::Mat&) inline Mat& Mat::operator = (const Mat& m) ^~~ In file included from gstcameracalibrate.h:51:0, from gstcameracalibrate.cpp:85: /usr/include/opencv2/core/core.hpp:1775:10: note: candidate: cv::Mat& cv::Mat::operator=(const Scalar&) Mat& operator = (const Scalar& s); ^~~~~~~~ gstcameracalibrate.cpp: In function ‘double camera_calibrate_calc_reprojection_errors(const std::vector<std::vector<cv::Point3_<float> > >&, const std::vector<std::vector<cv::Point_<float> > >&, const std::vector<cv::Mat>&, const std::vector<cv::Mat>&, const cv::Mat&, const cv::Mat&, std::vector<float>&, bool)’: gstcameracalibrate.cpp:606:11: error: ‘cv::fisheye’ has not been declared cv::fisheye::projectPoints(objectPoints[i], imagePoints2, ^~~~~~~ gstcameracalibrate.cpp: In function ‘bool camera_calibrate_calibrate_full(GstCameraCalibrate*, cv::Size&, cv::Mat&, cv::Mat&, std::vector<std::vector<cv::Point_<float> > >, std::vector<cv::Mat>&, std::vector<cv::Mat>&, std::vector<float>&, double&)’: gstcameracalibrate.cpp:674:15: error: ‘cv::fisheye’ has not been declared rms = cv::fisheye::calibrate(objectPoints, imagePoints, imageSize, ^~~~~~~ Makefile:1155: recipe for target 'libgstopencv_la-gstcameracalibrate.lo' failed make: *** [libgstopencv_la-gstcameracalibrate.lo] Error 1
Conclusion, this element requires a newer OpenCV library then what is shipped by your Debian. We could just bump the requirement, but I guess Debian will refuse to move and drop OpenCV support.
> Conclusion, this element requires a newer OpenCV library then what is > shipped by your Debian. We could just bump the requirement, but I guess > Debian will refuse to move and drop OpenCV support. Debian stretch is the current debian stable release, shipping GStreamer 1.10. They won't upgrade to 1.16 anyway, so I think it's fine to bump the requirement. It would be nice if the newer version required would be in debian sid though. Not sure what's required exactly - https://packages.debian.org/search?keywords=libopencv might give some hints what's available
I'm bumping to 3.0.0, SID got 3.2. https://packages.debian.org/source/sid/opencv
Might require more tuning, but this should bring us closer now: commit d2ee9b16fa7d4d3c2ded648a7606152209813a34 (HEAD -> master) Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Thu Aug 2 11:19:07 2018 -0400 opencv: Fixup include to follow new standard This uses the new path for OpenCV headers. OpenCV now have master headers files per modules, which reduce the amount of required includes. Note that HIGHGUI was included to get the imgcodecs includes, which I fixed, though the master header is missing the C headers, so I included that directly. All the image stuff should be ported to C++ eventually. Finally, this patch also update the header checks to reflect the modules that are really being used. commit 9c91282ba64d1750c9c1500b06f826d565ace859 Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Thu Aug 2 11:03:47 2018 -0400 opencv: Updated to use new header path commit cabf8750984afb662d4401b4316ea96d0bc1ecc8 Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Thu Aug 2 10:39:14 2018 -0400 opencv: Bump requirement to 3.0.0+ And removes all the ifdef code to support the 2.X APIs.
While doing this, I notice that the C++ porting is a bit incomplete. But it should not be too much of an issue atm.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/622.