After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 629244 - [opencv] Add motion detection element
[opencv] Add motion detection element
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: 0.10.23
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2010-09-10 09:18 UTC by Nicola
Modified: 2011-07-28 08:31 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
python opencv motion detection sample (3.56 KB, text/x-python)
2010-09-14 14:27 UTC, Nicola
  Details
fix gstreamer version check (327 bytes, patch)
2011-03-16 18:21 UTC, Robert Jobbagy
none Details | Review
motioncells patch for gst-plugins-bad (123.35 KB, patch)
2011-03-17 15:08 UTC, Robert Jobbagy
none Details | Review
motion cells patch for gst-plugins-bad (91.68 KB, patch)
2011-04-29 11:54 UTC, Robert Jobbagy
none Details | Review
motion cells patch for gst-plugins-bad fixed some warnings (91.88 KB, patch)
2011-05-02 21:57 UTC, Robert Jobbagy
reviewed Details | Review
motion cells patch for gst-plugins-bad fixed all warnings (100.36 KB, patch)
2011-05-03 12:21 UTC, Robert Jobbagy
none Details | Review
add some small fixes (115.44 KB, patch)
2011-05-18 21:48 UTC, Robert Jobbagy
none Details | Review
remove testing line (101.95 KB, patch)
2011-05-21 16:24 UTC, Robert Jobbagy
none Details | Review
accurate timestamp patch (5.89 KB, patch)
2011-06-10 18:55 UTC, Nicola
none Details | Review
fix a valgrind warning (734 bytes, patch)
2011-06-10 18:58 UTC, Nicola
none Details | Review
add a postnomotion property (5.39 KB, patch)
2011-06-10 22:34 UTC, Nicola
none Details | Review
motioncells mutex patch (10.45 KB, patch)
2011-06-21 08:08 UTC, Robert Jobbagy
none Details | Review
big patch (102.92 KB, patch)
2011-06-21 20:23 UTC, Robert Jobbagy
none Details | Review
Cleaned up the patch a little to compile cleanly (86.66 KB, patch)
2011-07-05 19:02 UTC, Roman Gaufman
none Details | Review
this patch contains some important fixes (89.80 KB, patch)
2011-07-05 23:49 UTC, Robert Jobbagy
none Details | Review
gst-plugins-bad-0.10.22-motioncells-v2.patch (87.74 KB, patch)
2011-07-06 03:27 UTC, Roman Gaufman
none Details | Review
motioncells dynamic property change test program (3.28 KB, text/x-csrc)
2011-07-06 07:29 UTC, Robert Jobbagy
  Details
my latest patch (94.83 KB, patch)
2011-07-13 19:51 UTC, Robert Jobbagy
none Details | Review
gst-plugins-bad-0.10.22-motioncells.patch (88.11 KB, patch)
2011-07-15 02:14 UTC, Roman Gaufman
reviewed Details | Review
accurate timestamp patch against latest global patch (6.06 KB, patch)
2011-07-16 14:54 UTC, Nicola
none Details | Review
added miminummotionframes prop as per Roman request (6.93 KB, patch)
2011-07-16 22:17 UTC, Nicola
none Details | Review
use g_mutex and not pthread mutex, solved a lock bug (8.79 KB, patch)
2011-07-17 16:47 UTC, Nicola
reviewed Details | Review
Replace long int with gint64 (2.37 KB, patch)
2011-07-17 20:57 UTC, Nicola
none Details | Review
Proper g_mutex patch (6.36 KB, patch)
2011-07-18 21:30 UTC, Nicola
none Details | Review
the latest big patch what contains all of earlier patches (96.37 KB, patch)
2011-07-19 21:32 UTC, Robert Jobbagy
needs-work Details | Review
gst-plugins-bad-0.10.22-motioncells.patch (latest) (87.61 KB, patch)
2011-07-21 15:42 UTC, Roman Gaufman
none Details | Review
latest motioncells patch (88.12 KB, patch)
2011-07-24 19:34 UTC, Robert Jobbagy
needs-work Details | Review
motioncells new element to detect areas of motion (88.86 KB, patch)
2011-07-27 17:36 UTC, Robert Jobbagy
committed Details | Review
fix a valgrind warning (728 bytes, patch)
2011-07-27 18:42 UTC, Nicola
committed Details | Review
gstmotioncells element dynamic change property test program (35.68 KB, patch)
2011-07-27 20:38 UTC, Robert Jobbagy
committed Details | Review

Description Nicola 2010-09-10 09:18:50 UTC
recently an opencv plugin was added to gstreamer, is possible to include motion detection too? For example would be really useful to post a messagge on the bus when motion is detected,

what do you think about?
Comment 1 Thiago Sousa Santos 2010-09-14 00:54:23 UTC
Yes, that would be nice.

Do you have experience with this? I don't, but I can help with the gstreamer bits and I've used some parts of OpenCV, so I know at least about its datatypes and basic functions :)

Perhaps we could use some cooperation?

I found this on a quicksearch: http://opencv.willowgarage.com/documentation/c/motion_analysis_and_object_tracking.html
Comment 2 Nicola 2010-09-14 14:27:37 UTC
Created attachment 170246 [details]
python opencv motion detection sample

sample motion detection with opencv and python found somewhere on the web
Comment 3 Thiago Sousa Santos 2010-10-28 23:46:41 UTC
Have you tried it? It doesn't look good to me.
Comment 4 Nicola 2010-10-29 06:16:52 UTC
yes, it seems to work on my laptop (python 2.7, opencv 2.1 + python bindings), what's wrong for you?
Comment 5 Robert Jobbagy 2011-01-25 20:49:44 UTC
(In reply to comment #1)
> Yes, that would be nice.
> 
> Do you have experience with this? I don't, but I can help with the gstreamer
> bits and I've used some parts of OpenCV, so I know at least about its datatypes
> and basic functions :)
> 
> Perhaps we could use some cooperation?
> 
> I found this on a quicksearch:
> http://opencv.willowgarage.com/documentation/c/motion_analysis_and_object_tracking.html

I'm working on a motion detection gst-opencv element.
it works in pipeline and has some properties.
it detect motion in real time with low cpu usage.

we need talk about this.
I'm waiting for your response what do you want.

Robert
Comment 6 Stefan Sauer (gstreamer, gtkdoc dev) 2011-02-14 20:24:42 UTC
Robert, it would be nice if you can post your element as a patch against git of gst-plugins-bad for review. Thanks!
Comment 7 Robert Jobbagy 2011-02-15 13:00:53 UTC
Hi guys,

I made two motion detect elements.

first,
motion boundingbox, this detect motion and draw around the moving peoples.
But I need work on it and add some properties and features.

second,
what I developind nowdays motioncells.
You can set a grid on frame (gridx,gridy) and this element compute motion in this cells. The element sen bus msg and draw moving cells if detect motion.

this element has follow properties:

gridx               : Motion Grid X
                        flags: readable, writable
                        Integer. Range: 8 - 32 Default: 10 Current: 10
  gridy               : Motion Grid Y
                        flags: readable, writable
                        Integer. Range: 8 - 32 Default: 10 Current: 10
  sensitivity         : Motion CELLS SENSITIVITY
                        flags: readable, writable
                        Double. Range:               0 -               1 Default:             0.5 Current:             0.5
  threshold           : Threshold of Motion CELLS Number
                        flags: readable, writable
                        Double. Range:               0 -               1 Default:            0.01 Current:            0.01
  display             : Motion Cells visible or not on Current Frame
                        flags: readable, writable
                        Boolean. Default: true Current: true
  datafile            : Location of motioncells data file (empty string means no saving)
                        flags: readable, writable
                        String. Default: "" Current: " "
  motionmaskcoords    : The upper left x, y and lower right x, y coordinates separated with ":", describe a region. Regions separated with ","
                        flags: readable, writable
                        String. Default: "" Current: ""
  motionmaskcellspos  : The line and column idx separated with ":" what cells want we mask-out, describe a cell. Cells separated with ","
                        flags: readable, writable
                        String. Default: "" Current: ""
  cellscolor          : The color of motion cells separated with ","
                        flags: readable, writable
                        String. Default: "" Current: "255,255,0"
  motioncellsidx      : The line and column idx separated with ":", describe a cell. Cells separated with ","
                        flags: readable, writable
                        String. Default: "" Current: ""
  gap                 : GAP between clips
                        flags: readable, writable
                        Integer. Range: 1 - 60 Default: 5 Current: 5
  calculatemotion     : If needs calculate motion on frame you need this property setting true otherwise false
                        flags: readable, writable
                        Boolean. Default: true Current: true
  postallmotion       : Element post bus msg for every motion frame or just motion start and motion stop
                        flags: readable, writable
                        Boolean. Default: false Current: false
  usealpha            : Use or not alpha blending because it cause flicking on frames, it has some bug
                        flags: readable, writable
                        Boolean. Default: false Current: false
  motioncellthickness : Motion Cell Border Thickness, if it's -1 then motion cell will be fill
                        flags: readable, writable
                        Integer. Range: -1 - 5 Default: 1 Current: 1

Sorry about gst-inspect snippet, but it was the easiest. 

we need talk about what do you need in gst-plugins-bad?
what kind of motion detect what do you want? 
My elements works in real time with 1-25 fps, reduced frame size (320x240 and 352 x 288) and at daylight , need some modification that it works fine at night.
I havent time made it yet. 

And I use OpenCv through wrapper class not directly in gst, I wrote C++ class and call functions through wrapper class.

I'm waiting for your response.

Robert
Comment 8 Stefan Sauer (gstreamer, gtkdoc dev) 2011-02-28 12:54:15 UTC
It would be easier to review if people can see the patch and play with it.

I believe people would like to integrate it with the opencv plugin in -bad.

Nicola, any comments from your side?
Comment 9 Nicola 2011-02-28 14:34:52 UTC
I would like to test the patch. Motioncells seems good, Robert can you attach to this bug report, please?
Comment 10 Robert Jobbagy 2011-03-01 20:12:14 UTC
Hi Guys,

Thanks for your interest.

I would like make some small modification and test with my motion element.
And I will send the patch in the near future.
Comment 11 Robert Jobbagy 2011-03-16 18:21:45 UTC
Created attachment 183560 [details] [review]
fix gstreamer version check
Comment 12 Robert Jobbagy 2011-03-16 18:25:50 UTC
Hi Guys,

I check out the latest version form svn and I found two error:

first : 


checking for GST... no
configure: Requested 'gstreamer-0.10 >= 0.10.32.1' but version of GStreamer is 0.10.32
configure: error: no gstreamer-0.10 >= 0.10.32.1 (GStreamer) found
  configure failed

I sent the patch.

second :


make[4]: Entering directory `/home/evil/Test/gst-plugins-bad/gst/colorspace'
  CC     libgstcolorspace_la-gstcolorspace.lo
gstcolorspace.c:81: error: expected ‘}’ before ‘GST_VIDEO_CAPS_r210’
gstcolorspace.c:88: error: expected ‘}’ before ‘GST_VIDEO_CAPS_r210’
make[4]: *** [libgstcolorspace_la-gstcolorspace.lo] Error 1
make[4]: Leaving directory `/home/evil/Test/gst-plugins-bad/gst/colorspace'
make[3]: *** [all] Error 2
make[3]: Leaving directory `/home/evil/Test/gst-plugins-bad/gst/colorspace'
make[2]: *** [colorspace] Error 2
make[2]: Leaving directory `/home/evil/Test/gst-plugins-bad/gst'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/evil/Test/gst-plugins-bad'
make: *** [all] Error 2

I will try fix this in these days, but if anyone have any idea pls share with me.

I integrate my motion detect element to gst-plugins-bad but I couldn't try it because I gave this errors :(

If I fix these bugs and test my motion detect element, I will send my motion detect patch.

Best Regards,

Robert
Comment 13 Robert Jobbagy 2011-03-16 18:27:47 UTC
I forget this sorry :


ii  gstreamer0.10-tools                              0.10.32-1~maverick1                                 Tools for use with GStreamer

So I have installed gstreamer 0.10.32.1 when I gave previous error.
Comment 14 Stefan Sauer (gstreamer, gtkdoc dev) 2011-03-16 22:12:42 UTC
Robert, you will probably need git versions of core and base right now :/
Comment 15 Robert Jobbagy 2011-03-17 15:06:48 UTC
I fixed all errors and tested my motion detect in gst-plugins-bad, it works fine.

I'm beginner with gstreamer, but I hope my motion detect element is useful.

I'm waiting for your advice and response about motion detect element.

I'm beginner with git too, so I made the diff with 

git diff HEAD command and I hope it's good.

I attached the patch.

Enjoy :)
Comment 16 Robert Jobbagy 2011-03-17 15:08:21 UTC
Created attachment 183637 [details] [review]
motioncells patch for gst-plugins-bad

I'm waiting for your response and advice.
Comment 17 Robert Jobbagy 2011-03-23 17:25:29 UTC
we tested it more and found some small bugs, I will resend a new patch on this week.
Comment 18 Robert Jobbagy 2011-04-13 10:25:34 UTC
I fixed bugs what we found but I gave some error when I updated to gst-plugins-bad HEAD version and try compile it with my latest motion detect version.
I will try fix it.
Comment 19 Nicola 2011-04-29 06:21:47 UTC
Hi Robert, any progress on this?

thanks
Nicola
Comment 20 Robert Jobbagy 2011-04-29 06:32:35 UTC
Sorry I'm very busy now, but I'm working on it. I will touch with you soon as possible and send a new patch.
Comment 21 Robert Jobbagy 2011-04-29 11:53:36 UTC
I attached my latest version of my motioncells element.
I made some modification and fixed some small bugs.
I would like make some modification for better motion detection at night in near future.

I'm waiting for your response about this patch.

Best Regards,
Robert
Comment 22 Robert Jobbagy 2011-04-29 11:54:59 UTC
Created attachment 186874 [details] [review]
motion cells patch for gst-plugins-bad

I hope it works better than earlier version :)

Enjoy
Comment 23 Nicola 2011-04-30 15:03:13 UTC
Hi Robert,

I applied your patch against 0.10.21.3 -bad prerelease but the motioncells plugin is not avaliable

Failed to load plugin '/usr/lib/gstreamer-0.10/libgstopencv.so': /usr/lib/gstreamer-0.10/libgstopencv.so: undefined symbol: gst_motioncells_plugin_init

which -bad version need I to use?

thanks
Nicola
Comment 24 Robert Jobbagy 2011-04-30 16:17:18 UTC
Hi Nicola,

Thanks your response.

I use this revision : 


Date:   Thu Apr 28 10:07:04 2011 +0200

decklink: Check for pthread.h and link with -lpthread

Maybe git diff HEAD patch contains too many mess, maybe you need try delete generated doc informations from patch.

I tried this patch on Ubuntu 10.10 and Redhat 6 and it works fine with this gst-plugins-bad revision.

I will review again my patch today and will response to you.

Robert
Comment 25 Robert Jobbagy 2011-04-30 16:26:22 UTC
Nicola,

I tested it now and this is the result : 


gst-inspect motioncells

Factory Details:
  Long name:	motioncells
  Class:	Filter/Effect/Video
  Description:	Performs motion detection on videos and images, providing detected motion cells index via bus messages
  Author(s):	Robert Jobbagy <jobbagy dot robert at gmail dot com>
  Rank:		none (0)

Plugin Details:
  Name:			opencv
  Description:		GStreamer OpenCV Plugins
  Filename:		/usr/local/lib/gstreamer-0.10/libgstopencv.so
  Version:		0.10.21.3
  License:		LGPL
  Source module:	gst-plugins-bad
  Source release date:	2011-04-29 11:32 (UTC)
  Binary package:	GStreamer Bad Plug-ins prerelease
  Origin URL:		Unknown package origin

ls -l /usr/local/lib/gstreamer-0.10/libgstopencv.so
-rwxr-xr-x 1 root root 645337 2011-04-30 18:22 /usr/local/lib/gstreamer-0.10/libgstopencv.so


it seems good I think.

Robert
Comment 26 Nicola 2011-04-30 18:21:40 UTC
sorry, my fault the patch modify the Makefile.am so it is needed to rerun autogen.sh, I'm now testing the element, thanks!
Comment 27 Robert Jobbagy 2011-04-30 18:30:22 UTC
No problem Nicola :)

I'm waiting for your test result and your response :)

Robert
Comment 28 Nicola 2011-04-30 19:36:30 UTC
It seems good and the cpu usage is very low, great work Robert!

some questions/impressions:

1) is the gap property the time of no motion, in seconds, after that a "motion finished" message is posted?
2) if there are massive light intensity changes the element report them as motion, some projects (for example http://www.lavrsen.dk/foswiki/bin/view/Motion/MotionGuideBasicFeatures#Motion_Detection_Settings) have a property to control this situation
3) is the motion detected comparing the last two frames? If yes maybe could be useful,to reduce false detections, to define the number of frames with motion that make a motion event begin

thanks again for your great work,

Nicola
Comment 29 Robert Jobbagy 2011-04-30 19:45:39 UTC
HI Nicola,

Thanks your quickly response.

1, yes, it's right.

2,you are right, I haven't time yet that I solve this situation.
But I have some idea for this, but thanks your link, I will check it.

3,
not exactly.
I build a background model or average frame form previous frames and compare (absdiff) current frame and this avg frame or bg model. Bg model or avg frame is a good thing ;)
This is the way what needs lowest cpu usage and it was my goal.

I'm waiting your feature request or other questions.

Thanks your response.

Robert
Comment 30 Robert Jobbagy 2011-04-30 19:49:13 UTC
I forget this:

2, you can reduce false motion detection what caused change lightining, you set lower sensitivity value :)


Robert
Comment 31 Nicola 2011-05-02 13:58:25 UTC
Hi Robert,

I noted this warnings while compiling motioncells (the element works fine, just warnings):

  CC     libgstopencv_la-gstmotioncells.lo
gstmotioncells.c: In function ‘gst_motioncells_set_property’:
gstmotioncells.c:364:9: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:375:11: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:407:13: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c: In function ‘gst_motioncells_chain’:
gstmotioncells.c:769:3: warning: suggest parentheses around ‘&&’ within ‘||’
gstmotioncells.c:789:3: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:794:3: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:806:4: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:815:4: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:822:3: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:834:4: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:836:4: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:843:4: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:849:5: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:851:5: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:860:5: warning: ISO C90 forbids mixed declarations and code
gstmotioncells.c:864:6: warning: ISO C90 forbids mixed declarations and code
  CXX    libgstopencv_la-motioncells_wrapper.lo
cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-Wmissing-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wold-style-definition" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wnested-externs" is valid for C/ObjC but not for C++
motioncells_wrapper.cpp: In function ‘int searchIdx(int)’:
motioncells_wrapper.cpp:59:39: warning: comparison between signed and unsigned integer expressions
motioncells_wrapper.cpp: In function ‘int motion_cells_free(int)’:
motioncells_wrapper.cpp:93:1: warning: no return statement in function returning non-void
motioncells_wrapper.cpp: In function ‘int searchIdx(int)’:
motioncells_wrapper.cpp:66:1: warning: control reaches end of non-void function
  CXX    libgstopencv_la-MotionCells.lo
cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-Wmissing-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wold-style-definition" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wnested-externs" is valid for C/ObjC but not for C++
MotionCells.cpp: In function ‘uint64_t ntohl64(uint64_t)’:
MotionCells.cpp:22:10: warning: no previous declaration for ‘uint64_t ntohl64(uint64_t)’
MotionCells.cpp: In function ‘uint64_t htonl64(uint64_t)’:
MotionCells.cpp:34:10: warning: no previous declaration for ‘uint64_t htonl64(uint64_t)’

the fixes seems trivial (cast, declare variable and then assign values ecc..) do you need help? I have same spare time the next weekend,

Nicola
Comment 32 Robert Jobbagy 2011-05-02 16:39:43 UTC
Hi Nicola,

I knew it, but I forget fix these warnigs.
I will make it now and send new patch.

But these warnings I cant fix :
I tried, but I cant :( 

gstmotioncells.c: In function ‘gst_motioncells_set_property’:
gstmotioncells.c:364:9: warning: passing argument 1 of ‘gst_element_get_state’
from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct
GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:375:11: warning: passing argument 1 of ‘gst_element_get_state’
from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct
GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:407:13: warning: passing argument 1 of ‘gst_element_get_state’
from incompatible pointer type
/usr/include/gstreamer-0.10/gst/gstelement.h:807:25: note: expected ‘struct
GstElement *’ but argument is of type ‘struct Gstmotioncells 

Can you help me ? 

Thanks,

Robert
Comment 33 Robert Jobbagy 2011-05-02 21:53:58 UTC
I fixed some warnings,but these remainings:

gstmotioncells.c: In function ‘gst_motioncells_set_property’:
gstmotioncells.c:351: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:362: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
gstmotioncells.c:394: warning: passing argument 1 of ‘gst_element_get_state’ from incompatible pointer type
/usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct GstElement *’ but argument is of type ‘struct Gstmotioncells *’
cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-Wmissing-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wold-style-definition" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wnested-externs" is valid for C/ObjC but not for C++
gstmotioncells.c: In function ‘gst_motioncells_chain’:
gstmotioncells.c:760: warning: suggest parentheses around ‘&&’ within ‘||’
cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-Wmissing-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wold-style-definition" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wnested-externs" is valid for C/ObjC but not for C++


if you can please fix these warnings.
Send patch.

Thanks,

Robert
Comment 34 Robert Jobbagy 2011-05-02 21:57:42 UTC
Created attachment 187089 [details] [review]
motion cells patch for gst-plugins-bad fixed some warnings

fixed some warnings
Comment 35 Stefan Sauer (gstreamer, gtkdoc dev) 2011-05-03 08:30:25 UTC
(In reply to comment #33)
> I fixed some warnings,but these remainings:
> 
> gstmotioncells.c: In function ‘gst_motioncells_set_property’:
> gstmotioncells.c:351: warning: passing argument 1 of ‘gst_element_get_state’
> from incompatible pointer type

Just add a GST_ELEMENT() cast to filter:
filter->ret= gst_element_get_state (GST_ELEMENT(filter),
		      &filter->state, &filter->pending, 250 * GST_NSECOND);


> /usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct
> GstElement *’ but argument is of type ‘struct Gstmotioncells *’
> gstmotioncells.c:362: warning: passing argument 1 of ‘gst_element_get_state’
> from incompatible pointer type
> /usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct
> GstElement *’ but argument is of type ‘struct Gstmotioncells *’
> gstmotioncells.c:394: warning: passing argument 1 of ‘gst_element_get_state’
> from incompatible pointer type
> /usr/local/include/gstreamer-0.10/gst/gstelement.h:807: note: expected ‘struct
> GstElement *’ but argument is of type ‘struct Gstmotioncells *’

> cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid
> for C/ObjC but not for C++

you need to declare your variable at the start of a { scope.

> cc1plus: warning: command line option "-Wmissing-prototypes" is valid for
> Ada/C/ObjC but not for C++

we require function prototypes before using the function. Either declare stuff in bottom up way or add prototypes preferably in one block at the top of the source.

> cc1plus: warning: command line option "-Wold-style-definition" is valid for
> Ada/C/ObjC but not for C++
> cc1plus: warning: command line option "-Wnested-externs" is valid for C/ObjC
> but not for C++

> gstmotioncells.c: In function ‘gst_motioncells_chain’:
> gstmotioncells.c:760: warning: suggest parentheses around ‘&&’ within ‘||’

just add extra () braces to make the evaluation order more explicit.

> 
> 
> if you can please fix these warnings.
> Send patch.
> 
> Thanks,
> 
> Robert
Comment 36 Stefan Sauer (gstreamer, gtkdoc dev) 2011-05-03 08:37:36 UTC
Review of attachment 187089 [details] [review]:

A few nitpick. Also please run your code through gst-indent (gstreamer/tools) before making the patch.

::: ext/opencv/gstmotioncells.c
@@ +113,3 @@
+
+GST_BOILERPLATE(Gstmotioncells, gst_motioncells, GstElement, GST_TYPE_ELEMENT);
+

The naming conventions would require to use:
GST_BOILERPLATE(GstMotionCells, gst_motion_cells, GstElement, GST_TYPE_ELEMENT);

probably this is fine too, but I'd prefer the former.
GST_BOILERPLATE(GstMotioncells, gst_motioncells, GstElement, GST_TYPE_ELEMENT);

@@ +189,3 @@
+	g_object_class_install_property(gobject_class, PROP_GRID_X,
+			g_param_spec_int("gridx", "GRID X", "Motion Grid X", GRID_MIN,
+					GRID_MAX, GRID_DEF, G_PARAM_READWRITE));

Add | G_PARAM_STATIC_STRINGS to the flags (e.g. G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS). This save a memcpy for all those parameter strings.
Comment 37 Robert Jobbagy 2011-05-03 08:43:04 UTC
Hi Stefan,

Thanks your advice and guide. 
I'm working that fix these warnings now and I will resend the patch as soon as possible.

Thanks your time.

Robert
Comment 38 Robert Jobbagy 2011-05-03 12:20:35 UTC
Hi again,

I eliminated all warnings.
And format the source files with (from GST website): 

indent \
  --braces-on-if-line \
  --case-brace-indentation0 \
  --case-indentation2 \
  --braces-after-struct-decl-line \
  --line-length80 \
  --no-tabs \
  --cuddle-else \
  --dont-line-up-parentheses \
  --continuation-indentation4 \
  --honour-newlines \
  --tab-size8 \
  --indent-level2

Add | G_PARAM_STATIC_STRINGS flags
and made this rename:
GST_BOILERPLATE(Gstmotioncells, gst_motioncells, GstElement,
GST_TYPE_ELEMENT);


I hope every things is fine now :)

attached the patch

Robert
Comment 39 Robert Jobbagy 2011-05-03 12:21:34 UTC
Created attachment 187119 [details] [review]
motion cells patch for gst-plugins-bad fixed all warnings

I fixed warnings what Stefan report me.
Comment 40 Nicola 2011-05-06 19:12:00 UTC
Hi Robert,

what do you think about an option to post on the bus the video frame with the motion? The motion will be outlined based on the "display" property.

This way it is easy to save the frame that triggered the motion if one need this
Comment 41 Robert Jobbagy 2011-05-06 19:18:24 UTC
Hi Nicola,

How can I post on the bus the video frame ? 
Do you can show me an example or give me more explanation ?
Comment 42 Nicola 2011-05-06 21:41:37 UTC
There is an example in the camerabin code:

http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/camerabin/gstcamerabin.c#n1911

this could be useful since the motion events posted on the bus are async, so if one capture a frame when this bus message is received the captured frame may differ from the motion frame, so the idea is to send the motion frame on the bus if a property is set
Comment 43 Robert Jobbagy 2011-05-06 21:46:26 UTC
I will try make it, but I'm very busy until next Friday.
So I can begin this next weekend.

Robert
Comment 44 Stefan Sauer (gstreamer, gtkdoc dev) 2011-05-09 08:09:07 UTC
(In reply to comment #41)
> Hi Nicola,
> 
> How can I post on the bus the video frame ? 
> Do you can show me an example or give me more explanation ?

Nicola, usualy bus message with analysis result will contain the timestamp of the frame they related to. One could also send the result as a event downstream and flag it so the the sink will emit the message with the same data as the event. This way elements and the app will get the data.
Sending the whole buffer might create quite a bit of overhead, if would definitely be great if this would be at least optional.
Comment 45 Nicola 2011-05-09 08:46:26 UTC
Stefan,

can you provide an example of a gstreamer element that send an event downstream?
Comment 46 Stefan Sauer (gstreamer, gtkdoc dev) 2011-05-09 09:28:25 UTC
(In reply to comment #45)
> Stefan,
> 
> can you provide an example of a gstreamer element that send an event
> downstream?

It should be used more often (e.g. in level and spectrum):
./gst-plugins-base/gst/playback/gstplaybin2.c:2813:        event = gst_event_new_sink_message (msg);
Comment 47 Robert Jobbagy 2011-05-18 21:48:26 UTC
Created attachment 188081 [details] [review]
add some small fixes

datafile property simplicity,add datafile mutex,add more error handling
Comment 48 Robert Jobbagy 2011-05-18 21:49:29 UTC
Hi Nicola , 

please test this patch too and 
I'm waiting your response.

Thanks,

Robert
Comment 49 Nicola 2011-05-19 21:40:39 UTC
Thanks Robert, I'll test this weekend. Is this a bug fix only release?
Comment 50 Robert Jobbagy 2011-05-20 07:18:03 UTC
yes,because I dont know that I made what you said or dont, because Stefan said it cause overhead. But if you give me guide what will be useful in this patch, I try make it when I have time.

Robert
Comment 51 Nicola 2011-05-20 20:28:15 UTC
Hi Robert,

after using the element for a while (about 30 minutes) I get a segfault:

Got message #8705 from element "motioncells0" (element): motion, motion_finished=(string)2848;
Got message #9795 from element "motioncells0" (element): motion, motion_cells_indices=(string)"0:0\,0:1\,0:2", motion_begin=(string)3241;
Got message #9922 from element "motioncells0" (element): motion, motion_finished=(string)3253;
Caught SIGSEGV accessing address (nil)

I'll try to investigate this with gdb (I'm using opencv 2.1), additionaly ps show a slow but costant memory increase I'll investigate this with valgrind,

the last patch give the following compile warnings:

gstmotioncells.c: In function ‘gst_motion_cells_chain’:
gstmotioncells.c:734:5: warning: ISO C90 forbids mixed declarations and code
  CXX    libgstopencv_la-motioncells_wrapper.lo
motioncells_wrapper.cpp: In function ‘char* getMotionCellsIdx(int)’:
motioncells_wrapper.cpp:57:16: warning: address of local variable ‘p_str’ returned

Nicola
Comment 52 Robert Jobbagy 2011-05-21 06:54:04 UTC
Hi Nicola,

Thanks your response.

I will fix this issue today and send patch.

Sorry about this.

Robert
Comment 53 Robert Jobbagy 2011-05-21 16:23:54 UTC
Sorry I was very tired and I forgot remove a testing line from code, and it caused seg fault and memleak :(

I tested this patch with gdb and valgrind , the pipeline ran more hours and it worked fine. 

Pls if you can test it with multiple pipeline and in other extreme situation.


Thanks your help and your time.

Robert
Comment 54 Robert Jobbagy 2011-05-21 16:24:47 UTC
Created attachment 188301 [details] [review]
remove testing line
Comment 55 Nicola 2011-05-21 20:42:30 UTC
Thanks Robert,

I'm retesting just now, I noticed that some parts of the patch fail to apply against bad 0.10.22 release (still compile and motioncells works):

root@e6500:/usr/local/src/gst-plugins-bad-0.10.22# patch -p 1 < gst-plugins-bad-motioncells.patch6 
patching file configure.ac
Hunk #1 FAILED at 52.
1 out of 2 hunks FAILED -- saving rejects to file configure.ac.rej
patching file docs/plugins/gst-plugins-bad-plugins.args
patching file docs/plugins/gst-plugins-bad-plugins.hierarchy
Hunk #2 FAILED at 12.
Hunk #3 succeeded at 66 (offset -5 lines).
Hunk #4 FAILED at 98.
Hunk #5 succeeded at 455 (offset -3 lines).
2 out of 5 hunks FAILED -- saving rejects to file docs/plugins/gst-plugins-bad-plugins.hierarchy.rej
patching file docs/plugins/gst-plugins-bad-plugins.interfaces
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file docs/plugins/gst-plugins-bad-plugins.interfaces.rej
patching file docs/plugins/gst-plugins-bad-plugins.prerequisites
patching file ext/opencv/Makefile.am
patching file ext/opencv/MotionCells.cpp
patching file ext/opencv/MotionCells.h
patching file ext/opencv/analitics.h
patching file ext/opencv/gstmotioncells.c
patching file ext/opencv/gstmotioncells.h
patching file ext/opencv/gstopencv.c
patching file ext/opencv/motioncells_wrapper.cpp
patching file ext/opencv/motioncells_wrapper.h

are these errors safe to ignore?
Comment 56 Robert Jobbagy 2011-05-21 20:50:45 UTC
I dont know , because I couldn't update my version to HEAD I gave some errors.


~/gstreamer/gst-plugins-bad$ git pull
remote: Counting objects: 123, done.
remote: Compressing objects: 100% (85/85), done.
remote: Total 85 (delta 65), reused 0 (delta 0)
Unpacking objects: 100% (85/85), done.
From git://anongit.freedesktop.org/gstreamer/gst-plugins-bad
   58ee65f..d25908c  master     -> origin/master
You are not currently on a branch, so I cannot use any
'branch.<branchname>.merge' in your configuration file.
Please specify which remote branch you want to use on the command
line and try again (e.g. 'git pull <repository> <refspec>').
See git-pull(1) for details.

how can I solve this ? 

and I have more patches in my working copy version and when I make a diff I need remove other patches from diff.

gstreamer core and base up-to-date.

Thanks your help.


Robert
Comment 57 Nicola 2011-05-23 13:53:30 UTC
Hi Robert,

is there a reason to limit the gridx and gridy range to 8-32? A 3x3 grid will give bad detection performance?

thanks
Nicola
Comment 58 Robert Jobbagy 2011-05-23 14:27:35 UTC
Hi Nicola,

I don't know , I never try it.
I think this is the right range.

Robert
Comment 59 Nicola 2011-05-24 18:17:52 UTC
Hi,

using gridx =9, gridy=9 and motionmaskcellspos="0:0,0:1,0:2,1:0,1:1,1:2,2:0,2:1,2:2,0:3,0:4,0:5,1:3,1:4,1:5,2:3,2:4,2:5,0:6,0:7,0:8,1:6,1:7,1:8,2:6,2:7,2:8"

I can see motion outlined in zone 2:7, 2:8

a 9x9 grid should be something similar to this:

0:0 0:1 0:2 0:3 0:4 0:5 0:6 0:7 0:8
1:0 1:1 1:2 1:3 1:4 1:5 1:6 1:7 1:8
....

and so on,

so seems there is something wrong with motionmaskcellspos parameter,

Am I misunderstanding something?

Nicola
Comment 60 Robert Jobbagy 2011-05-24 18:43:54 UTC
Hi,

you can use motionmaskcellspos property if you want masking-out some area from frame , if dont want detect motion in this area what you set in motionmaskcellspos.

And if you check motioncellsidx what give back element and you see an idx what you want mask-out then something wrong in the element.

example 

if you set 

motionmaskcellspos="2:3,2:4,2:5"

and element give back motioncellsidx in bus msg motion_cells_indices="0:0,0:1,2:3"  then it's wrong because the element I didn't want detect motion in 2:3 cell.

I hope it's clear.

Robert
Comment 61 Nicola 2011-05-24 19:03:58 UTC
Yes is what I'm saying, I have:

gchar *mask;
g_object_get(G_OBJECT(detector),"motionmaskcellspos",&mask,NULL);

and mask content is as follow:

0:0,0:1,0:2,1:0,1:1,1:2,2:0,2:1,2:2,0:3,0:4,0:5,1:3,1:4,1:5,2:3,2:4,2:5,0:6,0:7,0:8,1:6,1:7,1:8,2:6,2:7,2:8

then I get this message on the bus:

"motion, motion_cells_indices=(string)"2:6\,3:5\,3:6\,3:7\,4:6\,5:5\,5:6\,6:6", motion=(string)5;"

as you can see 2:6 should not be there
Comment 62 Nicola 2011-05-24 19:10:42 UTC
the problem is more evident if you set sensitivity=0.9
Comment 63 Robert Jobbagy 2011-05-24 19:41:54 UTC
I'm very busy now but I'll try find the solution this problem on this weekend or next week. Pls if you have more time test it and tell me more details about any issues what you find.

Thanks your time Nicola.

Robert
Comment 64 Nicola 2011-05-25 06:17:17 UTC
You can easily reproduce the issue with a pipeline like this:

gst-launch -m v4l2src ! video/x-raw-yuv,width=320,height=240,framerate=5/1 ! videorate ! video/x-raw-yuv,width=320,height=240,framerate=3/1 ! ffmpegcolorspace ! motioncells postallmotion=true gridx=9 gridy=9 motionmaskcellspos="0:0,0:1,0:2,1:0,1:1,1:2,2:0,2:1,2:2,0:3,0:4,0:5,1:3,1:4,1:5,2:3,2:4,2:5,0:6,0:7,0:8,1:6,1:7,1:8,2:6,2:7,2:8" sensitivity=0.9 ! ffmpegcolorspace ! xvimagesink

after a short while I see:

"motioncells0" (element): motion, motion_cells_indices=(string)"2:5\,3:3\,3:4\,3:5\,3:6\,4:3\,4:4\,4:5\,4:6\,5:3\,5:4\,5:5\,5:6\,6:3\,6:4\,6:5\,6:6\,7:4\,7:5\,7:6\,7:7\,8:5\,8:6\,8:7", motion=(string)5;

where 2:5 is a masked cell,

thanks
Nicola
Comment 65 Nicola 2011-05-25 06:24:25 UTC
The "motioncellsidx" property instead seems to work correctly, I'll do some more tests this evening or this weekend,

thanks
Nicola
Comment 66 Robert Jobbagy 2011-05-25 19:04:02 UTC
Hi Nicola,

thanks your help.

and why use it in your pipeline ?

video/x-raw-yuv,width=320,height=240,framerate=5/1 ! videorate ! video/x-raw-yuv,width=320,height=240,framerate=3/1 

I think enough the first or second scale and framerate settings

anyway I can reproduce the issue and I try detect the location of bug and fix it.


Robert
Comment 67 Nicola 2011-05-25 21:07:17 UTC
Thanks Robert,

yes the two capsfilter are redundant I mixed two pipeline ...

as future improvement I think would be useful a property to fill the masked cell with a color, what do you think about?


Nicola
Comment 68 Robert Jobbagy 2011-05-26 06:34:07 UTC
Nicola , 

I made the usealpha property that motion cells filled with alpha blending 
but something doesnt works fine , you can see it .cpp //FIXME alpha has bug

I haven't time fix it yet, but I will.

Robert
Comment 69 Robert Jobbagy 2011-05-26 06:35:10 UTC
but you can use this : 


motioncellthickness : Motion Cell Border Thickness, if it's -1 then motion cell will be fill
                        flags: readable, writable
                        Integer. Range: -1 - 5 Default: 1 Current: 1



Robert
Comment 70 Nicola 2011-05-26 06:52:21 UTC
Yes Robert I saw motioncellthickness property but I think it is useful to fill the masked cells (the ones a user exclude from the detection) and not the cells where motion is detected

Nicola
Comment 71 Robert Jobbagy 2011-05-26 07:22:34 UTC
Ok, I understand you now, I misunderstand earlier.

Robert
Comment 72 Robert Jobbagy 2011-05-30 06:57:34 UTC
I will make it, maybe this week(end) or next week.

Robert
Comment 73 Nicola 2011-06-01 21:35:00 UTC
Hi Robert,

I did a small modification to the file gstmotioncells.c:

I changed line 900 to:

s = gst_structure_new ("motion", "motion_cells_indices", G_TYPE_STRING,
            detectedmotioncells, "motion_begin", G_TYPE_UINT64,
            GST_BUFFER_TIMESTAMP (buf), NULL);

this way when there is a motion begin the message you post on the bus contains the exact buffer timestamp, with this information you can get the exact frame where motion is detected, here is a sample bus message

"motioncells0" (element): motion, motion_cells_indices=(string)2:6, motion_begin=(guint64)2165376560;

please note motion_begin=(guint64)2165376560

in the previous version the bus message tell us that there is motion at second 21 not precise.

My opinion is that motion_begin_timestamp last_motion_timestamp ecc.. should be changed from (long int) to (guint64) so you have an exact information and not an approximated one.

This change is enough for my request at comment 40 too. If you agree with the change I can make a proper patch,

thanks
Nicola
Comment 74 Nicola 2011-06-02 10:49:13 UTC
Hi again,

valgrind show the following warnings:

==4516== Thread 1:
==4516== 12 bytes in 1 blocks are definitely lost in loss record 484 of 4,345
==4516==    at 0x4C28FAC: malloc (vg_replace_malloc.c:236)
==4516==    by 0x616EA62: g_malloc (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.2800.6)
==4516==    by 0xF52B68A: ??? (gstmotioncells.c:782)
==4516==    by 0x52E2DB1: gst_pad_chain_data_unchecked (gstpad.c:4247)
==4516==    by 0x52E362A: gst_pad_push_data (gstpad.c:4479)
==4516==    by 0x52E642C: gst_pad_push (gstpad.c:4704)
==4516==    by 0x506A3C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4516==    by 0x52E2DB1: gst_pad_chain_data_unchecked (gstpad.c:4247)
==4516==    by 0x52E362A: gst_pad_push_data (gstpad.c:4479)
==4516==    by 0x52E642C: gst_pad_push (gstpad.c:4704)
==4516==    by 0xEA61208: gst_jpeg_dec_chain (gstjpegdec.c:1544)
==4516==    by 0x52E2DB1: gst_pad_chain_data_unchecked (gstpad.c:4247)

==4359== 230,424 bytes in 1 blocks are possibly lost in loss record 4,342 of 4,344
==4359==    at 0x4C28FAC: malloc (vg_replace_malloc.c:236)
==4359==    by 0x104BEB24: cv::fastMalloc(unsigned long) (in /usr/lib/libcxcore.so.2.1.0)
==4359==    by 0x104F2447: cvCreateData (in /usr/lib/libcxcore.so.2.1.0)
==4359==    by 0x10504E92: cvCloneImage (in /usr/lib/libcxcore.so.2.1.0)
==4359==    by 0xF52EDF2: MotionCells::performDetectionMotionCells(_IplImage*, double, double, int, int, long, bool, bool, int, motionmaskcoordrect*, int, motioncellidx*, cellscolor*, int, motioncellidx*, long, char*, bool, int) (MotionCells.cpp:253)
==4359==    by 0xF52C747: perform_detection_motion_cells (motioncells_wrapper.cpp:37)
==4359==    by 0xF52B8D2: ??? (gstmotioncells.c:827)
==4359==    by 0x52E653B: gst_pad_push (gstpad.c:4684)
==4359==    by 0x506A3C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4359==    by 0x52E2DB1: gst_pad_chain_data_unchecked (gstpad.c:4247)
==4359==    by 0x52E362A: gst_pad_push_data (gstpad.c:4479)
==4359==    by 0x52E642C: gst_pad_push (gstpad.c:4704)
Comment 75 Robert Jobbagy 2011-06-05 17:16:15 UTC
HI Nicola,


I always run valgrind and check memory leaks but usually log file is empty ... :(

I tried this command :

valgrind --log-file=/var/tmp/motioncells.out --track-fds=yes --leak-check=full --show-reachable=yes --tool=memcheck gst-launch v4l2src ! video/x-raw-yuv,width=320,height=240,framerate=5/1 ! ffmpegcolorspace ! motioncells ! ffmpegcolorspace ! xvimagesink

but it didnt give any warning or mem leak, it made an empty log file.

what is your command what found this leak ? 

Anyway I check 253 line and near lines but I didnt found any bug yet.


Robert
Comment 76 Robert Jobbagy 2011-06-05 18:52:19 UTC
And pls post your small patch for timestamp stuff.

Thanks.

Robert
Comment 77 Stefan Sauer (gstreamer, gtkdoc dev) 2011-06-05 19:05:36 UTC
(In reply to comment #75)
> HI Nicola,
> 
> 
> I always run valgrind and check memory leaks but usually log file is empty ...
> :(
> 
> I tried this command :
> 
> valgrind --log-file=/var/tmp/motioncells.out --track-fds=yes --leak-check=full
> --show-reachable=yes --tool=memcheck gst-launch v4l2src !
> video/x-raw-yuv,width=320,height=240,framerate=5/1 ! ffmpegcolorspace !
> motioncells ! ffmpegcolorspace ! xvimagesink
> 
> but it didnt give any warning or mem leak, it made an empty log file.
> 

A few comments:
- gst-launch spawns gst-launch-0.10, when valgrinding call gst-launch-0.10 directly or use --trace-children=yes.
- also export some env-vars for good reports. I use a shell alias:
alias vg_memcheck='G_SLICE=always-malloc G_DEBUG=gc-friendly GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 valgrind --tool=memcheck --leak-check=full --leak-resolution=high --trace-children=yes --num-callers=20 -v'
Comment 78 Robert Jobbagy 2011-06-05 19:13:26 UTC
I found the right valgrind command , I forget add a flag to valgrind,

I see now the possibly lost in log file, but I dont know what's wrong.

Because I release m_pprevFrame and m_pcurFrame too.

if you have any idea pls share with me.


Stefan thanks your response:

I use this now : 


valgrind --log-file=/var/tmp/motioncells.out --track-fds=yes --leak-check=full --show-reachable=yes --tool=memcheck --trace-children=yes gst-launch v4l2src ! video/x-raw-yuv,width=320,height=240,framerate=5/1 ! ffmpegcolorspace ! motioncells ! ffmpegcolorspace ! xvimagesink


Thanks,

Robert
Comment 79 Stefan Sauer (gstreamer, gtkdoc dev) 2011-06-05 19:39:56 UTC
Robert, it's better to call gst-launch-0.10 directly and not use --trace-children=yes. Then you don't get eveual reports for the gst-launch wrapper.

Also definitely do G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind ...
(the GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 you only need to c++ code).
Comment 80 Robert Jobbagy 2011-06-05 19:48:48 UTC
Stefan I never use this what you post.

I need put this :

alias vg_memcheck='G_SLICE=always-malloc G_DEBUG=gc-friendly
GLIBCPP_FORCE_NEW=1 GLIBCXX_FORCE_NEW=1 valgrind --tool=memcheck
--leak-check=full --leak-resolution=high --trace-children=yes --num-callers=20
-v'

in my .bashrc ?

and how can I use it ?

Thanks your help.

Robert
Comment 81 Stefan Sauer (gstreamer, gtkdoc dev) 2011-06-05 19:54:57 UTC
Robert,

yes you can add the line to your bashrc and then use valgrind by just running

vg_memcheck gst-launch-0.10 v4l2src !
video/x-raw-yuv,width=320,height=240,framerate=5/1 ! ffmpegcolorspace !
motioncells ! ffmpegcolorspace ! xvimagesink

But you can also just use the command as I've shown there. You can prefix any command in a shell with environment variable for just this command. E.g.
HELLO="robert" /bin/echo $HELLO

The env-vars mentioned in my previous comment are meant to make glib or libc more valgrind friendly.
Comment 82 Robert Jobbagy 2011-06-05 20:01:47 UTC
Thanks Stefan,

it works fine just I mistype something.


Robert
Comment 83 Nicola 2011-06-06 04:34:54 UTC
Thanks Robert,

I'll retest the timestamp patch and I'll post it the next weekend,

Nicola
Comment 84 Nicola 2011-06-10 18:55:33 UTC
Created attachment 189667 [details] [review]
accurate timestamp patch

With the attached patch the timestamp posted on the bus are guint64 and not more string rounded to seconds. This way you can extract the exact buffer with the motion event. I hope this patch doesn't introduce regressions
Comment 85 Nicola 2011-06-10 18:58:29 UTC
Created attachment 189668 [details] [review]
fix a valgrind warning
Comment 86 Nicola 2011-06-10 22:34:47 UTC
Created attachment 189698 [details] [review]
add a postnomotion property

This property, if enabled, post a no_motion event on the bus if for N seconds nothing is detected. I think is useful if you have to syncronize the analysis results with another app. Infact if for some reason you miss a motion_finished the state will be not in sync until the next motion_begin event
Comment 87 Robert Jobbagy 2011-06-20 08:31:03 UTC
Hi Guys,

I'm here again so what's wrong with this element yet ?

What can I fix ?

Robert
Comment 88 Nicola 2011-06-20 09:02:03 UTC
Hi Robert,

please take a look at the patches I posted, particularly the one at comment 85 (fix a memory leak).

I think are still open:

1) bug reported at comment 64
2) the following compile warning:

gstmotioncells.c: In function ‘gst_motion_cells_chain’:
gstmotioncells.c:734:5: warning: ISO C90 forbids mixed declarations and code
  CXX    libgstopencv_la-motioncells_wrapper.lo

(the line number may be different if you applied my patches).

3) look at the second valgrind warning at comment 74, however this is a "possibly lost" and not a "definitely lost" as the one fixed by patch at comment 85, could be something normal (maybe an opencv.supp file may placate valgrind)

If you need help I can look at these on the next weekend,

apart these three remaining (minor) issues I'm very happy with this element

Nicola
Comment 89 Robert Jobbagy 2011-06-21 08:06:13 UTC
Hi Nicola,

I checked you patches.

comment 85 is ok, but I changed motioncellscolor pointer
comment 64 I'll try fix it as soon as possible

I made some modification with mutex, I attached a patch, pls check it.

If I fix any bug I'll send the patch.

Robert
Comment 90 Robert Jobbagy 2011-06-21 08:08:16 UTC
Created attachment 190345 [details] [review]
motioncells mutex patch

I found a bug with set property and add mutex locks
Comment 91 Robert Jobbagy 2011-06-21 19:59:05 UTC
Hi Nicola,

I fixed comment 64 bug.

but I don't know what's wrong with this : 


gstmotioncells.c: In function ‘gst_motion_cells_chain’:
gstmotioncells.c:751: warning: ISO C90 forbids mixed declarations and code


/* this function handles the link with other elements */
static gboolean
gst_motion_cells_set_caps (GstPad * pad, GstCaps * caps)
{
  GstMotioncells *filter;
line 751  GstPad *otherpad;
  gint width, height;
  GstStructure *structure;
  int numerator, denominator;


I'll attache a big patch what contains all earlier patches.

Pls test this, if you have time.

Thanks your help.

Robert
Comment 92 Robert Jobbagy 2011-06-21 20:23:52 UTC
Created attachment 190390 [details] [review]
big patch

it contains all previous patches, I hope
Comment 93 Nicola 2011-06-22 06:39:11 UTC
Thanks Robert, I'll take a look at the last patch on weekend,

Nicola
Comment 94 Nicola 2011-06-23 20:25:43 UTC
your last patch apply against 0.10.22 but doesn't build anymore for me:

make[3]: Entering directory `/usr/local/src/gst-plugins-bad-0.10.22/ext/opencv'
make[3]: *** No rule to make target `gstmotioncells.c', needed by `libgstopencv_la-gstmotioncells.lo'.  Stop.
make[3]: Leaving directory `/usr/local/src/gst-plugins-bad-0.10.22/ext/opencv'
make[2]: *** [opencv] Error 2
make[2]: Leaving directory `/usr/local/src/gst-plugins-bad-0.10.22/ext'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/local/src/gst-plugins-bad-0.10.22'
make: *** [all] Error 2

I'll try to understand why as soon as I have some spare time
Comment 95 Nicola 2011-06-23 20:51:30 UTC
ok here is the problem:

patching file ext/opencv/MotionCells.cpp
patch: **** malformed patch at line 1640: diff --git a/ext/opencv/MotionCells.h b/ext/opencv/MotionCells.h
Comment 96 Roman Gaufman 2011-07-01 03:52:56 UTC
How does motioncells compare to something like this? < https://gitorious.org/gstreamer-motion-plugin
Comment 97 Robert Jobbagy 2011-07-01 06:47:27 UTC
I don't know this element , but I will check it.

thanks your link
Comment 98 Nicola 2011-07-01 07:00:39 UTC
I quickly looked at that element code and it seems to lack a lot of features compared to motioncells, however I'll compile and test it as soon as I have some time and I'll give my impressions
Comment 99 Roman Gaufman 2011-07-04 23:08:40 UTC
Trying to use the latest "big patch", I'm getting:

gst-plugins-bad-0.10.22 $ patch -p1 < ../gst-plugins-bad-motioncells.patch 
patching file configure.ac
Hunk #1 FAILED at 52.
1 out of 2 hunks FAILED -- saving rejects to file configure.ac.rej
patching file docs/plugins/gst-plugins-bad-plugins.args
patching file docs/plugins/gst-plugins-bad-plugins.hierarchy
Hunk #2 FAILED at 12.
Hunk #3 succeeded at 34 (offset -5 lines).
Hunk #4 FAILED at 61.
Hunk #5 succeeded at 224 (offset -3 lines).
2 out of 5 hunks FAILED -- saving rejects to file docs/plugins/gst-plugins-bad-plugins.hierarchy.rej
patching file docs/plugins/gst-plugins-bad-plugins.interfaces
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file docs/plugins/gst-plugins-bad-plugins.interfaces.rej
patching file docs/plugins/gst-plugins-bad-plugins.prerequisites
patching file ext/opencv/Makefile.am
patching file ext/opencv/MotionCells.cpp
patch: **** malformed patch at line 1640: diff --git a/ext/opencv/MotionCells.h b/ext/opencv/MotionCells.h

Anyone?
Comment 100 Robert Jobbagy 2011-07-05 06:24:21 UTC
I made this patch same what earlier patches, but I will send new patch on this week. I made some small fixes.

Robert
Comment 102 Roman Gaufman 2011-07-05 19:02:10 UTC
Created attachment 191355 [details] [review]
Cleaned up the patch a little to compile cleanly
Comment 103 Roman Gaufman 2011-07-05 19:03:26 UTC
Also have a homebrew formula and going to be posting all my experimentations with this patch here: https://github.com/hackeron/gstreamer_experiments
Comment 104 Robert Jobbagy 2011-07-05 19:15:34 UTC
Hi,

Thanks your response, I'm making now the new patch for gst-motioncells.
I'll post it in this evening.
If you wait a little bit you can try the new patch.

Robert
Comment 105 Robert Jobbagy 2011-07-05 23:49:52 UTC
Created attachment 191375 [details] [review]
this patch contains some important fixes

- sensitivity 1 bug fix
- motionmaskidx bug fix (reported Nicola)
- alpha blending bug fix
- some bus msg bug fix
- add some extra check
- when grid decrease and motioncellsidx or/and motionmaskidx will be out of bound crash bug fix
- empty idx list bug fix
- small bug fixes
Comment 106 Robert Jobbagy 2011-07-05 23:53:26 UTC
Hi Guys,

Thanks Nicola bug reports and lot of tests and Roman thanks your patch.
Here is the latest patch for motion cells what contains some bug fixes.
I hope it will be useful.

Nicola and others please test it and I'm waiting for your feedback :)

Enjoy.

Robert.

P.S: I hope you can use this patch easy.
Comment 107 Roman Gaufman 2011-07-06 03:27:11 UTC
Created attachment 191382 [details] [review]
gst-plugins-bad-0.10.22-motioncells-v2.patch

Based on "this patch contains some important fixes" by Robert Jobbagy - I couldn't get the patch to apply on gst-plugins-bad-0.10.22 - modified it a bit to make it apply cleanly :)
Comment 108 Robert Jobbagy 2011-07-06 07:00:41 UTC
Hi Roman,

Thanks your patch, I don't understand what's wrong with the patch.

I made this with git diff HEAD command just like all earlier patches ...

Thanks again.

Robert
Comment 109 Robert Jobbagy 2011-07-06 07:29:07 UTC
Created attachment 191389 [details]
motioncells dynamic property change test program

you can run motioncells with this program and can change properties in runtime.
Comment 110 Robert Jobbagy 2011-07-06 07:30:02 UTC
I uploaded a test program, I hope it will be useful that you can make more and flexible tests.

Robert
Comment 111 Robert Jobbagy 2011-07-06 07:40:51 UTC
Nicola,

Sorry, your comment 84 and comment 86 patches missing my latest patch :(
I just now realize it.

Robert
Comment 112 Roman Gaufman 2011-07-06 17:18:40 UTC
@Robert

Maybe because I'm patching against the release tarball source? - not sure.

Comment 86 patch appears to be included in your latest patch, I'm seeing POST_NO_MOTION.

I'm also seeing parts of the patch in Comment 84 included - for instance I'm seeing:
(gint64) (GST_BUFFER_TIMESTAMP (buf) / GST_MSECOND) -

But then I'm also seeing:
filter->diff_timestamp = (long int) (GST_BUFFER_TIMESTAMP(buf)/GST_MSECOND);
In the patch it is:
filter->diff_timestamp = (gint64) (GST_BUFFER_TIMESTAMP (buf) / GST_MSECOND);

Is there any reason to use a long int instead of gint64?
Comment 113 Robert Jobbagy 2011-07-06 17:26:55 UTC
Roman,

you are right with Comment 86 , sorry I was very tired and I'm confused.

it's my mistake long int , you can use gint64 too :)

Robert
Comment 114 Robert Jobbagy 2011-07-13 19:48:46 UTC
Hi Guys,

I checkout the latest gst-plugins-bad and use Roman patch.
And after I made some little modification.

Here is my latest patch.

Enjoy and I'm waiting your response :)

Robert
Comment 115 Robert Jobbagy 2011-07-13 19:51:14 UTC
Created attachment 191920 [details] [review]
my latest patch

- unused element ides store in a vector and reuse it, if it's necessary
- set empty datafile fix 
- some code simplicity
Comment 116 Roman Gaufman 2011-07-15 01:58:47 UTC
I'm trying to compile the latest patch, I'm getting: 

PIC -o .libs/libgstopencv_la-gstmotioncells.o
gstmotioncells.c: In function 'gst_motion_cells_init':
gstmotioncells.c:322: error: 'PTHREAD_MUTEX_RECURSIVE_NP' undeclared (first use in this function)
gstmotioncells.c:322: error: (Each undeclared identifier is reported only once
gstmotioncells.c:322: error: for each function it appears in.)
make[3]: *** [libgstopencv_la-gstmotioncells.lo] Error 1
make[2]: *** [opencv] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any ideas?
Comment 117 Roman Gaufman 2011-07-15 02:14:48 UTC
Created attachment 192016 [details] [review]
gst-plugins-bad-0.10.22-motioncells.patch

Latest version cleaned up to patch against 0.10.22 and using PTHREAD_MUTEX_RECURSIVE instead of PTHREAD_MUTEX_RECURSIVE_NP (as I understand it should work the same?)
Comment 118 Robert Jobbagy 2011-07-15 06:55:59 UTC
Roman,

Comment 116 I dont know why doesnt compile it.
I always try recompile gst-plugins-bad with my patch.
And it worked fine.

and check this :

http://forums.minegoboom.com/viewtopic.php?t=2076

comment 117 thanks your cleaner work :)

but if you share with me what did you do with my patch ,maybe I can do it too :)
And I can send a fine patch :)

Robert
Comment 119 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-15 07:04:48 UTC
Robert, please use the threading function from glib (gthread). This will make the plugin portable.

Please also mark old version of the patch as obsolete, where it applies. That helps to see what is still under discussion.

In order to get this in, I'd suggest to get the build related things fixed, agree that the api and functionality is sort of whats desired and apply the patch. Performance and additional features can be addressed in followup patches.
Comment 120 Robert Jobbagy 2011-07-15 07:43:37 UTC
Stefan,

Thanks your guide.
I'm beginner with bugzilla.
I marked all older patch as obsolete except Nicola's patches.

Okay,it's my mistake that I replace gthread with pthread.


Robert
Comment 121 Roman Gaufman 2011-07-15 12:26:09 UTC
Robert, 

Try to download http://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-0.10.22.tar.bz2 and apply your patch to it - you will see it doesn't apply :(

I basically went through your patch, made the minimum changes required manually (for instance why are you re-arranging docs/plugins/gst-plugins-bad-plugins.hierarchy?) - maybe after you do autogen and the rest, that moves some stuff around? - not sure.

Just using your patch with an MPEG4 stream from a Panasonic IP camera, it seems to falsely detect motion everywhere once in a while when the image goes blocky for a frame or two. Can you add a feature "minimum_motion_frames" that will let you set the minimum consecutive frames of motion before a motion event is triggered? -- this is also useful for when a dual lens or dual sensor camera switches from day to night time mode.

Roman
Comment 122 Robert Jobbagy 2011-07-15 12:50:49 UTC
Hi Roman,

Thanks your feedback.
I always use gst-plugins-bad HEAD version from git repo.
In last time I used your patch on it.( comment 117 )
And I made some modification, use autogen and make , I try it and work fine.
After I made make clean and made patch with git diff HEAD command.
I dont know why put these stuffs( docs/plugins/gst-plugins-bad-plugins.hierarchy) in the patch. I forget one thing on my patch , I dont use this :

indent \
  --braces-on-if-line \
  --case-brace-indentation0 \
  --case-indentation2 \
  --braces-after-struct-decl-line \
  --line-length80 \
  --no-tabs \
  --cuddle-else \
  --dont-line-up-parentheses \
  --continuation-indentation4 \
  --honour-newlines \
  --tab-size8 \
  --indent-level2

it was my mistake :(

Your feature request seems useful , I can do it this weekend or next week.


Robert
Comment 123 Nicola 2011-07-15 20:33:11 UTC
Hi,

I take a look at the latest big patch, my accurate timestamp patch get lost,the timestamp posted on the bus are string rounded to second precision again and not gint64 anymore and there are some "long int" again I think is better to use gint64 or guint64 where appropriate, I'll try to fix these as soon as I have some spare time.

Robert I think what Roman requested could be done easily as follow:

- add a new property "minimun_motion_frame" default to 1, actual behaviour
- add in the header file a new int "consecutive_motion"
- when motion is detected increment consecutive_motion by 1 and set filter->previous_motion = true and send the bus message only when consecutive_motion>=minimum_motion_frame

If you agree with the above approach I can do this patch too,

Nicola
Comment 124 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-16 07:41:08 UTC
(In reply to comment #120)
> Stefan,
> 
> Thanks your guide.
> I'm beginner with bugzilla.
> I marked all older patch as obsolete except Nicola's patches.
> 
> Okay,it's my mistake that I replace gthread with pthread.
> 
> 
> Robert

Don't worry. Thanks for working on the plugin, its getting easier over time :)
Comment 125 Nicola 2011-07-16 14:54:57 UTC
Created attachment 192087 [details] [review]
accurate timestamp patch against latest global patch
Comment 126 Robert Jobbagy 2011-07-16 15:05:06 UTC
Nicola,

Comment 123, It's okay ;)
Comment 127 Nicola 2011-07-16 21:20:53 UTC
Robert,

seems the propset_mutex in the last patch make this pipeline:

gst-launch -m v4l2src ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace ! motioncells  gridx=10 gridy=10 sensitivity=0.8 ! ffmpegcolorspace ! xvimagesink

no work anymore, to make it to work I removed the propset_mutex, is this working for you?
Comment 128 Nicola 2011-07-16 22:17:45 UTC
Created attachment 192104 [details] [review]
added miminummotionframes prop as per Roman request

In this patch I temporarily disabled the propset_mutex too, it need some work in any case since is needed to convert it to gmutex. Robert your purpose is a global mutex to allow dynamic property change in a safe manner? I'll try to take a look at this tomorrow.

With this patch I included my name in the files gstmotioncells.c and gstmotioncells.h, I discussed this with Robert in a private email,

Roman, please test this patch and report any issues, I did minimal testing,

thanks
Nicola
Comment 129 Nicola 2011-07-17 10:20:28 UTC
Hi,

I fixed the mutex things too, before post the patch I would like to ask for some advices:

1) actually we lock in gst_motion_cells_chain, so every time we process a frame and when we set a property (gst_motion_cells_set_property), this allow to dynamically change a property in a safe way, however I think not all will use the plugin this way and a lock for every frame could be expensive, I would like to make a property to enable this, I think the default should be disabled. Robert, Stefan, what do you think about?
2) Robert setted a mutex in gst_motion_cells_set_property before the "switch (prop_id)" this lock the pipeline posted above where I set multiple property using gst-launch, so to solve this I setted the muxex for each property and this work but is less convenient. Stefan, is there a way to make it work the Robert way, or a way to protect with a mutex the whole set_property method? I would like not lock/release every property but the whole method,

thanks
Nicola
Comment 130 Tim-Philipp Müller 2011-07-17 11:19:09 UTC
> 1) actually we lock in gst_motion_cells_chain, so every time we process a frame
> and when we set a property (gst_motion_cells_set_property), this allow to
> dynamically change a property in a safe way, however I think not all will use
> the plugin this way and a lock for every frame could be expensive, I would like
> to make a property to enable this, I think the default should be disabled.

Please just don't worry about this too much. I really don't think we want or need yet another property for this. It shouldn't be that expensive really, esp. in comparison to any image processing algorithms, and also there's no contention on that lock. In any case, make it work first, optimise later.
Comment 131 Nicola 2011-07-17 16:47:08 UTC
Created attachment 192132 [details] [review]
use g_mutex and not pthread mutex, solved a lock bug

Hi,

I tested the plugin again and it seems to work fine, there are no compile warnings anymore, valgrind show two possibly lost

==4129== 168 bytes in 1 blocks are possibly lost in loss record 2,177 of 2,304
==4129==    at 0x4C28FAC: malloc (vg_replace_malloc.c:236)
==4129==    by 0xBBD0B24: cv::fastMalloc(unsigned long) (in /usr/lib/libcxcore.so.2.1.0)
==4129==    by 0xBC16E21: cvCloneImage (in /usr/lib/libcxcore.so.2.1.0)
==4129==    by 0xAC406C7: MotionCells::performDetectionMotionCells(_IplImage*, double, double, int, int, long, bool, bool, int, motionmaskcoordrect*, int, motioncellidx*, cellscolor*, int, motioncellidx*, long, char*, bool, int) (MotionCells.cpp:218)
==4129==    by 0xAC3D5D9: perform_detection_motion_cells (motioncells_wrapper.cpp:44)
==4129==    by 0xAC3C698: ??? (gstmotioncells.c:979)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B613C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B613C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B59697: gst_base_src_loop (gstbasesrc.c:2516)
==4129==    by 0x4EB2BEF: gst_task_func (gsttask.c:318)
==4129==    by 0x55D7B15: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.2800.6)
==4129==    by 0x55D53E3: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.2800.6)
==4129==    by 0x5860D8B: start_thread (pthread_create.c:304)
==4129==    by 0x5B5E04C: clone (clone.S:112)
==4129== 
==4129== 921,624 bytes in 1 blocks are possibly lost in loss record 2,304 of 2,304
==4129==    at 0x4C28FAC: malloc (vg_replace_malloc.c:236)
==4129==    by 0xBBD0B24: cv::fastMalloc(unsigned long) (in /usr/lib/libcxcore.so.2.1.0)
==4129==    by 0xBC04447: cvCreateData (in /usr/lib/libcxcore.so.2.1.0)
==4129==    by 0xBC16E92: cvCloneImage (in /usr/lib/libcxcore.so.2.1.0)
==4129==    by 0xAC406C7: MotionCells::performDetectionMotionCells(_IplImage*, double, double, int, int, long, bool, bool, int, motionmaskcoordrect*, int, motioncellidx*, cellscolor*, int, motioncellidx*, long, char*, bool, int) (MotionCells.cpp:218)
==4129==    by 0xAC3D5D9: perform_detection_motion_cells (motioncells_wrapper.cpp:44)
==4129==    by 0xAC3C698: ??? (gstmotioncells.c:979)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B613C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B613C5: gst_base_transform_chain (gstbasetransform.c:2458)
==4129==    by 0x4E8C53B: gst_pad_push (gstpad.c:4684)
==4129==    by 0x8B59697: gst_base_src_loop (gstbasesrc.c:2516)
==4129==    by 0x4EB2BEF: gst_task_func (gsttask.c:318)
==4129==    by 0x55D7B15: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.2800.6)
==4129==    by 0x55D53E3: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.2800.6)
==4129==    by 0x5860D8B: start_thread (pthread_create.c:304)
==4129==    by 0x5B5E04C: clone (clone.S:112)

however I monitored the ram usage in /proc/<pid>/status for about an hour and the memory usage is constant, maybe we need some opencv suppression file.

I think if the latest property I added for the Roman use case works fine this plugin is "bad" enough for the inclusion. Stefan, Tim could you please review the plugin and tell us what need to be done to have motioncells in -bad?

thanks
Nicola
Comment 132 Nicola 2011-07-17 20:57:58 UTC
Created attachment 192146 [details] [review]
Replace long int with gint64
Comment 133 Robert Jobbagy 2011-07-18 20:15:12 UTC
Nicola,

Thanks your feedback and your patches.
I made a simple demo program what show the cvCloneImage possibility memory leak and sent it OpenCV mail list and I'm waiting for the answer.

I found some little bugs/mistakes in the code and I'll post patch maybe tomorrow evening.

I hope gst-motioncells will be part of the gst-plugins-bad in the near future :)

Stefan and Tim, we are waiting for your review result and what's missing from gst-motioncells that it'll be useful for everyone :)

Robert
Comment 134 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-18 21:03:51 UTC
Review of attachment 192132 [details] [review]:

If I recall right, we used an existing lock in some other elements, but I can't recall one right now.

::: ext/opencv/gstmotioncells.c
@@ +397,3 @@
   switch (prop_id) {
     case PROP_GRID_X:
+      g_mutex_lock(filter->propset_mutex);

I would move that before the switch and put the unlock below to avoid the repetition.
Comment 135 Nicola 2011-07-18 21:16:09 UTC
Stefan, what you suggest is exactly what Robert did before but this way this pipeline works:

gst-launch -m v4l2src ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace !
motioncells ! ffmpegcolorspace ! xvimagesink

while this one remain locked:

gst-launch -m v4l2src ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace !
motioncells  gridx=10 gridy=10 sensitivity=0.8 ! ffmpegcolorspace ! xvimagesink

am I missing something?
Comment 136 Nicola 2011-07-18 21:30:36 UTC
Created attachment 192219 [details] [review]
Proper g_mutex patch

Sorry Stefan, my bad, attached is the proper g_mutex patch,

Nicola
Comment 137 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-18 21:35:54 UTC
Review of attachment 192016 [details] [review]:

A few random comments. It would really be good to just run gst-indent (from core) over the code before making the patch too.

::: orig/ext/opencv/gstmotioncells.c
@@ +390,3 @@
+    case PROP_GRID_X:
+      retgridx = gst_element_get_state (GST_ELEMENT (filter),
+          &filter->state, &filter->pending, 250 * GST_NSECOND);

If you don't need pending, you can pass NULL. Also it seems that instead of filter->state you could use a local variable.

@@ +398,3 @@
+        filter->prevgridx = filter->gridx;
+      } else {
+        filter->prevgridx = filter->gridx;

If you assign filter->prevgridx = filter->gridix in both cases, I would move it out of the branches.

@@ +493,3 @@
+     if (tmpux > -1 && tmpuy > -1 && tmplx > -1 && tmply > -1) {
+       filter->motionmaskcoords =
+           (motionmaskcoordrect *) g_malloc (filter->motionmaskcoord_count *

you can use g_new for better readability here.

@@ +506,3 @@
+       filter->motionmaskcoord_count = 0;
+     }
+   }

indentation seems to be off.

@@ +550,3 @@
+      sscanf (colorstr[2], "%d", &b);
+      //check right RGB color format
+      (r < 0) ? r = 0 : (r > 255) ? r = 255 : 0;

r=CLAMP(r, 0, 255)

@@ +683,3 @@
+        else
+          g_string_append_c (str, ',');
+        g_string_append (str, substr->str);

just do:
  g_string_append_printf(str, "%d:%d", filter->motionmaskcellsidx[i].lineidx,
	            filter->motionmaskcellsidx[i].columnidx);
Comment 138 Robert Jobbagy 2011-07-19 07:13:47 UTC
Comment 135

Nicola,

Your pipeline is wrong, this is the right one:


gst-launch -m v4l2src ! videorate ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace ! motioncells  gridx=10 gridy=10 sensitivity=0.8 ! ffmpegcolorspace ! xvimagesink

or 


gst-launch -m v4l2src ! videorate ! videoscale ! video/x-raw-yuv,width=320,height=240,framerate=5/1 ! ffmpegcolorspace ! motioncells  gridx=10 gridy=10 sensitivity=0.8 ! ffmpegcolorspace ! xvimagesink

Robert
Comment 139 Nicola 2011-07-19 07:28:32 UTC
No Robert,

my v4l2src device negotiate 5 fps and 320x240 natively, however my last patch use only one mutex for all properties as you initially did,

Nicola
Comment 140 Robert Jobbagy 2011-07-19 07:34:54 UTC
Interesting, because this pipeline :

gst-launch -m v4l2src ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace !
motioncells  gridx=10 gridy=10 sensitivity=0.8 ! ffmpegcolorspace ! xvimagesink

doesnt works for me without videorate

Robert
Comment 141 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-19 15:34:32 UTC
It would be cool to see a consolidated patch (and marking the included patches as obsolete). I would have some time to play with the actual element (so far I only checked the code).
Comment 142 Nicola 2011-07-19 15:41:57 UTC
Hi Stefan,

We (Robert and I) are working on this, we hope to push a global patch soon,

Nicola
Comment 143 Robert Jobbagy 2011-07-19 21:32:34 UTC
Created attachment 192268 [details] [review]
the latest big patch what contains all of earlier patches 

- change long int to gint64
- cellscolor with alpha fix
- add minimum motion frames property (thanks Nicola)
- replace g_free with GFREE macro
- replace pthread with gthread (thanks Nicola)
- replace g_malloc with g_new0 (reported by Stefan)
- code simplification
- remove pending (reported by Stefan)
- replace string append (reported by Stefan)
- replace ternary conditional with CLAMP (reported by Stefan)
- add more property check
- fix motionmaskcoords
- more accurate timestamps (thanks Nicola)
- reuse deleted object id's
Comment 144 Robert Jobbagy 2011-07-19 21:37:12 UTC
Hi Guys,

Stefan,

We (Nicola and I ) tried fix all of them what you reported. And I made some other modifications too.

Nicola,

Thanks your help, feedbacks and bug reports.

Roman,

Please check this latest patch and if you find any bug please report it or if 
you have any feature request please share it with us.

This latest patch contains all of earlier patches.

We are waiting feedbacks.

Thanks your time and enjoy.

Robert
Comment 145 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-20 07:44:24 UTC
Could you please make this a git-formatted patch. Its easy:

1.) just do a diffstat gst-plugins-bad-motioncells.patch10
2.) do a "git add" for all the files in the patch
3.) git commit and enter a descriptive message, mention the bug number at the end
4.) git format-patch origin
5.) you get a 0001-<some-text>.patch which you can attach here

Be sure to to init git once:
git config --global user.name "Your Name"
git config --global user.email you@example.com
Comment 146 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-20 08:14:23 UTC
Review of attachment 192268 [details] [review]:

A few more comments:
1) For MotionCells.cpp MotionCells.h analitics.h motioncells_wrapper.cpp motioncells_wrapper.h we'd like to have the same comment block at the top as in gstmotioncells.c (the license is important).
2) would it be possible to merge the analitics.h into one of the other headers? I'd like to make sure that it is easy to see which files in the opencv plugin belong to which element

These are cosmetic. The element itself builds fine and first experiments show that it is working. Would be great if you could join IRC, then we can discuss there too - I am there as ensonic.

::: configure.ac
@@ +57,1 @@
 

its good to update to git head before making the patch

@@ +117,2 @@
 AS_PROG_OBJC
 

why do you need to set the OPENCV_FLAGS here?

::: ext/opencv/gstmotioncells.c
@@ +42,3 @@
+ * Boston, MA 02111-1307, USA.
+ */
+

Please add a gtk-doc blob here. Have a look at another element. This would also help to understand the properties. Please include some example launch lines (those that you use for testing).

@@ +217,3 @@
+  g_object_class_install_property (gobject_class, PROP_GRID_X,
+      g_param_spec_int ("gridx", "GRID X", "Motion Grid X", GRID_MIN,
+          GRID_MAX, GRID_DEF, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

For the 1st arg of g_param_spec_xxx() we usualy use a '-' to separate words.
Don't make the 2nd arg of g_param_spec_xxx() all caps. The 2nd arg should be a slighly more read able version of 1st arg (e.g. use a ' ' instead of a '-'). Here is an example:

  g_object_class_install_property (gobject_class, PROP_CHECK_PERFECT,
      g_param_spec_boolean ("check-perfect", "Check For Perfect Stream",
          "Verify that the stream is time- and data-contiguous. "
          "This only logs in the debug log.  This will be deprecated in favor "
          "of the check-imperfect-timestamp/offset properties.",
          DEFAULT_CHECK_PERFECT, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

@@ +232,3 @@
+  g_object_class_install_property (gobject_class, PROP_GAP,
+      g_param_spec_int ("gap", "GAP", "GAP between clips", GAP_MIN, GAP_MAX,
+          GAP_DEF, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

A better description needed. Also write "Gap" instead of "GAP" - it is not an abbreviation.

@@ +259,3 @@
+      g_param_spec_long ("date", "Motion Cell Date",
+          "Current Date in milliseconds", DATE_MIN, DATE_MAX, DATE_DEF,
+          G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

What is the date used for?

@@ +267,3 @@
+      g_param_spec_string ("datafileextension", "DataFile Extension",
+          "Extension of datafile", DEF_DATAFILEEXT,
+          G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

What is the datafile used for. If it is output of detected motion, then leave that to the application. If you want add a tests/examples/motioncells_logger.c that write motion-bus-messages to a log file.

::: ext/opencv/motioncells_wrapper.h
@@ +6,3 @@
+#ifndef motioncells_WRAPPER_H
+#define motioncells_WRAPPER_H
+

this should be all caps (MOTIONCELLS_WRAPPER_H)

@@ +49,3 @@
+#endif
+
+#endif                          /* BGDETECT_WRAPPER_H */

wrong header name in comment
Comment 147 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-20 08:36:57 UTC
Review of attachment 192268 [details] [review]:

::: ext/opencv/MotionCells.cpp
@@ +185,3 @@
+                  motioncellscolor.R_channel_value), p_thickness);
+        }
+

Is it intentional that in the alpha version its filled (and thickness is ignored)? I was looking at this because the rectangle is flickering a lot for me. Wonder if you have an idea why.
Comment 148 Robert Jobbagy 2011-07-20 13:33:04 UTC
Hi Stefan ,

I don't give emails about your comments, I dont know why ...

So 

I formatted the patch with this : 

indent \
  --braces-on-if-line \
  --case-brace-indentation0 \
  --case-indentation2 \
  --braces-after-struct-decl-line \
  --line-length80 \
  --no-tabs \
  --cuddle-else \
  --dont-line-up-parentheses \
  --continuation-indentation4 \
  --honour-newlines \
  --tab-size8 \
  --indent-level2

what I found gstreamer site.

I'll fix what you reported this evening.

Thanks your feedbacks.

Robert
Comment 149 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-20 17:50:50 UTC
regarding gst-indent:
http://cgit.freedesktop.org/gstreamer/gstreamer/tree/tools/gst-indent
just copy to $HOME/bin
Comment 150 Roman Gaufman 2011-07-21 15:42:49 UTC
Created attachment 192384 [details] [review]
gst-plugins-bad-0.10.22-motioncells.patch (latest)

Cleaned up version against 0.10.22 based on Robert's latest patch (2011-07-19 21:32 UTC). 

Also, sorry for the late reply, was away. Just trying out the new minimummotionframes now, thanks guys :)
Comment 151 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-21 15:52:26 UTC
Roman, we agreed to work with git style patches against git-head.
Comment 152 Roman Gaufman 2011-07-21 15:57:15 UTC
I'm not quite sure how to use minimummotionframes - I have this pipeline:

uridecodebin uri=rtsp://192.168.0.252/nphMpeg4/g726-640x480 name=dec ! videorate ! videoscale ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace ! motioncells postallmotion=true threshold=0.5 gridx=32 gridy=32 minimummotionframes=60 ! ffmpegcolorspace ! ximagesink

Whenever the image gets blocky for a frame or two from the IP camera, motioncells reports a motion event.

What am I doing wrong?
Comment 153 Roman Gaufman 2011-07-21 16:02:03 UTC
Stefan, I'm not quite sure what to do with git-head. I cloned the repo and I don't see any opencv stuff in there at all - the source seems nothing like what's in 0.10.22?

For example: diff -rupN gst-plugins-bad-0.10.22/ gst-plugins-bad-git/ | wc
429171 1613128 15739626

I'm cloning from git://anongit.freedesktop.org/gstreamer/gst-plugins-bad - is that wrong?
Comment 154 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-21 16:14:47 UTC
gst-plugin-bad certainly has opencv plugins:
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/opencv
Please also read my comment #145. Now lets stick to the topic and get the patch finalized. it would be good to discuss the use of it in the mailing list or on irc as well.
Comment 155 Robert Jobbagy 2011-07-21 16:16:40 UTC
Roman and Nicola,

I think minimummotionframes don't will works fine.
Because I made frame drop in C++ side.

And If you set a high fps then 2 motion frame chance is very low.
Example :

You set 25 fps then element compute on every 5. frame and drop four frame.
Calculate on 0.,5.,10. etc ...

I made some little bugfix yesterday.

This evening I'll test your pipeline and I'll add my bug fixes to latest patch
and post it again.

And fix what Stefan said.

Robert
Comment 156 Nicola 2011-07-21 16:45:33 UTC
Roman, what do you mean for "motioncells reports a motion event" the visual motion effect is ever reported however with that minimumotionframes configuration you should not see message on the bus. I think this is the right thing to do, use gst-launch with -m option and report is you see a motion_begin event on the bus,

Nicola
Comment 157 Nicola 2011-07-21 16:49:19 UTC
To do a finer debug please remove postallmotion=true from your pipeline too, 

thanks
Nicola
Comment 158 Roman Gaufman 2011-07-21 17:27:33 UTC
I'm using this script: https://github.com/hackeron/gstreamer_experiments/blob/master/ruby/gst_test.rb

Maybe I'm misunderstanding how it works. I tried setting minimumotionframes to 2, 3, 5, 10, 25, etc - I see a motion begin event very unreliably when I set it to 2, any other number I don't see a motion begin event at all. 

I think this is because of frames being dropped as Robert said - will try again when Robert posts the latest fixes.

Also it seems other than the dropped frames issue, postallmotion=true doesn't respect minimumotionframes - any way to make it respect minimummotionframes?

I basically want to see all motion_cells_indices from when motion starts to when motion ends (respecting minimummotionframes).
Comment 159 Nicola 2011-07-21 19:53:34 UTC
Roman, I'll explain what minimummotionframes do with an example:

- suppose you set the proerty to 10, this mean that to have a motion_begin event motion need to be detected in 10 consecutive frame, so if motion is detected in 9 frames then there is a frame without motion and then other 9 frames with motion, no motion_begin event is triggered.

If you set postallmotion=true each motion event is posted on the bus anyway, I think it is easy for an app discard unwanted motion (all you need is discard motion until a motion_begin event), however it is safe to dinamycally change motioncells property, so you can for example set postallmotion=true when you receive a motion_begin message and postallmotion=false when you receive a motion_finished event. The same apply for display motion cells on the video.

I can do what you ask after the inclusion,

Nicola
Comment 160 Roman Gaufman 2011-07-22 12:13:10 UTC
Nicola, I see - yes, I'm able to set postallmotion=true live, that does what I need, however minimummotionframes is not working correctly. I tried setting it to 5 and then panning the camera so there is constant motion - no motion_begin event was triggered.
Comment 161 Nicola 2011-07-23 06:54:01 UTC
Yes Roman, You are right it works fine here with this pipeline:

gst-launch -m v4l2src ! videoscale ! video/x-raw-yuv,width=320,height=240 ! videorate ! video/x-raw-yuv,framerate=2/1  ! ffmpegcolorspace ! motioncells  gridx=9 gridy=9 sensitivity=0.8 usealpha=true postnomotion=10 minimummotionframes=5 ! ffmpegcolorspace ! xvimagesink sync=false

but it is not reliable with higher fps, I think it is related to what Robert said in comment 155,

Nicola
Comment 162 Robert Jobbagy 2011-07-23 14:47:41 UTC
Roman ,

Sorry for tha late reply , but I was very busy on this week.
So I checked now your pipeline what you post in comment 152

uridecodebin uri=rtsp://192.168.0.252/nphMpeg4/g726-640x480 name=dec !
videorate ! videoscale ! video/x-raw-yuv,framerate=5/1 ! ffmpegcolorspace !
motioncells postallmotion=true threshold=0.5 gridx=32 gridy=32
minimummotionframes=60 ! ffmpegcolorspace ! ximagesink

and you set wrong threshold value. 

Or it isnt wrong just it means if we detect motion in 0.5 * 32 * 32 cells then we detect motion and we post a bus msg like motion begin :)

Please check again the all properties if I'll post the latest patch ( soon )and if something isn't clear for you, just ask from me.

Robert
Comment 163 Robert Jobbagy 2011-07-24 19:34:30 UTC
Created attachment 192574 [details] [review]
latest motioncells patch

- fix all of them what Stefan reported
- fix alpha flicking
- some other modifications
Comment 164 Robert Jobbagy 2011-07-24 19:36:10 UTC
Hi Guys,

I think , I fix all mistakes what Stefan reported.

Roman and Nicola please test it if you have time.

Thanks,

Robert
Comment 165 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-25 08:38:37 UTC
Comment on attachment 191389 [details]
motioncells dynamic property change test program

Please include it in the patch too (put it under tests/examples/opencv). Add the copyright header to the source. Could be done as a separate patch too.
Comment 166 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-25 08:58:52 UTC
Review of attachment 192574 [details] [review]:

This now looks quite good to me. Please do one more git commit --amend and edit the commit message to contain something useful and be similar to other commit messages. Its should be e.g:
"""
motioncells: new element to detect areas of motion

<longer description here>
"""

I still think it would be worth to kick out the datafile stuff if that could be done as a example under e.g. tests/examples/opencv/motioncell-logger.c.
The top-level doc blob could be improved and some of the property description (I still don't get what "gap" is about). The display-property should probably be FALSE by default (and set to true in the example launch line in the docs). I would be fine with doing such fine-tuning in follow up commits. But lets fix the commit message and agree on the data-file functionality.

::: ext/opencv/gstmotioncells.c
@@ +282,3 @@
+      g_param_spec_string ("datafile", "DataFile",
+          "Location of motioncells data file (empty string means no saving)",
+          " ", G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));

It is better to use NULL for no-saving. The empty string would get copied.

@@ +355,3 @@
+  filter->prev_datafile = g_strdup (" ");
+  filter->cur_datafile = g_strdup (" ");
+  filter->basename_datafile = g_strdup (" ");

What about just using NULL for empty?
Comment 167 Robert Jobbagy 2011-07-25 20:53:19 UTC
Stefan,

Sorry I'm very tired.
I'll post the patch tomorrow evening, because I would like review it once again 

Thanks help everyone.

Robert
Comment 168 Robert Jobbagy 2011-07-27 17:32:57 UTC
Hi Guys,

This is the latest patch.
We ( Nicola and I) made lot of test and we didnt find any bug in this version.
I hope we can fix any bug what you reported and made all feature and other request.

Please test it and if you find any bug please report it.

Thanks your help and your time.

Enjoy the motioncells element :)

Robert
Comment 169 Robert Jobbagy 2011-07-27 17:36:32 UTC
Created attachment 192767 [details] [review]
motioncells new element to detect areas of motion

- we fix all bug what testers reported (reported by Stefan,Nicola,Roman)
- reformat/indent this version for gst (reported by Stefan)
- some small modification for gst code style (reported by Stefan)
Comment 170 Nicola 2011-07-27 18:42:18 UTC
Created attachment 192768 [details] [review]
fix a valgrind warning

We hope this is the latest patch :-)
Comment 171 Robert Jobbagy 2011-07-27 18:52:53 UTC
Thanks Nicola this patch.

Sorry Guys this stupid mistake :(

Robert
Comment 172 Robert Jobbagy 2011-07-27 20:38:09 UTC
Created attachment 192773 [details] [review]
gstmotioncells element dynamic change property test program 

I made test program , what you can use with motioncells element and you can dynamic change motioncells properties
Comment 173 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-28 08:30:30 UTC
commit 75a6072db8a8225f8cb2ade960a8eb3d89ee5bf3
Author: Robert Jobbagy <jobbagy.robert@gmail.com>
Date:   Wed Jul 27 22:34:23 2011 +0200

    gstmotioncells_dynamic_test: test tool what can to do dynamic change properties

commit 4723a5d90ce02cb4d720b1be7ebdf365f81910e4
Author: Robert Jobbagy <jobbagy.robert@gmail.com>
Date:   Wed Jul 27 18:58:15 2011 +0200

    motioncells: new element to detect areas of motion
Comment 174 Stefan Sauer (gstreamer, gtkdoc dev) 2011-07-28 08:31:19 UTC
Comment on attachment 192768 [details] [review]
fix a valgrind warning

I squashed that on the big patch.