After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 667183 - Request for new algorithm: following the brightest object visible
Request for new algorithm: following the brightest object visible
Status: RESOLVED WONTFIX
Product: mousetrap
Classification: Other
Component: General
unspecified
Other Linux
: Normal enhancement
: ---
Assigned To: mousetrap-maint
Flavio Percoco Premoli
gnome[unmaintained]
Depends on:
Blocks:
 
 
Reported: 2012-01-03 14:30 UTC by swaroop
Modified: 2018-08-17 19:45 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
This patch adds a plugin to track a bright object to move the mouse. (3.37 KB, patch)
2015-08-31 00:52 UTC, Maxwell Lemkin
needs-work Details | Review

Description swaroop 2012-01-03 14:30:08 UTC
An algorithm for simply following the brightest object visible will be really helpful when IR is used in place of normal light and the camera is physically tweaked to allow only IR inside. I'm sure many many projects will be infinitely thankful for this.. please can you add this or instruct me to do it myself?

- swaroopkml@gmail.com

Thank you
Comment 1 Maxwell Lemkin 2015-06-11 00:47:54 UTC
James Colla and are are working on this.
Comment 2 Maxwell Lemkin 2015-08-31 00:52:06 UTC
Created attachment 310323 [details] [review]
This patch adds a plugin to track a bright object to move the mouse.

This plugin is not enabled by default. To enable the plugin, uncomment the respective line in the config YML file. Disable other tracking plugins to prevent any interference with the bright object tracking.
Comment 3 Kevin Brown 2015-08-31 13:54:33 UTC
Review of attachment 310323 [details] [review]:

For now I've done a general review, I'll look over the algorithms that are used to calculate the delta later.

::: src/mousetrap/plugins/bright.py
@@ +5,3 @@
+
+
+LOGGER = logging.getLogger(__name__)

You're defining a logger up here but you don't appear to actually be using it.

@@ +20,3 @@
+
+    def run(self, app):
+        if self.height_image == None:

When checking if a value is `None`, you generally use

    if self.height_image is None:

This is because `None` is a singleton, so you should check be checking to ensure the instances are the same.

@@ +26,3 @@
+        pointer_location = app.pointer.get_position()
+        gray = app.image.to_cv_grayscale()
+        (minVal, maxVal, minLoc, maxLoc) = cv2.minMaxLoc(gray)

You only appear to be using `maxLoc` here, which is the last index, and that appears to be a two-value tuple (a point). Instead of unpacking all of the values, you can instead just unpack that one index.

    brightest_x, brightest_y = cv2.minMaxLoc(gray)[3]

Also remember that Python tends to use snake_case instead of camelCase like other languages.

@@ +43,3 @@
+        pointer_location = (pointer_location[0] + delta_x, pointer_location[1] + delta_y)
+        app.pointer.set_position(pointer_location)
+        app.pointer.get_position()

This line of code doesn't appear to be doing anything.
Comment 4 Kevin Brown 2015-08-31 16:52:44 UTC
Review of attachment 310323 [details] [review]:

After checking on a whiteboard with the deltas, I noticed a few things.

1. There are quite a few calculations you keep using that can be saved in variables and reused
2. You have an issue with the dead zone that makes it larger than expected.

I've left in-line comments that address these issues in more detail.

::: src/mousetrap/plugins/bright.py
@@ +2,3 @@
+import logging
+import cv2
+import math

You don't appear to be using anything from the `math` module in here.

@@ +29,3 @@
+        delta_x = 0
+        delta_y = 0
+        if not ((self.width_image//2 + self.DEAD_ZONE) > maxLoc[0] > (self.width_image//2 - self.DEAD_ZONE)):

Right now these calculations work, but they can be simplified quite a bit.

You know how to get the center of the image, you might as well store that somewhere because it's being used a lot

    center_x = self.width_image // 2
    center_y = self.height_image // 2

And you also know how to get the difference between the light location and the center of the image

    difference_x = brightest_x - center_x
    difference_y = center_y - brightest_y

Note that the calculation for the Y difference is reversed because the screen's (0, 0) is the top left instead of the bottom left.

So because the dead zone is 10 pixels around the center, you know can test that with

    if abs(difference_x) > 10:

Which is similar to what is being done in the nose locator.

@@ +31,3 @@
+        if not ((self.width_image//2 + self.DEAD_ZONE) > maxLoc[0] > (self.width_image//2 - self.DEAD_ZONE)):
+            if maxLoc[0] < self.width_image//2:
+                delta_x = (self.width_image//2 - self.DEAD_ZONE - maxLoc[0]) * 0.1

This is also considering the dead zone, which was probably intentional, and because of that the dead zone is actually about 20 pixels around the center of the image. This is because for anything less than 20 pixels, the delta will be a fraction of a pixel (< 1.0) and the pointer will not be able to move.

It's also good to note that the difference (possibly excluding the dead zone) should directly relate to the delta. So you should see the following

    difference = (20, 20)
    delta = (2, 2) or (1, 1)

    difference = (-20, -20)
    delta = (-2, -2) or (-1, -1)

So you can set up the delta to be

    delta_x = int(difference_x * 0.1)
    delta_y = int(difference_y * 0.1)

And you should be able to get the expected delta from that.
Comment 5 André Klapper 2018-08-17 19:45:38 UTC
mousetrap is not under active development anymore since 2015.
Its codebase has been archived:
https://gitlab.gnome.org/Archive/mousetrap/commits/master

Closing this report as WONTFIX as part of Bugzilla Housekeeping to reflect
reality. Please feel free to reopen this ticket (or rather transfer the project
to GNOME Gitlab, as GNOME Bugzilla is deprecated) if anyone takes the
responsibility for active development again.