2

I have a script that accesses my webcam and displays the captured video on the screen. I can also very easily draw a circle (a rectangle, it doesn't matter) over the video.

When I move the camera physically, the video also "moves". My webcam is attached to the top left corner of my monitor. So when I slide the camera across the monitor to the right, the video itself, and so all obejcts in it move left. But, it's obvious, that the drawn circle stays unmoved.

Now I want the circle be moved when the camera moves.

G1. When the circle contains an object (as in pic1), the goal can be reached using the object tracking algorithms. (I am working with python and using openCV lib, and there are several builtin algorithms (Boosting, MedianFlow, MIL), also the meanshift algorithm, etc.)

G2. There can be cases, when the circle doesn't contain an object (doesn't have any underlying object, as in pic2). In these cases the object tracking algorithms don't work.

I hope I managed to explain the problem. Now I have 2 questions:

Q1. Is there a better way to achieve the Goal1?

Q2. How can I achieve the Goal2? (Maybe there is a way to determine that the video and objects (the wall watch in pic2) moved left, and move the circle proportionally.)

Any help would be appreciated.

P. S. I'm new to Stack Exchange platform, so if the current topic isn't the right place for my question (problem), please tell me where should I post it.

Pic1 Pic2

Lorenzo Villani
  • 39
  • 3
  • 3
    I don't know what I'm taking about here, but does your tracking really need to track objects *within* the circle? Can't you track "the most trackable object" and apply some offset to place your circle? BTW, I think your question and your wording is just fine. Welcome! – KlaymenDK Jun 17 '19 at 19:10
  • Thank you! I just want the circle be moved when the underlying object moves. That object can be anything, it isn't always a "thing", that one can see and understand what is it, so I can't use the object detection algorithms for tracking. And on the other hand, the object tracking algorithms (like missed, kcf, mil) don't work when part of the picture, that the circle contains, isn't different from the neighborhood. (As in pic2.) – Lorenzo Villani Jun 19 '19 at 12:25
  • 1
    And what about zoom/distance (objects becoming bigger/smaller in the video)? – Bernhard Hiller Jun 26 '19 at 07:46
  • What is your ultimate goal? Do you want the camera to follow moving things to keep them in your picture? Do you want to compensate for camera movements in order to keep you video still? – Martin Maat Jul 27 '19 at 07:26

3 Answers3

1

Thanks everyone for replying!

After researching, I found a better solution for G1, and so an answer to Q1.

Now I use Lucas-Kanade method (optical flow estimation method) for object tracking. I just need to mention a point inside the object (for example one can draw a very small circle), and it sticks to that object, that is the circle moves with the object. One of the advantages of this method is the performance. This method is much more faster than the object tracking algorithms, that were mentioned in the question. And if needed, the circle size can be changed, so it can be like the circle in pic1, because only the center point is important.

This method also helps in the 2-nd case, so if I mention a point on the white wall, it works better than the object tracking algorithms, and not only when comparing performance, but also when comparing the result. When using Lucas-Kanade method the result is more accurate, the circle stays closer to the selected point, than in case of object tracking algorithms. But this method doesn't completely solve the problem (G2), because very often the circle doesn't stick to the selected point on the wall, and jumps to other points. However, this is the best result I've achieved for now.

Lorenzo Villani
  • 39
  • 3
0

You can combine G1 and G2 into one algorithm:

  1. Create a map of objects in the view / image (coordinates, sizes...): clock, orchid, books, wallpapers...
  2. Draw the circle (rectangle, whatever) relative to some of the objects in the image:
    • exactly around the clock;
    • XY pixels above the orchid;
    • (use your imagination :) )

What is not clear to me (and not really relevant for the answer) is how you decide the initial position of the circle. Do you use an algorithm with some criteria, or just place it by hand?

virolino
  • 572
  • 4
  • 12
0

Without any references (known objects/markers) this is virtually impossible. It would be expensive for sure. You would have to scan the whole image and pick some distinct areas as references. Then follow these around. You will lose some over time because of changing light conditions, orientation, zoom or simply because they moved out of the frame. Then you will need new ones. So with each iteration you will have to do a fresh scan for orientation items and use them for as long as they work, perhaps matching "new" items with lost ones that probably just look a bit different this time around, updating properties on each iteration. The combination of tracking data for the different reference items will have to provide confidence. It is going to be "fuzzy".

Martin Maat
  • 18,218
  • 3
  • 30
  • 57