Extremely Fast and Simple WebGL Motion Detector to Rotate 3D Graphic

BY MARKUS SPRUNCK

In this WebGL experiment the browser captures the video signal and compares subsequent frames to detect motion of the user. With this signal the program finds the viewpoint and rotates the head, body and eyes of a teddy bear.

You may start the web application with a WebGL enabled browser here https://webgl-examples.appspot.com/teddy-regards-you/index.html.

The algorithm is very simple, robust and extremely fast compared with classic face detection approaches. The code is based on:

  • Three.js (r76),

  • glfx.js and

  • JavaScript.

Tested Browser under macOS Mojave (v10.14)

  • Chrome (v70),

  • Safari (v12) and

  • Firefox (v63)

Expected Result

Open the web page https://webgl-examples.appspot.com/teddy-regards-you/index.html with a WebGL compatible Browser.

It should support getUserMedia/Stream API (see http://caniuse.com/#search=stream) and webGL (see http://caniuse.com/#search=webgl).

With Chrome you should see something like this:

Figure 1: Browser asks for allowance to use the camera

Allow the site to access your camera. Now move your head in the direction of the camera Frame. You may also use your hand to move the Teddy bear.

Figure 2: The teddy moves his body and eyes dependent from your movements

The movement of your head should be large enough or use your hand. To control the visible Frame of your camera and to change the detection parameter open the GUI. In the right bottom corner a control window appears.

Figure 3: The teddy moves his body and eyes dependent from your movements

Motion Detection Algorithm

The underlying algorithm for the used motion detection is extremely simple:

  • First the video Signal is mirrored on the Vertical Axis. Just all x-Coordinates are swapped and the y-Coordinate stays the same. This is needed to get the viewer perspective, usually the camera of your PC will create a reversed left to right picture.

  • After This always subsequent frames of the video signal are compared. If the viewer moves his/her head the frames show small differences. WebGL provides the possibility to compare these frames with HW acceleration, so this is extremely fast.

  • Then the average position of all the changed pixels will be computed and displayed as small red cross. This moving average shows the average position of all movements in the screen.

To improve the results the lower part of the screen will not be analyses and this detection border is changeable in the GUI.

The algorithm is independent from the moving object, so you may also use a hand and/or other objects. Maybe you try is with a small bar of chocolate.

Motion Detection Color Codes

  • Pixels in blue indicate changes between frames (above the detection Border)

  • Pixels in Green indicate movements which are not detected (below the detection Border)

  • Pixels in red are the center of Movements

  • pixels in Orange indicate two times the standard deviation around the Center of Movements

Eclipse Project and Source Code

fork the project on GitHub or download the zipped project from GitHub. Open the project in Eclipse (Luna) and You should see something like this:

Figure 4: Project Structure in eclipse

Start from Local Host

You can not directly open the page from file system, because of the security settings of the browsers. To start the web application from a local folder you should start a HTTP server.

The simplest way of doing this is With Python.:

python -m simpleHTTPServer

As alternative you may start cHrome with the options

  • enable-media-stream

  • allow-file-access-from-files

But, be aware that these options are not secure. This start option means that Pages originating from anywhere (local and/or web) have access to local resources And this is really not a good thing.

Start with Code Coverage Measurement

Basic support of Code Coverage measurement is included in the project. The code coverage is done by:

  • blanket.js in combination with

  • qunit.js Library.

For about five seconds a simple test case measures all activities in the application. The expected result looks like this:

Figure 5: Result of Code Coverage measurement

Run with Orbit contrOls

During development it is helpful to start the 3D GUI also with orbit controls to move the graphic with the mouse:

on your development machine: