Embedded Files
BY MARKUS SPRUNCK
BY MARKUS SPRUNCK
In this WebGL experiment the browser captures the video signal and compares subsequent frames to detect motion of the user. With this signal the program finds the viewpoint and rotates the head, body and eyes of a teddy bear.
You may start the web application with a WebGL enabled browser here https://webgl-examples.appspot.com/teddy-regards-you/index.html.
The algorithm is very simple, robust and extremely fast compared with classic face detection approaches. The code is based on:
Three.js (r76),
glfx.js and
JavaScript.
Tested Browser under macOS Mojave (v10.14)
Tested Browser under macOS Mojave (v10.14)
Chrome (v70),
Safari (v12) and
Firefox (v63)
Expected Result
Expected Result
Open the web page https://webgl-examples.appspot.com/teddy-regards-you/index.html with a WebGL compatible Browser.
It should support getUserMedia/Stream API (see http://caniuse.com/#search=stream) and webGL (see http://caniuse.com/#search=webgl).
With Chrome you should see something like this:
Figure 1: Browser asks for allowance to use the camera
Allow the site to access your camera. Now move your head in the direction of the camera Frame. You may also use your hand to move the Teddy bear.
Figure 2: The teddy moves his body and eyes dependent from your movements
The movement of your head should be large enough or use your hand. To control the visible Frame of your camera and to change the detection parameter open the GUI. In the right bottom corner a control window appears.
Figure 3: The teddy moves his body and eyes dependent from your movements
Motion Detection Algorithm
Motion Detection Algorithm
The underlying algorithm for the used motion detection is extremely simple:
First the video Signal is mirrored on the Vertical Axis. Just all x-Coordinates are swapped and the y-Coordinate stays the same. This is needed to get the viewer perspective, usually the camera of your PC will create a reversed left to right picture.
After This always subsequent frames of the video signal are compared. If the viewer moves his/her head the frames show small differences. WebGL provides the possibility to compare these frames with HW acceleration, so this is extremely fast.
Then the average position of all the changed pixels will be computed and displayed as small red cross. This moving average shows the average position of all movements in the screen.
To improve the results the lower part of the screen will not be analyses and this detection border is changeable in the GUI.
The algorithm is independent from the moving object, so you may also use a hand and/or other objects. Maybe you try is with a small bar of chocolate.
Motion Detection Color Codes
Motion Detection Color Codes
Pixels in blue indicate changes between frames (above the detection Border)
Pixels in Green indicate movements which are not detected (below the detection Border)
Pixels in red are the center of Movements
pixels in Orange indicate two times the standard deviation around the Center of Movements
Eclipse Project and Source Code
Eclipse Project and Source Code
fork the project on GitHub or download the zipped project from GitHub. Open the project in Eclipse (Luna) and You should see something like this:
Figure 4: Project Structure in eclipse
Start from Local Host
Start from Local Host
You can not directly open the page from file system, because of the security settings of the browsers. To start the web application from a local folder you should start a HTTP server.
The simplest way of doing this is With Python.:
python -m simpleHTTPServer
As alternative you may start cHrome with the options
enable-media-stream
allow-file-access-from-files
But, be aware that these options are not secure. This start option means that Pages originating from anywhere (local and/or web) have access to local resources And this is really not a good thing.
Start with Code Coverage Measurement
Start with Code Coverage Measurement
Basic support of Code Coverage measurement is included in the project. The code coverage is done by:
blanket.js in combination with
qunit.js Library.
For about five seconds a simple test case measures all activities in the application. The expected result looks like this:
Figure 5: Result of Code Coverage measurement
the same measurement is also possible from local host, Just start with the following URL:
or on your development machine:
Run with Orbit contrOls
Run with Orbit contrOls
During development it is helpful to start the 3D GUI also with orbit controls to move the graphic with the mouse:
on your development machine:
Google Sites
Report abuse