Simi MotionCapture3D Workflow

Specification

The first step is to define the objects and the movements which you would like to capture. For a human being, for example, these are generally the movements of the joints. A so-called specification is produced for these objects, consisting of the points which are to be tracked later on the video image. This means that all the required points are specified and, if required, then connected with each other. In this way a hierarchical structure can be created, for example, a human skeleton. It is important that the file format for export of the data later is taken into account here. For data export in BVH format, for example, a particular set of points is required for a human body. Suitable templates for this purpose are supplied with the software.

Camera setup

The movement or object is recorded by several cameras simultaneously. At least two cameras are required for the subsequent calculation of three-dimensional data. The accuracy of the captured data can be improved by using additional cameras and choosing their position carefully. The greater the number of cameras used the less likelihood of a particular point being hidden in all the recordings.

Types of camera

Any kinds of camera can be used, as well as image and video data from other recording systems, as long as they show the objects to be captured from at least two different angles and they can be synchronized in time. Simi MotionCapture3D provides integrated support for synchronized recording with DV cameras and with high-speed FireWire cameras.

Calibration

Each camera must be calibrated once before recording starts. This means that its position and its perspective relative to the other cameras is determined. A so-called calibration system, which is offered by Simi in various configurations, is used for this purpose. Arbitrary objects in the room or a self-made calibration system can be used as a makeshift substitute. It is important that the system incorporates a sufficient number of points, that these are visible in each camera recording and that the system encompasses the entire space in which the motion you want to capture is performed.

Markers

Capture of movements with Simi MotionCapture3D is marker based. Points to be captured must therefore show up in a different color from the background. Special marker sets can be supplied by Simi. It is also possible to use any kind of adhesive labels or other markers on the points to be captured. The software can differentiate between different colored markers. Pattern matching can be supplied as an option. Distinctive objects or points can be tracked automatically on the video image without any additional markers.

Real-time

Simi MotionCapture3D does not process the video data in real-time. This means that the video recordings made by all the cameras are not processed until the second step of the procedure. This tracking process can be performed relatively quickly and is supported by automatic tracking and pattern matching.

Capture

The points from which the motion data is to be acquired are marked by clicking on them with the mouse in the first video frame. If the points have been identified by markers and are always visible, the software can then track movements automatically. If the points have not been marked, you can either use pattern matching or capture the points manually in each video frame. This last option is also available for points which are temporarily hidden or which are missed by automatic tracking until it can pick up again a few frames later.

Post-processing

Manipulation, correction and smoothing of the 2D data can now be carried out for each camera separately. If required the data can also be exported in a text file and then processed or used externally. In the same way processed data can also be re-imported to Simi MotionCapture3D.

Calculations and export

The software calculates three-dimensional movements of joints or markings based on the two-dimensional information from various camera recordings and the calibration. These can then be exported in a file and used, for instance, with any animation application. Simi MotionCapture3D supports various file formats for this purpose. Before exporting the 3D data, it is possible to manipulate it, smooth it or to perform calculations with other data on it or even to analyze it using optional software modules.