Motion Capture Path 1: Setup & Calibration
Last update: 9/11/2008

Outline of the process (For more details see Step by step instructions and why do what we do below)

Step 1. Starting iQ and opening a project folder
Step 2. Hardware configuration: Camera setup
Step 3. System calibration: a wand and a L-frame
Step 4. Placing markers on a performer
Step 5. Capturing a range of motion (ROM) trial
Step 6. Capturing a F.B. Pose
Step 7. Reconstructing ROM trial's 3D trajectories from 2D camera data
Step 8. Manually labeling a single frame in the ROM trial
Step 9. Auto labeling the rest of the frames in the ROM trial
Step 10. Subject calibration using a T-pose in the ROM trial
Step 11. Testing the quality of the subject calibration.

Step by step instructions and why we do what we do

1. Starting iQ and opening a project folder

Grayscale images captured by the Vicon cameras are image-processed by the processors on the cameras. 2D camera data are collected by the MX Net boxes in the corner where the computers are. The MX Net boxes power and synchronize the cameras as well. 2D camera data are handled by Tarsus in realtime and passed to Vicon iQ which lets you interface with the system and database. We run Tarsus on one machine and iQ on another machine.

Check if Tarsus is running on the computer that is on your right side when you sit at the desk with two computers in the mocap room (HRA 1.304). If not, run tarsus.exe. Start Vicon iQ 2.5 on the left computer. Click on Eclipse (in the upper left corner), select Open Database in the pop up menu, and select utd_mocap. And then go to Data Management (in the upper left corner). You can create folders for a new project, a new capture day, and a new session by clicking the following icons . Each group should have a project folder.

Notes:

2. Hardware configuration: Camera setup

The goal here is to let the maximum number of cameras see markers in the capture volume so that you can get clean data without missing markers, ghost markers, or mislabeled markers. Let every part of the capture volume be seen by at least three cameras. Make sure in each corner at least two cameras see markers well.

a). Mark capture volume. Place markers on the floor along the edges of the capture space and the line that divides the space into two halves (see the diagram below). Also place markers on top of a 5' or 6' tripod and place the tripod at a corner and at the center so that you will know if the markers on a performer's head are visible to the cameras at the corners.

b). Turn on cameras. In iQ, go to Setup (which is next to Data Management in the upper left corner) and select Hardware Config (the top vertical tab on the right side). Select Vicon MX as System Type. (If you don't see System Type, click on the Hardware Config tab. If the cameras are off, turn them on by flipping the switches on the back of Vicon Link and Vicon MX boxes in the grad lab (HR 1.306). If any camera is off, select Reboot All in Hardware Config. If any camera is still off after rebooting, flip the power switch of the box to which the camera’s cable goes in.

c). See through cameras. Click on the red triangle Play/Stop Realtime button  (below Log window where iQ's feedback text shows up, near the bottom of the iQ window. If your iQ window is too small, you won't see it!) You can see what each camera is seeing by selecting Camera (below Data Management, Setup, Calibrate, Capture, etc.) and then select Cams in a red box.  (If you can’t find them, see the screen capture below). You can select a camera by clicking on the camera’s number or using the left/right arrows in the interface. Selecting a camera number and shift-selecting another camera number let you select multiple cameras.

Camera

Cams

d). Aim cameras. Make sure that each camera is aiming at the part of the volume as follows and its coverage overlaps with the neighboring camera coverages:

Note: You could rotate cameras to maximize with the instructor or Mocap TA's permission. For instance, if what a camera is seeing is like this , then you can rotate the camera so that a bigger area of the capture volume will be visible like this.

Note: If you move cameras, you need to create new masks for cameras so that the lights from the cameras are ignored: go to Thresholds (the vertical tab second from bottom, on the right side). Click on Start Recording Background and then Stop Recording Background in Auto Threshold Creation.

e). Adjust camera attributes. Go to Cameras (the vertical tab third from top, on the right side) and adjust each camera’s Strobe Intensity, Camera Threshold, Gain, and Circle Quality:

While adjusting the camera attributes use All as Grayscale Mode (which is above Gain and Circle Quality). Zoom into markers with the right mouse button and the alt key and try to make the center of a marker white and gray around the edge. Turn Grayscale Mode back to Normal when you are done adjusting the attribute. If nothing works (i.e., many markers are still invisible to the camera), check the camera’s focal length and f-stop (ask the Mocap Lab TA to do that).

f). Mark capture volume. Replace markers on the floor with velcro tapes to mark the edges of the capture volume if your capture volume is different from the default one.

3. System calibration: a wand & a L-frame

A wand is used to calibrate cameras. The goal of camera calibration is to let the system figure out each camera's location and orientation in the global coordinate system (world space)so that when a marker is seen from at least two cameras, the location of the marker in the global coordinate system is determined and the marker is tracked through time and space.

A L-frame is used to set the capture volume's origin and axes. In other words, the L-frame placed on the capture space's floor allows the system figure out the location and orientation of the floor in the global coordinate system.

a). Calibrate cameras. Go to Calibrate. In Calibrate Cameras select the wand that you will be using (390_130_Wand). When the person with the wand is ready, click on Start Wand Wave. Click on Stop Wand Wave when all the cameras turn green in Status Report after waving the wand a few times. In Status Report, Blue is awesome. Green is excellent. Yellow is good. Red is bad. Two or three goods are acceptable. To see a waving wand, select Camera (below Data Management, Setup, Calibrate, Capture, etc. ). Make sure that the wand is waved in the center of the volume.

b). Set the capture volume's origin and orientation. To set the capture volume’s origin and axes, place the L-Frame, select Live 3D Workspace (below Data Management, Setup, Calibrate, Capture, etc.) to see what’s seen in the capture volume and click on Track L-Frame and Set Origin. For especially a game production, it is good to place the L-Frame at the same location for all capture sessions. If Set Origin doesn’t work, go to Setup and select Realtime Config (the vertical tab second from top, on the right side). Turn on Kinematics in RealTime Engine Output.

c). Test your calibration's quality. Go to Capture and capture the following: move a wand around, place it on the floor, and move a hand around the wand. Click on Load Into Post in Review Last Capture. Run CircleFit, Reconsruct... in Pipeline. Play it back. If you see ghosting, the calibration is poor. Also if you see markers vibrate more than 1 mm when the wand is on the floor, the calibration is bad. (The marker's diameter is 15 mm). Turn on Rays in View Options (in the left side of the interface) will show you the number of cameras that are looking at the markers. When markers vibrate on the floor, there are only few cameras that are seeing the markers.

Note: Calibrate Floor Plane may be used if you find the floor plane of captured data is tilted (e.g, a performer's feet float or go below the floor plane in MotionBuilder or Maya.)

4. Placing markers on a performer

Markers are used to capture the motion of the performer's skeleton. So, it is the best to place the markers on the spots that have the least amounts of soft tissues since soft tissues (skin and muscles) deform and slide when the performer's skeleton moves. If you are capturing secondary motions, you will place additional markers on soft tissues.

Most markers on limbs (e.g., wrist markers, elbow markers, ankle markers, knee markers) are placed on a joint not to measure the rotation angle of the joint itself but that of the parent joint (i.e., the joint that is hierarchically one node up.) For instance, the wrist markers are used to measure the rotation at the elbow. It is a good idea to ask the performer to rotate his/her hands so that you can place the wrist markers where the markers move at least when the hands are rotated.

Place markers on the performer so that the real markers correspond to the markers of the 3D skeleton. There are 43 or 44 markers. (See the pictures for John's marker set & UTD standard set. )

To view a marker set in iQ, go to Modeling and select Builder (the virtical tab on the right side). Go to File (in the upper left corner) -> Open..., select Vicon Skeleton Template File (*.vst) as Files of type, and open johns_base_marker.vst or UTD_standard_main.vst in the VICONiQ2.5 folder. (Local Disk (C:) -> Program Files -> Vicon -> VICONiQ2.5). In 3D Workspace, to tumble use Alt + LMB, to track use Alt + MMB, and to dolly use Alt + RMB (as in Maya.)

Notes:
  • Wrist markers are used to measure the rotation of the elbow. The wrist markers should be placed where they are moved at least by the wrist rotation.
  • Elbow markers are used to measure the rotation of the shoulder. The elbow markers should be placed where they are moved at least by the elbow rotation.
  • Shoulder markers are used to measure the shoulder translation. The shoulder markers should be placed where they are moved at least by the shoulder rotation.
  • Markers on the back should not pick up the neck rotation.
  • Side toe markers ("inside toe" and "outside toe") and the heal marker are used to measure the rotation of the ankle. The side toe markers should be placed where they are moved at least by bending the toes.
  • Ankle marker is used to measure the rotation of the knee. The ankle marker should be placed where they are moved at least by the ankle rotation.
  • Knee markers are used to measure the rotation of the leg at the hip. The knee markers should be placed where they are moved at least by the knee rotation.
  • Leave at least two marker diameters between two markers.
  • If a performer's size is between two sizes, e.g., medium and large, ask s/he to wear the smaller mocap suit, e.g., medium instead of large, so that the suit will be sufficiently tight and markers will not slide.
  • If a performer has long hair ask him/her to tie his/her hair so that no markers will be occulted by the performer's hair.

 

5. Capturing a range of motion (ROM) trial

A range of motion trial is used to change the proportions of a generic skeleton (VST) that has generic marker positions to the physical dimensions of the captured subject (VSK) in Step 10: Subject Calibration.

Go to Capture. Type in a file name in the box next to Name and select R.O.M. as Type. 2D Camera Data in Choose Record Type(s) should be selected, but not Realtime Output.

Click on the Play/Stop Realtime button (near the bottom of iQ window) if iQ is still playing realtime.

Click on Start in Capture! to start capturing a performer’s range of motion. Have the performer start with a T-pose, rotate all the joints (the neck, shoulders, elbows, hands, waist, hips, knees, and ankles) one by one, and end with a T-pose (See photos and notes on T-pose below).

Click on Load Into Post in Review Last Capture and go to 6. b).

Notes on T-pose:

6. Capturing a F.B. Pose

The F.B. Pose is useful when you test how your rigged character's skin behaves (e.g., skin weights) when mocap data is applied to it in MotionBuilder and Maya. See photos of the F.B. Pose.

Go to Capture. Type in a file name in the box next to Name. Click on Start in Capture! to start capturing. Have the performer start with a T-pose with the limbs bent at the knees and elbows, walk around in a small circle, reach the floor with one hand while raising the other hand in the air, and end with a T-pose with the limbs bent.

The F.B. Pose can be processed as described in Step 2. Processing Data in Motion Capture Path 2: Capturing and Processing Data.

7. Reconstructing ROM trial's 3D trajectories from 2D camera data

Image processing methods are applied to 2D images captured by cameras to minimize the noise and separate the markers from the background. Circles are fitted to the markers (and sometimes other things in a camera view...). The two-dimensional coordinates of each marker for each camera view (2D camera data) are determined. And then, using the 2D camera data and camera coordinates, the trajectory of each marker throughout a sequence is identified in the global 3D coordinate system.

2D camera data

Reconstructed 3D trajectories

When markers get too close, the reconstructed trajectories of the markers may be wrong. When markers are swapped, you need to find the frame in which the problem starts and relabel markers.

When markers get too close...
even if the real trajectories are like these...
the trajectories may be reconstructed like these.

Similarly when markers are occluded for a period of time, the reconstructed trajectories of the markers will be discontinuous and may be wrong. Missing trajectories can be generated using kinematic fit and other methods. When markers are swapped, you need to find the frame in which the problem starts and relabel markers.

When markers go missing...
even if the real trajectories are like these...
the trajectories may be reconstructed like these.

a). Open the ROM trial file. If the ROM file is not loaded, go to Data Management. Right-click on the ROM trial. Select Open from the pop-up window. Go to Capture and click on Load Into Post in Review Last Capture.

b). Go to Pipeline. Go to Post Processing and select Pipeline (the bottom vertical tab on the right side). If CircleFit, Reconstruct, ... is not in Pipeline, click on the add button (near the bottom of Pipeline Control above Operation Control) and select CircleFit, Reconstruct, ... from the pop-up window.

c). Set parameters for Circle Fit, Reconstruct, Trajectory Fit Frame Range. Double-click on CircleFit, Reconstruct, ... in Pipeline Control shows you the list of its parameters. Selecting Show Advanced Parameters lists additional parameters Try the following:

d). Run CircleFit, Reconstruct, ... Right-click on CircleFit, .. and select Run Selected Op from the pop-up window. If you don't see trajectories, go to View Options, and turn on Reconstruction Entities, Reconstructions, and Trajectories.

If a marker is missing a lot, try decreasing Min Cameras to Start Trajectory and Min Cameras to Reconstruct from 3 to 2. Min Cameras to Reconstruct can be decreased to 1. When Min Cameras to Reconstruct is 2, the two cameras that make the tightest angle are used: when it is 1, any pair of cameras are used. Watch out for ghost markers when you decrease Min Cameras to Reconstruct.

Notes on Pipeline Control:

8. Manually labeling a single frame in the ROM trial

Since iQ does not know which marker belongs to which part of a human body, you need to label markers in a single frame in the Range of Motion trial.

Go to Post Processing and select Subjects (the top vertical tab on the right side). Create a new skeleton by clicking on Create Vicon Skeleton Template (VST) and selecting johns_base_markes.vst or UTD_standard_main.vst as a Template.

Go to Labeling (the second vertical tab on the right side). Choose Sequence in Modes and Whole in Rules. Go to the first frame by dragging the time slider to frame 1. (If frame 1 is missing a marker, go forward in time to find the first frame that contains all the markers.) In 3D Worksapce manually label them.

9. Auto labeling the rest of the frames in the ROM trial

iQ labels the rest of the frames in your Range of Motion trial. (Aren't you glad?)

Set the current frame to frame 1 (or the frame that was labeled in the previous step). This is important because Autolabel Range Of Motion, by default, assumes that the current frame is correctly labeled. Go to Pipeline. Select Autolabel Range Of Motion and run it. (If ghosting occurs, increase noise and recalculate. Also run Delete Unlabelled Trajectories.) Continuity Chart (see the capture below) is helpful. Play through the trial and make sure there are no labeling mistakes. Correct any labeling mistakes with the labeling tools. When you correct labels, use Single in Modes and Forward in Rules.


10. Subject calibration using a T-pose in the ROM trial

Subject calibration creates a skeleton that matches the physical dimensions of the captured subject (VSK) by changing the proportions of a generic skeleton (VST) with generic marker positions. A VSK file will be useful when you process and clean data. First of all, labeling markers is much easier with a VSK file: you can simply use "Trajectory Labeler" in Pipeline, instead of manually labeling all the markers for each shot. Also you can run a "Kinematic Fit" that will place the skeleton inside the markers to correct data. See Motion Capture Path 2: Capturing and Processing Data.

Make sure you are in a T-pose frame. Go to Post Processing and select Subjects (the top vertical tab on the right side). Select (General Events) and T (T-pose) in Events (in the right lower corner). Click on to open Calibrate Subject's option window (see the screen capture below). In the option window select Medium Calibration Quality and turn on Autosave. Run Calibrate Subject. When calibration is done, VST (Subject Template) becomes VSK (Calibrated Subject) in Active Subjects list and VSK is saved automatically if Autosave is turned on.



VST. Subject Template servers as an un-calibrated generic model of the subject we are going to be capturing and processing. This is called VST (Vicon Skeletal Template) and exists as a VST file on disk, with the file extension ".vst." VST files can be created and edited using iQ's Modeling mode.

Subject Calibration. In order for iQ to accurately process data, a subject-specific model file must be generated. This process consists of collecting a Range of Motion (ROM) trial for the subject with a marked known pose, typically a T-pose, and then specifying a VST file that has the same general marker locations. Subject Calibration optimizes the marker and skeleton relationships in the VST file to best match the actual locations of the markers on the subject. The proportions of the subject template are changed to match the physical dimensions of the captured subject.

VSK. The result of the Subject Calibration is a calibrated subject file, which exists as a .VSK file in disk. Once a VSK file is produced for each captured subject, it is used for labeling, gap filling, and producing skeletal animation from captured and processed optimal marker data.

Note: When you capture multiple subjects, you can use the same .vst file for all subjects. But you need to create a .vsk file for each subject.

11. Testing the quality of the subject calibration

We can test the quality of the subject calibration by unlabeling all the markers and applying the auto-labeling.

In Post Processing, go to Labeling (the second vertical tab) and Unlabel All. Return to Pipeline and run Trajectory Labeler. This will completely unlabel the trial, and then attempt to re-label using your newly calibrated subject. If this works, then you can be confident that your subject calibration was good. You can also try to run Kinematic Fit at this point, to verify the quality of the subject calibration. To see the kinematically fit skeleton, go to View Options (on your left) and turn on Subjects and Bones (under Subjects).

Next step is capturing and processing data.