Tracking
The aboriginal footfall is anecdotic and tracking features. A affection is a specific point in the angel that a tracking algorithm can lock assimilate and chase through assorted frames (SynthEyes calls them blips). Generally actualization are called because they are bright/dark spots, edges or corners depending on the accurate tracking algorithm. Accepted programs use arrangement analogous based on NCC account and RMS error. What is important is that anniversary affection represents a specific point on the apparent of a absolute object. As a affection is tracked it becomes a alternation of two-dimensional coordinates that represent the position of the affection beyond a alternation of frames. This alternation is referred to as a track. Already advance accept been created they can be acclimated anon for 2D motion tracking, or again be acclimated to account 3D information.
edit Calibration
The additional footfall involves analytic for 3D motion. This action attempts to acquire the motion of the camera by analytic the inverse-projection of the 2D paths for the position of the camera. This action is referred to as calibration.
To explain further: back a point on the apparent of a three dimensional article is photographed its position in the 2D anatomy can be affected by a 3D bump function. We can accede a camera to be an absorption that holds all the ambit all-important to archetypal a camera in a absolute or basic world. Therefore a camera is a agent that includes as its elements the position of the camera, its orientation, focal length, and added accessible paremeters that ascertain how the camera focuses ablaze assimilate the becloud plane. Absolutely how this agent is complete is not important as continued as there is a accordant bump action P.
The bump action P takes as its ascribe a camera agent (denoted camera) and addition agent the position of a 3D point in amplitude (denoted xyz) and allotment a 2D point that has been projected assimilate a alike in advanced of the camera (denoted XY). We can accurate this:
XY = P(camera, xyz)
An analogy of affection projection. About the apprehension of a 3D structure, red dots represent credibility that are called by the tracking process. Cameras at anatomy i and j activity the actualization assimilate a alike depending on the ambit of the camera. In this way actualization tracked in 2D accord to absolute credibility in a 3D space. Admitting this analogy is computer generated, bout affective is commonly done on absolute objects.
The bump action transforms the 3D point and strips abroad the basic of depth. Without alive the abyss of the basic an changed bump action can alone acknowledgment a set of accessible 3D points, that anatomy a band appearing from the centermost of the camera and casual through the projected 2D point. We can accurate the changed bump as:
xyz ∈ P'(camera, XY)
or
{xyz :P(camera, xyz) = XY}
Let's say we are in a bearings area the actualization we are tracking are on the apparent of a adamant article such as a building. Since we apperceive that the absolute point xyz will abide in the aforementioned abode in absolute amplitude from one anatomy of the angel to the abutting we can accomplish the point a connected alike admitting we do not apperceive area it is. So:
xyzi = xyzj
where the subscripts i and j accredit to approximate frames in the attack we are analyzing. Since this is consistently accurate again we apperceive that:
P'(camerai, XYi) ∩ P'(cameraj, XYj) ≠ {}
Because the amount of XYi has been bent for all frames that the affection is tracked through by the tracking program, we can break the about-face bump action amid any two frames as continued as P'(camerai, XYi) ∩ P'(cameraj, XYj) is a baby set. Set of accessible camera vectors that break the blueprint at i and j (denoted Cij).
Cij = {(camerai,cameraj):P'(camerai, XYi) ∩ P'(cameraj, XYj) ≠ {})
So there is a set of camera agent pairs Cij for which the circle of the changed projections of two credibility XYi and XYj is a non-empty, hopefully small, set absorption about a abstract anchored point xyz .
In added words, brainstorm a atramentous point amphibian in a white abandoned and a camera. For any position in amplitude that we abode the camera, there is a set of agnate ambit (orientation, focal length, etc) that will photograph that atramentous point absolutely the aforementioned way. Since C has an absolute cardinal of members, one point is never abundant to actuate the absolute camera position.
As we alpha abacus tracking points, we can attenuated the accessible camera positions. For archetype if we accept a set of credibility {xyzi,0,...,xyzi,n} and {xyzj,0,...,xyzj,n} area i and j still accredit to frames and n is an basis to one of abounding tracking credibility we are following. We can acquire a set of camera agent brace sets {Ci,j,0,...,Ci,j,n}.
In this way assorted advance acquiesce us to attenuated the accessible camera parameters. The set of accessible camera ambit that fit, F, is the circle of all sets:
F = Ci,j,0 ∩ ... ∩ Ci,j,n
The beneath elements are in this set the afterpiece we can appear to extracting the absolute ambit of the camera. In absoluteness errors alien to the tracking action crave a added statistical access to chargeless a acceptable camera agent for anniversary frame, access algorithms and array block acclimation are generally utilized. Unfortunately there are so abounding elements to a camera agent that back every constant is chargeless we still ability not be able to attenuated F bottomward to a distinct achievability no amount how abounding actualization we track. The added we can bind the assorted parameters, abnormally focal length, the easier it becomes to ascertain the solution.
In all, the 3D analytic action is the action of absorption bottomward the accessible solutions to the motion of the camera until we ability one that apparel the needs of the blended we are aggravating to create.
edit Point-cloud projection
Once the camera position has been bent for every anatomy it is again accessible to appraisal the position of anniversary affection in absolute amplitude by changed projection. The consistent set of credibility is generally referred to as a point billow because of its raw actualization like a nebula. Since point clouds generally acknowledge some of the appearance of the 3D arena they can be acclimated as a advertence for agreement constructed altar or by a about-face affairs to actualize a 3D adaptation of the absolute scene.
edit Ground-plane determination
The camera and point billow charge to be aggressive in some affectionate of space. Therefore, already arrangement is complete, it is all-important to ascertain a arena plane. Normally, this is a assemblage alike that determines the scale, acclimatization and agent of the projected space. Some programs attack to do this automatically, admitting added generally the user defines this plane. Since alive arena planes does a simple transformation of all of the points, the absolute position of the alike is absolutely a amount of convenience.
edit Reconstruction
Reconstruction is the alternate action of recreating a photographed article application tracking data. This address is accompanying to photogrammetry. In this accurate case we are apropos to application bout affectivecomputer application to reconstruct a arena from accidental footage.
A about-face affairs can actualize three-dimensional altar that actor the absolute altar from the photographed scene. Application abstracts from the point billow and the user's estimation, the affairs can actualize a basic article and again abstract a arrangement from the footage that can be projected assimilate the basic article as a apparent texture.
edit 2D vs. 3D
Match affective has two forms. Compositing programs, such as Shake, Adobe After Furnishings and Discreet Combustion, accept two-dimensional motion tracking capabilities. This affection translates images in two-dimensional amplitude and can add furnishings such as motion becloud in an attack to annihilate about motion amid two actualization of two affective images. This address is acceptable to actualize color back the two images do not accommodate above changes in camera perspective. For archetype a advance abysmal in the accomplishments of a attack can generally be replaced application two-dimensional tracking.
Three-dimensional bout affective accoutrement accomplish it accessible to extrapolate three-dimensional advice from two-dimensional photography. Programs able of 3D bout affective include:
Blender (open source; uses libmv)
Voodoo (freeware)
Scenespector VooCAT
Icarus (University of Manchester analysis project, now discontinued but still popular)
Maya MatchMover 2010
The Pixel Farm PFTrack, PFMatchit, PFHoe (based on PFTrack algorithms)
REALVIZ MatchMover (Autodesk bought and re-released as allotment of Maya 2010 bundle)
Science.D.Visions 3DEqualizer (which won an Academy Accolade for Technical Achievement)
Andersson Technologies LLC's SynthEyes
Boujou (which won an Emmy accolade in 2002)
NukeX from The Foundry
CameraTracker (a constituent for Adobe After Effects) from The Foundry.
VideoTrace from Punchcard (innovativecomputer application for breeding 3D models from video and images)
IXIR 2D Clue Editor It’s able of 2D advance and Mask files of softwares like 3D Equalizer, PFTrack, Boujou, SynthEyes, Matchmover, Movimento, Nuke, Shake, Fusion, After Effects, Combustion, Mocha, Silhouette
These programs acquiesce users to acquire camera movement and added about motion from approximate footage. The tracking advice can be transferred to computer graphicscomputer application such as Blender, 3ds Max, Maya or LightWave and acclimated to breathing basic cameras and CGI objects.