Matching data for 3D reconstruction :

This directory tree contains data that can be used as input for algorithms that perform 3D reconstruction from image features that have been tracked along a sequence.Three short sequences have been acquired and visible objects have been selected. Some 3D points on each objects have been tracked and the data stored in files.

We hope that this data will be useful to people testing reconstruction algorithms : using these files should be easier than acquiring images and tracking points.

Contents description :

This dataset consists in :

Three uncalibrated greylevel image sequences, in "tif" format, of five to twelve images.
The result of hand-tracking some points along these sequences : some 3D points have been selected for their visibility and identifiability, and the pixel coordinates of their projections are recorded. The result is stored ascii files readable by "matlab".
Some auxiliary data, such as tentative reconstructions obtained from that data, in VRML 2.0 and "matlab" formats.

Directory structure :

Each of the following directories contains data relevant to a single image sequence :

close_range: Five close-range views of calibration grid. Distance is appoximately twice the object size. (see the first image)
long_range : Twelve views of objects seen from long range. Distance is 5 to 10 times the object sizes.(see the first image)
noodle_drain : Twelve views of a noodle (see the first image)

Each directory contains a SEQUENCE/image subdirectory containing the "tif" images, as well as at least one SEQUENCE/OBJECT directory that contains the data pertaining to a given object :

close_range/grid: Five close-range views of calibration grid. (48 points).
long_range/grid : Twelve views of the calibration grid seen from long range (48 points).
long_range/box : Ten views of the bottom plastic box. (14 points).
long_range/cardboard : Ten views of the cardboard box. (16 points).
long_range/all : Twelve views of all three objects in the "long_range" sequence. Not all observations are available. (78 points).
noodle_drain/drain : Twelve views of a noodle drain.(45 points).

Naming conventions used for the files :

The above directories contain at the following files :

matches.m contains the matching data, in the form of "matlab" code (maybe "octave" can run it too). In this file are defined the following variables :
- N the number of images from the sequence that have been used.
- P the number of points that have been tracked.
- imsz a 1-by-2 matrix containing the height and width of the original images.
- u0 a 2N-by-P matrix containing the matching data itself : each column contains the pixel coordinates of the projection of a given 3D point. Each pair of successive rows contain the X and Y coordinates of the projection in a given image of the sequence. Missing observations are given the value 9999.
  Important Note : the coordinates are taken from bottom to top, left to right. In order to obtain a direct frame whose Z-axis points forward (towards the object), one of the coordinate axes in the image plane must be inverted. It is customary to invert the Y-axis, so that coordinates are taken from top to bottom.
ml_OBJECT.m contain a tentative reconstruction obtained from the data in the corresponding matches.m file. It is a minimum of the likelihood function when one assumes the observations are subject to Gaussian noise and obtained by perspective projection. No radial distortion is assumed. This file is provided for illustration purpose only and there is *no warranty* that this corresponds to the global maximum likelihood solution. These values may e.g. be used as initial values for an estimation algorithm. In this file, the following variables are defined :
- x, a 3-by-P estimate of the positions of the 3D points. This estimate is normalised to have the origin of coordinates as center of mass (tnat is, the sum of each row is 0). Also, the mean squared value of the components is zero.
- a, a 3N-by-P matrix, whose consecutive 3-by-3 blocks are orthogonal matrices representing the estimated orientation of the camera.
- t, 3N-by-1, represents the position of the world origin in the camera frame.
- k, 1-by-5 represents the estimated intrinsic parameters.
One assumes that the observations u0(2*f-i,p), for f=1..N, p=1..P and i=1..2, are obtained through the perspective projection model. We first define the simplified projection model :
```
               a(3*f-i,:)*(x(:,p)+t(3*f-2:3*f))
V(2*f-i,p)  =  -------------------------------- 
               a(3*f,:)  *(x(:,p)+t(3*f-2:3*f))
```
This is the projection of the 3D point on the image plane, assuming that the focal length is 1, the sensor is skewless, the aspect ratio is 1 and the coordinates are taken relative to the principal point.

The full projection model also integates the skew, aspect ratio, focal length, principal point and noise.
```
u0(2*f-[1,0],p)  = exp(-k(5)) * ( K * V(2*f-[1,0],p) + C ) + Noise
```
Where
```
K           = [   1              0  ]        Represents the skew of the
              [ 10*k(1)  10*k(2)+1  ]        sensor and its aspect ratio.

and

C           = 10*[ k(3) ]                    Is the image center 
                 [ k(5) ]                    ("principal point")
```
The focal length of the camera is exp(k(5)).

More details can be found in many documents on computer vision, for example, at CVonline.
ml_OBJECT.wrl contain the same data as ml_OBJECT.m, but represented as vrml code. This file is provided only for illustration purpose.
Additionaly, since the SEQUENCE/grid directories contain information pertaining to points of a known 3D object (the calibration grid), a file priorx.m is present which defines a 3-by-P matrix that represents the 3D points on the calibration grid. This file can e.g. be used as an initial value for an estimation algorithm.

Downloads

The whole directory tree, tar and gzipped. (~6MB)
The "light" version with only a few images showing the matched points, tar and gzipped. (~720KB)

Acknowledgments

This data has been produced within the INCO - Copernicus Project 960174 (VIRTUOUS)

Disclaimer/Author

The data in this directory may contain some errors. Please contact Etienne Grossmann for any comments and questions.