METHODOLOGY
The methodology is dividedinto two major steps:
* Learning epipolar geometry.
* Using epipolar geometry to achieve 3-d reconstruction.
The methodology is explained for the examples shown previously
Part I : Learning epipolar geometry
The steps involved in finding fundamental matrix from
uncalibrated images are :
Part II : Using Epipolar geometry
CORNER DETECTION
Basics
Our primary job is to search point which can be identified in both the
images. Searching these points would have been difficult if we had gone searching any arbitary point.
However, what we do is that we search for corners which we hope can be
identified easily in both the images. That is, for a corner in one image we will be searching only the corner points of the other image. This reduces the search. To find the corner points we search for points with high curvature.
Theory
Following operator is applied on the image(.pgm format)
R(x,y) = det[C] - k trace [C]
Îx² Îxy
where C =
Îxy Îy²
Ix, Iy are the derivatives of the image in x & y direction respectively.
Î is the smoothened image.
k = .04 and threshold is applied on the R matrix.
Source Codes
Results
The 'Tripod' image
TOP
MATCHING
Basics
For each of the corner point in first image we have to find the correcponding point in the second image and vice-versa. Now what does this correspondence means ? Two points( one in each image) are called matches if they correspond to same point on the viewed object.
This is a difficult problem since we have no information about the the scenery. However,
it can be tackled by considering the correlation score of the two points. Correlation score determines how close a portion in picture matches with the portion in other. This value
is 1 if it matches perfectly and -1 if it does not match at all. A threshold is applied
on the value of score found.
Theory
For each corner point (u1,v1) in first image we draw a search window of quarter the image
size about this point in second image. We assume that the match is there in this region.
To find the correlation score we consider window of size 15×15(painted red in fig. below) about (u1,v1) and (u2,v2) and apply the standard correlation technique.
Source Codes
Results
TOP
DISAMBIGUATING THE MATCHES
Basics
During the matching procedure we discussed in last section it may so happen that a point may correspond to more than one point. We need to remove this ambiguity in matching. To do so we calculate what is called strength of match. A potential match having highest strength is selected. The concept of strength of match is that if m1 and m2 are good match then we expect to find more good matches in neighbourhood of m1 and m2.
After the end of this procedure we would have unique corespondence from both images.
Theory
To find the strength of match of points m1 & m2 we do the following. For each point n1 in the neighbourhood of m1 we find a point n2 in neighbourhood of m2 such that n1 & n2 are
good matches. By good matches we not only mean having high corr. score but also having similar orientation w.r.t. the base points. This level is measured in mathematical terms and
is summed for each point in neighbourhood of m1.
Source Codes
Results
The 'Tripod' image(Points of same color correspond)
FINDING FUNDAMENTAL MATRIX
Basics
From the set of points of correspondence,
we calculate the fundamental matrix by Least Median
of Squares method.This method requires finding fundamental matrix for large sample of
eight points. From these matrices obtained we select that which minimizes error for wlole of the matching subspace.
This method is quite robost as it takes into account
the unmatched pair also.
Theory
If (u1,v1) & (u2,v2) are the two corresponding points in two images then the fundamental matrix is found by solving :
Af=0
where f = { F11 , F12 , F13 , F21 , F22 , F23 , F31 , F32 , F33 }
and ith row of A={ u1*u2 , v1*v2 , v1 , u1*v2 , v1*v2 , v2 , u1 , u2 }
where (u1,v1)&(u2,v2) are the ith matches.
Source Codes
Results
Fundamental matrix obtained for the 'tripod' example is :
0.000000 -5.987453 341.905914
4.871140 .477927 -255.715240
-165.803834 94.000000 -1
TOP
ESTIMATING RELATIVE CAMERA POSITIONS
Estimation of Relative camera position
from uncalibrated images using a non-iterative tecnique by first finding the singular value decomposition of the estimated fundamental matrix.The algorithm which we have used non-iteratively finds the focal lengths of the two cameras along with there relative displacement,in the combined form of camera projection matrix , as long as other internal parameters of the cameras are known.
If the estimated fundamental matrix has rank =2 then there are four possible solutions for projection matrix for the second camera , which are found my using matrices formed by
by the orthogonal eigen vectors obtained in the singular value decomposition.If the rank is not equal to 2 then the closest matrix to estimated fundamental matrix which has rank 2 is found and it is then used to get the projection matrices for the two cameras.The major part of the algorithm is concerned with finding this closest matrix so that a good estimate of projection matrix & hence the relative camera positions can be made.It is the projection matrix which helps us find co-ordiantes of any object point in a world frame.
Out of the four probable solutions for the projection matrix for the second camera ,one is set as the required matrix according to the requirement that the matched points must lie in front of both the cameras.
Source Codes
TOP
SEARCHING FOR POINT CORRESPONDENCE
using epipolar constraint i.e. once the fundamental matrix has been been found the point corresponding to a point in one image can lie only on a line, called the epipolar line .In our approach,for each point on first image we first find the epipolar line corresponding to it & then its cooresonding point is found using template matching for all points lying on the line.The epipolar lines found are shown in the following images.Theoretically all epipolar lines must be concurrent ,which is what we have obtained.
Results
The red lines show the epipolar lines.
By looking at the pair of tripod images it seems that only camera has been translated to get the two images , hence there must not be any change in y-cordinates of the image points.Notice that y-cordinate of epipoles in the two images is the same.
TOP
USING DEPTH MAP
Depth maps are one of the ways of conveying the 3D information about a scene.In our case we are showing the results using depth maps, in which the range of depth of points has been divided into intervals & the points with depth in each depth-interval are shown in same colour.Thus different colours in deth maps represent different different degree of closeness.
Results
