Augmented Reality with Planar Homographies · Projects

Overlaying one image onto another in correct perspective is the classic augmented-reality effect. A book cover in a video becomes a screen: find where the plane sits in every frame, warp the overlay to match, and composite.

Two views of the same flat surface are related by a $3 \times 3$ homography acting on homogeneous coordinates,

x^{'} \sim H x, H = h_{1} h_{4} h_{7} h_{2} h_{5} h_{8} h_{3} h_{6} h_{9},

defined up to scale — eight degrees of freedom, so four point correspondences pin it down. Each correspondence contributes two linear constraints, and the direct linear transform stacks them into the $2 N \times 9$ system $A h = 0$ , solved by the smallest singular vector of $A$ . computeH builds $A$ row by row and reads off the last row of $V$ from the SVD; computeH_norm first conditions each point set (shift the centroid to the origin and scale so the mean distance to it is $2$ ), runs the DLT, then undoes the two similarity transforms, which keeps the linear system well-behaved.

A planar homography H maps the four corners of the source plane (the flat book cover) to their positions in the image plane, where the same surface appears in perspective. Four correspondences fix H.

matchPics detects FAST corners in both grayscale images and describes each with a BRIEF binary descriptor, then keeps a match only when the nearest descriptor beats the runner-up by a ratio test. Real matches are polluted with outliers, so computeH_ransac estimates the homography with RANSAC: fit candidate models to minimal random samples and keep the one with the most inliers:

Algorithm:

\textsc{Ransac-Homography}(M, N, \tau)

1
input: putative matches $M$ , iteration budget $N$ , inlier threshold $\tau$
2
best $\gets$ empty set
3
repeat $N$ times
4
sample 4 matches from $M$ at random; fit $H$ by the direct linear transform
5
inliers $\gets$ matches in $M$ with reprojection error $\lVert \mathbf{x}' - H\mathbf{x} \rVert < \tau$
6
if $\lvert$ inliers $\rvert > \lvert$ best $\rvert$ then best $\gets$ inliers
7
refit $H$ on best by least squares
8
return $H$

The implementation samples $4$ correspondences per iteration over a budget of $100$ iterations, counts a point as an inlier when its reprojection error falls under $5$ pixels, and refits the homography on the full inlier set at the end. A sample of $4$ points is all-inlier with probability $w^{4}$ at inlier rate $w$ , so a principled budget follows from $N \geq log (1 - p) / log (1 - w^{4})$ for a target confidence $p$ .

compositeH warps the overlay and an all-ones mask by $H$ with warpPerspective, then blends the warped template over the scene wherever the mask lands — dropping a new cover onto the book in cv_desk.png. Frames for the moving version come from loadVid, and an extra-credit panorama script reuses the same match-and-warp pipeline to stitch overlapping photos. The full write-up is available as a PDF report.

Every frame repeats the same move: the flat overlay is warped by that frame's homography H into the exact perspective quadrilateral where the book cover sits in the video, then composited wherever the warped mask lands. Re-estimating H per frame keeps the overlay pinned to the cover as it moves.

References

Project repository
Reference notes: Linear Algebra

╌╌ END ╌╌