Module 5: Geometric Vision and Camera Models

1. Learning Objectives

• Understand the pinhole camera model and camera calibration
• Learn intrinsic and extrinsic parameters of cameras
• Understand homographies and epipolar geometry
• Explore stereo vision and depth estimation

2. The Pinhole Camera Model

The pinhole camera model is a mathematical abstraction describing how a 3D point in the world maps to a 2D point on the image plane.

x = K [R | t] X

• X: 3D world coordinate
• [R | t]: Extrinsic parameters (rotation and translation)
• K: Intrinsic matrix (focal length, principal point, etc.)
• x: 2D pixel coordinates

3. Intrinsic Parameters

• Focal length (fx, fy)
• Optical center (cx, cy)
• Skew coefficient (rare)

4. Extrinsic Parameters

• Rotation matrix (R)
• Translation vector (t)
• Describes how camera is placed in the world

5. Camera Calibration

Used to determine intrinsic and extrinsic parameters using images of a known object (like a checkerboard).

6. Homography

A homography is a transformation that maps points from one plane to another using a 3x3 matrix:

x' = Hx

Used for planar transformations, image stitching, and motion estimation.

7. Epipolar Geometry

• Describes geometric relationship between two views
• Key terms: epipole, epipolar line, fundamental matrix

8. Stereo Vision and Depth Estimation

• Uses two cameras to compute depth from disparity
• Applications: 3D reconstruction, robotics, AR

9. Hands-On Lab: Camera Calibration (OpenCV)

import cv2
import numpy as np
import glob

# Define checkerboard dimensions
CHECKERBOARD = (6, 9)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

objp = np.zeros((CHECKERBOARD[0]*CHECKERBOARD[1], 3), np.float32)
objp[:, :2] = np.mgrid[0:CHECKERBOARD[1], 0:CHECKERBOARD[0]].T.reshape(-1, 2)

objpoints = []
imgpoints = []

images = glob.glob('*.jpg')  # use uploaded checkerboard images

for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD, None)
    if ret:
        objpoints.append(objp)
        corners2 = cv2.cornerSubPix(gray, corners, (11,11), (-1,-1), criteria)
        imgpoints.append(corners2)
        img = cv2.drawChessboardCorners(img, CHECKERBOARD, corners2, ret)
        cv2.imshow('img', img)
        cv2.waitKey(500)

cv2.destroyAllWindows()

ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
print("Camera matrix:
", mtx)

10. Assignment