Module 6: Classical Machine Learning for Vision

1. Learning Objectives

• Understand the difference between supervised and unsupervised learning
• Apply KNN, SVM, and Decision Trees to visual features
• Evaluate model performance using accuracy, precision, recall

2. Supervised vs. Unsupervised Learning

Supervised: Training data has input-output pairs (e.g. image + label)

Unsupervised: No labels provided, models find patterns (e.g. clustering)

3. Common Classical Algorithms

• K-Nearest Neighbors (KNN): Classifies based on majority vote of neighbors
• Support Vector Machines (SVM): Finds decision boundary that maximizes margin
• Decision Trees: Recursive splitting based on feature values

4. Feature Extraction

Classical methods require hand-crafted features. Example:

from skimage.feature import hog
features, _ = hog(image, pixels_per_cell=(8, 8), cells_per_block=(2, 2), visualize=True)

5. Example Pipeline

1. Load image dataset (e.g., digits or CIFAR-10)
2. Extract features (HOG, grayscale histograms, etc.)
3. Train a classical model (SVM, KNN, etc.)
4. Evaluate accuracy on test set

6. Hands-On Lab

from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import classification_report

digits = load_digits()
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size=0.3, random_state=42)

clf = SVC(kernel='linear')
clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))

7. Evaluation Metrics

• Accuracy: (TP + TN) / Total
• Precision: TP / (TP + FP)
• Recall: TP / (TP + FN)
• F1 Score: Harmonic mean of precision and recall

8. Assignment