Image Basics (Week 1-3) |
Mon, Aug. 30 |
Lec. 1: Introduction
Course: logistics
Computer vision: goal, applications
|
Motivation: why computer vison is so cool
|
ps1 out (prerequisite)
lab1 out
final project (ideas out)
|
Wed, Sep. 1 |
Lec. 2: Image Formation
Light: EM wave, metamer, gamma curve
Material: diffusion, specular, BRDF
|
Imaging: optics and human perception
|
|
Fri, Sep. 3 |
Lab 1: Linear Algebra for Digital Images
Links: [pdf], [Colab]
Representation: matrix (NumPy)
Transformation: patch-wise function
|
Maths review: linear algebra
|
ps1 due
|
No Class (Labor Day) |
Wed, Sep. 8 |
Lec. 3: Cameras
Basic: Pin-hole and perspective geometry
Improvement: lens, digital sensor
|
Imaging: a brief history of cameras
|
ps2 out (formation)
lab2 out
|
Fri, Sep. 10 |
Lab 2: Calculus for Function Optimization
Links: [pdf], [Colab]
Optimization 101
Gradient: multivariate, gradient descent
|
Maths review: calculus
|
survey |
Mon, Sep. 13 |
Lec. 4: Image Processing I (Filtering)
Linear/non-linear filters
Edge, denoising
|
Filtering: simple image filters
|
|
Wed, Sep. 15 |
Review Session
Pset2, Lec. 4
|
|
|
Fri, Sep. 17 |
Lec. 5: Image Processing II (Decomposition)
Fourier transform: 1 layer, N filters
|
Filtering: filter basis (breadth)
|
ps2 due |
Mon, Sep. 20 |
Lec. 5 (part 2)
Image pyramid: N layers, 1 filter
|
Filtering: filter recursion (depth)
|
ps3 out (filter) |
Deep Learning (Week 4-6) |
Wed, Sep. 22 |
Lec. 6: Machine Learning I
Machine learning overview
|
AlexNet: linear layer
|
|
Fri, Sep. 24 |
Lec. 7: Machine Learning II
Linear regression
Logistic regression, loss function
|
AlexNet: loss layer
|
|
Mon, Sep. 27 |
Lab. 3: Machine Learning
|
AlexNet: Softmax layer
|
lab3 out |
Wed, Sep. 29 |
Lec. 8: Neural Networks I
Handcrafted features
Activation functions
|
AlexNet: fully-connected layer
|
|
Fri, Oct. 1 |
Lec. 9: Neural Networks II
Computation graphs
Backpropagation
|
AlexNet: optimization
|
|
Mon, Oct. 4 |
Lab 4: Multilayer Perceptron
Links: [pdf], [Colab]
Backpropagation as dynamic programming
CIFAR10: dataloader, tensorboard
|
|
ps3 due
lab4 out
final project (team info due)
|
Wed, Oct. 6 |
Lec. 10: Convolutional Networks
Convolution and pooling layers
Normalization and dropout layers
|
AlexNet: layers for efficient learning from images
|
|
Fri, Oct. 8 |
Lec. 11: AlexNet and Beyond
AlexNet visualization
VGGNet, BatchNorm, ResNet
|
Beyond AlexNet: three tricks to crack ImageNet
|
ps4 out (ML+NN)
|
Image Understanding (Week 7-8) |
No Class (Fall Break)
|
Tue, Oct. 12 |
Lec. 12: Image Prediction
Applications and challenges
MVP and design choices
|
Target domain: scalar label
|
final project (guideline)
|
Wed, Oct. 13 |
Lec. 13: Object Detection I
Problem formulation
Two-stage and single-stage
|
Target domain: bounding boxes
|
|
Fri, Oct. 15 |
Lec. 14: Object Detection II
R-CNN-based, YOLO
|
Target domain: bounding boxes
|
|
Mon, Oct. 18 |
Lab 5: Image Prediction and Object Detection in PyTorch
|
|
ps4 due
lab5 out
|
Wed, Oct. 20 |
Lec. 15: Image Segmentation
Generic/Semantic/Instance segmentation
FCN, SegNet, U-Net
|
Target domain: grid label
|
ps5 out (finetune) |
Fri, Oct. 22 |
Lec. 16: Image Generation
Links: [video]
GAN and conditional GAN
Style transfer
|
Target domain: natural images
|
final project (proposal due)
|
Mon, Oct. 25 |
Lab 6: Image Segmentation and Genereation in PyTorch
|
|
lab6 out |
Video Understanding (Week 9-10) |
Wed, Oct. 27 |
Lec. 17: Motion
Links: [video]
Motion representation
Optical flow
|
Target domain: grid label
|
|
Fri, Oct. 29 |
Lec. 18: Video Prediction
Links: [video]
Action recognition
3D CNN and RNNs
|
Target domain: scalar label
|
ps5 due |
Mon, Nov. 1 |
Lec. 19: Object Tracking I
Links: [video]
Single-object tracking, Siamese NN
|
Target domain: temporally-linked bounding boxes
|
check-in sign-up |
Wed, Nov. 3 |
Lec. 20: Object Tracking II
Links: [video]
Multi-object tracking, Graph NN
|
Target domain: temporally-linked bounding boxes
|
|
Fri, Nov. 5 |
Lab 7: Tips For The Final Project
|
|
|
3D Understanding (Week 11-12) |
Mon, Nov. 8 |
Lec. 21: Image Transformation
Links: [video]
Homogeneous coordinate, homography
|
Target domain: transformation matrix
|
ps6 out (motion)
final project (check-in) |
Wed, Nov. 10 |
Lec. 22: Image Alignment
Links: [video]
Feature matching, RANSAC
|
Target domain: transformation matrix
|
|
Fri, Nov. 12 |
Lec. 23: Image Feature
Links: [video]
Feature detection, SIFT
|
Target domain: invariant feature
|
|
Mon, Nov. 15 |
Lec. 24: Stereo Vision
Perspective geometry
Stereo matching, depth prediction
|
Target domain: grid label
|
|
Wed, Nov. 17 |
Lec. 25: Point Cloud
Links: [audio]
Epipolar geometry, Structure from motion
PointNet
|
Source domain: point cloud
|
ps6 due |
Frontiers in Vision (Week 13-14)
|
Fri, Nov. 19 |
Guest Lecture: Andrea Vicini, S.J.
Professor @ BC
Ethics in AI/Biomedical Research
|
|
ps7 out (stitch) |
Mon, Nov. 22 |
Guest Lecture: James Tompkin
Assistant Professor @ Brown University
computer vision for creative media
|
|
final project (draft slide due)
|
No Class (Thanksgiving Break) |
Mon, Nov. 29 |
Lec. 26: Multimodal Learning
Vision and soud
Vision and text
|
|
|
Wed, Dec. 1 |
Guest Lecture: Yue Wang
Final-year PhD @ MIT
Point cloud recognition, autonomous driving
|
|
|
Fri, Dec. 3 |
Guest Lecture: Kristen Grauman
Professor @ UT Austin
(BC alum!)
Vision and sound
|
|
ps7 due |
Final Projects (Week 15-16) |
Mon, Dec. 6 |
Presentation I
|
|
final project (final slide due)
|
Wed, Dec. 8 |
Presentation II
|
| |
Mon, Dec. 20 |
|
| final project (report/code due) |