Overview

CSCI 3343 is an advanced undergraduate-level computer vision class covering fundamental image basics (week 1-3), state-of-the-art deep learning (week 4-7) methods, and a wide range of real-world applications (week 8-14).

Schedule

  • [R]eading: read through the materials before the lecture.
  • [C]oding: coding assignment after the lecture.
ps4 out (CNN)
Theme Date Topic Materials Assignments
Image Basics
(Week 1-3) 		Tue, Aug. 30 Lec. 1: Introduction
Logistics
Course overview
 WHY: computer vison applications final project (fp): ideas
Thu, Sep. 1 Lec. 2: Image Formation
Light: gamma curve
Material: BRDF
Camera: geometry, color
 WHAT: optics, and human vision, digitalization
Tue, Sep. 6 Lec. 3: Point Operations
Transfer function
HOW: pixel-level
Thu, Sep. 8 Lec. 4: Filtering
Linear and nonlinear filters
HOW: patch-level
  • [R] CV: 3.2-3.3
  • [C] OpenCV
 lab1 due
ps1 out (formation)
Tue, Sep. 13 Lec. 5: Image Features
Edge, line, corner
 HOW: feature-level
  • [R] CV: 7.2
Thu, Sep. 15 Lec. 6: Image Decomposition
Frequency: Fourier basis, antialiasing
Scale: pyramid, blending
HOW: frequency/pyramid-level lab2 due
Deep Learning
(Week 4-7)		 Tue, Sep. 20 Lec. 7: Machine Learning I
Machine learning overview
Linear regression
AlexNet: linear layer
  • [R] D2L: 1, 3
 ps1 due
ps2 out (processing)
Thu, Sep. 22 Lec. 8: Machine Learning II
Logistic regression
Stochastic gradient descend
 AlexNet: loss layer lab3 due
Tue, Sep. 27 Lec. 9: Neural Networks I
Activation functions
Pytorch Tutorial
 AlexNet: fully-connected layer
Thu, Sep. 29 Lec. 10: Neural Networks II
Computation graphs
Backpropagation
 AlexNet: optimization lab4 due
fp: sign-up due
ps2 due
Tue, Oct. 4 Lec. 11: Convolutional Networks
Convolution and pooling layers
Normalization and dropout layers
 AlexNet: layers for efficient learning from images Mon: ps3 out (NN)
Thu, Oct. 6 Lec. 12: AlexNet and Beyond
VGGNet, BatchNorm, ResNet
Network visualization
Beyond AlexNet: three tricks to crack ImageNet
  • [R] D2L: 8
 lab5 due
Tue, Oct. 11 No Class (Monday schedule)
Thu, Oct. 13 Lec. 13: Review CNN
Lab 6: Forward and backward pass
Pset 3: Dataset and dataloader
fp: proposal check-in (in-person)
Image Understanding
(Week 8-9)		 Tue, Oct. 18 Lec. 14: Hacker's Guide
Data: preprocessing, augmentation
Model: activation,
Optimization: optimizer, fine-tuning
AlexNet: data and optimization tricks lab6 due
ps3 due
fp: proposal due
Thu, Oct. 20 Lec. 15: Object Detection
Problem formulation
Two-stage and single-stage
 Target domain: bounding boxes
Tue, Oct. 25 Lec. 16: Image Segmentation
Generic/Semantic/Instance segmentation
FCN, SegNet, U-Net
 Target domain: grid label
Thu, Oct. 27 Lec. 17: Image Generation
GAN and conditional GAN
Target domain: natural images
  • [R] D2L: 18
 lab8 due
Video Understanding
(Week 10-11)		 Tue, Nov. 1 Lec. 18: Motion
Motion representation
Optical flow
Target domain: grid label
  • [R] CV: 8
 fp: data check-in
ps4 due
Thu, Nov. 3 Lec. 19: Video Classification
Action recognition
Recurrent Neural Network
Target domain: temporally-linked bounding boxes lab9 due
Tue, Nov. 8 Lec. 20: Object Tracking
Single-object tracking, Siamese NN
Multi-object tracking, Graph NN
Target domain: temporally-linked bounding boxes
3D Understanding
(Week 11-14)		 Thu, Nov. 10 Lec. 21: Image Transformation and Alignment
Homogeneous coordinate, homography
Feature matching, RANSAC
 Target domain: transformation matrix
  • [R] CV: 2.1, 3.6.1
  • [R] CV: 7.1
 lab10 due
Tue, Nov. 15 Guest Lecture: Dr. Pengchuan Zhang
Research scientist @ Meta AI
Affliated assistant prof. @ UW
Vision-language multimodal intelligence
fp: model check-in
Thu, Nov. 17 Guest Lecture: Dr. Deqing Sun
Staff Research Scientist and Manager
Google
3D Understanding
(Week 11-14)		 Tue, Nov. 22 Lec. 22: Stereo Vision
Perspective geometry
Stereo matching, depth prediction
Target domain: grid label
  • [R] CV: 12
 ps5 out (stitching)
No Class (Thanksgiving Break)
Tue, Nov. 29 Lec. 23: Geometric Deep Learning
PointNet
 Source domain: point cloud
Multi-modal Understanding
(Week 14)		 Thu, Dec. 1 Lec. 24: Vision + X
Lanugage: GPT-3
Audio
Tue, Dec. 6 Guest Lecture: Dr. Ce Liu
Chief Architect for Computer Vision
Microsoft Azure AI
ps5 due
Thu, Dec. 8 Guest Lecture: James Tompkin
Assistant Professor @ Brown University
computer vision for creative media
fp: in-person slide check-in
Final Projects
(Week 16)		 Tue, Dec. 13 Final Week
fp: slide due
Thu, Dec. 15 Final Week
Mon, Dec. 19 final project (report/code due)


Staff & Office Hours


Instructor
Jerry Hou
TA
Nicholas Arangio
TA
Name Office hours
Donglai Wei (W/Th) 3-4 pm @ Rm 528F, 245 Beacon ST
Jerry Hou (Tu) 2-3 pm, (W) 6-7 pm @ Rm 122, 245 Beacon ST
Nicholas Arangio (W) 1-3 pm @ Zoom
  • Office hours will take place in person (or Zoom if needed).
  • Donglai will hold additional one-on-one AMA office hours (M) 1-2 pm, (F) 2-3pm (15-min by appointment)


Course information

This is a challenging course and we are here to help you become a more-AI version of yourself. Please feel free to reach out if you need help in any form.

1. Get help (besides office hours)

  • Discord: For labs/psets/final projects, we will create dedicated channels for you to ask public questions. If you cannot make your post public (e.g., due to revealing problem set solutions), please create a room with instructors and TAs separately, or come to office hours. Please note, however, that the course staff cannot provide help debugging code, and there is no guarantee that they'll be able to answer last-minute assignment questions before the deadline. We also appreciate it when you respond to questions from other students! If you have an important question that you would prefer to discuss over email, you may email the course staff, or the instructor directly.
  • Support: The university counseling services center provides a variety of programs and activities.
  • Accommodations for students with disabilities: If you are a student with a documented disability seeking reasonable accommodations in this course, please contact Kathy Duggan, (617) 552-8093, dugganka@bc.edu, at the Connors Family Learning Center regarding learning disabilities and ADHD, or Rory Stein, (617) 552-3470, steinr@bc.edu, in the Disability Services Office regarding all other types of disabilities, including temporary disabilities. Advance notice and appropriate documentation are required for accommodations.

2. Assignment

All programming assignments are in Python on Colab, always due at midnight (11:59 pm) on the due date.
  • Install Colab on the browser: Sign in to your Google account, follow the [link] to the folder of assignments, click on lab1.ipynb, click on "Open with" and "Connect more apps", install "Colaboratory".
  • Submission: You need save a copy of the file in your own Google drive, so that you can save your edits. Afterwards, you can download the ipynb file and submit it to Canvas.
  • Lab (1 per week): Every lecture has a lab exercise to help you gain the hands-on understanding about the material. The lab on previous week's lectures is due on Thursday. We will go through some code in class and you need to finish up the exercises.
  • Pset (1 per 1.5 week): In each pset, we will build a prototype component for the self-driving car with the learned materials.
  • Final project: (guideline) In lieu of a final exam, we'll have a final project. This project will be completed in small groups during the last weeks of the class. The direction for this project is open-ended: you can either choose from a list of project ideas that we distribute, or you can propose a topic of your own. A short project proposal will be due approximately halfway through the course. During the final exam period, you'll turn in a final report and give a short presentation. You may use an ongoing research work for your final project, as long it meets the requirements.

3. Academic policy

  • Late policy: You'll have 10 late days each (counting weekends) for labs and psets respectively over the course of the semester. Each time you use one, you may submit an assignment one day late without penalty. You are allowed to use multiple late days on a single assignment. For example, you can use all of your days at once to turn in one assignment a week late. You do not need to notify us when you use a late day; we'll deduct it automatically. If you run out of late days and still submit late, your assignment will be penalized at a rate of 10% per day. We will not provide additional late time, except under exceptional circumstances, and for these we'll require documentation (e.g., a doctor's note). Please note that the late days are provided to help you deal with minor setbacks, such as routine illness or injury, paper deadlines, interviews, and computer problems; these do not generally qualify for an additional extension.
  • Academic integrity: While you are encouraged to discuss homework assignments with other students, your programming work must be completed individually. Thus it is acceptable to learn from another student the general idea for writing program code to perform a particular task, or the technique for solving a mathematical problem, but unacceptable for two students to prepare their assignments together and submit what are essentially two copies of identical work. If you have any uncertainty about the application of this policy, please check with me. Failure to comply with these guidelines will be considered a violation of the University policies on academic integrity. Please make sure that you are familiar with these policies. We will use moss.pl tool to check each lab and pset for plagriasm detection.

4. Additional resource
Acknowledgements: This course draws heavily from MIT's 6.869: Advances in Computer Vision by Antonio Torralba, William Freeman, and Phillip Isola. It also includes lecture slides from other researchers, including Andrew Owens , Svetlana Lazebnik, Alexei Efros, Fei-fei Li, Carl Vondrick, David Fouhey, Justin Johnson, and Noah Snavely. Special thanks to Ce Liu for the insightful and generous advice.

© 2022 Boston College. Website based on a template from HTML5 Up and Andrew Owen's course at U. Michigan.