Overview

CSCI 3397/PSYC 3317 is an advanced undergraduate-level class covering both digital image processing (week 2-6), state-of-the-art deep learning (week 7-14) methods, and the recent foundation modeling (week 15).

Schedule

Theme Date Topic Materials Assignments
Intro
[Week 1]		 Wed, Jan. 17 Lec. 1: Introduction
Overview, syllabus, logistics
lab0 out
Fri, Jan. 19 Lec. 2: Digital Image
patch, spatial resolution, bit depth, channel
Mon, Jan. 22 Lec. 3: Image Formation
Natural image: pinhole
Microscopy and medical images

Module I: Digital Image Processing (DIP)

Image Filtering
[Week 2-3] 		Wed, Jan. 24 Lec. 4: Pixel-level Processing
Transfer function
  • [T] Chap. 4.2.1
 lab0 due
Fri, Jan. 26 Lec. 5: General Filtering
Filtering, padding
Mon, Jan. 29 Lec. 6: Linear Filtering
Linear: Convolution
Impulse, Box, Sharpen
  • [T] Chap. 4.3.1
 ps1 out (dip)
Wed, Jan. 31 Lec. 7: More Linear Filtering
Gaussian filter, edge filter
  • [T] Chap. 4.3
 lab1 due
Fri, Feb. 2 Lec. 8: Nonlinear Filtering
match filter, bilateral
Edge detection
  • [T] Chap. 4.2.3, 5.1
Image Registration
[Week 4-5] 		Mon, Feb. 5 Lec. 9: Image Transformation
Image transformation
Wed, Feb. 7 Lec. 10: Template Matching
Intensity-based, translation
  • [T] Chap. 10.2
Fri, Feb. 9 Guest Lecture: Microscopy Image Analysis
Guest: Dr. Clarence Yapp (@Harvard Medical School)
lab2 due
Mon, Feb. 12 Lec. 11: Scale Invariant Feature Transformation
SIFT-based
 ps1 due
fp team sign-up
Wed, Feb. 14 Lec. 12: Transformation Estimation
Linear regression, RANSAC
ps2 out (dip+image)
Image Segmentation
[Week 5-6] 		Fri, Feb. 16 Lec. 13: Image Segmentation I
Why: drug discovery, connectomics
What: semantic, instance, panoptic
  • [T] Chap 6.1, 6.5, 6.6
Mon, Feb. 19 Lec. 14: Image Segmentation II
Semantic: Ostu
Instance: connected component, Watershed
  • [T] Chap 6.7, 8.1
 lab3 due
fp sign-up due
Wed, Feb. 21 Lec. 15: Morphological Operation and Analysis
graph-based segmentation: NCut, MRF
Operation: dilation, erosion
Analysis: statistics

Module II: Deep Learning (DL)

Machine Learning
[Week 7]		 Fri, Feb. 23 Lec. 16: Overview
ML: pipeline, tasks
Overview
  • [F] Chap. 1.1
  • Coding: Scikit-learn
 fp check-in (topic)
Mon, Feb. 26 Lec. 17: Unsupervised Learning
Kmeans, PCA
fp check-in (topic)
Wed, Feb. 28 Lec. 18: Supervised Learning
Linear regression, basis function
lab4 due
ps2 due
Fri, Mar. 1 Lec. 19: Linear Classification
Data: discrete label
Layer: Linear, sigmoid
  • [F] Chap. 4.1-4.3
 fp proposal due
No Class (Happy Spring Break)
MLP Model
[Week 9]		 Mon, Mar. 11 Lec. 20: Linear Classification II
Loss: cross-entropy
Optimization: SGD, momentum
  • [F] Chap. 4.1-4.3
Wed, Mar. 13 Lec. 21: Linear Classification III
Implementation: Numpy and Pytorch
lab5 due
Fri, Mar. 15 Lec. 22: Multilayer Perceptron (MLP) I
Layer: Softmax, ReLU
  • [F] Chap. 3.4, 4.5
 fp check-in (data)
Mon, Mar. 18 Lec. 23: Multilayer Perceptron (MLP) II
Layer: Dropout
Optimization: backpropagation
  • [F] Chap. 3.4, 4.5
 ps3 out (dl)
CNN Model
[Week 10]
Wed, Mar. 20 Lec. 24: LeNet and AlexNet
Layer: convolution, pooling
lab6 due
Fri, Mar. 22 Lec. 25: VGGNet, BatchNorm, and ResNet
3x3 Filter, BatchNorm, ResNet
fp check-in (model)
Transformer Model
[Week 11] 		Mon, Mar. 25 Lec. 26: Attention Layers and Encodings
M: Attention module
Wed, Mar. 27 Lec. 27: Transformer Model
M: Attention layer, token, embedding, positional encoding
lab7 due
No Class (Good Friday)
No Class (Easter Monday)
Wed, Apr. 3 Lec. 28: Vision Transformer
GPT, ViT
ps3 due
Image Applications
[Week 12-13]		 Fri, Apr. 5 Lec. 29: Image Classification: Recent Advances
D: augmentation
O: transfer learning
L: label smoothing
Mon, Apr. 8 Lec. 30: Object Detection
CNN approach: R-CNN, Fast/Faster R-CNN, YOLO
Transformer approach: DETR
Target domain: bounding boxes
  • [F] Chap. 6.3
 lab8 due
Wed, Apr. 10 Lec. 31: Image Segmentation
FCN, Encoder-Decoder, UNet
Target domain: segmentation map
  • [F] Chap. 6.4
 fp check-in (exp)
Fri, Apr. 12 Lec. 32: Image Generation
GAN and conditional GAN
Diffusion model
Target domain: natural images
  • [F] Chap. 7.2
Multimodal Applications
[Week 14]		 Tue, Apr. 16
(Mon class) 		Guest Lec.: Medical Volume Processing
Guest: Jiancheng Yang (Postdoc@EPFL)

fp check-in (in-person)
Wed, Apr. 17 Lec. 32: Text-Image Representation
image sampling, image translation, DALL.E
  • [F] Chap. 6.4
 lab9 due
Fri, Apr. 19 Lec. 34: Video Applications
Action recognition
Recurrent Neural Network
  • raw data -> measurement
 ps4 due

Module III: Foundation Modeling

Foundational Modeling
[Week 15]		 Mon, Apr. 22 Lec. 35: Foundation Modeling
Multimodal input, Task convergence
SAM, Florence, GPT4-V
Wed, Apr. 24 Guest Lec.: Medical Foundation Model
Guest: Oishi Banerjee (PhD@Harvard)

Final Projects

Presentation Fri, Apr. 26 Presentation I
lab10 due
fp slide due
Mon, Apr. 29 Presentation II
Wed, May. 1 Presentation III
Submission Sun, May. 12 fp report/code due


Staff & Office Hours


Instructor
TA
TA
Name Office hours
Donglai Tue 3-4pm (zoom), Wed 3-4pm @ 245 Beacon Rm. 528F
Jason Thu/Fri 3-4pm @ 245 Beacon Rm. 122
Ryan Mon 1:50-2:50pm, Tue 10:15-11:15am @ 245 Beacon Rm. 122
  • Office hours will take place in person (or Zoom if needed).
  • Donglai will hold additional one-on-one AMA office hours Tue 4-5pm/Wed 3-4pm (15-min by appointment)


Course information

This is a challenging course and we are here to help you become a more-AI version of yourself. Please feel free to reach out if you need help in any form.

1. Get help (besides office hours)

  • Dropbox: The lecture pdfs will be uploaded to Dropbox (follow the link) and you can ask questions there by making comments on the slides directly.
  • Discord: For labs/psets/final projects, we will create dedicated channels for you to ask public questions. If you cannot make your post public (e.g., due to revealing problem set solutions), please directly DM TAs or the instructor separately, or come to office hours. Please note, however, that the course staff cannot provide help debugging code, and there is no guarantee that they'll be able to answer last-minute homework questions before the deadline. We also appreciate it when you respond to questions from other students! If you have an important question that you would prefer to discuss over email, you may email the course staff, or you can contact the instructor by email directly.
  • Support: The university counseling services center provides a variety of programs and activities.
  • Accommodations for students with disabilities: If you are a student with a documented disability seeking reasonable accommodations in this course, please contact Kathy Duggan, (617) 552-8093, dugganka@bc.edu, at the Connors Family Learning Center regarding learning disabilities and ADHD, or Rory Stein, (617) 552-3470, steinr@bc.edu, in the Disability Services Office regarding all other types of disabilities, including temporary disabilities. Advance notice and appropriate documentation are required for accommodations.

2. Homework submission

All programming assignments are in Python on Colab, always due at midnight (11:59 pm) on the due date .
  • Install Colab on the browser: Sign in to your Google account, follow the [link] to the folder of assignments, click on lab0.ipynb, click on "Open with" and "Connect more apps", install "Colaboratory".
  • Submission: You need save a copy of the file in your own Google drive, so that you can save your edits. Afterwards, you can download the ipynb file and submit it to Canvas.
  • Lab (1 per week): Every lecture has a lab exercise to help you gain the hands-on understanding about the material. The lab on previous week's lectures is due on Wednesday. We will go through some code in class and you need to finish up the exercises.
  • Pset (4 in total): In each pset, we will build a working prototype for a biology lab or a healthcare startup.
  • Final project: (guideline) In lieu of a final exam, we'll have a final project. This project will be completed in small groups during the last weeks of the class. The direction for this project is open-ended: you can either choose from a list of project ideas that we distribute, or you can propose a topic of your own. A short project proposal will be due approximately halfway through the course. During the final exam period, you'll turn in a final report and give a short presentation. You may use an ongoing research work for your final project, as long it meets the requirements.

3. Academic policy

  • Late days: You'll have 10 late days each for labs and psets respectively over the course of the semester. Each time you use one, you may submit a homework assignment one day late without penalty. You are allowed to use multiple late days on a single assignment. For example, you can use all of your days at once to turn in one assignment a week late. You do not need to notify us when you use a late day; we'll deduct it automatically. If you run out of late days and still submit late, your assignment will be penalized at a rate of 10% per day. If you edit your assignment after the deadline, this will count as a late submission, and we'll use the revision time as the date of submission (after a short grace period of a few minutes). We will not provide additional late time, except under exceptional circumstances, and for these we'll require documentation (e.g., a doctor's note). Please note that the late days are provided to help you deal with minor setbacks, such as routine illness or injury, paper deadlines, interviews, and computer problems; these do not generally qualify for an additional extension.
  • Academic integrity: While you are encouraged to discuss homework assignments with other students, your programming work must be completed individually. You may not search for solutions online, or to use existing implementations of the algorithms in the assignments. Thus it is acceptable to learn from another student the general idea for writing program code to perform a particular task, or the technique for solving a mathematical problem, but unacceptable for two students to prepare their assignments together and submit what are essentially two copies of identical work. If you have any uncertainty about the application of this policy, please check with me. Failure to comply with these guidelines will be considered a violation of the University policies on academic integrity. Please make sure that you are familiar with these policies. We will use moss.pl tool to check each lab and pset for plagriasm detection.

4. Additional resource
Acknowledgements: This course is a biomedical version of CSCI 3343: Computer Vision with concrete biomedical image analysis applications. See the "Acknowledge" section in that class.

© 2023 Boston College. Website based on a template from HTML5 Up and Andrew Owen's course at U. Michigan.