Introduction to Deep Learning

Dept. of Electrical Engineering
University of Notre Dame

Spring 2025- Enroll in Course Number EE 60572-01
Location/Time DBRT 303 - TTh 2pm-3:15pm
Office Hours - Fitz 266 - T/Th 10-12

Description: This is a survey course on deep learning for first year engineering graduate students. The course topics include 1) training pipelines for neural network models, 2) Convolutional Neural Networks for Image Processing, 3) Recurrent and Transformer Models for Sequence Data, 4) Generative Models (Autoencoders, GANs, and Diffusion), and 5) Security/Fairness in Machine Learning. The course will make extensive use of TensorFlow/Keras deep learning libraries.

Prerequisites:Students should have prior course work in linear algebra and random processes. Students must have prior Python programming experience. Students will use TensorFlow/Keras deep learning libraries and will be customizing class objects used to instantiate and train deep neural network models.

Project Option: Students may elect to do a project with instructor's approval. Projects will be individual student efforts (no teams). See instructor as soon as possible if you are interested in the project option. Instructor approval will be contingent on a satisfactory project proposal (due before Spring Break). The project grade will be used in place of the final exam grade. Project proposal and final report must be written using ICML latex style files (zip) (docx)

• Introduction - Learning by Example
  • Homework 1 solution (zip) : Due 1/21
• Generalization - a statistical approach
  • Homework 2 solution (zip)
• Neural Networks Models
  • Homework 3 solution (zip)
• Deep Neural Network Training Pipeline
  • Homework 4 solution(zip) : Due 2/20
  • Training Pipeline Lecture Notebook (zip)
  • Midterm 1 (zip) (take home 2/27: 8am - 6pm) - solution
• Convolutional Neural Networks
  • Homework 5 solution (zip) : Due 3/18
• Recurrent Models and Transformers in Natural Language Processing
  • Homework 6 solution(zip) updated 3/14 - DUE April 3
    Lecture 17 ipynb file, Lecture 17 data pickle file, Transformer Lecture
    zip file of my NMT eng-spa transformer notebook. It is about 100MB. Model takes 3.5 hours to train, but I've included saved weights from my model.
• Models for Generative AI
  • Homework 7 (Optional), HW7 keras model (zip)1.6GB
    
  • solution
  • 4/1 autoencoder ((zip), 4/3 VAE/GPT (zip), 4/8 GAN (zip), 4/10 DDM ( zip) ), 2024 GenAI slides pdf
• Security/Fairness in Machine Learning
  • 4/15 (Reinforcement Learning), (zip, DeepRL pdf),
  • 2024 notes on DL/Society (pdf), Adversarial Examples (zip)
  • Fairness (pdf) (Yuying Duan guest lecture)
  • On-device learning (pdf) (Gelei Xu guest lecture)
  • Final Exam consists of two parts:
    • Take home notebook practicum (zip) (released COB April 28, due morning April 30)
    • In-class short answer exam (closed-book) (May 5 - 7:30-9:30 pm - 303 DBRT)