Computational Creativity

ECSE 4964/6964, Rensselaer Polytechnic Institute, Fall 2023

RPI students can view class notes from each lecture by clicking the ✏️ emojis and watch Mediasite video recordings by clicking the 🎥 emojis. Outside visitors can follow along with the Youtube playlist here.

Jump to the current week

Introduction and Background

Mon Aug 28

LEC 1 Introduction to generative modeling

Mon Aug 28

HW 0 HW 0 released (due Thursday 9/7)

Mon Aug 28

HW 1 HW 1 released (due Thursday 9/21)

Thu Aug 31

LEC 2 Algorithms for making art, 1960-2010

Mon Sep 4

No class Labor Day

Tue Sep 5

LEC 3 Probability and machine learning review

Thu Sep 7

LEC 4 Deep learning crash course

Thu Sep 7

Due HW 0 due

Variational autoencoders

Mon Sep 11

LEC 5 Variational autoencoders (VAEs)

Thu Sep 14

LEC 6 VQ-VAEs and image quality metrics

Generative Adversarial Networks (GANs)

Mon Sep 18

LEC 7 Generative Adversarial Networks (GANs)

Mon Sep 18

HW 2 HW 2 released (due Thursday 10/12)

Thu Sep 21

LEC 8 Advanced GANs (e.g., ProGAN, StyleGAN, GANSpace)

Thu Sep 21

DUE HW 1 due

Mon Sep 25

LEC 9 Image-to-Image GANs (e.g., CycleGAN, pix2pix); GAN artists

Thu Sep 28

No class Radke out of town

Various generative models

Mon Oct 2

LEC 10 DeepDream and neural style transfer

Thu Oct 5

LEC 11 Denoising diffusion models

Mon Oct 9

No class Columbus Day

Thu Oct 12

Guest Aaron Hertzmann, Adobe Research: Can Computers Create Art?

Thu Oct 12

DUE HW 2 due

Thu Oct 12

Project Project progress report released (due Monday 10/23)

Thu Oct 12

HW 3 HW 3 released (due Thursday 11/2)

Mon Oct 16

LEC 12 Normalizing flow models

Language models

Thu Oct 19

LEC 13 Neural language models and word embeddings

Mon Oct 23

LEC 14 Attention and transformers

Mon Oct 23

DUE Project progress report due

Thu Oct 26

LEC 15 Large language models and their implications

Mon Oct 30

LEC 16 CLIP and its applications

Diffusion models for image generation

Thu Nov 2

LEC 17 DALLE-2 and Stable Diffusion

Thu Nov 2

DUE HW 3 due

Thu Nov 2

HW 4 HW 4 released (due Thursday 11/16)

Mon Nov 6

LEC 18 Diffusion developments (inpainting, Dreambooth, ControlNet)

Beyond text and images

Thu Nov 9

LEC 19 Generative models for music

Mon Nov 13

Guest Pamela Samuelson, UC Berkeley: Legal Challenges to Generative AI

Thu Nov 16

LEC 20 3D representations and neural rendering fields (NeRFs)

Thu Nov 16

DUE HW 4 due

Sat Nov 18

HW 5 HW 5 released (due Thursday, 11/30)

Sat Nov 18

PROJECT Project final report guidelines released (due Friday, 12/8)

Mon Nov 20

LEC 21 Generative models for 3D

Thu Nov 23

No class Thanksgiving

Mon Nov 27

LEC 22 Generative models for X (vector graphics, layouts, animation)

Thu Nov 30

LEC 23 Final class discussion

Thu Nov 30

DUE HW 5 due

Class presentations

Mon Dec 4

Project Class presentations 1

Thu Dec 7

Project Class presentations 2

Fri Dec 8

DUE Final project materials due