Syllabus for MME 596 Seminar on Deep Learning

Summer 2019

Course description

A reading seminar on deep learning with the central text Introduction to Deep Learning by Eugene Charniak. Chapter exercise solutions and some supplemental reading will be required. Programming in Python with Tensorflow will be required, but experience with any programming language is sufficient.

General information

Instructor
Rico AR Picone, PhD
Office Hours (CH 103C)
Tu 12–2 and Th 1–2
Office Hours (CSS)
W 3–4
Office Hours (Pan 107)
W 4–5
Office location
CH 103C
Moodle
moodle.stmartin.edu

secretssssssssss

Textbooks

Eugene Charniak. Introduction to Deep Learning. MIT Press, 2018. (Required.)

Schedule

The following schedule is tentative. All assignments will be set one week before the due date.

perceptrons, neural nets, feed-forward neural nets, Python EC Ch 1 get started on Assignment 1
feed-forward neural nets continue Assignment 1
Tensorflow EC Ch 2 Assignment 1, Assignment 2 Exercises
convolutional neural nets EC Ch 3 Assignment 2, Assignment 3
word embeddings and recurrent neural nets EC Ch 4 Assignment 4
sequence-to-sequence learning EC Ch 5 Assignment 5
deep reinforcement learning EC Ch 6 Assignment 6
unsupervised neural-network models EC Ch 7 Assignment 7

Assignments

Assignment 1

• Install the latest Python 3 at www.python.org. This site has good documentation for installation on all platforms.
• Open a terminal window (Windows: cmd.exe, MacOS: terminal.app) and verify your Python installation by typing python --version. It should return your version, e.g. Python 3.7.3.
• In a terminal window, install the scipy, numpy, and mnist modules using the syntax pip install <module name>. The documentation for the mnist module will help you use this dataset in your program. For more info in the mnist module, see this repo.
• In Python, write the feed-forward stochastic gradient descent program described and verify it can perform to 91-92% accuracy on the Mnist development data. I recommend using the linear algebra-based algorithm, which is more performant and easier to write. I also recommend seeding the random number generator for easier debugging. Submit this as a class definition .py file and an execution script .py file on Moodle.
• Read and consider each of the Written Exercises. Be ready to discuss Exercise 1.4 at the weekly meeting.

Assignment 2

• Complete the Written Exercises 2.1, 2.2, 2.3, 2.4, 2.5, and 2.6.
• In Python, write the feed-forward stochastic gradient descent program from Assignment 1, but use TensorFlow. Submit this as a class definition .py file and an execution script .py file on Moodle.

Assignment 3

• In TensorFlow, write a two-layer convolution neural network for Mnist image recognition. Submit this as a class definition .py file and an execution script .py file on Moodle.

Assignment 4

• In TensorFlow, write a bigram or trigram word embedding feedforward neural network and train and test it on the Penn Treebank Corpus. Submit this as a class definition .py file and an execution script .py file on Moodle. Here’s a code snippet that will get you started working with the data.
from nltk.corpus import treebank  as tb # PTB
from nltk import FreqDist # to determine word frequency
import numpy as np # for numpy arrays, etc.

frequency_list = FreqDist(i.lower() for i in words)
my_dictionary  = frequency_list.most_common()[:10000] # most common 10k
my_dictionary = np.array(my_dictionary) # convert to np array
my_dictionary = my_dictionary[:,0] # select words, ignore frequency
print(my_dictionary[:10]) # print the ten most frequent words


Homework, quiz, & exam policies

Homework & homework quiz policies

Weekly homework will be due on Fridays.

Working in groups on homework is strongly encouraged.