Syllabus for MME 595
Special Topics
Machine Learning and State-Space Control Systems

• Course description
• General information
• Discord
• Textbooks
  • Core Textbooks
  • Additional Textbooks
• Schedule
• Assignments
  • Assignment 1
  • Assignment 2
  • Assignment 3
  • Assignment 4
  • Assignment 5
  • Assignment 6
• Development environment
  • Anaconda
  • Git
  • VS Code
  • Machine Learning Packages
• Grading policies
• Academic integrity policy
• Acceptable use of AI in coursework
• Access and accommodations
• Sexual misconduct/sexual harassment reporting
• University sanctioned activities
• Counseling and Wellness Center
• Religious Accommodation
• Center for Student Success

Course description

An introduction to machine learning and modern (i.e., state-space) control systems. Modern control systems techniques such as optimal control and optimal estimation are introduced. Machine learning techniques such as deep feedforward and convolutional artificial neural networks (ANNs) and reinforcement learning (RL) are introduced and applied to control systems.

General information

Instructor

Rico AR Picone, PhD

Actual office hours (CH 103C)

Tue, Thu 3:20–5:50

Virtual office hours (zoom, make appointment!)

By appointment

Office hours appointments

make appointment

Office location

CH 103C

Classroom location

Panowicz Hall 107

Times

MW 2:00–4:20 pm

Moodle

moodle.stmartin.edu

secretssssssssss

Discord

Everyone is required to join the messaging service called “Discord.” We’ll use it to communicate with each other during the semester. The Discord server you need to join is called drico. That’s an invitation link. Be sure to join the channel #ml-ss-control.

Textbooks

No single book covers all the material we will discuss in sufficient detail. We will use two core books in detail. In fact, the supplement (Murray) to the secondary text (Åström) will be used more than the secondary text itself.

Core Textbooks

Primary: [Brunton] Brunton, S. L., & Kutz, J. N. (2022). Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (2nd ed.). Cambridge: Cambridge University Press. (Cambridge, libgen)

Secondary: [Åström] Åström, K. J., Murray, R. (2021). Feedback Systems: An Introduction for Scientists and Engineers, (2nd ed.). United Kingdom: Princeton University Press. (Princeton, digital, wiki)

Secondary Supplement: [Murray] Richard M. Murray (2023). Optimization-Based Control (Version v2.3h, 12 March 2023). California Institute of Technology. (Cal Tech, wiki)

Additional Textbooks

There may be occasional readings from the following books (I will supply them, as needed).

Control Theory

Lewis, F. L., Vrabie, D., Syrmos, V. L. (2012). Optimal Control. Germany: Wiley. (digital)

Kirk, D. E. (2004). Optimal Control Theory: An Introduction. United States: Dover Publications.

William L. Brogan. Modern Control Theory. Prentice-Hall, 1991. (libgen)

Machine Learning

Ian Goodfellow, Yoshua Bengio, Aaron Courville (2016). Deep Learning. MIT Press. (digital)

Stuart J. Russell and Peter Norvig (2020). Artificial Intelligence: A Modern Approach (4th ed.). Pearson Education. (website)

Control Theory and Machine Learning

Li, J., Lewis, F. L., Fan, J. (2023). Reinforcement Learning: Optimal Feedback Control with Industrial Applications. (n.p.): Springer International Publishing.

Schedule

The following schedule is tentative. All assignments will be set one week before the due date.

Assignments

Your work should be turned in on Moodle each Friday night before midnight. Sunday night is the cutoff, but it’s preferable to have it done before then so you can prepare for the coming week’s classes.

Assignment 1

• Solution
  • Published
  • Solution code (for Brunton and Kutz)
  • Solution code (for special problems): directory mnist-nn
• Brunton Exercises 4.3, 4.4
• Special Exercise: Create and train a feedforward NN for the MNIST data used in Brunton 4.4. Report plots of the training and testing loss for each training epoch. Report the testing accuracy of the trained model. Hand-optimize hyperparameters such as the learning rate and metaparameters such as the number of layers, number of units per layer, etc. Use Keras with TensorFlow.

Assignment 2

• Solution
  • Published
  • Solution code (for Brunton and Kutz)
• Brunton exercises 6.1d, 7.5, 7.8
• Optional: Brunton exercise 6.2

Assignment 3

• Solution
  • Published
  • Solution code (for Brunton and Kutz)
• Brunton exercises 8.2, 8.4

Assignment 4

• Solution
  • Published
  • Solution code (for special problems): directories murray_3_10, murray_3_12
• Murray exercises 3.10, 3.12 (pvtol.py)

Assignment 5

• Solution
  • Published
  • Solution code (for Brunton and Kutz)
  • Solution code (for special problems): directory murray_4_6
• Murray exercise 4.6
• Brunton exercise 10.1

Assignment 6

• Solution
  • Published
  • Solution code (for special problems): directory reinforcement-learning-cart-pole
• Special Exercise: Write a deep reinforcement learning program to learn a policy network $\pi_\theta(s, a)$ to regulate the angular position of an inverted pendulum on a cart to be vertical. Use two networks, an actor (policy $\pi_\theta$) and a critic (value $V_\phi(s)$). Use a temporal difference TD(0) method to update the networks. Use the Gymnasium cart-pole environment. Note that this has a discrete action space, able to push the cart with a force a unit leftward (0) or rightward (1). Train for several episodes and display the results.

Development environment

The following development environment will be used for all class examples and assignments. The three primary pieces of software, Anaconda (Python), Git, and VS Code are available for free on all major operating systems. All of this except VS Code is installed on engineering lab computers (including those in the classroom).

Anaconda

Anaconda is a distribution of Python that includes the conda package and environment manager.

Download and install Anaconda. The default Anaconda environment is base, which is fine to use as long as you don’t want to work on multiple projects.

To edit or create an environment, on Windows open the Anaconda Prompt app, or on macOS, open the Terminal app. To create a new environment, use the command

conda create --name <environment-name> python=3.11 numpy scipy matplotlib sympy

The Python version is specified and the final arguments here are a list of packages to install (those given here are what I recommend as a default).

To install additional packages, make sure your environment is activated first (base is automatically activated):

conda activate <environment-name>

To install additional packages to the activated environment, use the installation formula (e.g., those here). For instance, to install the sympy package,

conda install conda-forge::sympy

All though it can cause compatibility issues if it is mixed with conda install packages, you can install packages with pip to an Anaconda environment. For instance,

pip install engcom

If an environments.yml file is provided by someone (maybe me) with a Python version and several packages to install, move that file to your current working directory (or cd to its location) and execute the command

conda env create -f environment.yml

Git

Git is an amazing version control tool for code development and collaboration. I use it for everything.

Install Git from the instructions here.

VS Code

This is currently the most popular code editor/integrated development environment (IDE) for many programming languages and tasks.

Setting up your VS Code environment

Do the following to set up your VS Code environment:

1. Download and install VS Code. It is available on all major platforms. It can be installed on your user account on a lab computer.
2. Open VS Code.
3. Under the “Extensions” tab (left-side navigation), install the Python extension.
4. Windows users: type ctrl+` to open a terminal. Click the dropdown next to the + sign and select “Select Default Profile.” From the dropdown, select “Git Bash.”
5. Type ctrl+shift+P (macOS cmd+shift+p), type Python: Select Interpreter, and press Enter. Select from the list the python.exe file in an anaconda directory with the name base (or an environment you created). This selects the Anaconda environment base.

6. Create a new file with ctrl+N. Write the following in the file:

    import numpy as np
    import sympy as sp
    import scipy
    import matplotlib.pyplot as plt
       
    print("Hello, world!")
   

7. Click the “play” button in the top right (Run Python File in Dedicated Terminal). The terminal should display the output Hello, world!.

Machine Learning Packages

Here are some helpful machine learning packages.

CVXPY for Convex Optimization

The convex optimization package CVXPY is excellent for optimization problems. (Here is the documentation.) Install it in your conda environment with the following command:

conda install -c conda-forge cvxpy

Keras with TensorFlow

I recommend using Keras with a TensorFlow backend for building neural networks.

Install these packages into your conda environment (unfortunately we have to use pip instead of conda for these, making the environment a bit fragile). The installation is different for each OS. The installation instructions are here.

Grading policies

Total grades in the course may be curved, but individual homework assignments and exams will not be. They will be available on moodle throughout the semester.

Homework

100%

secrets

Academic integrity policy

Cheating or plagiarism of any kind is not tolerated and will result in a failing grade (“F”) in the course. I take this very seriously. Engineering is an academic and professional discipline that requires integrity. I expect students to consider their integrity of conduct to be their highest consideration with regard to the course material.

What is academic integrity? Saint Martin’s University is a community of faculty, students and staff engaged in the exchange of ideas in the ongoing pursuit of academic excellence. Essential to our mission is a focused commitment to scholarly values and intellectual integrity, and a respect for the ideas, beliefs and work of others. This commitment extends to all aspects of academic performance. All members are expected to abide by ethical standards both in their conduct and their exercise of responsibility to themselves and toward other members of the community. As an expression of our shared belief in the Benedictine tradition, we support the intellectual, social, emotional, physical and spiritual nurturing of students.

What is academic dishonesty? Saint Martin’s University defines academic dishonesty as violating the academic integrity of an assignment, test and/or evaluation of any coursework. This dishonest practice occurs when you seek to gain for yourself or another an academic advantage by deception or other dishonest means. You have a responsibility to understand the requirements that apply to particular assessments and to be aware of acceptable academic practice regarding the use of material prepared by others. Therefore, it is your responsibility to be familiar with the policies surrounding academic dishonesty as these may differ from other institutions.

Acceptable use of AI in coursework

Any use of technology that misleads a reviewer in assessing the student’s mastery of a specific set of skills or knowledge is a type of intellectual dishonesty, that is, a type of cheating. Students who are unsure about the appropriateness of using an artificial intelligence tool (or “AI”) should check with the instructor before using it. This includes the use of tools that generate text, images, video, code, and other works. If you are permitted by your instructor to use one or more AI tools in producing your work, you should disclose the use of that tool. You should say which tool you used and how you used it. Then if you use specific AI generated content (text, images, videos, audio, code, and so on) you should cite it in the style (APA, MLA, and so on) specified by your instructor.

Access and accommodations

Your experience in this class is important to me. If you have already established accommodations with Disability Support Services (DSS), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.

If you have not yet established services through DSS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but are not limited to mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DSS at 360-438-4580 or dss.testing@stmartin.edu or smu.dss@stmartin.edu. DSS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DSS. It is the policy and practice of the Saint Martin's University to create inclusive and accessible learning environments consistent with federal and state law.

Sexual misconduct/sexual harassment reporting

Saint Martin’s University is committed to providing an environment free from sex discrimination, including sexual harassment and sexual violence. There are Title IX/sexual harassment posters around campus that include the contact information for confidential reporting and formal reporting. Confidential reporting is where you can talk about incidents of sexual harassment and gender-based crimes including sexual assault, stalking, and domestic/relationship violence. This confidential resource can help you without having to report your situation to the formal reporting process through the Interim Dean of Students – Ms. Ann Adams MBA, Associate VP of Human Resources – Ms. Cynthia Johnson, Public Safety – Ms. Sharon Schnebly, or the Office of the Provost – Dr. Tanya Smith Brice, unless you request that they make a report. Please be aware that, in compliance with Title IX and under the Saint Martin’s University policies, educators must report incidents of sexual harassment and gender-based crimes including sexual assault, stalking, and domestic/relationship violence. If you disclose any of these situations in class, on papers, or to me personally, I am required to report it.

University sanctioned activities

If you are absent from class due to university-sanctioned activities, such as sports, it is your responsibility to request that the absence be excused; otherwise, the absence will be recorded as unexcused. Absent students are responsible for catching up with the class, and if any assignments are due on the day of the absence, it is your responsibility to turn in the assignments on time (prior to class). Assignments may be submitted as an attachment to an email.

Counseling and Wellness Center

There may be times, as a college student, when personal stressors interfere with your academic performance and your daily life. The Counseling and Wellness Center supports students by addressing mental and emotional well-being with FREE and confidential services. To schedule an appointment, call 360-412-6123 or email counselingcwc@stmartin.edu or stop by the CWC (1st floor St. Raphael Center).

If you would rather not go to the CWC or need support in the evenings and weekends, please consider using the TimelyCare app (timelycare.com/smusaints) to speak with a mental health provider, free, 24/7, from your phone or computer.

Religious Accommodation

Saint Martin's University, in honor of the sacredness of the individual, and being deeply rooted in the Catholic Benedictine tradition of higher education, values the many religious and spiritual practices of our campus community. Saint Martin's University supports our students in their ongoing journey of becoming. In compliance with Washington State Law RCW 28B.137.010, Saint Martin's University reasonably accommodates students for reasons of religious observances.

Center for Student Success

The Center for Student Success offers free academic services for all Saint Martin's students. The Center provides subject-area peer tutoring in science, technology, nursing, engineering, math, business, accounting, economics, world languages and other subjects. At the Writing Center, students meet with writing tutors to discuss their academic, personal, and professional writing. The Advising Center works with students on academic advising, connecting with campus support resources, transition and self-exploration guidance, personalized academic improvement plans, learning workshops, and support for changing majors. Disability Support Services is also located in the Center for any student with a disability who needs accommodations. For more information on the Center for Student Success, or to sign up for a tutoring, advising, or DSS meeting, see the website: stmartin.libcal.com/appointments.