ACMS40760 Introduction to Stochastic Modeling - Fall 2021

Course Syllabus PDF

Course Description
Randomness is an essential component in models designed to increase our understanding of natural and industrial processes. In particular, memoryless phenomena are often encountered in practice, where the future state of a system only depends on its present state, with no recollection of the past. In these cases, Markov chain models offer a formidable tool to answer questions related to the probability of events of practical interest in finance, biology, physics and engineering. This course provides an overview of stochastic and Markov chain modeling by alternating mathematical concepts and proofs with applications in several disciplines.

Course Objectives
By the end of this course, the students should be able to:

  • Leverage discrete and continuous Markov chains models to compute the probability of events in cases of practical relevance.
  • Formulate and solve problems by computing the long-term probabilities of a Markov chain model.
  • Write Python code to simulate Markov chains, and compute probabilities of events that may be difficult to derive by hand.
  • Apply Poisson processes to model the occurrence of events in various applications.

ACMS 30440 or ACMS 30530 or MATH 30530

Class Info

  • Instructor: D. Schiavazzi
  • Office: 101G Crowley Hall
  • Email: dschiavazzi (at)
  • Phone: (574) 631-0420
  • Class Location: Hayes Healy Center 117
  • Class Schedule: Monday, Wednesday - 12:30PM - 1:45PM
  • Office Hours: After class in 101G Crowley Hall or by appointment

Textbook and other references

  • Weekly notes and videos from the instructor available on Sakai/resources or Panopto.
  • Pinsky, M. and Karlin, S., An introduction to stochastic modeling 4th Edition, Academic press, 2010.
  • Sheldon M. Ross, Introduction to Probability Models 11th Edition (or online edition), Academic press, 2010.

Required Work and Grading Criteria
The required work consists of homework problems, a midterm exam and a final exam (or project). Attendance and participation in class is also taken into account. The breakdown is:

  • Participation during online meetings: 10%.
  • Homeworks: 30%.
  • Midterm: 30%.
  • Final exam or project: 30%.

Weekly online meetings - Class meetings will be in person. Online material and written notes will be available through Sakai. Monday class meetings will be partly dedicated to answering homework questions, while Wednesday meetings will be fully focused on reviewing the theory or solving problems together.

Homework Assignments - Homework assignments will be based on the material either presented in class or available online. The students will be asked to solve problems, answer questions from the theory or to develop Python code. Weekly homework assignments will be made available through Sakai on Wednesday starting from the second week of class, and are expected to be returned by Wednesday of the following week. There is no late return policy for this class and a homework not returned will received an F. However, the two lowest homework grades will be dropped.

Python programming - If you are unfamiliar with Python, I strongly recommend you to go through the online tutorial at this link, specifically: 1. Getting started with Python for science, 1.1. Python scientific computing ecosystem, 1.2. The Python language, 1.4. NumPy: creating and manipulating numerical data and 1.5. Matplotlib: plotting. A video review on these topics is available through Panopto and the first homework will allow you to familiarize with the language.

Midterm Exam - An in-class 75 minutes closed-book midterm exam is scheduled for Wednesday October 13th from 12:30PM to 01:45PM.

Final Exam - An online 2 hours final exam is scheduled for Thursday December 16th from 8:00AM to 10:00AM.

Honor Code - All students must familiarize themselves with the Honor Code on the University's website and pledge to observe its provisions in all written and oral work, including oral presentations, quizzes and exams, and drafts and final versions of essays. While discussion in small groups in doing homework is permitted (and strongly encouraged) in this course, the work should be your own. Exams are expected to be completed only by yourself with no help from others.

Program (tentative)

Week n. Tentative Content
Week 1 Python tutorial.
Week 2 Review of probability theory (Ch.1) and conditional probability (Ch.2).
Week 3 Review of the most common probability distributions (Ch.3).
Week 4 Random Sums (Ch.4).
Week 5 Markov chains (Ch.5).
Week 6 Absorbing Markov chains and First Step Analysis (Ch.6).
Week 7 Midterm preparation.
Week 8 Special Markov chains (Ch.7).
Week 9 One dimensional random walk (Ch.8).
Week 10 Branching Process (Ch. 9).
Week 11 Long run behavior of Markov chains (Ch. 10).
Week 12 The classification of states (Ch.11).
Week 13 Applications. The PageRank algorithm (Ch.12).
Week 14 Poisson processes (Ch.13).