Randomness is an essential component in models designed to increase our understanding of natural and industrial processes. In particular, memoryless phenomena are often encountered in practice, where the future state of a system only depends on its present state, with no recollection of the past. In these cases, Markov chain models offer a formidable tool to answer questions related to the probability of events of practical interest in finance, biology, physics and engineering. This course provides an overview of stochastic and Markov chain modeling by alternating mathematical concepts and proofs with applications in several disciplines.
By the end of this course, the students should be able to:
ACMS 30440 or ACMS 30530 or MATH 30530
Textbook and other references
Required Work and Grading Criteria
The required work consists of homework problems, a midterm exam and a final exam (or project). Attendance and participation in class is also taken into account. The breakdown is:
Weekly online meetings - Class meetings will be in person. Online material and written notes will be available through Sakai. Monday class meetings will be partly dedicated to answering homework questions, while Wednesday meetings will be fully focused on reviewing the theory or solving problems together.
Homework Assignments - Homework assignments will be based on the material either presented in class or available online. The students will be asked to solve problems, answer questions from the theory or to develop Python code. Weekly homework assignments will be made available through Sakai on Wednesday starting from the second week of class, and are expected to be returned by Wednesday of the following week. There is no late return policy for this class and a homework not returned will received an F. However, the two lowest homework grades will be dropped.
Python programming - If you are unfamiliar with Python, I strongly recommend you to go through the online tutorial at this link, specifically: 1. Getting started with Python for science, 1.1. Python scientific computing ecosystem, 1.2. The Python language, 1.4. NumPy: creating and manipulating numerical data and 1.5. Matplotlib: plotting. A video review on these topics is available through Panopto and the first homework will allow you to familiarize with the language.
Midterm Exam - An in-class 75 minutes closed-book midterm exam is scheduled for Wednesday October 13th from 12:30PM to 01:45PM.
Final Exam - An online 2 hours final exam is scheduled for Thursday December 16th from 8:00AM to 10:00AM.
Honor Code - All students must familiarize themselves with the Honor Code on the University's website and pledge to observe its provisions in all written and oral work, including oral presentations, quizzes and exams, and drafts and final versions of essays. While discussion in small groups in doing homework is permitted (and strongly encouraged) in this course, the work should be your own. Exams are expected to be completed only by yourself with no help from others.
|Week n.||Tentative Content|
|Week 1||Python tutorial.|
|Week 2||Review of probability theory (Ch.1) and conditional probability (Ch.2).|
|Week 3||Review of the most common probability distributions (Ch.3).|
|Week 4||Random Sums (Ch.4).|
|Week 5||Markov chains (Ch.5).|
|Week 6||Absorbing Markov chains and First Step Analysis (Ch.6).|
|Week 7||Midterm preparation.|
|Week 8||Special Markov chains (Ch.7).|
|Week 9||One dimensional random walk (Ch.8).|
|Week 10||Branching Process (Ch. 9).|
|Week 11||Long run behavior of Markov chains (Ch. 10).|
|Week 12||The classification of states (Ch.11).|
|Week 13||Applications. The PageRank algorithm (Ch.12).|
|Week 14||Poisson processes (Ch.13).|