ACMS80770 Topics in Applied Mathematics

UQ for Computational Science and Engineering: Monte Carlo and spectral stochastic methods

Course Syllabus PDF

Course Description
Inputs of numerical models are often uncertain and quantification of how this variability affects the outputs may be challenging, particularly when 1) the number of inputs is large (high dimensionality); 2) model evaluation is a computationally intensive task; 3) change in the outputs is non-smooth, i.e., characterized by sharp gradients or even discontinuities. A number of approaches to deal with this difficulties will be discussed in class. From the solution of random differential equations through Monte Carlo and Quasi-Monte Carlo methods to adaptive numerical integration in high dimensions using sparse grids. Approaches to build parametric surrogates of expensive computational models will be discussed with reference to simpler polynomial-based and more complex adaptive and multi-resolution representations which will also be leveraged to quantify indices of local and global sensitivity. Applications will include non-smooth maps, dynamical systems, scalar transport, passive vibration control of structures, robust optimization in CFD, numerical hemodyamics. Discussions on the mathematical derivations will be complemented by examining computer code through python interactive notebooks.

Course Objectives
By the end of this course, the students will be able to:

  • Solve random PDEs with Monte Carlo and Quasi-Monte Carlo methods.
  • Use advanced numerical integration for cubature of high-dimensional stochastic responses.
  • Construct parametric meta-models using polynomial families and more advanced adaptive representations.
  • Manipulate random inputs through Gaussian approximations or probability transformations.
  • Generate samples from random fields.
  • Perform local and global sensitivity analysis.

Class Info

  • Instructor: D. Schiavazzi
  • Office: 242 Hayes-Healy
  • Email: dschiavazzi@nd.edu
  • Phone: (574)631-0420
  • Class Location: B016 Haggar Hall
  • Class Schedule: Monday, Wednesday, Friday 9:25AM - 10:15AM
  • Office Hours: Friday 3:00PM - 4:00PM

Academic Calendar - Fall Semester 2017
Class start: 08/22. Mid-term break 10/14 - 22. Thanksgiving Holiday 11/22-26. Last class day 12/07. Reading days 12/08-10. Final examinations 12/11-15.

Textbook and other references
There is no suggested textbook for the class. However, material from the literature and the following sources will likely be discussed.

  • Le Maître, O. and Knio, O.M., Spectral methods for uncertainty quantification: with applications to computational fluid dynamics, Springer Science & Business Media, 2010.
  • Ghanem, R.G. and Spanos, P.D., Stochastic finite elements: a spectral approach, Dover Publications, 2003.

Guest lecturers
The following guest lecturers will present in class on their area of expertize.

  • Gianluca Geraci, Ph.D., Sandia National Laboratories, Multi-level/Multi-fidelity Monte Carlo Uncertainty Propagation.
  • Justin S. Tran, Stanford University, UQ in Cardiovascular Modeling.

Required Work and Grading Criteria
The required work consists of homework problems, one midterm exam, and one final project. The breakdown of marks is:

  • Homeworks: 30%
  • Midterm: 30%
  • Final Project: 40%

Homework Assignments - Homework assignments will be based on the material presented in class. Most of the homeworks will require to develop python code and to answer questions from the theory.

Midterm Exam - There will be a take home mid-term exam.

Final Project - Students will be asked to propose a project that combines their research activities with some of the topics discussed in class. This may involve, for example, reproducing (and critically reviewing) the results of a paper in the literature. Alternatively, a project will be assigned by the instructor. Each project may be performed individually or in groups, and will be presented to the rest of the class during the last class meeting.

Honor Code - All students must familiarize themselves with the Honor Code on the University’s website and pledge to observe its provisions in all written and oral work, including oral presentations, quizzes and exams, and drafts and final versions of essays.

Program (tentative)

Week n. From/To Tentative Content
Week 1 Aug 22nd - Aug 25th Introduction and Background. Uncertainty in engineering systems. Motivating problems. Forward problem. Parameter estimation under uncertainty. Robust optimization. Sensitivity analysis. Types of uncertainty. Rudiments of Probability Theory. Elements of measure theory in probability. Random variables. Common discrete and continuous probability distributions.
Week 2 Aug 28th - Sept 1st Rudiments of Probability Theory. Useful inequalities. Convergence of random variables. Conditional probability and independence. Monte Carlo integration. Plain Monte Carlo estimator and its variance.
Week 3 Sept 4th - Sept 8th Monte Carlo integration. Variance reduction. Stratified sampling. Optimal partition sampling size. Proportional allocation with one and multiple samples per stratum. Sampling from arbitrary distributions. Latin hypercube sampling. Importance sampling. Control variates.
Week 4 Sept 11th - Sept 15th Quasi-Monte Carlo Integration. Discrepancy of point clouds and error bounds for QMC integration. Van der Corput sequences. Halton sequences. Faure sequences. Sobol sequences. Grey code implementation. Practical generation with primitive polynomials and direction numbers.
Week 5 Sept 18th - Sept 22nd Numerical integration. Newton-Cotes formulas and Runge's phenomenon. Orthogonal polynomials and three-term recurrence. Jacobi matrices and Golub-Welsch theorem. Jacobi matrices for Gauss-Radau and Gauss-Lobatto quadrature.
Week 6 Sept 25th - Sept 29th Numerical integration. Gauss-Kronrod and Gauss-Patterson quadrature. Laurie's algorithm. Clenshaw-Curtis quadrature. Monomial orderings. Lex, grlex and grevlex orderings. Full and partial tensor products of polynomials. Total order and hyperbolic cross multi-indices.
Week 7 Oct 2nd - Oct 6th Numerical integration. Smolyak sparse grids with examples. Adaptive sparse quadrature. Neighbor and admissible multi-indices. Generalized sparse grid algorithm.
Week 8 Oct 9th - Oct 13th Parametric meta-models. Multivariate Lagrange polynomials. Admissible lattices. Hierarchical interpolation on sparse grids. Radial basis function interpolation. Interpolation of mixed data. Stability and regularization.
Midterm
Week 9 Oct 23rd - Oct 27th Parametric meta-models. Representation through orthogonal polynomials. Advanced Monte Carlo Integration. Multi-level/Multi-fidelity Monte Carlo estimators.
Week 10 Oct 30th - Nov 3rd Advanced/Adaptive representations. Multi-element approaches. Multi-resolution expansion and Multiwavelets. Determining expansion coefficients. Numerical integration and sparse pseudo-spectral approximation method. Stochastic regression. Ordinary least squares.
Week 11 Nov 6th - Nov 10th Determining expansion coefficients. Rudiments of compressed sensing. Greedy heuristics. Orthogonal matching pursuit. Tree-based orthogonal matching pursuit. Phase transition analysis. Design of experiments. Full factorial design. Main effects and interactions. Fractional factorial design.
Week 12 Nov 13th - Nov 17th Screening methods. One-at-a-time designs. Morris OAT designs. Local sensitivity analysis. Measures of local sensitivity. Brute force and direct methods. Derived sensitivities. Normalization.
Week 13 Nov 20th Variance-based sensitivity analysis. Law of total expectation. Law of total variance. Correlation ratio. Estimators for the variance of the conditional expectation.
Week 14 Nov 27th - Dec 1st Variance-based sensitivity analysis. Sobol' decomposition of the stochastic response. Direct sensitivity indices. Total sensitivity indices. Monte Carlo estimates. Variance-based sensitivity indices from polynomial chaos expansions.
Week 15 Dec 4th - Dec 8th Applications. Uncertainty quantification in hemodynamics. Student project presentations.