I will post slides on Moodle after each lecture. The lecture materials are based on a list of reference books that we will use in this class.
• Sheldon M. Ross, Stochastic Processes. ISBN-13: 978-0471120629, ISBN-10: 0471120626
We will cover from Chapter 1 -- Chapter 8 on most of the topics in this book.
• Sheldon M. Ross, Simulation., ISBN-13: 978-0124158252, ISBN-10: 0124158250
• Art B. Owen , Monte Carlo theory, methods and examples,
• [online] J. Pitman and M. Yor, "A guide to Brownian motion and related stochastic processes.
• [online] R. Durett, Probability: Theory and Examples, Cambridge UP, 2019
• [online] O. Kallenberg, Foundations of Modern Probability, 1st ed., Springer, 1997.
• Stochastic Simulation: Algorithms and Analysis (Stochastic Modelling and Applied Probability, No. 57)
The course will cover the following topics in the core of the theory of Random and Stochastic Processes. We will introduce the students to some fundamental stochastic processes such as discrete state Markov chains, Poisson processes, and Brownian motion, as well as an array of important stochastic models. Computer programming will be a central part of this course. The theoretical part of this course includes analysis of random walks, convergence of discrete and continuous-time Markov processes to stationarity, Poisson processes and other point processes, renewal theory, Brownian motion and Martingale processes. A major focus of the course is on solving problems extending the scope of the lectures, developing analytical skills and probabilistic intuition.
• Review of probability theory: probability space, random variables, expectations, independence, conditional expectations.
• Random walks and finite state Markov chains: Transition matrix, transience and recurrence, limiting distributions, convergence of Markov Chains.
• Poisson processes: definition, inter-arrival and waiting time. • *Renewal process: introductions, renewal equation, renewal theorems (include of time permits) • Continuous Markov chains: Strong Markov properties, Chapman Kolmogorov equations, irreducible and recurrence. Long time behaviors • Random walks and Brownian Motions: Definitions, scaled random walk, Brownian motion. • Martingales: introduction, optional sampling theorem, Martingale inequalities.
In addition, we also offer a parallel series of lectures on topics of stochastic simulations -- introduction to Monte Carlo methods and computer modeling of stochastic systems. Monte Carlo topics that we will cover include random variable generation, expectation estimation with confidence interval formation, importance sampling, stochastic optimization, MCMC algorithms and sampling of Brownian motion. Tentatively, the following topics on Stochastic Simulations will be discussed during the semester:
• Review of basic probability, generation of pseudorandom numbers.
• Monte Carlo integration Simulation of random samples from discrete distributions and continuous distributions
• Use simulations to estimate integrals and properties of random variables together with corresponding measures of accuracy
• Calculate analytically the properties of random walk, Brownian motion, Poisson process and Markov Chains.
• Simulate the realizations of the stochastic processes introduced in the course and use them to estimate properties of the underlying random processes
• Analysis, verification, and validation of simulation results
• Variance reduction techniques for MCMC.
Prerequisites are familiarity with (measure-theoretic) probability theory as it is treated in the course “Advanced Probability Theory" (Math 605). Note that if you only took Stat 515 or other probability theory courses that did not use measure theory, these will NOT be counted as the prerequisites. To check if you have enough background, try to answer the following questions:
• Given a sequence of random variables X_n. What are the definitions of X_n converges to X in probability, or in distribution, or almost surely? Which is stronger than others? Does convergence in mean implied almost sure convergence, why?
• Give the proof of CLT and the Weak Law of Large numbers.
• What is the sigma-algebra generated by a random variable?
If you have trouble solving these questions, then you should consider taking Math 605 or another Stochastic Processes course that will be offered in Fall 2022 for the undergraduate students.
• Lecture: Lectures will be delivered alternatively between these two topics; one will be about theory of stochastic processes and models and the other one will be about statistical computing and Monte Carlo simulation. Concepts will be introduced in an interactive lecture format with discussion of Python or Matlab, where appropriate. You will be expected to learn the concepts found in both the lectures and assigned/associated readings. Lecture time includes Python exercises that are conducted with follow-up discussion. Completed lecture content will be available online.
• Homework: Homework assignments will be both mathematical and data analytic in nature. The assignments will be found on the web and due (preferably typed) by the beginning of lecture on the associated due date (Sunday night) listed on Moodle. Be sure to include your name (and section number, if applicable) on all assignments! NO LATE HOMEWORKS WILL BE ACCEPTED. I encourage you to find a teammate to do your homework, especially if you do not have much coding experience. My teaching experiments tell me that students learn more in team works or discussions. You are allowed and encouraged to discuss the assignments with each other, but the work that you hand in MUST BE YOUR OWN. This means that each student must perform all analyses on his/her own computer, and must independently write up the analysis. Plagiarism will be swiftly dealt with to the full extent allowed under UMass policies on cheating and plagiarism. For students who do not have any programing background, and reluctant to code, you can choose to finish all the written homework for each assigned set to get full credit. For students who prefer to do the coding assignment, you can choose to do half of the written homework, as well as the simulation assignment to get full credit.
• Project report for final. You need to form a team together with 2 more students (discuss with me if you want to form a team with 4 students). Your team will need to select a topic for your project at the end of February. Once you have some idea, or need some ideas, contact me, I will discuss it with you.
• There will be no midterm exam, instead, we will have midterm presentations on your project plans, Each team will discuss 20-30 minites to explain what your project will be.
• The outcome of your project should contain a project report, as well as a in-class presentation at the end of the semester.
• Moodle Webpage: The course outline, lecture notes, necessary data, homework assignments and solutions, information for the exams, and supplementary material for this course can be found on Moodle (located at Homework assignments will be posted one week prior to their due date. Any additional announcements that are made over the course of the semester will also be found / updated on Moodle.
• Grading:
Homework: 60%
Final project & presentation: 40%
100%