MATH 697U: Introduction to Stochastic Processes, Fall 2014
Meeting : TuTh 8:30--9:45 LGRT 123
Instructor : Luc
Rey-Bellet
Office : 1423 J LGRT
Phone : 545-6020
E-Mail : luc <at> math.umass.edu
Office Hours : Mo 11:00--12:30, Th 11:00--12:30, or by appointment.
Classnotes
Ch1--Ch2
Text: Introduction to Stochastic Processes, 2nd edition (2007) by Gregory F. Lawler, Chapman&Hall.
Further references:
Introduction to Probability Models, 8-th Edition, by Sheldon M. Ross, Academic Press
Lectures on Monte-Carlo Methods, by Neal N. Madras, American Mathematical Society
Introduction to Stochastic Processes, by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.
Stochastic Processes, by Sheldon M. Ross, Wiley.
Adventures in Stochastic processes, by Sidney I. Resnick, Birkhauser.
A first course in Stochastic Processes, by Samuel Karlin and Howard M. Taylor, Academic Press.
An introduction to Markov processes. by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.
Syllabus: This course is an introduction to stochastic processes and Monte-Carlo methods. Prerequisite are a good knowledge of calculus and elementary probability as in Stat 515 or Stat 607. And we will use from time to time some more advanced concepts from analysis and linear algebra. One of the main goal in the class is to develop a "probabilist intuition and way of thinking". We will present some proofs and we will skip some others in order to provide a reasonably broad range of topics, concepts and techniques. We emphasize examples both in discrete and continuous time from a wide range of disciplines, for example branching processes, queueing systems, population models, chemical reaction networks and so on. We will also discuss the numerical implementation of Markov chains and discuss the basics of Monte-Carlo algorithms. Among the topics treated in the class are
Brief review of probability concepts. The limit theorems for sums of independent random variables.
Simulation of random variables. A first look at Monte-Carlo algorithms.
Markov chains on discrete state spaces (both finite and countable). Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, applications and examples.
Continuous-Time Markov chains. Definition and basic properties. Poisson Process, Birth and Death Process, Queueing models.
Reversible Markov processes and Monte-Carlo Markov algorithms.
Martingales.
Brownian motion and applications. Elementary stochastic analysis.
Grade :
Regular weekly homework will be assigned.
You are expected to pick (with my help) a project, but you have a fairly wide latitude to pick your project which fits your tastes and research interests. Your project will consists of a written research style paper and an oral presentation to the class.
Homework :
Homework Week #1 (due on 9/16):
Week 1
Week 1 solutions
Homework Week #2 (due on 9/23):
Week 2
Week 2 solutions
Homework Week #3 (due on 9/30):
Week 3
Week 3 solutions
Homework Week #4 (due on 10/7):
Week 4
Week 4 solutions
Homework Week #5 (due on 10/16): Week 5
Homework Week #6 (due on 10/21): Week 6
Homework Week #7 (due on 10/28):
Week 7
Homework Week #8 (due on 11/11): Week 8
Homework Week #9 (due on 11/18): Week 9