**MATH 697U: Introduction to Stochastic Processes, Fall 2014**

**Meeting :**
TuTh 8:30--9:45 LGRT 123

**Instructor :** Luc
Rey-Bellet

**Office :** 1423 J LGRT

**Phone :** 545-6020

**E-Mail : **luc <at> math.umass.edu

**Office Hours :** Mo 11:00--12:30, Th 11:00--12:30, or by appointment.

**Classnotes **
Ch1--Ch2

**Text**: *Introduction
to Stochastic Processes, *2nd edition (2007) by Gregory F.
Lawler, Chapman&Hall.

**Further references:**

*Introduction to Probability Models,*8-th Edition, by Sheldon M. Ross, Academic Press*Lectures on Monte-Carlo Methods,*by Neal N. Madras, American Mathematical Society*Introduction to Stochastic Processes,*by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.*Stochastic Processes,*by Sheldon M. Ross, Wiley.*Adventures in Stochastic processes,*by Sidney I. Resnick, Birkhauser.*A first course in Stochastic Processes,*by Samuel Karlin and Howard M. Taylor, Academic Press.*An introduction to Markov processes.*by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.

**Syllabus: **This course is an
introduction to stochastic processes and Monte-Carlo methods.
Prerequisite are a good knowledge of calculus and elementary
probability as in Stat 515 or Stat 607. And we will use from time to
time some more advanced concepts from analysis and linear algebra.
One of the main goal in the class is to develop a "probabilist intuition and way of thinking".
We will present some proofs and we will skip some others in order to provide
a reasonably broad range of topics, concepts and techniques.
We emphasize examples both in discrete and continuous time from a wide range of
disciplines, for example branching processes, queueing systems, population models, chemical reaction networks and so on.
We will also discuss the numerical implementation of Markov chains and
discuss the basics of Monte-Carlo algorithms. Among the topics
treated in the class are

Brief review of probability concepts. The limit theorems for sums of independent random variables.

Simulation of random variables. A first look at Monte-Carlo algorithms.

Markov chains on discrete state spaces (both finite and countable). Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, applications and examples.

Continuous-Time Markov chains. Definition and basic properties. Poisson Process, Birth and Death Process, Queueing models.

Reversible Markov processes and Monte-Carlo Markov algorithms.

Martingales.

Brownian motion and applications. Elementary stochastic analysis.

**Grade :**

Regular weekly homework will be assigned.

You are expected to pick (with my help) a project, but you have a fairly wide latitude to pick your project which fits your tastes and research interests. Your project will consists of a written research style paper and an oral presentation to the class.

**Homework :**

**Homework Week #1**(due on 9/16): Week 1 Week 1 solutions

**Homework Week #2**(due on 9/23): Week 2 Week 2 solutions

**Homework Week #3**(due on 9/30): Week 3 Week 3 solutions

**Homework Week #4**(due on 10/7): Week 4 Week 4 solutions

**Homework Week #5**(due on 10/16): Week 5

**Homework Week #6**(due on 10/21): Week 6

**Homework Week #7**(due on 10/28): Week 7

**Homework Week #8**(due on 11/11): Week 8

**Homework Week #9**(due on 11/18): Week 9