**MATH 697U: Introduction to Stochastic Processes, Spring 2010**

**Meeting :**
MWF 10:10--11:00 LGRT 123

**Instructor :** Luc
Rey-Bellet

**Office :** 1423 J LGRT

**Phone :** 545-6020

**E-Mail : **luc <at> math.umass.edu

**Office Hours :** Monday 2:00--3:15, Wednesday 2:30--4:00, or by appointment.

**Classnotes **
Ch1--Ch2

**Text**: *Introduction
to Stochastic Processes, *2nd edition (2007) by Gregory F.
Lawler, Chapman&Hall.

**Further references:**

*Introduction to Probability Models,*8-th Edition, by Sheldon M. Ross, Academic Press*Lectures on Monte-Carlo Methods,*by Neal N. Madras, American Mathematical Society*Introduction to Stochastic Processes,*by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.*Stochastic Processes,*by Sheldon M. Ross, Wiley.*Adventures in Stochastic processes,*by Sidney I. Resnick, Birkhauser.*A first course in Stochastic Processes,*by Samuel Karlin and Howard M. Taylor, Academic Press.*An introduction to Markov processes.*by Daniel W. Stroock, Graduate Texts in Mathematics, Springer.

**Syllabus: **This course is an
introduction to stochastic processes and Monte-Carlo methods.
Prerequisite are a good knowledge of calculus and elementary
probability as in Stat 515 or Stat 607. We present general concepts
and techniques of the the theory of stochastic processes in
particular Markov chains in discrete and continuous time. We
emphasize examples from various disciplines, for example branching
processes, queueing systems, population models, and so on. We will
also discuss the numerical implementation of Markov chains and
discuss the basics of Monte-Carlo algorithms. Among the topics
treated in the class are

Brief review of probability concepts. The limit theorems for sums of independent random variables.

Simulation of random variables. A first look at Monte-Carlo algorithms.

Markov chains on discrete state spaces (both finite and countable). Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, applications and examples.

Continuous-Time Markov chains. Definition and basic properties. Poisson Process, Birth and Death Process, Queueing models.

Reversible Markov processes and Monte-Carlo Markov algorithms.

Martingales.

Brownian motion and applications.

**Grade :**

1/3 Midterm (Take-home)

1/3 Final Exam

1/3 Homework, attendance and participation

**Exams:**

Midterm: Midterm exam

Final: Final exam

**Homework :**

**Homework #1**(due on February 15): Homework #1

**Homework #2**(due on March 8 ): Homework #2

**Homework #3**(due on April 19 ) Homework #3

**Homework #4**(due on May 3 ): Homework #4