Discrete Stochastic

Processes

Robert Gallagher - MIT

1. Introduction & Probability Review

2. More Review: The Bernoulli Process

3. Law of Large Numbers, Convergence

4. Poisson (The Arrival Process)

5. Poisson Combining & Splitting 

6. From Poisson to Markov

7. Finite-state Markov Chains, The Matrix Approach

8. Markov Eigenvalues & Eigenvectors

9. Markov Rewards & Dynamic Processing

10. Renewals & The Strong Law of Large Numbers

11. Renewals: Strong Laws & Rewards

12. Renewal Rewards, Stopping Trials, and Wald's Inequality 

13. Little M/G/1, Ensemble Averages

14. Review

15. The Last Renewal

16. Renewals & Countable-state Markov

17. Countable-state Markov Assumptions

18. Countable-state Markov Chains & Processes

19. Countable-state Markov Processes

20. Markov Processes & Random Walks

21. Hypothesis Testing & Random Walks

22. Random Walks & Thresholds

23. Martingales (Plain, Sub, Super)

24. Martingales: Stopping & Converging

25. Putting it All Together