INSEAD Library

Includes glossary and index

Digitized

Introduction to Stochastic Processes
Table of Contents
1 Markov Chains 1
1.1 Markov chains having two states 2
1.2 Transition function and initial distribution 5
1.3 Examples 6
1.4 Computations with transition fonctions 12
1.4.1 Hitting times 14
1.4.2 Transition matrix 16
1.5 Transient and recurrent states 17
1.6 Decomposition of the state space 21
1.6.1 Absorption probabilities 25
1.6.2 Martingales 27
1.7 Birth and death chains 29
1.8 Branching and queuing chains 33
1.8.1 Branching chain 34
1.8.2 Queuing chain 36
Appendix
1.9 Proof of results for the branching and queuing chains 36
1.9.1 Branching chain 38
1.9.2 Queuing chain 39
2 Stationary Distributions of a Markov Chain 47
2.1 Elementary properties of stationary distributions 47
2.2 Examples 49
2.2.1 Birth and death chain 50
2.2.2 Particles in a box 53
2.3 Average number of visits to a recurrent state 56
2.4 Null recurrent and positive recurrent states 60
2.5 Existence and uniqueness of stationary distributions 63
2.5.1 Reducible chains 67
2.6 Queuing chain 69
2.6.1 Proof 70
2.7 Convergence to the stationary distribution 72
Appendix
2.8 Proof of convergence 75
2.8.1 Periodic case 77
2.8.2 A result from number theory 79
3 Markov Pure Jump Processes 84
3.1 Construction of jump processes 84
3.2 Birth and death processes 89
3.2.1 Two-state birth and death process 92
3.2.2 Poisson process 94
3.2.3 Pure birth process 98
3.2.4 Infinite server queue 99
3.3 Properties of a Markov pure jump process 102
3.3.1 Applications to birth and death processes 104
4 Second Order Processes 111
4.1 Mean and covariance functions 111
4.2 Gaussian processes 119
4.3 The Wiener process 122
5 Continuity, Integration, and Differentiation of Second
Order Processes 128
5.1 Continuity assumptions 128
5.1.1 Continuity of the mean and covariance functions 128
5.1.2 Continuity of the sample functions 130
5.2 Integration 132
5.3 Differentiation 135
5.4 White noise 141
6 Stochastic Differential Equations, Estimation Theory,
and Spectral Distributions 152
6.1 First order differential equations 154
6.2 Differential equations of order n 159
6.2.1 The case n 2 166
6.3 Estimation theory 170
6.3.1 General principles of estimation 173
6.3.2 Some examples of optimal prediction 174
6.4 Spectral distribution 177
Answers to Exercises 190
Glossary of Notation 199
Index 201