STUDY OUTLINE FOR STOCHASTIC Ph.D. EXAMINATION
Probability and Statistics
(References: Ross [3], chpts. 1-3; Ross [4], chapt. 1; Kulkarni [2] , appendix.)
Probability and Random Variables
Sample space and events
Probability measures
Combinatorial analysis
Joint and conditional probability
Bayes’ theorem
Random variables
Expectation and moments
Variance, coefficient of variation
Independence
Covariance, correlation
Conditional expectation
Law of total probability
Probability distributions
Discrete random variables
Bernoulli distribution
Binomial distribution
Multinomial distribution
Geometric distribution
Hypergeometric distribution
Poisson distribution
Negative binomial distribution
Generating functions
Continuous random variables
Exponential distribution
Uniform distribution
Gamma distribution
Beta distribution
Normal distribution
Laplace transforms
Laplace-Stieltjes transforms
Convergence and Limit Theorems (Concepts, not proofs)
Convergence with prob. 1 (a.s., a.e.)
Strong law of large numbers
Convergence in probability
Weak law of large numbers
Convergence in distribution
Central limit theorem
Binomial ---- Poisson
Binomial ---- Normal
Discrete-Time Markov Chains
(References: Ross [4] chpt. 4; Heyman and Sobel [1], chpt. 7; Kulkarni [2] chpts. 2-4, Resnick [5] chpt 2.)
Transient Behavior
Chapman-Kolmogorov equations
k-step transition probabilities
Classification of states
Transience
Null recurrence
Positive recurrence
Periodicity, aperiodicity
Reducibility, irreducibility
Criteria for recurrence, transience
First-passage, recurrence times
Limiting and Stationary Behavior
Stationary equations
Stationary probabilities
Limiting probabilities
Computational Techniques
Direct methods
Iterative methods
Applications
M/GI/1 queue
GI/M/1 queue
Markov Chains with Rewards
Exponential Distribution and Poisson Processes
(References: Ross [3] chpt. 5; Ross [4] chpt. 2; Heyman and Sobel [1] chpt 4; Kulkarni [2] chpt. 5, Resnick [5] chpt. 4.)
Properties of Exponential Distribution
Poisson Process
Counting processes
Event times
Inter-arrival and waiting times
Conditional distributions
Splitting and Superposition
Generalizations of Poisson Process
Non-homogeneous Poisson process
Compound Poisson process
Continuous-Time Markov Chains
(References: Ross [4] chpt. 5; Heyman and Sobel, [1] chpt. 8; Kulkarni [2] chpt. 6, Resnick [5] chpt. 5.)
Markov property
Birth-Death Process
Limiting distribution
Applications to queues
Kolmogorov Equations
Forward and Backward Differential Equations
Stationary (Balance) Equations
Stationary Distributions
First-Passage Times
Uniformization
CTMC with Rewards
Renewal Processes
(References: Ross [4] chpt. 3; Heyman and Sobel [1] chpts. 5, 6; Kulkarni [2] chpt. 8, Resnick [5] chpt. 3.)
Counting process
Elementary renewal theorem: strong law
Renewal function
Renewal-type equations and solutions
Elementary renewal theorem: expectation
Key renewal theorem, Blackwell theorem
Delayed renewal processes
Alternating renewal processes
Forward and backward recurrence times
Renewal reward and cumulative processes
Regenerative processes
Markov Renewal Processes
(References: Heyman and Sobel [1] chpt. 9; Kulkarni [2] chpt. 9.)
Properties of Markov Renewal Process
Markov renewal kernel
Counting processes
Markov renewal functions
Generalized Markov Renewal Equations
Semi-Markov Process
Limiting behavior
Markov Regenerative Process
Queueing Processes
(References: Ross [4] chpt. 8; Heyman and Sobel [1] chpt. 11; Kulkarni [2] chpt. 7.)
Little’s Law
PASTA
Birth-Death Queues
M/GI/1 and GI/M/1 Queues
References
1. D. Heyman and M. Sobel. Stochastic Models in Operations Research, Vol. I, McGraw-Hill Book Co., New York, 1982.
2. V. G. Kulkarni, Stochastic Processes: Methods and Applications, Chapman and Hall, London, 1995.
3. S. M. Ross, Introduction to Probability, Academic Press, Inc., Orlando, Florida, 1985, 3rd Edition.
4. S. M. Ross, Stochastic Processes, John Wiley, New York, 1983
5. S. I. Resnick, Adventures in Stochastic Processes, Birkhouser.
5