Stability of stochastic jump parameter semi-Markov linear systems of difference equations
Efraim Shmerling
Department of Computer Science and Mathematics,
ArielUniversityCenter of Samaria, Ariel, Israel
Abstract. The asymptotic stability of stochastic Itỏ-type jump parameter semi-Markov systems of linear difference equations is examined. A system of matrix equations is presented for which the existence of a positive definite solution of the system implies the asymptotic stability of the stochastic semi-Markov system. Finally, an illustrative example is given.
1. Introduction
Linear systems of differential and difference equations whose coefficient matrices depend on finite-valued Markov processes have been studied intensively during the last decades (see [3] and [4] ) and have been shown to have broad applicability. We started to study more general semi-Markov systems in [1,2].
Here, we consider the following stochastic semi-Markov system of difference equations
, (1)
where w(t) , t= 0,1,2,… isa discrete analogue of the standard Wiener process , ξ(t), t= 0,1,2,… is a finite-valued discrete-time semi-Markov process with n possible states that jumpsfrom state to stateat times tj, j =0,1,2, . . . , with t0 = 0. Note that ‘jumps’ to the same state are possible, in which case k = s. The random sequence { ξ(tj, j =0,1,2, . . .} is a stationary Markov chain with known transition probabilities
; k, s = 1, 2, . . . , n.
The duration of time during which the process belongs to state before it jumps to state , k, s = 1, 2, . . . , n, is given by a discrete integer-valued random variable Tkswhose distribution function Fks(t)= P{Tks ≤ t} and corresponding probability density function pks(t) exist and are known. The intensities qksof the jumps from state to state are given by
qks(t) = πksp ks(t), k, s = 1, 2, . . . , n, t = 1, 2, . . .
and the probability density of the elapsed time Tkbetween two jump times tj and tj+1,
provided that the process jumps to state at time tj, is given by
t = 1, 2, . . .
If Fk(t) denotes the probability distribution function of Tk and ψk(t) denotes the
probability of the event that no jump takes place during the interval (tj, tj + t), provided that the process jumps to state at time tj, then
,, t = 1, 2, . . .
In Section 2 of this paper, we state as an assertion a sufficient condition for
asymptotic stability of the zero solution of system (1). The assertion is analogous to a theorem proved for stochastic semi-Markov systems of differential equations (the proof of the theorem is given in [1]). We assume that the assertion for system (1) can be proved in a similar way. Then, in Section 3, we give an example which illustrates how this assertion can be used to examine asymptotic stability.
2. Formulation of assertion
Mean square asymptotic stability of the zero solution of the system of stochastic difference equations(1) follows from the convergence of the series
and the existence of a positive definite solution of the system of matrix equations
(2)
where are matrix variables, are some positive matrices, and are some positive definite symmetric matrices. The linear transformations are defined by the equalities
, where are the solutions ofthe matrix difference equations
(3)
3. Example
Consider the stochastic difference equation
,
where w(t) designates a discrete analogue of the standard Wiener process, ξ(t) is a semi-Markov process which takes two values, and , and is defined by the intensities
We use the notations:
We have
Obviously , the series and converge iff satisfy the inequalities and .
System (2) takes the form :
(4)
and can be rewritten as
(5)
The condition is satisfied, provided that the inequality
(6)
holds. The inequality (6) is a sufficient condition for the mean square asymptotic stability of the zero solution.
References
[1] E. Shmerling, K.Hochberg
Stability of Stochastic Jump Parameter Semi-Markov Linear Systems of Differential Equations, to appear in: Stochastics: An International Journal of Probability and Stochastic Processes.
[2] E. Shmerling and K.J. Hochberg, Solution of Jump Parameter Systems of Differential and Difference Equations with Semi-Markov Coefficients, J. Appl. Probab. 40 (2003), pp. 442–454.
[3] M. Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker, New York, 1990.
[4] D.D. Sworder and J.E. Boyd, Estimation Problems in Hybrid Systems, CambridgeUniversity Press,New York, 1999.
7-1