Mixing
Simulation of a continous Markov chain in Matlab?
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
Can anyone help me with the code to simulate a continuous time Markov chain? I could not figure out how to write code for a continuous time Markov chain. I would appreciate any help. Thank you in advance.
stochastic-processes markov-chains matlab markov-process stochastic-analysis
edited Jul 16 at 15:35
RayDansh
882215
asked Jul 16 at 15:20
picantonica
61
add a comment |Â
up vote
0
down vote
favorite
Can anyone help me with the code to simulate a continuous time Markov chain? I could not figure out how to write code for a continuous time Markov chain. I would appreciate any help. Thank you in advance.
stochastic-processes markov-chains matlab markov-process stochastic-analysis
edited Jul 16 at 15:35
RayDansh
882215
asked Jul 16 at 15:20
picantonica
61
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Can anyone help me with the code to simulate a continuous time Markov chain? I could not figure out how to write code for a continuous time Markov chain. I would appreciate any help. Thank you in advance.
stochastic-processes markov-chains matlab markov-process stochastic-analysis
edited Jul 16 at 15:35
RayDansh
882215
asked Jul 16 at 15:20
picantonica
61
Can anyone help me with the code to simulate a continuous time Markov chain? I could not figure out how to write code for a continuous time Markov chain. I would appreciate any help. Thank you in advance.
stochastic-processes markov-chains matlab markov-process stochastic-analysis
edited Jul 16 at 15:35
RayDansh
882215
asked Jul 16 at 15:20
picantonica
61
edited Jul 16 at 15:35
RayDansh
882215
edited Jul 16 at 15:35
RayDansh
882215
edited Jul 16 at 15:35
RayDansh
882215
882215
asked Jul 16 at 15:20
picantonica
61
asked Jul 16 at 15:20
picantonica
61
asked Jul 16 at 15:20
picantonica
61
61
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23
add a comment |Â
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
1
down vote
The standard way to simulate a time-homogeneous CTMC is to convert it to the associated "jump chain", which is a DTMC, with associated holding times. To do this, you consider a DTMC with jump probabilities $p_ij=-q_ij/q_ii$, where $Q$ is the generator matrix, and $p_ii=0$. Then you sample the holding time at $i$ as an exponential random variable with rate $-q_ii$.
As for simulating the DTMC, that is basically just a matter of sampling from a discrete distribution many times.
In the time-inhomogeneous case, in general you have to take a step size $h$ (which might itself depend on time) and say that you stay at $i$ with probability $1+hq_ii$ and go to $j$ with probability $hq_ij$. In the time-homogeneous case, this approach wastes samples (because when $h$ is small, most time steps are not jumps) and unnecessarily discretizes the holding time distribution.
answered Jul 16 at 15:39
Ian
65.1k24681
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
The standard way to simulate a time-homogeneous CTMC is to convert it to the associated "jump chain", which is a DTMC, with associated holding times. To do this, you consider a DTMC with jump probabilities $p_ij=-q_ij/q_ii$, where $Q$ is the generator matrix, and $p_ii=0$. Then you sample the holding time at $i$ as an exponential random variable with rate $-q_ii$.
As for simulating the DTMC, that is basically just a matter of sampling from a discrete distribution many times.
In the time-inhomogeneous case, in general you have to take a step size $h$ (which might itself depend on time) and say that you stay at $i$ with probability $1+hq_ii$ and go to $j$ with probability $hq_ij$. In the time-homogeneous case, this approach wastes samples (because when $h$ is small, most time steps are not jumps) and unnecessarily discretizes the holding time distribution.
answered Jul 16 at 15:39
Ian
65.1k24681
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
add a comment |Â
up vote
1
down vote
The standard way to simulate a time-homogeneous CTMC is to convert it to the associated "jump chain", which is a DTMC, with associated holding times. To do this, you consider a DTMC with jump probabilities $p_ij=-q_ij/q_ii$, where $Q$ is the generator matrix, and $p_ii=0$. Then you sample the holding time at $i$ as an exponential random variable with rate $-q_ii$.
As for simulating the DTMC, that is basically just a matter of sampling from a discrete distribution many times.
In the time-inhomogeneous case, in general you have to take a step size $h$ (which might itself depend on time) and say that you stay at $i$ with probability $1+hq_ii$ and go to $j$ with probability $hq_ij$. In the time-homogeneous case, this approach wastes samples (because when $h$ is small, most time steps are not jumps) and unnecessarily discretizes the holding time distribution.
answered Jul 16 at 15:39
Ian
65.1k24681
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
add a comment |Â
up vote
1
down vote
up vote
1
down vote
The standard way to simulate a time-homogeneous CTMC is to convert it to the associated "jump chain", which is a DTMC, with associated holding times. To do this, you consider a DTMC with jump probabilities $p_ij=-q_ij/q_ii$, where $Q$ is the generator matrix, and $p_ii=0$. Then you sample the holding time at $i$ as an exponential random variable with rate $-q_ii$.
As for simulating the DTMC, that is basically just a matter of sampling from a discrete distribution many times.
In the time-inhomogeneous case, in general you have to take a step size $h$ (which might itself depend on time) and say that you stay at $i$ with probability $1+hq_ii$ and go to $j$ with probability $hq_ij$. In the time-homogeneous case, this approach wastes samples (because when $h$ is small, most time steps are not jumps) and unnecessarily discretizes the holding time distribution.
answered Jul 16 at 15:39
Ian
65.1k24681
The standard way to simulate a time-homogeneous CTMC is to convert it to the associated "jump chain", which is a DTMC, with associated holding times. To do this, you consider a DTMC with jump probabilities $p_ij=-q_ij/q_ii$, where $Q$ is the generator matrix, and $p_ii=0$. Then you sample the holding time at $i$ as an exponential random variable with rate $-q_ii$.
As for simulating the DTMC, that is basically just a matter of sampling from a discrete distribution many times.
In the time-inhomogeneous case, in general you have to take a step size $h$ (which might itself depend on time) and say that you stay at $i$ with probability $1+hq_ii$ and go to $j$ with probability $hq_ij$. In the time-homogeneous case, this approach wastes samples (because when $h$ is small, most time steps are not jumps) and unnecessarily discretizes the holding time distribution.
answered Jul 16 at 15:39
Ian
65.1k24681
edited Jul 16 at 17:35
answered Jul 16 at 15:39
Ian
65.1k24681
answered Jul 16 at 15:39
Ian
65.1k24681
answered Jul 16 at 15:39
Ian
65.1k24681
65.1k24681
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
add a comment |Â
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
Thank you. do you know where i can find some codes for Matlab ?? please.
– picantonica
Jul 16 at 15:52
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
@picantonica I doubt you'll find much online, this is sufficiently simple that most people just DIY it (especially since in general making it run fast in a large system necessarily requires custom code).
– Ian
Jul 16 at 15:53
thank you very much :)
– picantonica
Jul 16 at 15:54
thank you very much :)
– picantonica
Jul 16 at 15:54
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2853500%2fsimulation-of-a-continous-markov-chain-in-matlab%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
mathworks.com/help/econ/dtmc.html
– user1949350
Jul 16 at 15:23