calculating the $P(X,Y,Z) , P(X,Z)$ and $P(X|Y,Z)$ when Z=X+Y

Clash Royale CLAN TAG#URR8PPP

up vote
0
down vote

favorite

Let $X$ and $Y$ be two independent binary random variable with the same alphabet $0,1$,ie,$Pr(0)=Pr(1)=frac12$

I have known the probability of P(X|Z) already.

X Y X+Y X⊕Y Prob
0 0 0 0 1/4 
0 1 1 1 1/4
1 0 1 1 1/4
1 1 2 0 1/4

For $mathbb P(X=x mid Z=z)$

• $mathbb P(X=0 mid X+Y=0)=1$, $,mathbb P(X=1 mid X+Y=0)=0$,


• $mathbb P(X=0 mid X+Y=1)=frac12$, $mathbb P(X=1 mid X+Y=1)=frac12$,


• $mathbb P(X=0 mid X+Y=2)=0$, $,mathbb P(X=1 mid X+Y=1)=1$

But i don't know how to calculate the $P(X,Y,Z) , P(X,Z)$ and $P(X|Y,Z)$? Is $P(X,Y,Z)=P(X,Z)$? Is $P(X|Y,Z)=P(X|Z)$ ?

probability

share|cite|improve this question

asked Jul 29 at 23:55

Shine Sun

1258

add a comment | 

up vote
0
down vote

favorite

Let $X$ and $Y$ be two independent binary random variable with the same alphabet $0,1$,ie,$Pr(0)=Pr(1)=frac12$

I have known the probability of P(X|Z) already.

X Y X+Y X⊕Y Prob
0 0 0 0 1/4 
0 1 1 1 1/4
1 0 1 1 1/4
1 1 2 0 1/4

For $mathbb P(X=x mid Z=z)$

• $mathbb P(X=0 mid X+Y=0)=1$, $,mathbb P(X=1 mid X+Y=0)=0$,


• $mathbb P(X=0 mid X+Y=1)=frac12$, $mathbb P(X=1 mid X+Y=1)=frac12$,


• $mathbb P(X=0 mid X+Y=2)=0$, $,mathbb P(X=1 mid X+Y=1)=1$

But i don't know how to calculate the $P(X,Y,Z) , P(X,Z)$ and $P(X|Y,Z)$? Is $P(X,Y,Z)=P(X,Z)$? Is $P(X|Y,Z)=P(X|Z)$ ?

probability

share|cite|improve this question

asked Jul 29 at 23:55

Shine Sun

1258

add a comment | 

up vote
0
down vote

favorite

up vote
0
down vote

favorite

Let $X$ and $Y$ be two independent binary random variable with the same alphabet $0,1$,ie,$Pr(0)=Pr(1)=frac12$

I have known the probability of P(X|Z) already.

X Y X+Y X⊕Y Prob
0 0 0 0 1/4 
0 1 1 1 1/4
1 0 1 1 1/4
1 1 2 0 1/4

For $mathbb P(X=x mid Z=z)$

• $mathbb P(X=0 mid X+Y=0)=1$, $,mathbb P(X=1 mid X+Y=0)=0$,


• $mathbb P(X=0 mid X+Y=1)=frac12$, $mathbb P(X=1 mid X+Y=1)=frac12$,


• $mathbb P(X=0 mid X+Y=2)=0$, $,mathbb P(X=1 mid X+Y=1)=1$

But i don't know how to calculate the $P(X,Y,Z) , P(X,Z)$ and $P(X|Y,Z)$? Is $P(X,Y,Z)=P(X,Z)$? Is $P(X|Y,Z)=P(X|Z)$ ?

probability

share|cite|improve this question

asked Jul 29 at 23:55

Shine Sun

1258

Let $X$ and $Y$ be two independent binary random variable with the same alphabet $0,1$,ie,$Pr(0)=Pr(1)=frac12$

I have known the probability of P(X|Z) already.

X Y X+Y X⊕Y Prob
0 0 0 0 1/4 
0 1 1 1 1/4
1 0 1 1 1/4
1 1 2 0 1/4

For $mathbb P(X=x mid Z=z)$

• $mathbb P(X=0 mid X+Y=0)=1$, $,mathbb P(X=1 mid X+Y=0)=0$,


• $mathbb P(X=0 mid X+Y=1)=frac12$, $mathbb P(X=1 mid X+Y=1)=frac12$,


• $mathbb P(X=0 mid X+Y=2)=0$, $,mathbb P(X=1 mid X+Y=1)=1$

But i don't know how to calculate the $P(X,Y,Z) , P(X,Z)$ and $P(X|Y,Z)$? Is $P(X,Y,Z)=P(X,Z)$? Is $P(X|Y,Z)=P(X|Z)$ ?

probability

share|cite|improve this question

asked Jul 29 at 23:55

Shine Sun

1258

share|cite|improve this question

share|cite|improve this question

asked Jul 29 at 23:55

Shine Sun

1258

asked Jul 29 at 23:55

Shine Sun

1258

asked Jul 29 at 23:55

Shine Sun

1258

1258

add a comment | 

add a comment | 

1 Answer
1

active

oldest

votes

up vote
1
down vote



accepted

Since $X perp Y$ then $P(X,Y) = P(X) , P(Y) = 1/4$.

$$
beginarrayccc
p_XY& 0 & 1 & p_Y \
0 & 1/4 & 1/4 & 1/2\
1 & 1/4 & 1/4 & 1/2\
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Let’s construct the table of the marginals to find $P(X=x,Z=z) = p_XZ(x,z)$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & ?\
1 & ? & ? & ?\
2 & 0 & ? & ? \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Observe that $p_Z(0) = p_XY(0,0) = 1/4$ as $Z = 0 iff X = 0 cap Y = 0$. The same holds for $p_Z(2) = p_XY(1,1) = 1/4$. Then $p_Z(1) = 1 - 1/4 - 1/4 = 1/2$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & 1/4\
1 & ? & ? & 1/2\
2 & 0 & ? & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

It is easy to resolve all the $?$ inside the table.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & 1/4 & 0 & 1/4\
1 & 1/4 & 1/4 & 1/2\
2 & 0 & 1/4 & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Note that $p_XZ = p_YZ$.

Note also that $p_XY(x,y) = p_XYZ(x,y,0)+p_XYZ(x,y,1)+p_XYZ(x,y,2)$, then:
[
beginalign
p_XY(0,0) = p_XYZ(0,0,0) = 1/4 \
p_XY(0,1) = p_XYZ(0,1,1) = 1/4 \
p_XY(1,0) = p_XYZ(1,0,1) = 1/4 \
p_XY(1,1) = p_XYZ(1,1,2) = 1/4 \
endalign
]

---

Supplement - The conditional mutual information $I(X,Y|Z)$ can be calculated using the formula shown in Wikipedia.

$$I(X,Y|Z)=sum_zin Zsum_yin Ysum_xin Xp_XYZ(x,y,z)logfracp_Z(z)p_XYZ(x,y,z)p_XZ(x,z)p_YZ(y,z)$$
Then
[
beginalign
I(X,Y|Z) &= frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1,/,4cdot p_Z(z)1,/,4 cdot 1,/,4 \
&= frac14sum_zin Zsum_yin Ysum_xin Xlogleft(4 , p_Z(z)right) \
&= sum_zin Zlogleft(4 , p_Z(z)right) & left[text$sum_ysum_x f(z)=4f(z)$right] \
&= logfrac44 + logfrac42 + logfrac44 \
&= log 2
endalign
]
If $log = log_2$ then
[I(X,Y|Z)= 1 text bit]

share|cite|improve this answer

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

• 
  
  
  

  
  

  
  
  
  
  
  

  
  but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
  – Shine Sun
  Jul 30 at 4:48
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  @ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
  – Giulio Scattolin
  Jul 30 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
  – Shine Sun
  Jul 30 at 11:31
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
  – Giulio Scattolin
  Jul 30 at 11:35
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
  – Giulio Scattolin
  Jul 30 at 14:06
  
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

 

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2866549%2fcalculating-the-px-y-z-px-z-and-pxy-z-when-z-xy%23new-answer', 'question_page');

);

Post as a guest

Name Email

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

active

oldest

votes

active

oldest

votes

up vote
1
down vote



accepted

Since $X perp Y$ then $P(X,Y) = P(X) , P(Y) = 1/4$.

$$
beginarrayccc
p_XY& 0 & 1 & p_Y \
0 & 1/4 & 1/4 & 1/2\
1 & 1/4 & 1/4 & 1/2\
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Let’s construct the table of the marginals to find $P(X=x,Z=z) = p_XZ(x,z)$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & ?\
1 & ? & ? & ?\
2 & 0 & ? & ? \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Observe that $p_Z(0) = p_XY(0,0) = 1/4$ as $Z = 0 iff X = 0 cap Y = 0$. The same holds for $p_Z(2) = p_XY(1,1) = 1/4$. Then $p_Z(1) = 1 - 1/4 - 1/4 = 1/2$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & 1/4\
1 & ? & ? & 1/2\
2 & 0 & ? & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

It is easy to resolve all the $?$ inside the table.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & 1/4 & 0 & 1/4\
1 & 1/4 & 1/4 & 1/2\
2 & 0 & 1/4 & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Note that $p_XZ = p_YZ$.

Note also that $p_XY(x,y) = p_XYZ(x,y,0)+p_XYZ(x,y,1)+p_XYZ(x,y,2)$, then:
[
beginalign
p_XY(0,0) = p_XYZ(0,0,0) = 1/4 \
p_XY(0,1) = p_XYZ(0,1,1) = 1/4 \
p_XY(1,0) = p_XYZ(1,0,1) = 1/4 \
p_XY(1,1) = p_XYZ(1,1,2) = 1/4 \
endalign
]

---

Supplement - The conditional mutual information $I(X,Y|Z)$ can be calculated using the formula shown in Wikipedia.

$$I(X,Y|Z)=sum_zin Zsum_yin Ysum_xin Xp_XYZ(x,y,z)logfracp_Z(z)p_XYZ(x,y,z)p_XZ(x,z)p_YZ(y,z)$$
Then
[
beginalign
I(X,Y|Z) &= frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1,/,4cdot p_Z(z)1,/,4 cdot 1,/,4 \
&= frac14sum_zin Zsum_yin Ysum_xin Xlogleft(4 , p_Z(z)right) \
&= sum_zin Zlogleft(4 , p_Z(z)right) & left[text$sum_ysum_x f(z)=4f(z)$right] \
&= logfrac44 + logfrac42 + logfrac44 \
&= log 2
endalign
]
If $log = log_2$ then
[I(X,Y|Z)= 1 text bit]

share|cite|improve this answer

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

• 
  
  
  

  
  

  
  
  
  
  
  

  
  but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
  – Shine Sun
  Jul 30 at 4:48
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  @ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
  – Giulio Scattolin
  Jul 30 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
  – Shine Sun
  Jul 30 at 11:31
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
  – Giulio Scattolin
  Jul 30 at 11:35
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
  – Giulio Scattolin
  Jul 30 at 14:06
  
  

  
  
  
  

add a comment | 

up vote
1
down vote



accepted

Since $X perp Y$ then $P(X,Y) = P(X) , P(Y) = 1/4$.

$$
beginarrayccc
p_XY& 0 & 1 & p_Y \
0 & 1/4 & 1/4 & 1/2\
1 & 1/4 & 1/4 & 1/2\
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Let’s construct the table of the marginals to find $P(X=x,Z=z) = p_XZ(x,z)$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & ?\
1 & ? & ? & ?\
2 & 0 & ? & ? \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Observe that $p_Z(0) = p_XY(0,0) = 1/4$ as $Z = 0 iff X = 0 cap Y = 0$. The same holds for $p_Z(2) = p_XY(1,1) = 1/4$. Then $p_Z(1) = 1 - 1/4 - 1/4 = 1/2$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & 1/4\
1 & ? & ? & 1/2\
2 & 0 & ? & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

It is easy to resolve all the $?$ inside the table.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & 1/4 & 0 & 1/4\
1 & 1/4 & 1/4 & 1/2\
2 & 0 & 1/4 & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Note that $p_XZ = p_YZ$.

Note also that $p_XY(x,y) = p_XYZ(x,y,0)+p_XYZ(x,y,1)+p_XYZ(x,y,2)$, then:
[
beginalign
p_XY(0,0) = p_XYZ(0,0,0) = 1/4 \
p_XY(0,1) = p_XYZ(0,1,1) = 1/4 \
p_XY(1,0) = p_XYZ(1,0,1) = 1/4 \
p_XY(1,1) = p_XYZ(1,1,2) = 1/4 \
endalign
]

---

Supplement - The conditional mutual information $I(X,Y|Z)$ can be calculated using the formula shown in Wikipedia.

$$I(X,Y|Z)=sum_zin Zsum_yin Ysum_xin Xp_XYZ(x,y,z)logfracp_Z(z)p_XYZ(x,y,z)p_XZ(x,z)p_YZ(y,z)$$
Then
[
beginalign
I(X,Y|Z) &= frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1,/,4cdot p_Z(z)1,/,4 cdot 1,/,4 \
&= frac14sum_zin Zsum_yin Ysum_xin Xlogleft(4 , p_Z(z)right) \
&= sum_zin Zlogleft(4 , p_Z(z)right) & left[text$sum_ysum_x f(z)=4f(z)$right] \
&= logfrac44 + logfrac42 + logfrac44 \
&= log 2
endalign
]
If $log = log_2$ then
[I(X,Y|Z)= 1 text bit]

share|cite|improve this answer

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

• 
  
  
  

  
  

  
  
  
  
  
  

  
  but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
  – Shine Sun
  Jul 30 at 4:48
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  @ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
  – Giulio Scattolin
  Jul 30 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
  – Shine Sun
  Jul 30 at 11:31
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
  – Giulio Scattolin
  Jul 30 at 11:35
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
  – Giulio Scattolin
  Jul 30 at 14:06
  
  

  
  
  
  

add a comment | 

up vote
1
down vote



accepted

up vote
1
down vote



accepted

Since $X perp Y$ then $P(X,Y) = P(X) , P(Y) = 1/4$.

$$
beginarrayccc
p_XY& 0 & 1 & p_Y \
0 & 1/4 & 1/4 & 1/2\
1 & 1/4 & 1/4 & 1/2\
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Let’s construct the table of the marginals to find $P(X=x,Z=z) = p_XZ(x,z)$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & ?\
1 & ? & ? & ?\
2 & 0 & ? & ? \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Observe that $p_Z(0) = p_XY(0,0) = 1/4$ as $Z = 0 iff X = 0 cap Y = 0$. The same holds for $p_Z(2) = p_XY(1,1) = 1/4$. Then $p_Z(1) = 1 - 1/4 - 1/4 = 1/2$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & 1/4\
1 & ? & ? & 1/2\
2 & 0 & ? & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

It is easy to resolve all the $?$ inside the table.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & 1/4 & 0 & 1/4\
1 & 1/4 & 1/4 & 1/2\
2 & 0 & 1/4 & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Note that $p_XZ = p_YZ$.

Note also that $p_XY(x,y) = p_XYZ(x,y,0)+p_XYZ(x,y,1)+p_XYZ(x,y,2)$, then:
[
beginalign
p_XY(0,0) = p_XYZ(0,0,0) = 1/4 \
p_XY(0,1) = p_XYZ(0,1,1) = 1/4 \
p_XY(1,0) = p_XYZ(1,0,1) = 1/4 \
p_XY(1,1) = p_XYZ(1,1,2) = 1/4 \
endalign
]

---

Supplement - The conditional mutual information $I(X,Y|Z)$ can be calculated using the formula shown in Wikipedia.

$$I(X,Y|Z)=sum_zin Zsum_yin Ysum_xin Xp_XYZ(x,y,z)logfracp_Z(z)p_XYZ(x,y,z)p_XZ(x,z)p_YZ(y,z)$$
Then
[
beginalign
I(X,Y|Z) &= frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1,/,4cdot p_Z(z)1,/,4 cdot 1,/,4 \
&= frac14sum_zin Zsum_yin Ysum_xin Xlogleft(4 , p_Z(z)right) \
&= sum_zin Zlogleft(4 , p_Z(z)right) & left[text$sum_ysum_x f(z)=4f(z)$right] \
&= logfrac44 + logfrac42 + logfrac44 \
&= log 2
endalign
]
If $log = log_2$ then
[I(X,Y|Z)= 1 text bit]

share|cite|improve this answer

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

Since $X perp Y$ then $P(X,Y) = P(X) , P(Y) = 1/4$.

$$
beginarrayccc
p_XY& 0 & 1 & p_Y \
0 & 1/4 & 1/4 & 1/2\
1 & 1/4 & 1/4 & 1/2\
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Let’s construct the table of the marginals to find $P(X=x,Z=z) = p_XZ(x,z)$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & ?\
1 & ? & ? & ?\
2 & 0 & ? & ? \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Observe that $p_Z(0) = p_XY(0,0) = 1/4$ as $Z = 0 iff X = 0 cap Y = 0$. The same holds for $p_Z(2) = p_XY(1,1) = 1/4$. Then $p_Z(1) = 1 - 1/4 - 1/4 = 1/2$.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & ? & 0 & 1/4\
1 & ? & ? & 1/2\
2 & 0 & ? & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

It is easy to resolve all the $?$ inside the table.

$$
beginarrayccc
p_XZ& 0 & 1 & p_Z \
0 & 1/4 & 0 & 1/4\
1 & 1/4 & 1/4 & 1/2\
2 & 0 & 1/4 & 1/4 \
hline
p_X & 1/2& 1/2 & 1
endarray
$$

Note that $p_XZ = p_YZ$.

Note also that $p_XY(x,y) = p_XYZ(x,y,0)+p_XYZ(x,y,1)+p_XYZ(x,y,2)$, then:
[
beginalign
p_XY(0,0) = p_XYZ(0,0,0) = 1/4 \
p_XY(0,1) = p_XYZ(0,1,1) = 1/4 \
p_XY(1,0) = p_XYZ(1,0,1) = 1/4 \
p_XY(1,1) = p_XYZ(1,1,2) = 1/4 \
endalign
]

---

Supplement - The conditional mutual information $I(X,Y|Z)$ can be calculated using the formula shown in Wikipedia.

$$I(X,Y|Z)=sum_zin Zsum_yin Ysum_xin Xp_XYZ(x,y,z)logfracp_Z(z)p_XYZ(x,y,z)p_XZ(x,z)p_YZ(y,z)$$
Then
[
beginalign
I(X,Y|Z) &= frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1,/,4cdot p_Z(z)1,/,4 cdot 1,/,4 \
&= frac14sum_zin Zsum_yin Ysum_xin Xlogleft(4 , p_Z(z)right) \
&= sum_zin Zlogleft(4 , p_Z(z)right) & left[text$sum_ysum_x f(z)=4f(z)$right] \
&= logfrac44 + logfrac42 + logfrac44 \
&= log 2
endalign
]
If $log = log_2$ then
[I(X,Y|Z)= 1 text bit]

share|cite|improve this answer

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

share|cite|improve this answer

share|cite|improve this answer

edited Jul 31 at 13:58

edited Jul 31 at 13:58

edited Jul 31 at 13:58

answered Jul 30 at 1:35

Giulio Scattolin

15619

answered Jul 30 at 1:35

Giulio Scattolin

15619

answered Jul 30 at 1:35

Giulio Scattolin

15619

15619

• 
  
  
  

  
  

  
  
  
  
  
  

  
  but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
  – Shine Sun
  Jul 30 at 4:48
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  @ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
  – Giulio Scattolin
  Jul 30 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
  – Shine Sun
  Jul 30 at 11:31
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
  – Giulio Scattolin
  Jul 30 at 11:35
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
  – Giulio Scattolin
  Jul 30 at 14:06
  
  

  
  
  
  

add a comment | 

• 
  
  
  

  
  

  
  
  
  
  
  

  
  but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
  – Shine Sun
  Jul 30 at 4:48
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  @ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
  – Giulio Scattolin
  Jul 30 at 10:05
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
  – Shine Sun
  Jul 30 at 11:31
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
  – Giulio Scattolin
  Jul 30 at 11:35
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
  – Giulio Scattolin
  Jul 30 at 14:06
  
  

  
  
  
  

but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
– Shine Sun
Jul 30 at 4:48

but if $I(X;Y|Z)=H(X|Z)-H(X|Y,Z)=[frac-14log_21+0+frac-14log_2frac12+frac-14log_2frac12]-[frac14log_2frac14 times 4]=frac12 -2=$ negative value,but the I(X;Y|Z) should be semipositive ,$gt$.
– Shine Sun
Jul 30 at 4:48

@ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
– Giulio Scattolin
Jul 30 at 10:05

@ShineSun just edited the answer to calculate what you requested. I'm going to investigate your formula right now.
– Giulio Scattolin
Jul 30 at 10:05

may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
– Shine Sun
Jul 30 at 11:31

may i ask why is $I(X,Y|Z)=frac14sum_zin Zsum_yin Ysum_xin Xlogfrac1/8[p_Z(x,z)]^2 =frac12sum_zin Zsum_xin Xlogfrac1/8[p_Z(x,z)]^2$ ?Why is $sum_y=2$
– Shine Sun
Jul 30 at 11:31

I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
– Giulio Scattolin
Jul 30 at 11:35

I think I did something wrong there, from another calculation it comes out $I(X,Y|Z)=2+2-2-1.5=0.5 text bits$. I'm going to figure out that and improve comments ASAP
– Giulio Scattolin
Jul 30 at 11:35

It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
– Giulio Scattolin
Jul 30 at 14:06

It's a bit embarassing, I made a mistake while copying the formula from Wikipedia. Moreover now I get $I(X,Y|Z)=1 text bit$ from the corrected version. I'm afraid the assumption $P(X|Y,Z)=P(X,Z)$ is wrong and I also misused a formula in the calculation I cited in my previous comment.. I've updated the answer in the meanwhile.
– Giulio Scattolin
Jul 30 at 14:06

add a comment | 

 

draft saved

draft discarded

 

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2866549%2fcalculating-the-px-y-z-px-z-and-pxy-z-when-z-xy%23new-answer', 'question_page');

);

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name Email

Name Email

Name

Email

Name Email

Name

Email