Mixing
Under what conditions does $H(Xmid f(Y))=H(Xmid Y)$?
Clash Royale CLAN TAG#URR8PPP
up vote
2
down vote
favorite
I have the problem that I cannot solve:
Under what conditions does $H(X∣f(Y))=H(X∣Y)$?
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$. Are they equal?
This is an exercise in the textbook. There is a solution, but I don't think it's correct (more precisely, it is not satisfactory enough). The provided solution is as below.
Suggested Solution (not satisfactory). If $H(X|g(Y )) = H(X|Y )$, then $H(X)−H(X|g(Y )) = H(X) − H(X|Y )$, i.e., $I(X; g(Y )) = I(X; Y )$. This is the condition for equality in the data processing inequality. From the derivation of the inequality, we have equality iff $X → g(Y ) → Y$ forms a Markov chain. Hence $H(X|g(Y )) = H(X|Y )$ iff $X → g(Y ) → Y$ . This condition includes many special cases, such as $g$ being one-to-one, and $X$ and $Y$ being independent. However, these two special cases do not exhaust all the possibilities.
probability statistics markov-chains entropy
asked Jul 30 at 0:14
khahuras
184
add a comment |Â
up vote
2
down vote
favorite
I have the problem that I cannot solve:
Under what conditions does $H(X∣f(Y))=H(X∣Y)$?
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$. Are they equal?
This is an exercise in the textbook. There is a solution, but I don't think it's correct (more precisely, it is not satisfactory enough). The provided solution is as below.
Suggested Solution (not satisfactory). If $H(X|g(Y )) = H(X|Y )$, then $H(X)−H(X|g(Y )) = H(X) − H(X|Y )$, i.e., $I(X; g(Y )) = I(X; Y )$. This is the condition for equality in the data processing inequality. From the derivation of the inequality, we have equality iff $X → g(Y ) → Y$ forms a Markov chain. Hence $H(X|g(Y )) = H(X|Y )$ iff $X → g(Y ) → Y$ . This condition includes many special cases, such as $g$ being one-to-one, and $X$ and $Y$ being independent. However, these two special cases do not exhaust all the possibilities.
probability statistics markov-chains entropy
asked Jul 30 at 0:14
khahuras
184
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04
add a comment |Â
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I have the problem that I cannot solve:
Under what conditions does $H(X∣f(Y))=H(X∣Y)$?
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$. Are they equal?
This is an exercise in the textbook. There is a solution, but I don't think it's correct (more precisely, it is not satisfactory enough). The provided solution is as below.
Suggested Solution (not satisfactory). If $H(X|g(Y )) = H(X|Y )$, then $H(X)−H(X|g(Y )) = H(X) − H(X|Y )$, i.e., $I(X; g(Y )) = I(X; Y )$. This is the condition for equality in the data processing inequality. From the derivation of the inequality, we have equality iff $X → g(Y ) → Y$ forms a Markov chain. Hence $H(X|g(Y )) = H(X|Y )$ iff $X → g(Y ) → Y$ . This condition includes many special cases, such as $g$ being one-to-one, and $X$ and $Y$ being independent. However, these two special cases do not exhaust all the possibilities.
probability statistics markov-chains entropy
asked Jul 30 at 0:14
khahuras
184
I have the problem that I cannot solve:
Under what conditions does $H(X∣f(Y))=H(X∣Y)$?
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$. Are they equal?
This is an exercise in the textbook. There is a solution, but I don't think it's correct (more precisely, it is not satisfactory enough). The provided solution is as below.
Suggested Solution (not satisfactory). If $H(X|g(Y )) = H(X|Y )$, then $H(X)−H(X|g(Y )) = H(X) − H(X|Y )$, i.e., $I(X; g(Y )) = I(X; Y )$. This is the condition for equality in the data processing inequality. From the derivation of the inequality, we have equality iff $X → g(Y ) → Y$ forms a Markov chain. Hence $H(X|g(Y )) = H(X|Y )$ iff $X → g(Y ) → Y$ . This condition includes many special cases, such as $g$ being one-to-one, and $X$ and $Y$ being independent. However, these two special cases do not exhaust all the possibilities.
probability statistics markov-chains entropy
asked Jul 30 at 0:14
khahuras
184
edited Jul 30 at 2:11
asked Jul 30 at 0:14
khahuras
184
asked Jul 30 at 0:14
khahuras
184
asked Jul 30 at 0:14
khahuras
184
184
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04
add a comment |Â
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
3
down vote
accepted
The data processing inequality lemma already provides the complete answer. $I(X;Y)=I(X;g(Y))$ holds if and only if $$tag1Xrightarrow Y rightarrow g(Y)$$ and $$tag2 Xrightarrow g(Y) rightarrow Y.$$
Now note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid yright)tag3,
endalign
$$
where the last equality holds since (1) implies that $g(Y)rightarrow Y rightarrow X$, i.e., $X$ is independent of $g(Y)$, when conditioned on $Y$. Also note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid g(y)right) tag4,
endalign
$$
where the last equality holds since (2) implies $Yrightarrow g(Y) rightarrow X$.
It follows from (3) and (4) that it must hold
$$
pleft(xmid g(y)right) = p(x mid y),
$$
for all valid values of $x$ and $y$. This is the most general condition you seek, which captures the cases of $g$ being one to one and $Y$ independent of $X$.
answered Jul 30 at 15:30
Stelios
2,051278
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
The data processing inequality lemma already provides the complete answer. $I(X;Y)=I(X;g(Y))$ holds if and only if $$tag1Xrightarrow Y rightarrow g(Y)$$ and $$tag2 Xrightarrow g(Y) rightarrow Y.$$
Now note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid yright)tag3,
endalign
$$
where the last equality holds since (1) implies that $g(Y)rightarrow Y rightarrow X$, i.e., $X$ is independent of $g(Y)$, when conditioned on $Y$. Also note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid g(y)right) tag4,
endalign
$$
where the last equality holds since (2) implies $Yrightarrow g(Y) rightarrow X$.
It follows from (3) and (4) that it must hold
$$
pleft(xmid g(y)right) = p(x mid y),
$$
for all valid values of $x$ and $y$. This is the most general condition you seek, which captures the cases of $g$ being one to one and $Y$ independent of $X$.
answered Jul 30 at 15:30
Stelios
2,051278
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
add a comment |Â
up vote
3
down vote
accepted
The data processing inequality lemma already provides the complete answer. $I(X;Y)=I(X;g(Y))$ holds if and only if $$tag1Xrightarrow Y rightarrow g(Y)$$ and $$tag2 Xrightarrow g(Y) rightarrow Y.$$
Now note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid yright)tag3,
endalign
$$
where the last equality holds since (1) implies that $g(Y)rightarrow Y rightarrow X$, i.e., $X$ is independent of $g(Y)$, when conditioned on $Y$. Also note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid g(y)right) tag4,
endalign
$$
where the last equality holds since (2) implies $Yrightarrow g(Y) rightarrow X$.
It follows from (3) and (4) that it must hold
$$
pleft(xmid g(y)right) = p(x mid y),
$$
for all valid values of $x$ and $y$. This is the most general condition you seek, which captures the cases of $g$ being one to one and $Y$ independent of $X$.
answered Jul 30 at 15:30
Stelios
2,051278
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
add a comment |Â
up vote
3
down vote
accepted
up vote
3
down vote
accepted
The data processing inequality lemma already provides the complete answer. $I(X;Y)=I(X;g(Y))$ holds if and only if $$tag1Xrightarrow Y rightarrow g(Y)$$ and $$tag2 Xrightarrow g(Y) rightarrow Y.$$
Now note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid yright)tag3,
endalign
$$
where the last equality holds since (1) implies that $g(Y)rightarrow Y rightarrow X$, i.e., $X$ is independent of $g(Y)$, when conditioned on $Y$. Also note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid g(y)right) tag4,
endalign
$$
where the last equality holds since (2) implies $Yrightarrow g(Y) rightarrow X$.
It follows from (3) and (4) that it must hold
$$
pleft(xmid g(y)right) = p(x mid y),
$$
for all valid values of $x$ and $y$. This is the most general condition you seek, which captures the cases of $g$ being one to one and $Y$ independent of $X$.
answered Jul 30 at 15:30
Stelios
2,051278
The data processing inequality lemma already provides the complete answer. $I(X;Y)=I(X;g(Y))$ holds if and only if $$tag1Xrightarrow Y rightarrow g(Y)$$ and $$tag2 Xrightarrow g(Y) rightarrow Y.$$
Now note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid yright)tag3,
endalign
$$
where the last equality holds since (1) implies that $g(Y)rightarrow Y rightarrow X$, i.e., $X$ is independent of $g(Y)$, when conditioned on $Y$. Also note that
$$
beginalign
p(x,g(y),y) &= p(y, g(y))pleft(xmid y, g(y)right)\
&=p(y, g(y))pleft(xmid g(y)right) tag4,
endalign
$$
where the last equality holds since (2) implies $Yrightarrow g(Y) rightarrow X$.
It follows from (3) and (4) that it must hold
$$
pleft(xmid g(y)right) = p(x mid y),
$$
for all valid values of $x$ and $y$. This is the most general condition you seek, which captures the cases of $g$ being one to one and $Y$ independent of $X$.
answered Jul 30 at 15:30
Stelios
2,051278
edited Jul 30 at 15:40
answered Jul 30 at 15:30
Stelios
2,051278
answered Jul 30 at 15:30
Stelios
2,051278
answered Jul 30 at 15:30
Stelios
2,051278
2,051278
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
add a comment |Â
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
What if it was extended to under which conditions $ mathbbE left[ X mid Y right] = mathbbE left[ X mid g left( Y right) right] $?
– Royi
Jul 30 at 17:05
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
Thanks for your solution @Stelios.
– khahuras
Jul 31 at 4:28
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2866558%2funder-what-conditions-does-hx-mid-fy-hx-mid-y%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
"it is not satisfactory enough" Why?
– Clement C.
Jul 30 at 0:35
I would like to draw a result about the relation between $p_X(cdot | g(Y))$ and $p_X(cdot | Y)$.
– khahuras
Jul 30 at 2:11
I would add when $ mathbbE left[ X mid Y right] = mathbbE left[ X mid f left( Y right) right] $ and $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $. Clearly when $ p_X left( cdot mid Y right) = p_X left( cdot mid f left( Y right) right) $ all are equal.
– Royi
Jul 30 at 4:51
@Royi, I don't understand your comment.
– khahuras
Jul 30 at 5:32
I said you should extend the question to the cases I wrote above in addition to what you wrote.
– Royi
Jul 30 at 6:04