Mixing
Solving Matrix equation $Y=WX$ for X
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I have the following equation of matrices (3 matrices):
$Y=WX$
I know that to solve for X we have to do:
$W^-1Y=W^-1WXxrightarrowW^-1Y=IXxrightarrowX=W^-1Y$
But... can someone let me know how the above equation is equal to:
$X=(W^TW)^-1W^TY$ ?
where $W^T$ is the matrix transposed and $W^-1$ the inverse of a matrix.
For reference, I found the following equation in page 11 of a PDF in this hyperlink.
matrices inverse transpose
asked Jul 25 at 14:16
ekalyvio
32
add a comment |Â
up vote
0
down vote
favorite
I have the following equation of matrices (3 matrices):
$Y=WX$
I know that to solve for X we have to do:
$W^-1Y=W^-1WXxrightarrowW^-1Y=IXxrightarrowX=W^-1Y$
But... can someone let me know how the above equation is equal to:
$X=(W^TW)^-1W^TY$ ?
where $W^T$ is the matrix transposed and $W^-1$ the inverse of a matrix.
For reference, I found the following equation in page 11 of a PDF in this hyperlink.
matrices inverse transpose
asked Jul 25 at 14:16
ekalyvio
32
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have the following equation of matrices (3 matrices):
$Y=WX$
I know that to solve for X we have to do:
$W^-1Y=W^-1WXxrightarrowW^-1Y=IXxrightarrowX=W^-1Y$
But... can someone let me know how the above equation is equal to:
$X=(W^TW)^-1W^TY$ ?
where $W^T$ is the matrix transposed and $W^-1$ the inverse of a matrix.
For reference, I found the following equation in page 11 of a PDF in this hyperlink.
matrices inverse transpose
asked Jul 25 at 14:16
ekalyvio
32
I have the following equation of matrices (3 matrices):
$Y=WX$
I know that to solve for X we have to do:
$W^-1Y=W^-1WXxrightarrowW^-1Y=IXxrightarrowX=W^-1Y$
But... can someone let me know how the above equation is equal to:
$X=(W^TW)^-1W^TY$ ?
where $W^T$ is the matrix transposed and $W^-1$ the inverse of a matrix.
For reference, I found the following equation in page 11 of a PDF in this hyperlink.
matrices inverse transpose
asked Jul 25 at 14:16
ekalyvio
32
asked Jul 25 at 14:16
ekalyvio
32
asked Jul 25 at 14:16
ekalyvio
32
asked Jul 25 at 14:16
ekalyvio
32
32
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14
add a comment |Â
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14
add a comment |Â
3 Answers
3
active
oldest
votes
up vote
1
down vote
accepted
We have
$$
X=W^-1Y=W^-1(W^T)^-1W^TY=(W^TW)^-1W^TY.
$$
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
add a comment |Â
up vote
0
down vote
As for all invertible matrices $F$, $G$ we have $(FG)^-1=G^-1F^-1$, it follows that $(W^T W)^-1=W^-1(W^T)^-1$ so $(W^T W)^-1W^T = W^-1(W^T)^-1 W^T=W^-1$, and so your two expressions are the same for any invertible $W$.
answered Jul 25 at 14:23
Kusma
1,107111
add a comment |Â
up vote
0
down vote
Your first solution
$$Y=WXimplies X=W^-1Y$$
is valid only when $W$ is a square and invertible matrix.
But in this case, as indicated in the paper, we are using the least square method and since in general matrix $W$ is not square we can show that the best solution is given by
$$x=(W^TW)^-1W^Ty implies X=(W^TW)^-1W^TY$$
The matter is related to projection onto subspaces, for a detailed explanation refer to Writing projection in terms of projection matrix.
answered Jul 25 at 14:27
gimusi
65k73583
add a comment |Â
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
We have
$$
X=W^-1Y=W^-1(W^T)^-1W^TY=(W^TW)^-1W^TY.
$$
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
add a comment |Â
up vote
1
down vote
accepted
We have
$$
X=W^-1Y=W^-1(W^T)^-1W^TY=(W^TW)^-1W^TY.
$$
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
add a comment |Â
up vote
1
down vote
accepted
up vote
1
down vote
accepted
We have
$$
X=W^-1Y=W^-1(W^T)^-1W^TY=(W^TW)^-1W^TY.
$$
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
We have
$$
X=W^-1Y=W^-1(W^T)^-1W^TY=(W^TW)^-1W^TY.
$$
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
answered Jul 25 at 14:21
Dietrich Burde
74.6k64184
74.6k64184
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
add a comment |Â
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
I'm not sure your answer is correct, the paper is referring to least square method and matrix W is, in general not square and not invertible.
– gimusi
Jul 25 at 14:44
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
@gimusi I am only relaying on what the OP wrote in his question "and $W^-1$ the inverse of a matrix". But I believe you, that for the properly formulated question this answer may be no longer correct.
– Dietrich Burde
Jul 25 at 15:02
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
Yes indeed, if $W$ were invertible why should we calculate $$X=(W^TW)^-1W^TY$$ and not by the simpler $$X=W^-1Y$$
– gimusi
Jul 25 at 15:16
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@gimusi, I wish I could vote both answers as they seem correct based on my (probably wrong) question. Dietrich proves that $W^−1Y$ could be written as $(W^TW)^-1W^TY$. But after you stating, my understanding is that the later operation is being used because we don't know if W is invertible and that is why we do not calculate the inverse of it (directly) but we do it with an intermediate calculation (transposing it, etc). As such, I think that: if $Y=WX$ then there exists a solution $X=AY$ in which the equation $A=W^-1$ is not necessary valid? (my math are totally weak) :(
– ekalyvio
Jul 25 at 16:18
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
@ekalyvio To understand what is going on you need to look deeper into the subject, refer to en.wikipedia.org/wiki/Ordinary_least_squares and ocw.mit.edu/courses/mathematics/… and ocw.mit.edu/courses/mathematics/…
– gimusi
Jul 25 at 16:24
add a comment |Â
up vote
0
down vote
As for all invertible matrices $F$, $G$ we have $(FG)^-1=G^-1F^-1$, it follows that $(W^T W)^-1=W^-1(W^T)^-1$ so $(W^T W)^-1W^T = W^-1(W^T)^-1 W^T=W^-1$, and so your two expressions are the same for any invertible $W$.
answered Jul 25 at 14:23
Kusma
1,107111
add a comment |Â
up vote
0
down vote
As for all invertible matrices $F$, $G$ we have $(FG)^-1=G^-1F^-1$, it follows that $(W^T W)^-1=W^-1(W^T)^-1$ so $(W^T W)^-1W^T = W^-1(W^T)^-1 W^T=W^-1$, and so your two expressions are the same for any invertible $W$.
answered Jul 25 at 14:23
Kusma
1,107111
add a comment |Â
up vote
0
down vote
up vote
0
down vote
As for all invertible matrices $F$, $G$ we have $(FG)^-1=G^-1F^-1$, it follows that $(W^T W)^-1=W^-1(W^T)^-1$ so $(W^T W)^-1W^T = W^-1(W^T)^-1 W^T=W^-1$, and so your two expressions are the same for any invertible $W$.
answered Jul 25 at 14:23
Kusma
1,107111
As for all invertible matrices $F$, $G$ we have $(FG)^-1=G^-1F^-1$, it follows that $(W^T W)^-1=W^-1(W^T)^-1$ so $(W^T W)^-1W^T = W^-1(W^T)^-1 W^T=W^-1$, and so your two expressions are the same for any invertible $W$.
answered Jul 25 at 14:23
Kusma
1,107111
answered Jul 25 at 14:23
Kusma
1,107111
answered Jul 25 at 14:23
Kusma
1,107111
answered Jul 25 at 14:23
Kusma
1,107111
1,107111
add a comment |Â
add a comment |Â
up vote
0
down vote
Your first solution
$$Y=WXimplies X=W^-1Y$$
is valid only when $W$ is a square and invertible matrix.
But in this case, as indicated in the paper, we are using the least square method and since in general matrix $W$ is not square we can show that the best solution is given by
$$x=(W^TW)^-1W^Ty implies X=(W^TW)^-1W^TY$$
The matter is related to projection onto subspaces, for a detailed explanation refer to Writing projection in terms of projection matrix.
answered Jul 25 at 14:27
gimusi
65k73583
add a comment |Â
up vote
0
down vote
Your first solution
$$Y=WXimplies X=W^-1Y$$
is valid only when $W$ is a square and invertible matrix.
But in this case, as indicated in the paper, we are using the least square method and since in general matrix $W$ is not square we can show that the best solution is given by
$$x=(W^TW)^-1W^Ty implies X=(W^TW)^-1W^TY$$
The matter is related to projection onto subspaces, for a detailed explanation refer to Writing projection in terms of projection matrix.
answered Jul 25 at 14:27
gimusi
65k73583
add a comment |Â
up vote
0
down vote
up vote
0
down vote
Your first solution
$$Y=WXimplies X=W^-1Y$$
is valid only when $W$ is a square and invertible matrix.
But in this case, as indicated in the paper, we are using the least square method and since in general matrix $W$ is not square we can show that the best solution is given by
$$x=(W^TW)^-1W^Ty implies X=(W^TW)^-1W^TY$$
The matter is related to projection onto subspaces, for a detailed explanation refer to Writing projection in terms of projection matrix.
answered Jul 25 at 14:27
gimusi
65k73583
Your first solution
$$Y=WXimplies X=W^-1Y$$
is valid only when $W$ is a square and invertible matrix.
But in this case, as indicated in the paper, we are using the least square method and since in general matrix $W$ is not square we can show that the best solution is given by
$$x=(W^TW)^-1W^Ty implies X=(W^TW)^-1W^TY$$
The matter is related to projection onto subspaces, for a detailed explanation refer to Writing projection in terms of projection matrix.
answered Jul 25 at 14:27
gimusi
65k73583
edited Jul 25 at 15:13
answered Jul 25 at 14:27
gimusi
65k73583
answered Jul 25 at 14:27
gimusi
65k73583
answered Jul 25 at 14:27
gimusi
65k73583
65k73583
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2862456%2fsolving-matrix-equation-y-wx-for-x%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
The paper linked is referring to Least Square Method, therefore in general $W$ is not square and not invertible and $W^-1$ doesn't exist.
– gimusi
Jul 25 at 15:14