Gradient derived from Jacobian?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
2
down vote

favorite












I was reading this wikipedia page on gradient descent (section: Solution of non-linear system) when I came across this formula:



$nabla F(mathbf x ^(0))=J_G(mathbf x ^(0))^mathrm T G(mathbf x ^(0))$



How did this equation come about? Sorry for my shaky calculus if this sounds stupid, but this is the only point in this section that I don't understand.




calculus nonlinear-optimization gradient-descent jacobian




share|cite|improve this question



















  • It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
    – littleO
    Jul 18 at 5:08










  • related question: math.stackexchange.com/questions/2818147/…
    – Chav Likit
    Jul 18 at 7:35














up vote
2
down vote

favorite












I was reading this wikipedia page on gradient descent (section: Solution of non-linear system) when I came across this formula:



$nabla F(mathbf x ^(0))=J_G(mathbf x ^(0))^mathrm T G(mathbf x ^(0))$



How did this equation come about? Sorry for my shaky calculus if this sounds stupid, but this is the only point in this section that I don't understand.




calculus nonlinear-optimization gradient-descent jacobian




share|cite|improve this question



















  • It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
    – littleO
    Jul 18 at 5:08










  • related question: math.stackexchange.com/questions/2818147/…
    – Chav Likit
    Jul 18 at 7:35












up vote
2
down vote

favorite









up vote
2
down vote

favorite











I was reading this wikipedia page on gradient descent (section: Solution of non-linear system) when I came across this formula:



$nabla F(mathbf x ^(0))=J_G(mathbf x ^(0))^mathrm T G(mathbf x ^(0))$



How did this equation come about? Sorry for my shaky calculus if this sounds stupid, but this is the only point in this section that I don't understand.




calculus nonlinear-optimization gradient-descent jacobian




share|cite|improve this question











I was reading this wikipedia page on gradient descent (section: Solution of non-linear system) when I came across this formula:



$nabla F(mathbf x ^(0))=J_G(mathbf x ^(0))^mathrm T G(mathbf x ^(0))$



How did this equation come about? Sorry for my shaky calculus if this sounds stupid, but this is the only point in this section that I don't understand.





calculus nonlinear-optimization gradient-descent jacobian





share|cite|improve this question










share|cite|improve this question




share|cite|improve this question









asked Jul 18 at 4:59









Chav Likit

495




495











  • It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
    – littleO
    Jul 18 at 5:08










  • related question: math.stackexchange.com/questions/2818147/…
    – Chav Likit
    Jul 18 at 7:35
















  • It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
    – littleO
    Jul 18 at 5:08










  • related question: math.stackexchange.com/questions/2818147/…
    – Chav Likit
    Jul 18 at 7:35















It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
– littleO
Jul 18 at 5:08




It would help to mention in the body of the question that $F(x) = (1/2) G(x)^T G(x)$.
– littleO
Jul 18 at 5:08












related question: math.stackexchange.com/questions/2818147/…
– Chav Likit
Jul 18 at 7:35




related question: math.stackexchange.com/questions/2818147/…
– Chav Likit
Jul 18 at 7:35










1 Answer
1






active

oldest

votes

















up vote
2
down vote



accepted










From the article



$$
F(bf x) = frac12G^T(bf x) G(bf x) = frac12sum_j G_j(bf x) G_j(bf x) tag1
$$



Take the derivative w.r.t to $x_i$



begineqnarray
fracpartial Fpartial x_i &=&
frac12sum_j fracpartialpartial x_ileft[G_j(bf x) G_j(bf x) right] = sum_j colorbluefracpartial G_j(bf x)partial x_i G_j(bf x) \
&=& sum_j colorblue[J_G(bf x)]_ij G_j(bf x) = [J_G(bf x) G(bf x)]_i tag2
endeqnarray



where the $(i,j)$-th component of matrix $J_G$ is defined as



$$
[J_G(bf x)]_ij = fracpartial G_j(bf x)partial x_i tag3
$$



Eq. (2) tells you what the $i$-th component of the gradient is, if you put them all together you get



$$
nabla F(bf x) = J_G(bf x) G(bf x) tag4
$$






share|cite|improve this answer





















    Your Answer




    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "69"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: false,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );








     

    draft saved


    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2855207%2fgradient-derived-from-jacobian%23new-answer', 'question_page');

    );

    Post as a guest

















    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    2
    down vote



    accepted










    From the article



    $$
    F(bf x) = frac12G^T(bf x) G(bf x) = frac12sum_j G_j(bf x) G_j(bf x) tag1
    $$



    Take the derivative w.r.t to $x_i$



    begineqnarray
    fracpartial Fpartial x_i &=&
    frac12sum_j fracpartialpartial x_ileft[G_j(bf x) G_j(bf x) right] = sum_j colorbluefracpartial G_j(bf x)partial x_i G_j(bf x) \
    &=& sum_j colorblue[J_G(bf x)]_ij G_j(bf x) = [J_G(bf x) G(bf x)]_i tag2
    endeqnarray



    where the $(i,j)$-th component of matrix $J_G$ is defined as



    $$
    [J_G(bf x)]_ij = fracpartial G_j(bf x)partial x_i tag3
    $$



    Eq. (2) tells you what the $i$-th component of the gradient is, if you put them all together you get



    $$
    nabla F(bf x) = J_G(bf x) G(bf x) tag4
    $$






    share|cite|improve this answer

























      up vote
      2
      down vote



      accepted










      From the article



      $$
      F(bf x) = frac12G^T(bf x) G(bf x) = frac12sum_j G_j(bf x) G_j(bf x) tag1
      $$



      Take the derivative w.r.t to $x_i$



      begineqnarray
      fracpartial Fpartial x_i &=&
      frac12sum_j fracpartialpartial x_ileft[G_j(bf x) G_j(bf x) right] = sum_j colorbluefracpartial G_j(bf x)partial x_i G_j(bf x) \
      &=& sum_j colorblue[J_G(bf x)]_ij G_j(bf x) = [J_G(bf x) G(bf x)]_i tag2
      endeqnarray



      where the $(i,j)$-th component of matrix $J_G$ is defined as



      $$
      [J_G(bf x)]_ij = fracpartial G_j(bf x)partial x_i tag3
      $$



      Eq. (2) tells you what the $i$-th component of the gradient is, if you put them all together you get



      $$
      nabla F(bf x) = J_G(bf x) G(bf x) tag4
      $$






      share|cite|improve this answer























        up vote
        2
        down vote



        accepted







        up vote
        2
        down vote



        accepted






        From the article



        $$
        F(bf x) = frac12G^T(bf x) G(bf x) = frac12sum_j G_j(bf x) G_j(bf x) tag1
        $$



        Take the derivative w.r.t to $x_i$



        begineqnarray
        fracpartial Fpartial x_i &=&
        frac12sum_j fracpartialpartial x_ileft[G_j(bf x) G_j(bf x) right] = sum_j colorbluefracpartial G_j(bf x)partial x_i G_j(bf x) \
        &=& sum_j colorblue[J_G(bf x)]_ij G_j(bf x) = [J_G(bf x) G(bf x)]_i tag2
        endeqnarray



        where the $(i,j)$-th component of matrix $J_G$ is defined as



        $$
        [J_G(bf x)]_ij = fracpartial G_j(bf x)partial x_i tag3
        $$



        Eq. (2) tells you what the $i$-th component of the gradient is, if you put them all together you get



        $$
        nabla F(bf x) = J_G(bf x) G(bf x) tag4
        $$






        share|cite|improve this answer













        From the article



        $$
        F(bf x) = frac12G^T(bf x) G(bf x) = frac12sum_j G_j(bf x) G_j(bf x) tag1
        $$



        Take the derivative w.r.t to $x_i$



        begineqnarray
        fracpartial Fpartial x_i &=&
        frac12sum_j fracpartialpartial x_ileft[G_j(bf x) G_j(bf x) right] = sum_j colorbluefracpartial G_j(bf x)partial x_i G_j(bf x) \
        &=& sum_j colorblue[J_G(bf x)]_ij G_j(bf x) = [J_G(bf x) G(bf x)]_i tag2
        endeqnarray



        where the $(i,j)$-th component of matrix $J_G$ is defined as



        $$
        [J_G(bf x)]_ij = fracpartial G_j(bf x)partial x_i tag3
        $$



        Eq. (2) tells you what the $i$-th component of the gradient is, if you put them all together you get



        $$
        nabla F(bf x) = J_G(bf x) G(bf x) tag4
        $$







        share|cite|improve this answer













        share|cite|improve this answer



        share|cite|improve this answer











        answered Jul 18 at 15:00









        caverac

        11k2927




        11k2927






















             

            draft saved


            draft discarded


























             


            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2855207%2fgradient-derived-from-jacobian%23new-answer', 'question_page');

            );

            Post as a guest
































































            Comments

            Post a Comment

            Popular posts from this blog

            What is the equation of a 3D cone with generalised tilt?

            Image
            Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite What is the equation of a 3D cone with generalized tilt? I've noticed that in most equations given to represent a cone, there is no parameter which defines the tilt of the cone in 3D space and that most of them have their point at the origin $(0,0,0)$ - I was wondering if anyone could give me a more generalized cone equation for a cone in any position in 3D space 3d share | cite | improve this question edited Jul 25 '16 at 0:05 ervx 9,425 3 13 36 asked Jul 24 '16 at 15:38 Charlene 11 2 2 Welcome to Math.SE! Could you please explain a few things to help people give an answer useful to you: 1. What do you mean by "generalized tilt"? 2. Are you asking about a right circular cone? Does the angle at the vertex matter? 3. What form of answer (implicit, parametric...?) are you looking for? 4. If this is homework, what tools do you have available, an...
            Read more

            Color the edges and diagonals of a regular polygon

            Image
            Clash Royale CLAN TAG #URR8PPP up vote 5 down vote favorite 1 Here is the problem: For what $n$ is it possible to color the edges and diagonals of an $n$-side regular polygon with $dfracbinomn23$ colors, such that you use every color exactly three times and for every color the three segments (edges or diagonals) with that color form a triangle? Trivially $nequiv 0 text or 1 pmod3$. I can also prove that if the statement is true for $k$ than it is true for $3k$ as well. How to finish? Please help! Thanks combinatorial-geometry share | cite | improve this question edited Aug 6 at 21:57 alcana 118 4 asked Aug 6 at 21:15 Leo Gardner 356 11 Even assuming "polyhpn" in the title is a typo for polygon , it is unclear what you mean by "the edges and sides". Aren't the side of a regular polygon the same as the edges? A regular $n$-sided polygon has $n$ edges. – hardmath Aug 6 at 21:20 3 $n$ ne...
            Read more

            Relationship between determinant of matrix and determinant of adjoint?

            Image
            Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite We are studying adjoints in class, and I was curious if there is a relationship between the determinant of matrix A, and the determinant of the adjoint of matrix A? I assume there would be a relationship because finding the adjoint requires creating a cofactor matrix and then transposing it. linear-algebra determinant adjoint-operators share | cite | improve this question asked Nov 17 '17 at 17:32 CluelessCoder 32 5 For a square matrix $A$ of order $n$, we have $A(operatornameadj A)=(operatornameadj A)A=|A|I_n$ where $I_n$ is the identity matrix of order $n$. This should tell you about the determinant of the adjoint in terms of that of $A$ (use multiplicative property and other elementary properties of determinants). – Prasun Biswas Nov 17 '17 at 17:35 1 @CluelessCoder: see here: math.stackexchange.com/questions/516127/…...
            Read more