Quasi Newton Methods Convergence

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
0
down vote

favorite












I am using quasi-newton(BFGS) method with wolfe line search conditions to find the optimum for a convex function. At times, when I run the code, it tends to get stuck at a point which is not optimum. I mean, I reach certain point using the quasi newton based descent direction where it's not possible to find the next point which satisfies wolfe conditions even with very small value of alpha to the range of ^-100 .



I am guessing this error may arise due to bad stopping criteria. I am using central differences to approximate the gradient, does the stopping criteria depends upon the step size of central differences?



What should be the stopping criteria for quasi-newton method?
Can anyone provide some suggestion about the convergence of quasi newton method?




numerical-methods convex-optimization




share|cite|improve this question



















  • Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
    – copper.hat
    2 hours ago










  • I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
    – Rimple Malik
    2 hours ago










  • That doesn't really answer my question...
    – copper.hat
    2 hours ago










  • What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
    – Rimple Malik
    2 hours ago














up vote
0
down vote

favorite












I am using quasi-newton(BFGS) method with wolfe line search conditions to find the optimum for a convex function. At times, when I run the code, it tends to get stuck at a point which is not optimum. I mean, I reach certain point using the quasi newton based descent direction where it's not possible to find the next point which satisfies wolfe conditions even with very small value of alpha to the range of ^-100 .



I am guessing this error may arise due to bad stopping criteria. I am using central differences to approximate the gradient, does the stopping criteria depends upon the step size of central differences?



What should be the stopping criteria for quasi-newton method?
Can anyone provide some suggestion about the convergence of quasi newton method?




numerical-methods convex-optimization




share|cite|improve this question



















  • Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
    – copper.hat
    2 hours ago










  • I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
    – Rimple Malik
    2 hours ago










  • That doesn't really answer my question...
    – copper.hat
    2 hours ago










  • What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
    – Rimple Malik
    2 hours ago












up vote
0
down vote

favorite









up vote
0
down vote

favorite











I am using quasi-newton(BFGS) method with wolfe line search conditions to find the optimum for a convex function. At times, when I run the code, it tends to get stuck at a point which is not optimum. I mean, I reach certain point using the quasi newton based descent direction where it's not possible to find the next point which satisfies wolfe conditions even with very small value of alpha to the range of ^-100 .



I am guessing this error may arise due to bad stopping criteria. I am using central differences to approximate the gradient, does the stopping criteria depends upon the step size of central differences?



What should be the stopping criteria for quasi-newton method?
Can anyone provide some suggestion about the convergence of quasi newton method?




numerical-methods convex-optimization




share|cite|improve this question











I am using quasi-newton(BFGS) method with wolfe line search conditions to find the optimum for a convex function. At times, when I run the code, it tends to get stuck at a point which is not optimum. I mean, I reach certain point using the quasi newton based descent direction where it's not possible to find the next point which satisfies wolfe conditions even with very small value of alpha to the range of ^-100 .



I am guessing this error may arise due to bad stopping criteria. I am using central differences to approximate the gradient, does the stopping criteria depends upon the step size of central differences?



What should be the stopping criteria for quasi-newton method?
Can anyone provide some suggestion about the convergence of quasi newton method?





numerical-methods convex-optimization





share|cite|improve this question










share|cite|improve this question




share|cite|improve this question









asked 5 hours ago









Rimple Malik

33




33











  • Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
    – copper.hat
    2 hours ago










  • I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
    – Rimple Malik
    2 hours ago










  • That doesn't really answer my question...
    – copper.hat
    2 hours ago










  • What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
    – Rimple Malik
    2 hours ago
















  • Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
    – copper.hat
    2 hours ago










  • I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
    – Rimple Malik
    2 hours ago










  • That doesn't really answer my question...
    – copper.hat
    2 hours ago










  • What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
    – Rimple Malik
    2 hours ago















Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
– copper.hat
2 hours ago




Are you resetting your BFGS periodically? If you restart from the non optimal point does the algorithm continue? Is this a numerical issue?
– copper.hat
2 hours ago












I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
– Rimple Malik
2 hours ago




I am starting with an initial guess and let the program to converge until gradient become sufficiently small.
– Rimple Malik
2 hours ago












That doesn't really answer my question...
– copper.hat
2 hours ago




That doesn't really answer my question...
– copper.hat
2 hours ago












What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
– Rimple Malik
2 hours ago




What do mean by resetting the BFGS periodically? I have not checked by restarting from the same point yet. I guess this is an numerical issue but I am unable to figure out what might be causing the problem. Can you please suggest what part might be causing numerical errors?
– Rimple Malik
2 hours ago















active

oldest

votes











Your Answer




StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);








 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2873292%2fquasi-newton-methods-convergence%23new-answer', 'question_page');

);

Post as a guest






















active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes










 

draft saved


draft discarded


























 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2873292%2fquasi-newton-methods-convergence%23new-answer', 'question_page');

);

Post as a guest
































































Comments

Post a Comment

Popular posts from this blog

What is the equation of a 3D cone with generalised tilt?

Image
Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite What is the equation of a 3D cone with generalized tilt? I've noticed that in most equations given to represent a cone, there is no parameter which defines the tilt of the cone in 3D space and that most of them have their point at the origin $(0,0,0)$ - I was wondering if anyone could give me a more generalized cone equation for a cone in any position in 3D space 3d share | cite | improve this question edited Jul 25 '16 at 0:05 ervx 9,425 3 13 36 asked Jul 24 '16 at 15:38 Charlene 11 2 2 Welcome to Math.SE! Could you please explain a few things to help people give an answer useful to you: 1. What do you mean by "generalized tilt"? 2. Are you asking about a right circular cone? Does the angle at the vertex matter? 3. What form of answer (implicit, parametric...?) are you looking for? 4. If this is homework, what tools do you have available, an...
Read more

Color the edges and diagonals of a regular polygon

Image
Clash Royale CLAN TAG #URR8PPP up vote 5 down vote favorite 1 Here is the problem: For what $n$ is it possible to color the edges and diagonals of an $n$-side regular polygon with $dfracbinomn23$ colors, such that you use every color exactly three times and for every color the three segments (edges or diagonals) with that color form a triangle? Trivially $nequiv 0 text or 1 pmod3$. I can also prove that if the statement is true for $k$ than it is true for $3k$ as well. How to finish? Please help! Thanks combinatorial-geometry share | cite | improve this question edited Aug 6 at 21:57 alcana 118 4 asked Aug 6 at 21:15 Leo Gardner 356 11 Even assuming "polyhpn" in the title is a typo for polygon , it is unclear what you mean by "the edges and sides". Aren't the side of a regular polygon the same as the edges? A regular $n$-sided polygon has $n$ edges. – hardmath Aug 6 at 21:20 3 $n$ ne...
Read more

Relationship between determinant of matrix and determinant of adjoint?

Image
Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite We are studying adjoints in class, and I was curious if there is a relationship between the determinant of matrix A, and the determinant of the adjoint of matrix A? I assume there would be a relationship because finding the adjoint requires creating a cofactor matrix and then transposing it. linear-algebra determinant adjoint-operators share | cite | improve this question asked Nov 17 '17 at 17:32 CluelessCoder 32 5 For a square matrix $A$ of order $n$, we have $A(operatornameadj A)=(operatornameadj A)A=|A|I_n$ where $I_n$ is the identity matrix of order $n$. This should tell you about the determinant of the adjoint in terms of that of $A$ (use multiplicative property and other elementary properties of determinants). – Prasun Biswas Nov 17 '17 at 17:35 1 @CluelessCoder: see here: math.stackexchange.com/questions/516127/…...
Read more