且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

很大的非线性最小二乘优化的收敛性

更新时间:2022-06-22 06:58:43

看来您的方法非常周到.约有500k变量和100k约束,您拥有很大的***度.内点法(其中IPOPT相当不错)的主要替代方法是主动集法.主动集方法通常在很少的***度下会更好,因此IPOPT是您***的选择.

It looks like you were very thorough in your approach. With ~500k variables and 100k constraints you have a lot of degrees of freedom. The primary alternative to interior point approaches (of which, IPOPT is quite good) is an active-set approach. Active-set methods tend to be better with few degrees of freedom, so IPOPT is your best bet.

IPOPT输出表明以下几点:

The IPOPT output indicates a few things:

  1. Hessian不是正定的(几乎每个迭代都需要正则化)

  1. The Hessian is not positive-definite (nearly every iteration requires regularization)

正则化后,问题在可变范围内足够凸(无回溯线搜索)

After regularization, the problem is sufficiently convex within variable bounds (no back-tracking line search)

IPOPT时间为>>函数调用时间(3403.326:214.977非常大).大部分时间是在矩阵分解中.

IPOPT time is >> function call time (3403.326 : 214.977 is very large). Most of this time is in matrix factorizations.

由于变量范围的限制,几乎每次迭代都被截断.

Nearly every iteration is truncated because of variable bounds.

据我了解,当您的问题太大时,它将超出硬件限制(CPU缓存),并且线性代数时间会变长.这可能是您的主要问题.

It is my understanding that when your problem gets too big it exceeds hardware limitations (CPU cache) and the linear algebra time blows up. This is probably your main problem.

由于这些原因,我建议您尝试 BFGS 方法( IPOPT选项).通过直接计算Hessian的近似逆,可以避免困难的矩阵分解/求解.此外,BFGS方法可以保证正定的Hessian.通常,当无法使用Hessian时使用BFGS近似值,因为确切的Hessian应该提供更准确的步长.但是,使用您的正则化,昂贵的因式分解和截断的步骤,BFGS可能几乎一样好.期望有更多的迭代(226很小),但是每个迭代都应该快得多.

For these reasons, I would recommend trying a BFGS approach (IPOPT option). By directly calculating the approximate inverse of the Hessian, you avoid difficult matrix factorizations/solves. Further, the BFGS approach can guarantee positive-definite Hessians. Usually, the BFGS approximation is used when the Hessian is not available since the exact Hessian should provide more accurate steps. But with your regularization, expensive factorization and truncated steps, BFGS will likely be almost as good. Expect more iterations (226 is very small), but each should be much faster.

您可能还想放宽变量范围.内点方法旨在避免越界.有这么多界限,可能会减慢进度.

You may also want to play with loosening your variable bounds. Interior-point methods are designed to avoid going to bounds. With that many bounds, it could be slowing down progress.