Re: optimisation

[Pedro Gonnet <pedro at vis dot ethz dot ch>]

Hi Nicholas,
Hi All,

If you have a problem for which you can only calculate numerical
gradients, nothing should prevent you from supplying a function that
calculates derivatives numerically... In my (albeit somewhat limited)
experience, numerical gradients are somewhat messy to generalize, since
a lot needs to be known of the function itself. I.e. for:

	f'(x) = (f(x + delta) - f(x - delta)) / (2 * delta)

how would you chose 'delta'? What values are too large to be precise and
what values are too small to avoid cancellation? There are more refined
methods, of course, for instance taking a taylor expansion of 'f(x)' and
feeding it an 'x' with an imaginary part (I can't remember what that was
called...), but this requires some re-coding of the function and would
be, hence, an ideal candidate for a user-supplied gradient-function...

I think the question is not so much one of derivates being computable,
but of people unwilling to calculate the derivatives by hand... This is,
of course, somewhat of a rant :)  With a bit of number-juggling and
MAPLE ('diff' and the package 'codegen' are _really_ useful here), I can
get analytical gradients for just about anything.

Then again, given that we all have gigaherz-machines, we could all use
random-walk type solvers for any kind of problem ;)

Cheers
Pedro



On Mon, 2003-01-06 at 16:52, Nicholas Cunliffe wrote:
> Hi,
> 
> Just thought Id cast an opinion. Having multi variable optimization 
> routines which REQUIRE user supplied derivetives is just LAME. 
> How many real problems are there which are so simple that the 
> derivatives can be supplied? None of any scientific or engineering 
> interest. 
> 
> Thanks
> 
> Nicholas