Optimization methods
One can choose among several optimisation methods which are coded
in the optimize command:
- steepest descent
- Newton's method
- Davidon-Fletcher-Powell's method
- random search
- grid search method
- search along coordinate axes
- Powell's method
- Hooke-Jeeves's method
- constrained simplex method
- simple genetic algorithm
- evaluating cost function across whole parameter space
Further description of a particular method can be found in the
literature. Here are brief descriptions of optimisation methods
and their parameters which can be manually adjusted:
- steepest descent (steepest_descent);
If the expression for the i-th component of the
gradient of the cost function is not given, then it is calculated
numerically. All parameters, that are to be optimised, have to be
given before this method is chosen. The parameters of the method
are:
r |
constant used in determining constraints where
the minimum lies in the negative gradient direction.
It must be between 1.5 and 2. |
method |
method used for searching in the negative gradient direction. Possible values are quadratic, golden and fibonacci. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the negative gradient direction. |
number_of_iterations |
maximum number of iterations when searching in the negative gradient direction. |
transformation |
type of transformation from explicitly constrained to unconstrained space. Possible values are no, sin and arcctg. |
gradient0 |
expression of the first component of the gradient |
gradient1 |
expression of the second component of the gradient |
gradient2 |
expression of the third component of the gradient etc. |
- Newton's method (newton); Gradient and Hesse matrix of the cost function are calculated numerically. The parameters of the method are:
r |
constant used in determining constraints where the minimum lies in the desired direction. It must be between 1.5 and 2. |
method |
method used for searching in the desired direction. Possible values are quadratic, golden and fibonacci. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the desired direction. |
number_of_iterations |
maximum number of iterations when searching in the desired direction. |
transformation |
type of transformation from explicitly constrained to unconstrained space. Possible values are no, sin and arcctg. |
- Davidon-Fletcher-Powell's method (davidon_fletcher_powell); If the expression for the i-th component of the gradient of the cost function is not given, then it is calculated numerically. All parameters, that are to be optimised, have to be given before this method is chosen. The parameters of the method are:
r |
constant used in determining constraints where the minimum lies in the desired direction. It must be between 1.5 and 2. |
method |
method used for searching in the desired direction. Possible values are quadratic, golden and fibonacci. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the desired direction. |
number_of_iterations |
maximum number of iterations when searching in the desired direction. |
modification |
modification of the Davidon-Fletcher-Powell's method. Possible values are no, modified, first and second. |
transformation |
type of transformation from explicitly constrained to unconstrained space. Possible values are no, sin and arcctg. |
gradient0 |
expression of the first component of the gradient |
gradient1 |
expression of the second component of the gradient |
gradient2 |
expression of the third component of the gradient etc. |
- random search (monte_carlo); Parameter distribution is regarded only if direction = no. On the other hand parameters r, epsilon and number_of_iterations are taken into account when direction != no. The parameters of the method are:
direction |
the direction of search is determined randomly. Possible values are no, quadratic, golden and fibonacci. |
distribution |
the distribution of the random parameter values. Possible values are linear and normal. |
r |
constant used in determining constraints where the minimum lies in the randomly determined direction. It must be between 1.5 and 2. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the randomly determined direction. |
number_of_iterations |
maximum number of iterations when searching in the randomly determined direction. |
- grid search method (grid_search); The parameters of the method are:
level |
grid level used. Possible values are 2 and 3. |
alpha |
the magnifying factor. |
epsilon |
satisfying relative grid size (in percentage). |
- search along coordinate axes (axis_search); The parameters of the method are:
r |
constant used in determining constraints where the minimum lies in the desired direction. It must be between 1.5 and 2. |
method |
method used for searching in the desired direction. Possible values are quadratic, golden and fibonacci. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the desired direction. |
number_of_iterations |
maximum number of iterations when searching in the desired direction. |
- Powell's method (powell); The parameters of the method are:
r |
constant used in determining constraints where the minimum lies in the desired direction. It must be between 1.5 and 2. |
method |
method used for searching in the desired direction. Possible values are quadratic, golden and fibonacci. |
epsilon |
satisfying relative distance (in percentage) between two points when searching in the desired direction. |
number_of_iterations |
maximum number of iterations when searching in the desired direction. |
transformation |
type of transformation from explicitly constrained to unconstrained space. Possible values are no, sin and arcctg. |
- Hooke-Jeeves's method (hooke_jeeves); The parameters of the method are:
alpha |
the magnifying factor. |
epsilon |
satisfying relative step (in percentage). |
- constrained simplex method (complex); The parameters of the method are:
k |
number of points in constrained simplex. The default value is 2 times number of parameters. If parameters are not specified yet, than the number of parameters is zero and the default value for k is zero. That will lead into an error. In this case k should be given manually. |
alpha |
the mirroring factor. |
size |
satisfying average relative distance (in percentage) of the points to the centre of the simplex. |
oscillation_detection |
detect and prevent an oscillation behavior around the lowest point of the constrained simplex. |
nmirror |
number of points mirrored in each iteration. |
- simple genetic algorithm (genetic); The parameters of the method are:
popsize |
population size. Must be an even number. |
lchrom |
chromosome length (number of bytes per dimension). Maximum length is 4 bytes per dimension. |
maxgen |
number of generations. |
pcross |
crossover probability. |
pmutation |
mutation probability. |
scaling |
cost function scaling factor. Must be greater than 1. |
coding |
parameter values coding. Possible values are binary and gray. |
elitism |
the best individual in each generation is copied to the next one. Possible values are yes and no. The default is no. |
- evaluating cost function across whole parameter space (parameter_space); The parameters of the method are:
outfile |
output filename, where the results are stored. |
npts0 |
number of points in the grid along the first parameter. |
npts1 |
number of points in the grid along the second parameter. |
npts2 |
number of points in the grid along the third parameter etc. |
|