The following command allows you to use the short form of the command names in the Optimization package.  

The commands in the Optimization package allow you to specify the objective function and the constraints in several different ways.  This worksheet deals with the algebraic form of the objective function and constraints.  The other forms are Matrix form and procedure form where Matrix form is covered in a separate example worksheet.

In general, an optimization problem is of the form

min  where  is in

subject to for  in ,  for  in

The Optimization package handles several types of objective and constraint functions.  Each type of problem is a subtype of the next one.

Objective Function/Constraint/Command

Linear/Linear/LPSolve

Quadratic/Linear/QPSolve

Nonlinear(General)/Nonlinear(General)/NLPSolve

There is also a minimize command that will attempt to automatically select the best algorithm for a given problem.

In addition, there is a command, LSSolve, which minimizes functions of the form

For a first example, we have a simplex two dimensional linear programming problem.

min

       subject to

The plot below shows the feasible region in yellow, the contours of the objective function, and the optimal solution as a green circle.

The  command returns the optimal function values, as well as the point at which the optimal value occurs.

Alternatively, we could use the first two constraints and the nonnegative option.

The first element of the solution is the minimum value that the objective function obtains, while satisfying the constraints.  The second element indicates a point where the minimum is reached.  This point is not necessarily unique.

We can also include equality constraint as the next example shows.  This example also demonstrates the maximize option.

For a first example of a quadratic program, we will use:

min

subject to

The plot below shows the feasible region in yellow and the contours of the objective function.  The green circle indicates the optimal point.

The QPSolve command returns the optimal function values, as well as the point at which the optimal value occurs.

The NLPSolve command allows us to solve problems of a fairly general nature.  As long as the derivative of the objective function and constraint function is continuous, we can use NLPSolve.

Our first basic example of nonlinear programming is a simple univariate minimization of a polynomial.

This plot shows the function and the minimum found by NLPSolve.

We can minimize this function in the range [0..6].

We can also maximize the function.

We can minimize more complex functions such as:

Below we plot the function f and the minimum point.  We can see that NLPSolve finds a local minimum, not necessarily a global one.

We can also minimize multivariate functions such as

with

We can use the least squares command to do curve fitting.  For example, if we have the following data points, ():

and we want to fit the data with a quintic polynomial.  Let our quintic be,

.

Now we need to find a, b, and c such that

is a minimum.

LSSolve takes as input a list of the residues .

Now we can call LSSolve with the residues.

Our polynomial becomes:

We can now plot the data and the curve