Application Center - Maplesoft

App Preview:

Optimal Stopping - Take Two

You can switch back to the summary page by clicking here.

Learn about Maple
Download Application




Optimal Stopping - Take Two

The following was implemented in Maple by Marcus Davidsson (2010)

davidsson_marcus@hotmail.com and is based upon

Hill, T (2001) Knowing When to Stop , Allen,S (2006) The Secretary Problem

 and Niederhoffer, V (2006)  Probability, Markets and Decision Making

 

The first game that we are going to discuss is as follows:

 

You are allowed to roll a dice a maximum of 6 times.

After each roll you will get the number of points on the face of the dice.
The objective is to determined when you should stop to ensure that

you maximize your expected value.

We will solve the above problem by using backward induction.The basic
rules that govern the dynamic are as follows:

 

 

i) If the outcome of a coin toss is larger or equal to the expected value

in the next period then you should stop otherwise you should continue

 

ii) The expected value in period n is given by the sum of the probability of a
certain payout multiplied by the specific payout for all payouts.  

 

iii) The expected value in period i is given by the probability of not stopping in period i

multiplied by the expected value in period i+1 plus the probability of stopping in period i

multiplied by the expected value of stopping in period i

 

 

 

 

 

 

 

The expected value for the last dice roll in period 6 is given by:

 

(1)

 

This means that if you get 1,2 or 3 in the dice roll in period 5 then you should continue
since your expected value in the next period is larger than the outcome in this period
If you get 4,5 or 6 then you should stop. Hence, P[not stop]=(3/6) and P[stop]=(3/6).  

 

 

The expected value in period 5 is given by:

 

 

(2)

This means that if you get 1,2,3 or 4 in the dice roll in period 4 then you should continue
since your expected value in the next period is larger than the outcome in this period
If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).  

 

 

The expected value in period 4 is given by:

 

 

(3)

 

This means that if you get 1,2,3 or 4 in the dice roll in period 3 then you should continue
since your expected value in the next period is larger than the outcome in this period
If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).  

The expected value in period 3 is given by:

 

 

(4)

 

This means that if you get 1,2,3 or 4 in the dice roll in period 2 then you should continue
since your expected value in the next period is larger than the outcome in this period
If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).  

The expected value in period 2 is given by:

 

 

(5)

 

This means that if you get 1,2,3,4 or 5 in the first dice roll then you should continue
since your expected value in the next period is larger than the outcome in this period
If you get 6 then you should stop. Hence, P[not stop]=(5/6) and P[stop]=(1/6).  

 

 

The expected value in period 1 is given by:

 

 

(6)


We can now plot the expected value and the lower bound for the stopping rule
for each time period as follows: