Decision Theory¶
Introduction¶
Decision theory spans a combination of problem-solving techniques to find the best decision in complex decision problems. In this unit, we will use the following terminology:
Alternatives: Decision variables which are controllable and depend on the decision maker’s decision.
Uncertainty and states of nature: External variables, are uncontrollable and need to be estimated or assumed.
Performances: Profit or Cost (utility) of the result of a decision
Hence, the objective is to find the alternative with the highest performance for the decision maker, possibly in situations where the performance may depend on external variables which cannot be controlled.
So far, this set up is not that different from the definitions seen in previous units, however, we will be focusing on the following aspects in this unit:
Impacts over time All the important consequences of a problem do not occur at the same instant in time
Uncertainty At the time the decision-maker must select an alternative, the consequences are not known with certainty.
Possibility of acquiring information Often we can acquire additional information to support decision-making at a cost. For instance, to collect seismic information to decide whether to drill for oil. Decision theory provides methods to evaluate if it is worth acquiring additional information or not.
Dynamic aspects The problem might not end immediately after an alternative is chosen but might require further analysis (e.g. further decisions)
Pay off Matrix¶
The pay-off matrix is a tool to represent the performances of the different alternatives against the possible future outcomes of an uncontrolled event, or states of nature. The different rows of the matrix represent the alternatives of the decision maker, and the different columns of the matrix, the different possible states of nature. Finally, every cell in the matrix contains the performance of each alternative given the occurrence of each state of nature. That is, let us note the decision maker’s alternatives as \(a_1, a_2, ..., a_m\), and the different states of nature as \(s_1, s_2, ..., s_n\). Let us note the performance of alternative \(a_i\) when \(s_j\) has occurred as \(u_{ij}\), then the pay-off matrix is:
\(\begin{bmatrix} u_{11} & u_{12} & ... & u_{1n}\\ u_{21} & u_{22} & ... & u_{2n}\\ u_{m1} & u_{m2} & ... & u_{mn} \end{bmatrix}\)
Note that the pay-off matrix represents all possible outcomes of our decision under uncertainty, given that the states of nature are exhaustive (together they describe all the possible outcomes of the uncontrolled variable) and mutually exclusive (if one occurs, then the rest cannot occur). The pay-off matrix can also be represented in tabular form as:
Decision Alternatives |
state 1 |
state 2 |
… |
state n |
|---|---|---|---|---|
\(a_1\) |
\(u_{11}\) |
\(u_{12}\) |
… |
\(u_{1n}\) |
\(a_2\) |
\(u_{21}\) |
\(u_{22}\) |
… |
\(u_{2n}\) |
… |
… |
… |
… |
… |
\(a_m\) |
\(u_{m1}\) |
\(u_{m2}\) |
… |
\(u_{mn}\) |
Let us, at this point, set one example to be used in the next sections of this presentation. In the example, the decision maker needs to select a financial product among a set of market alternatives: Gold, bonds, stock options, deposit, or hedge fund. Each product provides different benefits or losses depending on the behaviour of the market. The behaviour of the market is not controlled by the decision maker, therefore, it is characterised in this example as a set of states of natures:
Accumulation Phase: The market is stable and characterised by a slow raise
Mark-up Phase: The market has been stable for a while, investors feel secure and the market is characterised by a fast raise.
Distribution Phase: Most investors start selling to collect benefits, the market stabilises and there is little or no change.
Mark-down Phase: Some investors try to hold positions as there is a large fall of the market
The pay-off matrix shows the expected performance in euros of the investment of each product under each market cycle phase:
Decision Alternatives |
Accumulation |
Mark-up |
Distribution |
Mark-down |
|---|---|---|---|---|
Gold |
-100 |
100 |
200 |
0 |
Bonds |
250 |
200 |
150 |
-150 |
Stock options |
500 |
250 |
100 |
-600 |
Deposit |
60 |
60 |
60 |
60 |
Hedge fund |
200 |
150 |
150 |
-150 |
Dominance¶
Dominance is a property of alternatives that provide a better performance than others for every state of nature. In the example above, it can be seen that the bond provides a higher benefit than the option hedge in any market scenario, therefore the bond option is dominant over the hedge fund option. Dominance allows us to simplify the decision-making process, eliminating the dominated alternatives in favour of the dominant alternatives. Taking into account dominance, the pay-off matrix above can be written as:
Decision Alternatives |
Accumulation |
Mark-up |
Distribution |
Mark-down |
|---|---|---|---|---|
Gold |
-100 |
100 |
200 |
0 |
Bonds |
250 |
200 |
150 |
-150 |
Stock options |
500 |
250 |
100 |
-600 |
Deposit |
60 |
60 |
60 |
60 |
Any rational decision maker would select bonds before stock options under any market phase and therefore, the latter can be ignored.
Decision Rules¶
Decision rules are criteria used by a rational decision maker to make systematic decisions, id est, to select the best alternative. This section describes some of the most important criteria in related literature:
MinMax¶
The MinMax criteria aims to minimise loss in the worst-case scenario. It is therefore a conservative, or pessimistic criteria. Also known as Maximin (minimise the maximum loss), this criteria first finds the minimum value of the performance of each decision variables in every scenario:
\(m_i = \min(u_{i1}, u_{i2}, ..., u_{in}) \quad \forall i=[1, 2, ..., m]\)
These values represent the worst case scenario in every decision alternative, then the minmax criteria selects the maximum of these values:
\(d = \text{argmax}(m_1, m_2, ..., m_m)\)
The function argmax above returns the index of the maximum value rather than the maximum value. Hence, it will return the index of the alternative for which the maximum is value:
In the example above, the minmax criteria yields:
Decision Alternatives |
Accumulation |
Mark-up |
Distribution |
Mark-down |
\(m_i\) |
|---|---|---|---|---|---|
Gold |
-100 |
100 |
200 |
0 |
-100 |
Bonds |
250 |
200 |
150 |
-150 |
-150 |
Stock options |
500 |
250 |
100 |
-600 |
-600 |
Deposit |
60 |
60 |
60 |
60 |
60 |
\(d = \text{argmax}(-100, -150, -600, 60) = 4\)
\(v = max(-100, -150, -600, 60) = 60\)
The MinMax criteria models the decisions of a conservative rational decision maker and in this example, a conservative investor will invest in deposits.
MaxMax¶
The MaxMax criteria on the other hand aims to maximise profit in the best-case scenario. It is therefore an aggressive, and optimistic criteria. This criteria first finds the maximum value of the performance of each decision variables in every scenario:
\(o_i = \max(u_{i1}, u_{i2}, ..., u_{in}) \quad \forall i=[1, 2, ..., m]\)
These values represent the best case scenario in every decision alternative, then the MaxMax criteria selects the maximum of these values:
\(d = \text{\argmax}(m_1, m_2, ..., m_n)\)
In the example above, the MaxMax criteria yields:
Decision Alternatives |
Accumulation |
Mark-up |
Distribution |
Mark-down |
\(o_i\) |
|---|---|---|---|---|---|
Gold |
-100 |
100 |
200 |
0 |
200 |
Bonds |
250 |
200 |
150 |
-150 |
250 |
Stock options |
500 |
250 |
100 |
-600 |
500 |
Deposit |
60 |
60 |
60 |
60 |
60 |
\(d = \text{\argmax}(200, 250, 500, 60) = 3\)
\(v = \max(200, 250, 500, 60) = 500\)
The MaxMax criteria models the decisions of a conservative rational decision maker and in this example, a conservative investor will invest in deposits.
Bayesian¶
The Bayesian criteria takes into account additional information about the likelihood of each state of nature. It is therefore the criteria applied by an informed decision maker which wants to factor in the decision the likelihood of each alternative. The probabilities of each state or nature as defined as \(p(s_j)\). The Expected Monetary Value (EMV) of each alternative is defined as:
\(\text{EMV}_i = \sum{ij}*p(s_j)\)
The EMV can be interpreted as the weighted average value of the alternative given the probabilities of occurrence of each state of nature.
The Bayesian criteria selects the alternative with the maximum EMV:
\(d = \text{argmax}(text{EMV}_1, text{EMV}_2, ..., text{EMV}_m)\)
Let us use again the example above to apply this decision criteria.
States of nature |
Accumulation |
Mark-up |
Distribution |
Mark-down |
EMV |
|---|---|---|---|---|---|
Probabilities (\(p(s_j)\)) |
0.2 |
0.3 |
0.3 |
0.2 |
EMV |
Gold |
-100 |
100 |
200 |
0 |
70 |
Bonds |
250 |
200 |
150 |
-150 |
125 |
Stock options |
500 |
250 |
100 |
-600 |
85 |
Deposit |
60 |
60 |
60 |
60 |
60 |
\(d = \text{argmax}(70, 125, 85, 60) = 2\)
\(v = \max(70, 125, 85, 60) = 125\)
The Bayesian criteria selects the decision alternative with index 2 (Bonds) and its EMV is 125.