Sensitivity Analysis

Sensitivity analysis shows the effect of each input distribution on an output. The primary goal is to identify the most “critical” inputs, the inputs to concentrate on most when making decisions.

@RISK displays sensitivity results in either graphical or tabular format. All the different sensitivity methodologies create Tornado Graphs. In addition, the Change in Output Statistic method can also create a Spider Graph. To see sensitivity results displayed in a tabular format, use the Sensitivities Window on the Explore Menu. Sometimes it is desirable to have sensitivity information written directly to a spreadsheet cell. The functions RiskSensitivity and RiskSensitivityStatChange accomplish this task.

Sensitivity Analysis Methods

There are four different kinds of sensitivity analysis, each of which perform this task in a different way and with a different emphasis:

  • Change in Output Statistic – This methods groups input values into a set of bins across its range of values. Then a particular output statistic is calculated for each bin. The magnitude of the change across all the bins is tabulated. This is sometimes also called the conditional sensitivity method.
  • Regression – This method creates the regression equation that best produces the output based on a weighted sum of the inputs, standardizes the results, and then ranks them by their magnitude.
  • Correlation – This method determines the correlation coefficients between every input and the output and ranks them by their magnitude.
  • Contribution to Variance – This method uses a sequential sum of squares technique to determine how much effect the input has on the variance of the output.

Change in Output Statistic

In the Change in Output Statistic methodology, input samples are grouped in a set of equal-probable bins (or “divisions”) ranging from the input’s lowest value to its highest. A value for a statistic of the output (such as its mean) is calculated for the output values in the iterations associated with each bin. Inputs are then ranked by the amount of positive or negative swing those different bins cause to the output statistic.

The process that determines the impact of each input is:

• @RISK sorts all iterations in ascending order based on the input’s value.

• It divides the sorted iterations into bins. For example, with 2500 iterations and 10 bins, the first bin would contain the 250 iterations with the 250 lowest values of this input; the second bin would contain the 250 iterations with the next lowest values of this input; and so on.

• It computes the selected statistic (usually the mean, but not necessarily) of the output values in each bin.

• The impact of the input is determined by looking at the swing in the output statistic across all the bins.

The statistic and number of bins to use in the analysis are both controllable using the Sensitivity Analysis Settings dialog.

Regression

The regression methodology determines the impact of inputs on an output by performing a multivariate, step-wise linear regression. The coefficients of the inputs in the resulting regression equation are standardized and ranked.

When viewing the regression results in the Sensitivities window, the overall quality of the regression analysis is measured using the R-squared value of the model. The lower this value, the less stable the reported sensitivity statistics are. If the fit is too low — beneath 0.5 — a similar simulation with the same model could give a different ordering of input sensitivities.

Regression results can be reported in two formats: as standardized equation coefficients or as “mapped values.” Because they are standardized, regression coefficients vary between -1 and 1. The mapped values are converted to the equivalent change in the output due to a one standard deviation change in the input.

Correlation

The correlation methodology determines the impact of inputs on an output by calculating the Spearman rank correlation coefficient between each input and the output. These coefficients are then ranked by their magnitudes.

Correlation coefficients vary between -1 and 1.

Contribution to Variance

The Contribution to Variance methodology is concerned with determining the effect of an input on the variability of the output. The sensitivity values it computes is the fraction of variance of the output attributable to each input.

This is accomplished using a technique called the sequential sum of squares, a sophisticated iterative form of regression analysis. The variance in an output is explained by adding a sequence of inputs one by one to a regression model. The selection of the variables and the order in which they are added is determined by a preliminary stepwise regression procedure. As with any regression technique, when input variables are correlated, there is ambiguity in how much variability is assigned each input. Thus, caution in interpreting the contribution to variance results is critical when inputs are correlated with one another.