Error Reasoning in Complex Systems : Training and Application Error for Decision Models
respectively , between 0 % and 100 % in increments of 1 %. The values of the other parameters were maintained as outlined in Table 1 . We are now able to calculate the first set of distributions , P
( l | e acc
) using the ABM . Additionally , we will assume that the population has a true GPA potential defined by a normal distribution
with mean 2.5 and standard deviation 1.5 .
Figure 4 . Probability of Outcome , P ( l | e acc ).
The result of the ABM simulations with parameters w 1
= . 25 , w 2
= . 5 is shown in Figure 4 . Because this is a stylistic model , the choice of weights is arbitrary and chosen only to be illustrative of the baseline choice argument . The blue line in the figure shows the expected value of GPA given group effects of the students at varying degrees of model accuracy . We see that each value of accuracy can result in a distribution of expected aggregated school GPAs , l . Error bars indicate the 95 % confidence intervals of these distributions .
5 . Discussion & Conclusions
We are now able to address the
research questions directly .
In the first question , we see that application error and model training error are not necessarily coincident . Figure 4 shows the expected model GPA by the orange horizontal line . This is the error when only individual GPA discrepancies are considered , and class-level and school-level effects are left out . In contrast , the application error measuring the same outcome , ag-
177