Estimating changes in gaps in application outcomes after taking other factors into account
95 . Analysis in this section shows that , even after taking into account differences in observed characteristics between learners from Uni Connect areas and those from other areas , the gap in application rates between these two groups has widened ( statistically significantly ) since the launch of the Uni Connect programme . 33
96 . The gap in placed rates and the gap in high tariff application rates are both estimated to have increased slightly since 2016 , although neither of these increases is found to be statistically significant .
97 . However , for the 2021 cohort in particular , it is impossible to definitively separate the impact of the pandemic from that of the Uni Connect programme . Ultimately , the extent to which the findings of this analysis are distorted by the impact of the pandemic will depend on the extent to which any pandemic-related behaviour is different for those living in Uni Connect areas and those not ( after these learners are matched on a set of characteristics as described below ).
Matched counterfactual analysis
98 . It is possible that the trends in application outcomes presented so far are the result of factors other than the Uni Connect programme itself . It might be that the composition and characteristics of the two groups of learners are changing over time , which is influencing the gaps in application outcomes .
99 . This section presents the findings of a statistical approach called ‘ exact matching ’, described in Annex D , which enables us to estimate the change in the gap in application rates between 2016 ( before Uni Connect started ) and 2021 ( four years after its launch ), after differences in characteristics between the two groups of learners are taken into account . This approach works by comparing learners from Uni Connect areas against a group of learners with the same mix of characteristics , thereby reducing underlying differences in characteristics between the two groups which might influence application outcomes over time .
Annex D provides a technical description of this statistical approach , as well as details of further population restrictions implemented at this stage of the analysis to minimise spillover effects .
100 . It should be noted that this approach can only account for underlying differences in characteristics between groups of learners to the extent that this information is available in the NPD data . There will of course be factors , such as family support or individual motivation , which we cannot control for because they are not captured in the data . Nonetheless , because we know that some underlying factors differ between learners from Uni Connect target areas and those from other areas – and that these factors are associated with application outcomes – it remains informative to account for these differences as far as possible .
33
Statistical significance is reported at the 95 per cent confidence level throughout this report .
38