By Year and Subject
We looked at a value added( progress) analysis by calculating the residuals based on the actual grade compared to the assessment at the end of the intervention and the actual grade and then compared to the mock grade. We used a one-sample t-test comparing the residuals to zero with the hypothesis that if the average residual is significantly greater than 0, the intervention helped.
We ran the same test comparing assessment to actual grade and mock to actual grade per cohort( sliced by subject and year group).
The one-sample t-tests looked at whether the mean gain was greater than 0( one-tailed) and different from 0( two-tailed), overall and by Year and Subject. We also then looked at the Cohens’ D effect and the correlation between pairs.
All two tailed tests showed a moderate to strong correlation with strongest being between the post-assessment and final grade.
Year group Subject |
Comparison to final grade T-value |
T critical |
|
|
|
one- tail |
Result |
p-value |
Result |
|
Mean difference Cohen ' s D Effect |
Two tail / P- |
|
|
value |
Result |
Cohen ' s D Effect |
|
Correlatio n between pairs Result |
Year 10 |
Maths |
Assessment |
2.085 |
1.761 |
Significant |
0.027902 Significant |
0.53333 |
0.54 |
Medium |
0.056 |
Not statistically significant |
0.662 |
|
|
|
|
|
|
|
|
|
|
|
Highly |
|
|
|
|
|
|
|
|
|
|
|
|
Statisticall |
|
|
|
|
|
|
|
Statiscally |
|
|
Very large |
|
y |
|
Year 10 |
Maths |
Mock |
6.95 |
1.761 |
Significant |
3.33E-06 |
Significant |
1.85996 |
1.8 |
effect |
6.66E-06 |
significant |
1.85 |
Year 10
Year 10
Year 11
Year 11
English |
|
|
|
|
|
|
|
|
|
|
Language |
Assessment |
2.5106 |
1.7709 |
Significant |
0.013 |
Significant |
0.571428 |
0.67 |
Medium |
0.021 |
English |
|
|
|
|
|
Language |
Mock |
1.8829 |
1.7709 |
Significant |
0.0411 |
Statsically Significant |
0.42857 |
0.52 |
Medium |
0.098 |
|
|
|
Upper |
|
|
|
|
medium / |
|
English |
|
|
|
|
|
|
|
|
Language |
Assessment |
2.5483 |
1.7959 |
Significant |
0.0135 |
Significant |
0.5833 |
0.76 |
English |
|
|
|
|
|
|
|
|
Language |
Mock |
2.602 |
1.7959 |
Significant |
0.0123 |
Significant |
0.6667 |
0.75 |
Large 0.027 Upper medium /
Large 0.0246
Statisticall y signficant 0.89
Not statistcally significant 33 Statisticall y significant 0.91 Statisticall y significant 0.94
Moderate to large r = 0.227
Very large effect r = 0.64
Large effect size r = 0.748
Small effect r = 0.79
Large effect size r = 0.676
Large effect size r = 0.57
Modest Positive Correlation
Moderately strong Moderate to Strong Correlation
Strong relationship Strong Positive Correlation Moderate to Strong Correlation
Did the intervention make a difference from post assessment to final grade?
For Year 10 group, it appears from the results that they improved from the initial assessment to mocks / actual exam results. As stated earlier, there is a huge amount happening to support the students at the school throughout their Year 11 studies, however combined with the positive reporting from both the students and the teachers to the raising levels of confidence for the students resulting from the intervention, it could be inferred that it had a positive effect. Recommendations would be to see if this result was mirrored again with further groups if funding allowed.
For the Year 11 group, based on the final assessment to final grade, their grades, on average were lower. However, if we actually take the mock results to the final grade( where the intervention happened in between) they did improve between the mock and the exam.
• Mean uplift: + 0.67 grades( GCSE minus mock).( N = 12)
• Significance: t ≈ 2.60, p( two-tailed) ≈ 0.0246, p( one-tailed, > 0) ≈ 0.0123 → statistically significant improvement.
• Effect size: Cohen’ s d ≈ 0.75 → moderate-to-large practical impact.
Page 10 of 22