Ipsos | Uni Connect Phase 3 : Attainment-raising Activity – Final Report
37
4.4 What has worked less well ? Summary : Challenges for monitoring and evaluation can be categorised as :
1 ) School-based challenges . These included issues like teaching staff struggling to find the time to support , and some schools were reluctant to share learner data
2 ) Resource-based challenges . These included insufficient evaluation expertise within partnerships , and issues identifying appropriate validated surveys and questionnaires , especially for assessing CPD activities
School staff have limited time to support data-collection activities . As previously noted , some teachers are often time-poor with limited capacity to support logistical tasks such as helping delivery staff book rooms for programme activities . Despite data-monitoring requirements being clearly communicated to teachers , additional requests for their time to support data collection has been challenging .
“ The reality of teachers fitting these projects into their busy school week and staffing them and getting the surveys completed and getting the data , is a lot trickier in practice than the principle . A lot of it is dependent on teachers replying to emails , because they ’ re very busy people of course . That ’ s a bit of a challenge but making that really clear at the beginning when they sign up is obviously really important to try and overcome that .” ( Partnership lead )
Often schools that partnerships had not worked with before were unable to share data , and partnerships had to work on setting up data-sharing agreements . In some cases , existing data-sharing agreements had to be revised to reflect attainment-raising , which was a highly time-consuming process – for example , when the terms of the agreement had to be escalated to the Academy Trust level . Some partnerships also experienced circumstances where schools would not share data even with datasharing agreements . For example , some partnerships have requested predicted grades for the learners they work with but were unsure whether schools would grant them access .
“ Data-sharing and obtaining pre-16 data ( that demands parental permission ) is challenging and , so far , only 6 % of students have been tracked compared to usually 50 % in previous years . The relationship with the school is vital for anything in terms of the quantifiable data , so anything to do with assessments , which is going to be dependent on the relationship with the schools . That ’ s why it ’ s so important to build those up before you start doing it . If you don ’ t have that relationship , you might hit a brick wall in trying to access the data that ’ s required to demonstrate impact .” ( Partnership lead )
“ I think we ’ re still questioning about getting any exam data . So , we ’ re still having conversations about that even now .” ( Partnership lead )
Some partnerships struggled to identify appropriate validated surveys . Key issues relayed included :
▪ Partnerships were typically able to find validated scales , tools and surveys to use to measure outcomes for learners , though some noted issues in sourcing validated tools to measure metacognition
▪ Some partnerships felt that the validated tools available to them do not capture the marginal gains often exhibited by learners
20-048464 | FINAL | Public | This work was carried out in accordance with the requirements of the international quality standard for Market Research , ISO 20252 , and with the Ipsos MORI Terms and Conditions which can be found at http :// www . ipsos-mori . com / terms .