Ipsos | Uni Connect Phase 3 : Attainment-raising Activity – Final Report
38
▪ Partnerships offering CPD activities need more support developing evaluation questions , outcomes and validated tools for teachers
“ We ’ ve got some lovely , validated scales and things that we ’ re going to be using to measure confidence levels , But I said , ‘ Oh , I do want to do some questions for the teachers too .’” ( Partnership lead )
“ We were doing some work around developing a validated survey , [ with ] scales or some kind of questionnaire scales to be able to measure intermediate outcomes . Things like metacognition , kind of cognitive study strategies to a degree , even like academic self-efficacy and sense of belonging have some predictive value over attainment from what we ’ re finding in the data … robust ways of measuring the intermediate outcomes that are linked to attainment and be able to build kind of plausible theories of change and evaluations where they can measure impact .” ( Stakeholder ).
Partnerships are taking care to tackle issues with data quality in completed surveys . Ensuring learners fill out forms properly ( in order to collect good quality data ) was a priority for partnership leads . However , pre / post surveys can prove challenging , especially if they are lengthy . Equally , partnerships have had more success when administering these face-to-face in an activity session , but in some cases they must rely on teachers administering these outside of school time . Electronic online surveys ( with paper as a backup ) helps provide flexibility but introduces other challenges as schools sometimes do not permit phone use while on site .
“ We physically went to every school to collect paper copies of the pre / post questionnaire and bring them back or utilised our iPads . We didn ’ t leave it to the risk of , ‘ If teachers get them , we ’ ll come and collect them at some point ’,’ or ‘ You [ students ] do this online form ’, then we open it and half of them haven ’ t done it right or , you know , said they were James Bond , which is quite a common one .” ( Partnership lead )
Some partnerships had no evaluator within their team and struggled to recruit one . Approximately half of the partnerships have a staff member with evaluation expertise within the delivery team . Programmes without evaluation leads rely on delivery staff and other programme staff to conduct evaluation activities . This can be somewhat detrimental to progress with evaluation because at times these staff do not have the necessary skills or time to effectively run evaluation activities .
20-048464 | FINAL | Public | This work was carried out in accordance with the requirements of the international quality standard for Market Research , ISO 20252 , and with the Ipsos MORI Terms and Conditions which can be found at http :// www . ipsos-mori . com / terms .