International Journal of Open Educational Resources
( P ), a critical aspect to consider are the outcomes ( O ). If students save money , usage is widespread and nuanced , and perception is favorable , but student learning is not on par with the use of traditional textbooks , then the benefit of OER is diminished .
Most studies of OER outcomes have shown that courses using OER have comparable learning outcomes with courses using traditional textbooks ( e . g .,
Clinton & Kahn , 2019 ). Sometimes the outcomes for OER are better and occasionally they are worse . Reviews by Hilton (
2016 ,
2019 ) concluded that students generally achieved the same learning outcomes in classes with OER , compared with students in classes with non-OER . Robinson (
2015 ) utilized a quasi-experimental design to compare student learning outcomes between sections in the treatment group ( OER ) and sections in the control group ( non-OER ) among seven different courses . Overall , five sections using OER showed similar or better outcomes than sections of the same courses using traditional textbooks . Two sections of courses showed better outcomes using traditional textbooks . The same mixed pattern can be also found in a multi-institutional study by Fischer et al . (
2015 ). The authors utilized propensity score matching to control for age , gender , and minority status in 15 courses . Each course had sections that used either a traditional textbook or OER . The majority of courses ( 10 ) showed no difference in student grades according to OER vs . traditional textbook used . Four courses showed better grades in OER sections and one course showed better grades in the section using the traditional textbook .
A meta-analysis that aggregated findings from 22 studies with a combined total of over 100,000 students in which OER textbooks were compared to traditional textbooks found that learning outcomes were equivalent (
Clinton & Kahn , 2019 ). However , there was substantial variability across studies in effect sizes of learning outcomes between OER vs . non-OER . All of the studies used quasi-experimental designs with varying levels of control for possible confounds , such as being taught by different instructors . The authors grouped the studies for three potential methodological confounds : whether or not there was the same instructor , whether or not the same learning measurement was used to measure outcomes , and whether or not prior knowledge or academic achievement was accounted for in the findings . The findings on learning outcomes did not vary based on whether those potential confounds were accounted for . Therefore , it is uncertain why there was so much variability in learning outcomes across studies . However , when considering the relatively small effect sizes attributed to textbooks in general (
Robinson 2015 ) and the typically low coefficients of determination , it becomes apparent that variability in student performance is associated with myriad unmeasured covariates .
The access hypothesis provides a useful understanding of the meta-analytic findings on open textbook adoption . According to the access hypothesis , having access to learning materials
12