International Journal of Open Educational Resources
on the high end with few low grades in the class . The students ’ expectation was that more difficult courses would have a normal distribution around a mean of 2.0 with fewer A grades . The idea of using distributions of grades seemed to be a reasonable approach to quantitatively estimate course difficulty . Indeed , Anderson et al . (
2018 ) estimated course difficulty using historical grades and withdrawal rate in two finance courses ( two sections each ). While the withdrawal rate did not accurately discriminate between the two courses , the historical grade distributions seemed to be an appropriate discriminator . Wladis and Hackey (
2014 ) estimated course difficulty simply by distinguishing between “ lower level ” courses and “ higher level ” courses based on the presence of credit-bearing pre-requisites . If a 200-level course had a credit-bearing pre-requisite , it was deemed to have higher difficulty . The authors did not find a significant effect of online versus face-to-face delivery on retention rates in higher level courses .
2019 ) called for better control of potential confounds as this lack of control is a valid critique of OER efficacy research ( see
Griggs & Jackson , 2017 ;
Gurung , 2017 ). Indeed , Clinton (
2018 ) found that differences in prior academic achievement likely explained differences in learning outcomes when comparing an introduction to psychology course with a traditional textbook to one with an OER textbook . Some studies have controlled for possible confounds . For example , Fischer et al . (
2015 ) used propensity score matching to control for age , gender , and minority status across all courses . In addition , Jhangiani et al . (
2018 ) measured prior knowledge preceding the study and found that students in different courses had comparable background knowledge .
The current study was a test of the interaction between OER and course difficulty in a robust sample of courses and students while controlling for potential confounds . The primary research questions were :
1 . What is the association of textbook type with students ’ course grades controlling for gender ( self-reported ), Pell grant eligibility ( as a proxy for student socioeconomic status , see Colvard et al ., 2018 , for a similar approach ), prior academic success , and course difficulty ?
2 . Does the association of textbook type
14