Proponents of open educational resources (OER) claim that significant cost savings are possible when open textbooks displace traditional textbooks in the college classroom. We investigated student and faculty perceptions of OER used in a community college context. Over 125 students and 11 faculty from seven colleges responded to an online questionnaire about the cost and quality of the open textbooks used in their classrooms. Results showed that the majority of students and faculty had a positive experience using the open textbooks, appreciated the lower costs, and perceived the texts as being of high quality. The potential implications for OER initiatives at the college level seem large. If primary instructional materials can in fact be made available to students at no or very low cost, without harming learning outcomes, there appears to be a significant opportunity for disruption and innovation in higher education.
Review of literature
Over the past 15 years, extensive efforts have been made to create open educational resources (OER) to improve education around the world (D’Antoni, 2009; OECD, 2007). These OER have been touted as having the ability to increase learning by making educational resources more accessible (Cape Town Open Education Declaration, 2007; National Knowledge Commission, 2007). Some have posited that OER have the potential to save substantial amounts of money by eliminating the need for students or school districts to purchase expensive textbooks (Hilton and Wiley, 2011). Numerous projects have been undertaken to develop OER, including the creation of OpenCourseWare at MIT and several other universities (see http://www.ocwconsortium.org), education modules like those available by Connexions (see http://cnx.org), openly available textbooks such as those offered by CK–12 or the Saylor Foundation (http://ck12.org; http://www.saylor.org; Hilton and Wiley, 2011; Wiley, et al., 2012), openly available classes (Fini, et al., 2008), and Massively Open Online Courses (MOOC) (Mackness, et al., 2010; Fini, 2009).
While great efforts have been undertaken to create OER, much less has been done to examine its use and impacts. Important questions remain: Are people actually using these OER? Are OER as effective as traditional educational resources? Does using OER actually save money? How do students and teachers perceive the OER that they use in the classroom?
The current study explores student and teacher perceptions of the cost and quality of one type of OER — open textbooks — in the context of an initiative called Project Kaleidoscope (http://www.project-kaleidoscope.org/).
Review of literature
Much of the research relating to open textbooks involves explanations of why open textbooks may be a partial solution to the high costs of education. Ochôa, et al. (2011) review the costs of textbooks for college freshmen in Brazil, and present these costs in terms of the percentage of income earned by a person making minimum wage. After showing that these percentages range from the low 50s to the upper 80s, they describe plans for a new open textbook initiative focused on Latin America. Wiley, et al. (2012) followed the efforts of a school district that uses open textbooks based on CK–12 materials for three high school science courses. They found that open textbooks cost over 50 percent less than traditional books, and that there was no change in student outcomes on statewide, standardized tests when students used the US$5 textbooks throughout the year in place of traditional textbooks.
But students can save money with open textbooks only if faculty are willing to adopt them. Harley, et al. (2010) surveyed university faculty in California about their attitudes toward open textbooks, finding that 95 percent of respondents would be willing to adopt and use open textbooks when they are the same quality and have functionality equivalent to traditional textbooks. While faculty attitudes toward these hypothetical books are interesting, there is a critical need for empirical work establishing faculty attitudes toward open textbooks that they have actually adopted and used for a semester or longer. There is also an equal need to understand student attitudes toward open textbooks. The benefits of cost savings may very well be outweighed by negative perceptions of quality, the digital nature of many open textbooks, or other factors.
The purpose of this study is to establish the perceptions of post–secondary faculty and students who used open textbooks in place of traditional textbooks in their courses.
This section describes the study design, the measurement instruments, and data analysis approach.
Study design. Project Kaleidoscope (PK) brings together eight community colleges serving predominantly at–risk students to create course designs and textbooks using OER. Project partners involved in PK include Cerritos College (Calif.), Chadron State College (Neb.), College of the Redwoods (Calif.), Mercy College (N.Y.), Palo Verde College (Calif.), Santiago Canyon College (Calif.), Santa Ana College (Calif.), and Tompkins Cortland Community College (N.Y.). These partners collectively serve over 100,000 students per year, 69 percent of which are designated as “at–risk” by each college’s internal evaluation. During Fall Term 2011, PK affected nearly 2,000 students across 40 teachers, with expansion planned in subsequent terms.
PK is unique in its focus on supporting institutional adoption of OER, rather than on the creation of new resources. Faculty teams from across the colleges identify and evaluate existing OER for incorporation in the Kaleidoscope course designs and textbook development. The emphasis on open resources is driven by two project objectives: (a) eliminating textbook costs as an obstacle to the success of low–income students; and, (b) allowing faculty greater flexibility in sharing and improving the course resources.
To evaluate student and teacher perceptions of the various open textbooks used in their courses, 27 instructors participating in the PK initiative were asked to complete an online questionnaire about their perception of these texts in terms of cost and quality. Requests were made via e–mail containing a link to the questionnaire in late November, near the end of the Fall term. Instructors were also provided with a link to a student version of the questionnaire and were asked to forward that link onto their students. Follow–up e–mail messages were sent to non–responding teachers and teachers with low numbers of responding students every seven days through the end of the semester. The questionnaires included multiple–choice, multiple–select, and constructed response items. The questionnaires were developed through consultation with a measurement expert to help ensure validity and reliability.
Instructor questionnaire. The first few items of the instructor questionnaire addressed basic demographics (e.g., gender, institution, teaching tenure) and details about the course in which PK texts were used. Specifically, these items asked about experience teaching the course, communication with students about the experimental nature of the texts, preparation time for the course, and teacher perceptions on student use of texts in previous semesters. We also asked teachers about student preparedness and use of open texts during the Fall 2011 semester compared to previous semesters. Four items directly addressed the quality of the open texts used in the course. These items asked instructors about student feedback related to the texts, likelihood of future use of open texts, and perception of text quality compared to non–open texts used in the past. The term quality was not explicitly defined and was left to individual interpretation by respondents.
Student questionnaire. The student questionnaire consisted of 19 items. The first few items addressed basic demographics (e.g., gender, course and instructor information, SES) and academic history. Some items also addressed typical spending on texts, usual credit–load, and general textbook use. The remainder of the questionnaire addressed the specific course that employed PK texts. Two items asked about money spent on textbooks and printing text–related materials. The final six items related to student perceptions on quality of open texts used in the course. These items asked students about their use of the texts during the semester, the quality of the open texts compared to texts in other courses, and the likelihood of registering for future courses that use open textbooks. As in the instructor questionnaire, the term quality was not explicitly defined but was left to individual interpretation by respondents.
Data analysis. Multiple–choice and multiple–select items on the questionnaires were analyzed quantitatively. However, because of the small sample sizes in this initial study, statistical analyses were not appropriate. A descriptive analysis of results from these items was conducted instead. Open–ended items were analyzed using thematic analysis (Benner, 1985; Leininger, 1985; Taylor and Bogdan, 1984). This qualitative analysis approach involves several steps including identification of initial topics, organization of topics into emergent themes, axial coding, and re–organization and finalization of themes.
Instructors. Eleven of the 27 instructors responded to our survey. These teachers represented 11 subject areas from seven different community colleges. Six instructors who completed the questionnaire had no students respond to the student version.
Instructor demographics. Of the 11 instructors who responded to the questionnaire, seven (64 percent) were female and four (36 percent) were male. Ten (91 percent) have taught at the college level for more than two years, eight (82 percent) have taught for more than five years, and four (36 percent) have taught for more than 18 years. Ten (91 percent) instructors have master’s degrees and one (nine percent) has a doctoral degree. Most instructors (82 percent) teach four or more courses (or sections of the same course) per semester. All instructors have taught the course in which PK texts were used at least three times prior to the Fall 2011 implementation. More than half (55 percent) have taught the course six or more times. Finally, all but one instructor discussed the experimental nature of PK texts with their students. In fact, six instructors (55 percent) discussed the project more than five times throughout the semester.
Instructor perception of cost. Some of the cost associated with open texts includes instructor preparation time. Since all instructors were using PK texts for the first time, we asked them to compare time spent preparing to teach during the Fall 2011 semester with time spent in previous semesters. No instructor indicated spending less time, and most (82 percent) felt they spent somewhat more or much more time preparing to teach in Fall 2011 than in previous semesters. This result points to a somewhat hidden cost of initial implementation of open texts for these instructors, but it is not known whether these costs are any different from those incurred when implementing any other kind of text for the first time. On future versions of the questionnaire, we will include an item that explicitly asks instructors about general textbook implementation.
Instructor perception of quality. Student use of textbooks can be an indication of text quality and usefulness, but can also simply be an artifact of teaching style and study requirements. Most instructors (70 percent) indicated that students used their PK texts at least two–three times a week, similar to the frequency with which students self–reported using previous textbooks. Student preparation is probably more important than student use as an indicator of text quality. In our study, 90 percent of instructors reported that their students were equally prepared (60 percent) or more prepared (30 percent) than were students in previous semesters. Only one instructor felt that students were less prepared. Perhaps the strongest endorsement of open textbooks is that all 11 instructors indicated that they would be very likely to use open texts in future courses.
Five of the responding instructors were personally involved in the development of PK texts used in their courses. Those instructors not involved in text development were asked to rate the quality of the open texts compared to texts they have used in previous semesters. All six of these instructors indicated that the PK texts were of about the same quality as texts used in other courses. Instructors who were involved in creating the texts were not asked to evaluate them.
Finally, instructors were asked about feedback from students regarding the PK texts. Positive student feedback, as reported by eight of the instructors, included
They liked the fact that the textbook was free, and also was accessible online.
No major complaints.
They like that it is free.
They liked the price ($0–25) ...
They were happy that they did not need to purchase a textbook for this course.
Some really liked the text, others did not.
Negative feedback that teachers reported students saying included
Many don’t use [the text] at all (40 percent) ...
One student reported that one weekend the ... website was down and she couldn’t access the free electronic version. One student reported she had to get her own textbook to understand the material better (she bought an AP psychology book at a local bookstore — perhaps it was an easier read?).
[Students] did not often like accessing the online version.
The comments above seem to indicate that most students appreciate the low cost of the texts and the ability to access texts online. Negative feedback appears to revolve almost entirely around frustration with the online format of the texts, and only a very small portion of students reported such problems to their instructors.
Students. One hundred and thirty–two students representing 11 subject areas from seven different community colleges started the student questionnaire. One hundred and twenty (91 percent) of these students completed all items. Hence, some items have response counts between 120 and 132. Five instructors who did not complete the instructor version had some students who responded to the student questionnaire.
Student demographics. Of the 132 students who responded to the questionnaire, 85 (65 percent) were female and 46 (35 percent) were male. The majority of these students (71 percent) had received Pell Grants or fee waivers to fund their education, and only 27 percent had received loans. These data indicate that many of the responding students faced economic hardship. Nearly 75 percent of the students had completed four college terms or fewer (23 percent were in their first term), and the average GPA for all students who had completed more than one term was between 2.6 and 3.0. The majority of students who responded to the questionnaire (93 percent) take three or more courses per semester. Just over two–thirds (69 percent) enroll in four or more courses per semester.
Student perception of cost. Most students (81 percent) reported that they often or always purchase required texts for the courses they take, and 82 percent claimed to typically spend more than US$200 on texts each semester (46 percent pay more than US$300/semester). For the courses in which PK texts were used, 90 students (70 percent) said they did not purchase any texts. These students were asked to select a reason for why they did not purchase texts. The majority (83 percent) selected The texts were available free of charge online, but eight percent indicated they couldn’t afford to purchase the texts (Figure 1).
Figure 1: Most students indicated online availability of PK texts as the reason for not purchasing texts for their course.
Of the 39 students who reported purchasing texts, 28 (72 percent) spent less than US$60, and nearly half spent less than US$40. Finally, some students (34 percent) printed text materials for their course. Most of these (87 percent) spent less than US$20 on printing. These data indicate that, in general, students perceive spending less money on PK texts (even counting cost of personal printing) compared to what they typically spend on required texts.
Student perception of quality. One indicator of text quality is student use. Most students (64 percent) reported using textbooks at least two to three times per week in a typical course. For courses with PK texts, a similar pattern was seen: 60 percent of students reported using PK texts at least two–three times per week.
Students also responded to a series of questions related directly to perception of the quality of the PK texts. First, students were asked to compare PK texts to other kinds of texts. Only four students (three percent) felt that PK texts were worse than the quality of texts in other courses, while 67 students (56 percent) selected about the same as the quality of texts in other courses and 49 students (41 percent) selected better than the quality of texts in other courses (Figure 2).
Figure 2: Nearly all students indicated that PK texts were better or about the same quality as texts they have used in other courses.
Similarly, 52 percent of students reported liking the online format more than traditional printed texts, 17 percent liked the online format less, and 31 percent had no preference. Many students (77 percent) also reported that they would be very likely or somewhat likely to register for a future course with online texts similar to PK texts. Finally, students were given an exhibit question that read “Imagine a future course you are required to take. If two different sections of this course were offered by the same instructor during equally desirable time slots, but one section used texts similar to those used in this course and the other used traditional published texts, which section would you prefer to enroll in?” A majority of students (74 percent) reported they would prefer to enroll in the section with texts like those used in the PK course. Only 13 percent would enroll in the traditional text section and the other 13 percent had no preference.
In order to more fully understand why students perceived PK text quality as they did, students were asked to provide open responses to a few items. Four students (three percent) indicated that the PK texts were worse than texts in other courses and were asked to explain why they felt that way. Three of the four students who felt the PK texts were worse responded to this item. One student cited screen fatigue, visual presentation of the text, and typographical errors as the major reasons for her negative perception. A second student simply prefers a physical copy. The third student saw a misalignment of the text with content covered in the classroom.
At the same time, students who indicated that the PK texts were better than texts in other courses cited presentation, economics, and online accessibility as major reasons for their positive view. Within the presentation theme, students indicated that the PK texts were easier to understand (16 students), well–organized (eight students), aligned with classroom content (seven students), interactive or searchable (six students), visually appealing (four students), and interesting (one student). Here are a few quotes representing these positive perceptions:
The information was available in many forms depending what I need at the time, online available anywhere, at home in a textbook, printing lecture note to take to study. It made it easier to study and complete my homework even with a full time job.
I did not have to purchase an over priced, under used book. Saved money and the text was available from any where. Loved having the text online for free.
I appreciate how “to the point” the online readings are. I often spend lots of time learning supplementary material that doesn’t pertain to what will be on the tests and do worst on the test cramming information I do not need to know.
It was very concise and aligned with exactly what we were working on in the class. Having the textbook catered to us by our teacher was perfect.
Finally, students were asked to respond to the open–ended question: “Overall, what do you think of the texts used in this course?” Of the 98 responses, only nine (10 percent) were completely negative, 14 (14 percent) were a mixture of positive and negative, and 75 (75 percent) were completely positive (Figure 3).
Figure 3: Most student responses were completely positive to a general, open–ended question about PK text quality.
The student comments containing negative perceptions included eight main themes: (a) low visual appeal; (b) Web site issues; (c) online navigation problems; (d) general text uselessness; (e) note–taking limitations; (f) lack of text options; (g) Internet access issues; and, (h) low readability. Commentary illustrating these themes include
... only problem was that the images necessary for some of the problems did not come through ...
The only thing is that my eyes hurt after being in front of a computer for hours.
I prefer the book to the internet text. I can highlight and make notes in margins for quick reference. Plus the online text would time me out after a short time so I would lose all the information I had just tried to learn. I had to purchase the printed text a couple weeks into the term so that I could actually read it.
Kind of confusing to start out with a lot of broken links. And a lot of things were hard to find.
It was nice not to pay for a textbook, but a pain I could not open a book to look up information because I did not always have access to the internet.
Positive student responses revolved around seven main themes: (a) feeling; (b) content; (c) presentation; (d) positive adjective; (e) text comparison; (f) online accessibility; and, (g) economics (Table 1). Each of these major themes had several sub–themes. For instance, students reported that they liked, loved, enjoyed, or recommend the PK texts (feeling theme); found the texts informative, useful, or effective (content theme); thought the PK texts were easy to understand, well–organized, clear, concise, engaging, or visually appealing (presentation theme); used one or two descriptive words to describe the texts, including excellent, great, fine, perfect, very good, high quality, convenient, and awesome (positive adjective theme); felt the PK texts were better presented, more convenient, better organized, or a more effective learning resource than other texts they have used (text comparison theme); mentioned PK text availability, mobility, searchability, or efficiency (online accessibility theme); and, indicated that PK texts were free, cheap, good for poorer students, or saved them money (economics theme).
Table 1: Positive themes emerging from student responses to a general question about PK text quality.
Note: 98 students responded to this item with 75 providing completely positive comments and another 14 providing a mixture of negative and positive responses.
Theme Frequency Percent of positive comments Feeling 12 13.48 Content 18 20.22 Presentation 23 25.84 Positive adjective 23 25.84 Text comparison 7 7.87 Online accessibility 11 12.36 Economics 12 13.48
The following comments serve to highlight these positive perspectives:
I liked it because it was easier to follow than normal textbooks.
For me the text was extremely helpful and easy to understand. I was very thankful for it.
Clear, concise, overall had a good variety of practice problems. There were some mistakes in the text but nothing that really impeded me.
I preferred this book over a traditional printed text.
We didn’t have to carry the physical book around. We can view the book where we receive internet access, so that was nice.
I enjoy having online texts provided for me because I’m poor. I spend the money I have left after rent on school, so having free online texts provided for me benefits me very much.
I would much rather have all my classes set up this way instead of just reading the normal text and having to guess what is important and what is not needed.
great. learned more online than I ever did with a written text.
I thought the online textbook was very convenient and effective for my studying.
They were high quality, easy to understand and navigate, and they were free!
I liked it because it is all online and easy to access!
Most faculty and students who participated in the study recognized and appreciated the low cost of PK textbooks and perceived them as being of high quality. Although we did not define the term quality for the survey respondents, the themes that emerged from the open–ended questions give us a picture of what aspects of an open text are important to both faculty and students. Specifically, the presentation of the text, including visual appeal, organization, and readability, the online accessibility, the alignment of text content with classroom content, and economic efficiency all seem to play a part. However, students who had negative perceptions of PK texts reveal that some of PK texts’ major strengths (in terms of quality) can also become weaknesses. When students have poor Internet access, when the texts are not well–maintained and organized, or when students experience screen fatigue, PK texts are not seen as being as useful or of as high quality as traditional print texts.
The potential implications for initiatives like Project Kaleidoscope seem large. If primary instructional materials can in fact be made available to students at no or very low cost, without harming learning outcomes, there appears to be a significant opportunity for disruption and innovation in higher education.
About the authors
TJ Bliss is a Ph.D. student in the Educational Inquiry, Measurement and Evaluation Program in the David O. McKay School of Education at Brigham Young University.
E–mail: tjbliss [at] gmail [dot] com
John Hilton III is an assistant professor of Ancient Scripture in the Department of Religious Education at Brigham Young University.
David Wiley is an associate professor of Instructional Psychology and Technology in the David O. McKay School of Education at Brigham Young University.
Kim Thanos is founder and principal of Thanos Partners, Inc.
P. Benner, 1985. “Quality of life: A phenomenological perspective on explanation, prediction, and understanding in nursing science,” Advances in Nursing Science, volume 8, number 1, pp. 1–14.http://dx.doi.org/10.1097/00012272-198510000-00004
A. Fini, 2009. “The technological dimension of a massive open online course: The case of the CCK08 course tools,” International Review of Research in Open and Distance Learning, volume 10, number 5, at http://hdl.handle.net/10515/sy5f47h70, accessed 1 January 2013.
A. Fini, A. Formiconi, A. Giorni, N.S. Pirruccello, E. Spadavecchia, and E. Zibordi, 2008. “IntroOpenEd 2007: An experience on Open Education by a virtual community of teachers,” Journal of e–Learning and Knowledge Society, volume 4, number 1, at http://services.economia.unitn.it/ojs/index.php/Je-LKS_EN/article/view/266, accessed 1 January 2013.
D. Harley, S. Lawrence, S.K. Acord, and J. Dixson, 2010. “Affordable and open textbooks: An exploratory study of faculty attitudes,” Research and Occasional Papers Series, Center for Studies in Higher Education, UC Berkeley, at http://escholarship.org/uc/item/1t8244nb.pdf, accessed 1 January 2013.
J. Hilton and D. Wiley, 2011. “Open access textbooks and financial sustainability: A case study on Flat World Knowledge,” International Review of Research in Open and Distance Learning, volume 12, number 5, at http://hdl.handle.net/10515/sy57s7j58, accessed 1 January 2013.
M.M. Leininger, 1985. “Ethnography and ethnonursing: Models and modes of qualitative data analysis.” In M.M. Leininger (editor). Qualitative research methods in nursing. Orlando, Fla.: Grune & Stratton, pp. 33–72.
J. Mackness, S.F.J. Mak, and R. Williams, 2010. “The ideals and reality of participating in a MOOC,” Second International Conference on Networked Learning, at http://www.lancs.ac.uk/fss/organisations/netlc/past/nlc2010/abstracts/Mackness.html, accessed 1 January 2013.
X. Ochôa, A.S. Sprock, and I.F. Silveira, 2011. “Collaborative open textbooks for Latin America — The LATIn project,” paper presented at the 2011 International Conference on the Information Society (i–Society), at http://ariadne.cti.espol.edu.ec/xavier/papers/Ochoa-iSociety2011.pdf, accessed 1 January 2013.
S.J. Taylor and R. Bogdan, 1984. Introduction to qualitative research methods: The search for meanings. Second edition. New York: Wiley.
D. Wiley, J. Hilton, S. Ellington, and T. Hall, 2012. “A preliminary examination of the cost savings and learning impacts of using open textbooks in high school science classes,” International Review of Research in Open and Distance Learning, volume 13, number 3, at http://hdl.handle.net/10515/sy5707x40, accessed 1 January 2013.
Received 27 February 2012; revised 7 August 2012 ; accepted 15 December 2012.
“The cost and quality of open textbooks: Perceptions of community college faculty and students” by TJ Bliss, John Hilton, David Wiley, and Kim Thanos is licensed under a Creative Commons Attribution 3.0 Unported License.
The cost and quality of open textbooks: Perceptions of community college faculty and students
by TJ Bliss, John Hilton III, David Wiley, and Kim Thanos
First Monday, Volume 18, Number 1 - 7 January 2013