Online Learning and Student Success: A Comparison of Student Retention and Progression Across Delivery Modes

Concurrent Session 2

Session Materials

Brief Abstract

Contrary to what has been widely reported in the media, the research reported in this presentation suggests that taking online courses does not necessarily negatively impact student success, and indeed can in some circumstances enhance retention and progression.

Presenters

Scott is a Data Scientist with a strong interest in utilizing analytics and statistical methodologies to enhance decision making and drive policy changes in higher education. Scott currently works for PAR Framework, a division of Hobsons, where he builds predictive models, develops visual representations of data, and tackles cross-institutional research questions for partner institutions.

Extended Abstract

Introduction

Several large scale studies comparing retention and progression between community college students taking online and on-ground classes have led to a widely held perception that taking courses online negatively impacts student success (Hart, Friedman & Hill,2015; Jaggers & Xu, 2010; Johnson & Cuellar Majia, 2014; Xu & Jaggers, 2011). The research reported in this presentation contests that perspective.  It suggests that taking online courses does not necessarily negatively impact student success, and indeed in some circumstances it can actually enhance retention and progression.

Methodology

The data for this research were 656,258 undergraduate student records collected through the Preditctive Analytics Reporting (PAR) Framework, a non-profit, multi-institutional collaborative that provides member institutions with tools and resources for identifying risks and improving student success. Data analyzed included records for all students who began their studies between September 2009 and December 2012 at 14 member institutions:

• 5 primarily on-ground community colleges (213,056 student records)
• 5 primarily on-ground 4-year universities (113, 036 student records)
• 4 primarily online institutions (330,166 student records)

Student retention and progression was compared between three groups – students taking only on-ground courses, students taking only online courses, and students taking some courses online and some courses on-ground (blended) within each of the institutional categories. Delivery mode categories were based on courses enrolled in up to and immediately following the first six months of enrollment. A student was considered retained if they were enrolled in any course (or graduated) 12 to 18 months after their start date.  Progression was measured by credit ratio (credits earned with a C or better divided by credits attempted).

Average retention rates, credit ratios, and credits attempted for students in each delivery group at each institution were aggregated across the three institutional categories with institution as the unit of analysis. In addition, logistic regression was used to compare retention rates at each institution across delivery modes while controlling for demographic and academic variables.

Findings

Primarily On-Ground Community Colleges
 
Table 1: Average retention, credit ratios, and credits attempted for community college students by delivery mode https://docs.google.com/document/d/1PvyULcjwCcYYmqmPBkNCGKNGEAsub6pL7FvW9aLq59c/edit?usp=sharing

Table 1 (as seen in the above link) compares retention and progression across delivery groups at PAR community colleges. It shows students taking some of their courses online and some on-ground had a retention rate of students taking all of their courses online were retained in the year following their first enrollment. After controlling for confounding variables, however, logistic regression found students taking all of their courses online were only moderately less (1.2 to 1.6 times) less likely to be retained than students taking all of their courses on-ground or students blending their course. Moreover, there were no statistical differences between the latter groups at four out of institutions studied.

The results suggest that while taking all courses online had a mild negative impact on PAR community college students’ retention, taking some online courses (blended) did not. Moreover, credit ratios were remarkably similar across delivery modes, indicating that community college students taking some or all of their courses online were as likely to complete and pass their courses as students taking all of their courses on-ground.  Indeed, students blending online and on-ground courses were slightly more likely to complete their courses with passing grades than either students taking all their courses on-ground or students taking their courses only online.

Primarily On-ground Four-Year Colleges
 
Table 2: Retention rates, credit ratios, & credits attempted for primarily on-ground 4-year college students by delivery mode 

https://docs.google.com/document/d/1PvyULcjwCcYYmqmPBkNCGKNGEAsub6pL7FvW9aLq59c/edit?usp=sharing

Table 2 (as seen in the above link) compares retention and progression across delivery groups at PAR primarily on-ground four-year colleges. The data show that students at primarily on-ground four-year colleges were retained at higher percentages than primarily on-ground community college students in the PAR database.  Although differences in retention percentages between delivery mode groupings were not as pronounced in this population, the ranking patterns were the same as those of primarily on-ground community college.  After controlling for possible confounding variables in a logistic regression model, however, results showed that in the majority of cases no group was at higher risk than any other of not being retained at twelve to eighteen months.

Credit ratios were higher for students at four-year universities in this study than for students at community colleges, and although the average credit ratios for students blending their courses and students taking all their courses on-ground were remarkably similar (.80 and .81 respectively), the average credit ratio for students taking only online courses was considerably smaller (.74), and this effect was compounded by solely online students taking considerably fewer credits.  It is possible that students taking only online courses chose to do so because they had busier lives than students taking some or all of their courses on-ground. Perhaps these students attempted fewer courses and had a more difficult time passing them for the same reason.  The finding clearly deserves further investigation.

Primarily Online Institutions

Institutions in the primarily online category included a public community college, a public four-year university, and two for-profit universities. The category was so constituted because within the PAR community we have found that institutions focusing on online programs are more like each other than not and less like primarily on-ground institutions.

The vast majority of the students in the primarily online category were, unsurprisingly, fully online students. However, three of the four institutions in this group had a sufficient number of fully on-ground students to ask similar questions of this category of institutions. Part of the rationale for doing so was to explore whether primarily online institutions have better retention rates for students taking online courses than primarily on-ground institutions. They did not. The patterns of retention across delivery modes shown in Table 3 (as seen in the below link), however, were somewhat different.
 
Table 3: Retention and credit ratios for students enrolled in primarily online institutions by delivery mode.

https://docs.google.com/document/d/1PvyULcjwCcYYmqmPBkNCGKNGEAsub6pL7FvW9aLq59c/edit?usp=sharing
The results suggest that among the PAR primarily online institutions, students taking a blend of courses were more likely to be retained, but there was little difference in retention odds between fully on-ground and fully online students after accounting for extraneous variables. The results suggest that students taking some courses online and some courses on-ground were more likely to be retained than students taking all their courses either online or on-ground. 

The average credit ratio for students blending courses at primarily online institutions (.66) was similar to that for students blending courses at community colleges, while it was lower than that for students taking only online courses at community colleges, and for all delivery categories at four-year universities. The average credits attempted by students enrolled in primarily online institutions were a good bit lower across all categories than average credits attempted at either community colleges or four-year universities.  This finding suggests that students enrolling at primarily online institutions may do so because of access issues, most likely time constraints.

 

References

Hart, C. M. D., Friedmann, E., & Hill, M. (April, 2015). Online course-taking and student outcomes in California community colleges.  Paper presented at the 2015 annual meeting of the American Educational Research Association, Chicago.

Jaggers, S. S., & Xu, D. (September, 2010). Online learning in the Virginia Community College system. New York: Community College Research Center, Teachers College, Columbia University.

Johnson, H., Cuellar Mejia, M., & Cook, K. (2015). Successful Online Courses in California’s Community Colleges. San Francisco, CA: Public Policy Institute of California.

Xu, D., & Jaggers, S. S. (March, 2011). Online and hybrid course enrollment and performance in Washington State community and technical colleges. New York: Community College Research Center, Teachers College, Columbia University