Online learning and the success of non-traditional students: What data analysis can tell us
Concurrent Session 3
This research explored the characteristics of students taking online courses to investigate whether the online setting itself or such characteristics primarily affected retention rates. Results from 108,637 university students revealed variables that predicted online course taking, and the effect online course taking had on student retention at seven four-year institutions.
Online learning continues to grow at post-secondary institutions across the United States. As of 2015, 14.3% of all students (2,902,765) were taking all their courses online while 15.4% of all students were taking a combination of online and on-ground courses, while traditional, fully on-ground enrollments continued to decline (Allen & Seaman, 2017).
While most researchers agree that learning outcomes from online courses are not significantly different from traditional courses (Bernard et al., 2009; Means er al, 2009), some research suggests that retention is poorer among students taking online courses (Jaggers, & Xu, 2010; Xu, & Jaggers, 2011; Hart, Friedmann, & Hill, 2015). Interestingly, however, there is some indication that graduation rates are higher for students taking online courses (Shea, P. & Bidjerano, 2014; Johnson, Cuellar Mejia, & Cook, 2015).
It has been suggested that one reason for this anomaly is that students may take online courses because in some sense they have to. Many online students may be older, non-traditional students attending college part time. Research indicates that adult learners are often juggling multiple responsibilities in addition to school (Lundberg, McIntire, & Creasman, 2008). Online learning may be one way such learners manage their education.
The research reported in this paper examined such issues by exploring possible differences in the characteristics of students enrolled in solely online, solely on-ground, or both online and on-ground courses at seven four-year universities, and how delivery mode affected retention for this population when such characteristics were controlled for. Research questions were:
- Are students taking online courses different from students taking only on-ground courses?
- What is the impact of delivery mode on student retention when extraneous variables are controlled?
The data for this study included 108,637 students who first enrolled between August 2009 and September 2014 at seven universities that were members of the PAR Framework, a division of Hobsons, Inc. Participating universities were bachelors granting institutions that were part of public university systems in the southeast and upper-Midwest. All but one of the seven universities studied were primarily residential, and all had a high percentage (aof transfer students. They ranged in size from large to very small, and in selectivity from selective to inclusive. Two of the institutions were historically black colleges. Data points were captured within a student’s first term at their institution and related to retention to a second year.
A number of variables were considered as possible predictors of course delivery mode, and as control variables in modeling the relationship between course delivery mode and stopping out. To avoid the possibility of data leakage, the values for each of these variables represent only what was known at the student’s first term at the university. Variables included: age at entry, Pell status, race, gender, high school GPA, transfer status, prior credits, part-time/full-time status, major, and delivery mode.
To answer the first question of our study, what predictors or student characteristics were associated with course delivery mode, we used multinomial logistic regression to model the probability of a student being fully online, fully on-ground, or mixing their courses for each institution separately. Delivery mode was modeled as a nominal variable in a multinomial logistic regression.
The second goal of the study was to estimate the impact of delivery mode on student retention when learner variables were controlled for, with special attention given to the interaction of delivery mode and part time/full time status. This was done using binary logistic regression models to compare the odds of students stopping out between chosen delivery modes and part-time/full time status for each institution.
Learner Characteristics and Delivery Mode
The first research question asked what variables might predict course delivery mode. In the full sample, 79% of students were fully on-ground during their first term, 13% mixed their courses and 8% were fully online, although these percentages varied somewhat among the institutions studied.
Among the demographic variables, female students had significantly greater odds of mixing their courses and of taking all their courses online than male students at six out of seven universities studied. African American students had lower odds of mixing courses at four universities and lower odds of being fully online at six. Pell recipients were more likely to be fully online at three. Student age was the strongest predictor of delivery mode among the demographic variables. Students who were 25 and older, a primary marker of non-traditional status, had greater odds of mixing their courses and of taking all of their courses online at all seven universities.
Transfer status was also one of the strongest predictors of delivery mode. Transfer students had 1.46 to 4.02 times greater odds of mixing their courses, and 1.54 to 24.93 times greater odds of being fully online. Additionally, students entering with more than 30 prior credits also had greater odds of mixing courses and of being fully online. Both high levels of prior credits and being a transfer student are also common characteristics of non-traditional students, adding to the evidence that non-traditional learners are more likely to enroll in online courses.
One of the strongest predictors of being a fully online student was part time status. This was a significant predictor at all seven institutions with odds ratios ranging from 2.83 to 21.29 when comparing fully online students to fully on-ground students. The relationship between part-time status and mixed delivery was less clear with students mixing their classes more likely to be attending part time at three institutions, and less likely to be attending part time at two. Once again, we suspect part time students particularly benefit from the flexibility of online courses as they likely have less time available to dedicate to coursework or commute to course offerings on campus.
The effects of high school GPA and major on delivery mode were inconsistent.
Delivery Mode and Retention
The second phase of analysis centered on the relationship between course delivery mode and student retention. Across all the institutions studied, retention rates were two percentage points lower for students mixing their courses than for fully on-ground students, and 13 percentage points lower for fully online students than for those who were fully on-ground. However, raw retention rates do not control for other factors that might be influencing retention. As observed above, many characteristics of non-traditional learners were significant predictors of taking online courses, indicating that differences in retention rate may reflect the different characteristics of online course takers rather than choice of delivery mode.
To test this, we examined the effect of delivery mode on stopping out using a binary logistic regression model that included the other predictors considered in this study as control variables. Because part time status was such a strong predictor of students being fully online, the interaction of delivery mode and part time/full time status was also included in the model and found to be significant at five of the seven universities.
Among full time students, both students mixing their courses and students taking only online courses had greater odds of stopping out than students taking only on-ground courses at all of the institutions studies although not all these effects were significant. Those mixing their courses had significantly greater odds of stopping out at four of the universities, however the odds ratios were fairly small for this effect ranging from 1.15 to 1.30. Fully online full-time students also had significantly greater odds of stopping out than fully on-ground full-time students at four institutions. Odds ratios were a little higher for this effect, but still moderate in size, ranging from1.42 to 1.92. These effect sizes are smaller than one might expect based on the overall differences in retention rates, indicating that differences in retention can mostly be explained by factors other than course delivery mode.
The relationship between delivery mode and student retention differed among part time students was inconsistent but differed with that of full-time students for most of the universities in our study. Both part time students who mixed their courses and part time students who were fully online had lower odds of stopping out than part time students who were fully on-ground at two institutions, greater odds of stopping out at one institution, and no difference in odds at the other four. In short, at five of the institutions studied, the effect of delivery mode on stopping out was moderated by part time status and taking courses online did not negatively influence the odds of retention for part time students at all but one institution.
The research revealed that students taking online courses were more likely to be older, female and part-time attendees. The finding supports the notion that many students are taking online courses to help them juggle multiple responsibilities It suggests that online courses and programs should recognize this fact and consider how they can more fully support such students by becoming more flexible and accessible. As the number of non-traditional undergraduate students is growing and non-traditional students now outnumber traditional ones (NCES, 2015), colleges should be planning for both more, and more flexible online offerings.
The findings also suggest that much of the observed differences in retention rates between students taking only online courses and students taking only on-ground ones at the schools we studied have more to do with learner characteristics than with delivery mode per se. Indeed, while full time students taking online courses had slightly greater odds of stopping out than full time students taking all of their courses on-ground, this result was reversed for part-time students at two of the colleges studied. At four additional institutions there was no difference in retention between delivery modes for part-time students. Thus, there is some indication of an interaction between part time status and delivery mode, namely that taking online courses doesn’t make a difference in odds of retention for most part-time students and seems to even be beneficial for some. Such findings suggest that there are often several interacting factors that make taking online courses alternatively beneficial or detrimental to student success. Such factors and interactions surely deserve further investigation.