Assessing Readiness for Online Education - Research Models for Identifying Students At Risk

Concurrent Session 4
Streamed Session

Watch This Session

Session Materials

Brief Abstract

This study explored the interaction between student characteristics and the online environment, to explore differential results between online versus face-to-face course performance and subsequent persistence.

Presenters

Lead Presenter: Katherine Conway, Borough of Manhattan Community College-CUNY

Dr. Conway is a Professor in the Business Management Dept. at the Borough of Manhattan Community College, CUNY. She has a Ph.D. in Higher Education, Administration, Leadership and Technology from NYU. Dr. Conway's research focuses on student access and persistence, online learning, and STEM education, with an emphasis on immigrant, minority and first-generation students. She has published in numerous journals including The Review of Higher Education, Community College Review, and the Journal of College Student Retention: Research, Theory & Practice. Dr. Conway has presented widely including at the Association for the Study of Higher Education and American Educational Research Association. Jointly with her co-authors, her current research on online education, is funded by an NSF grant. Previous support included AERA, CUNY C3IRG and PSC CUNY awards. She serves as a reviewer for the Community College Journal of Research and Practice, The Internet and Higher Education, Computers and Education and and Diaspora, Indigenous and Minority Education.

Extended Abstract

Problem:
In 2013, over 40 million college students took online classes worldwide; by 2017, that number should triple (Atkins, 2013). Online courses potentially increase college access, but higher attrition rates (for reasons not yet well understood), may also be a stumbling block to degree completion. Yet restricting access to online courses may impede the college progression of "non-traditional" students who need the flexibility that online learning affords. To maximize the benefits of online learning while avoiding the pitfalls, institutions need research-based information about which students are at higher risk online.

This presentation will describe a study that compared online and face-to-face course outcomes taken by students at a large urban university system in the northeastern U.S. We will describe which student characteristics were the most consistent predictors of differential online versus face-to-face course outcomes, and subsequent college persistence. Additionally, we will discuss the implications of these findings and the potential of these kinds of studies to impact policies and programs related to online learning at the college level.

Question:
Identifying at-risk online students can only be done by considering the interaction of student outcomes with the online environment, by identifying factors that predict differential performance between comparable online and face-to-face courses rather than factors (e.g. G.P.A.) which predict academic outcomes more generally. Few studies have tested this interaction, resulting in a dearth of information about factors impacting online outcomes explicitly.
This study specifically sought to answer the question:
When controlling for differences in student characteristics, what differences exist between online and face-to-face course and subsequent outcomes; and which student characteristics are the strongest predictors of such differences?

Methodology:
This study used multi-level logistic regression models and propensity score matching to compare the differential online versus face-to-face performance of students with different characteristics. Models were run comparing the same course taken online versus face-to-face by different students, as well as models that compared online versus face-to-face courses taken by the same student. Factors considered include: whether a student has a child (and age of youngest child); gender; race/ethnicity; age; work hours; income; parental education; developmental course placement; marital/cohabitation status; immigration generational status; native speaker status; college level (two-year, four-year, or graduate); G.P.A; and number of credits/classes taken that semester, as well as scales measuring: motivation to complete the course; course enjoyment/engagement; academic integration (i.e. interaction with faculty/students outside class); self-directed learning skills; time management skills; preference for autonomy; and grit (i.e. perseverance and passion for long-term goals).

Results:
Preliminary results indicate that immigration generational status was a consistently significant predictor of differential online course performance. Foreign-born students were more likely to succeed in the online environment versus the face-to-face environment compared to other students with similar characteristics. However, many of the characteristics often associated with immigrants: autonomy, motivation etc. were not consistently significant predictors across different operalizations of the data set, and the effect sizes were small.

In addition, students with young children were significantly less likely to do well in online versus face-to-face courses compared to other students with similar characteristics. Further models are being run separating community college and results to determine if these patterns might be different for community college students.