Stay-at-Home Learning: Online Course Outcomes at a Hispanic-Serving Institution and What This Means for You
Concurrent Session 5
This session will examine original research suggesting that online students have non-significantly different grades but higher withdrawal rates than face-to-face students after controlling for 15 student characteristics. These results have important implications about how we provide education and support services online. Come discuss best practices for online education!
Topic: In this session I will present original research using a large, institutional dataset recording student outcomes over two academic years at a Hispanic-Serving Institution. This research is important because it uses a propensity score analysis to control for 15 different student covariates to control for imbalance between online and face-to-face enrollment groups and to control for selection bias. Results indicate that online students have non-significantly different grades but higher withdrawal rates than face-to-face students.
This topic is important as we understand what characteristics are associated with online or face-to-face enrollment patterns and what this means for our students, faculty, and administration. As we offer online courses in rapidly increasing numbers due to the COVID-19 pandemic, we need to ensure that these courses provide equitable educational access for all students.
Session Overview: I will start the session by introducing myself and asking participants to introduce themselves using a Poll Everywhere format to understand who is participating in the session. Then, I will introduce my research study and present the results or original research examining a large institutional data set of online and face-to-face course enrollments. After presenting the methods and results of the study, the remainder of the presentation will be an opportunity for participants to engage in a discussion about implications of this research at an HSI, and a group discussion about best-practices at individual institutions.
Introductions: 5 minutes
Presentation: 25 minutes
Questions and Discussion: 15 minutes
Research Summary: As online course enrollments are increasing in higher education in the United States, it is increasingly important to understand student course outcomes in these classes, particularly at Hispanic-Serving Institutions (HSI), where there has been limited previous research. This current study examines online course outcomes in the form of student course grades and student withdrawal rates as compared to outcomes in face-to-face courses. The setting for the study is Russell University (pseudonym), a public university in the Rocky Mountain west, and an HSI. Data used in this study came from a large, de-identified data set of all enrollments in any course offered in both online and face-to-face formats during the 2017-2018 and 2019-2019 academic years.
Baseline results of this study indicate that students in online classes have significantly higher course grades, and non-significantly different withdrawal rates than do students in face-to-face classes. However, enrollment patterns based on 15 different student covariates indicates a significant imbalance between online and face-to-face enrollment groups, which introduces selection bias into the results. To control for these covariates and balance the two enrollment groups, this study tests three different propensity score methods for validity and sensitivity to select a statistical method that is the best match for the data. The final statistical method chosen is a near-neighbor 1:2 propensity score analysis to control for these confounding covariates in order to balance the online and face-to-face enrollment groups. After balancing the groups using the near-neighbor 1:2 propensity score method, results indicate that there is a non-significant difference between online and face-to-face course enrollments in terms of student grades. However, after balancing, there is a significantly higher withdrawal rate among online students than face-to-face students. While promising, these results need additional confirmation from future research, as they remain highly sensitive to hidden bias from missing variables.
These results have important implications for students, faculty and administrators at an HSI to ensure equitable access to education in all course modalities. Online faculty should ensure that they intentionally build community in online classes and invite students to participate in high-impact practices such as research with their instructors. Administration should provide access to online student services such as advising and mental health resources in a personal, connected way to best meet the needs of students. However, helping students in online classes feel connected with the institution is something that all attendees can share, and this session will allow time for discussion and engagement around effective best practices at participants' institutions.
Engagement and Interaction: This session will provide details of the research study and context, and allow for reflection and discussion among attendees about how to best engage online students with the institutional community. Attendees will participate in an interactive introductions activity using a tool like Poll Everywhere. After presenting information from this study, there will be time for audience participation by inviting contributions of effective best practices that build community for online students at participants' institutions, and practical suggestions on how to reduce withdrawal rates in online classes. Engaging students and ensuring equitable access to education is a topic that requires a multitude of voices, and this discussion is an essential element of the presentation session.
Materials: I will provide slides from the session to the conference website and will also submit those slides to the conference proceedings.
Attendee Takeaways: In this session participants will learn about the value of a robust study that controls for student covariates and selection bias in course enrollment. Participants will understand the impact of online course enrollment at a Hispanic-serving institution. Finally, attendees will share practical suggestions for providing instructional and institutional support in online classes.