Comparing Online & Other Delivery Modes: Bending Rules to Improve Outcomes for Intro Psych Students

Concurrent Session 9
Streamed Session

Watch This Session

Session Materials

Brief Abstract

University rules may have outlived their usefulness in the world of online learning. We bent the rules for students enrolled in online Intro Psych courses.


Bruce has 30 years of UX design, product management, and consulting experience for several iconic technology companies as well as some start-ups that you never heard of. Among other things, he is a co-inventor on 9 patents. Seeking change, Bruce accepted a tenure-track appointment in psychology and human-computer interaction at Carleton University in 2014 where his research includes how to help first responders in terrorist attacks; urban navigation by people who are visually impaired; the nature of expertise in understanding data; and teaching and learning online. To the university he brings a bias to action and indifference to authority. Nonetheless, he likes what he does very well.

Extended Abstract

After the initial hoopla attending Massive Open Online Courses (MOOCs), serious assessments have shown that completion rates are often very low.  However, a consideration of students’ intentions allows a more optimistic view since those who may have no intention of completing a MOOC may nevertheless obtain benefits from that course. Our approach was to experiment with online course delivery together with some different rules around admissions and timetabling to see if we could overcome some common problems.
In 2014 we developed MOOCs in cooperation with our Registrar’s Office in order to provide a more responsive experience for students.  We took Introduction to Psychology I and II courses that would normally occupy 3-hours per week for one semester each and turned them into totally online courses comprised of approximately 65 video modules averaging 11-minutes each.  We added self-tests after each module and quizzes after each chapter, 3 online exams, 1 written assignment, and participation in (online) psychological research.
The two original face-to-face courses were transformed into six courses: PSYC1001 and PSYC1002 Web Courses, Open Access, and Flex Term. Web Courses were the online equivalent of face-to-face.  Students paid tuition fees, obtained credit for passing and the normal pre-requisites applied.  Open Access courses are the same as MOOCs.  Their content was identical to Web Courses but anyone could “enroll” and no tuition fees were charged but no credit was granted either.  Students could enroll in Open Access courses at any time.  What we call Flex Term courses are for academic credit and intended for those who are unsure of their abilities or their prospects of success, and those hoping to enroll in a course outside of the standard registration period.  The only way to enroll in Flex Term is by successfully completing at least 20% of one of our MOOCs.  Regardless of the time of year or time of day, a student can transfer from the MOOC to Flex Term and become formally enrolled as a Special Student upon receipt of appropriate tuition fees.  Finally, depending on the exact date that a student enrolls in Flex Term, they may be accorded the remainder of the semester, or up to one additional semester in order to complete.
These six courses were “soft launched” between September 2014 and January 2015.  Enrollment was limited to 200 students or less in each course in order to control the impact if problems were encountered.  The comparisons critical to us were: Was student achievement comparable across Web Course, Open Access, and Flex Term compared to their face-to-face and other online equivalents?  The DFW rate is the proportion of D’s, F’s, and withdrawals in each course as a function of total initial enrollment.   Was the DFW rate comparable?  Were student evaluations comparable?
The face-to-face course comparisons were based on the first author’s courses in the Fall of 2013 and to the previous five-years’ data from all Carleton University Introduction to Psychology courses irrespective of instructor.  The other comparisons were made to the first author’s video-on-demand courses in the Fall of 2012 where face-to-face lectures were videotaped and presented to distance students.  In the latter, approximately 350 students were in the videotaped classroom and another 350 were the recipients of the video-on-demand lectures.
Overall results are captured in Table 1:
Table 1: Comparing online, face-to-face, video on demand, & historical Intro Psych courses.  ¹Tsuji, n=162; ²Tsuji, n=120; ³Tsuji, n=345; ?All, n=29,679

Web Course F2014¹
F2F F2013²
Video on Demand F2012³
Intro Psych 2007-2012?
Student Evaluations
4.58 / 5.0
4.63 / 5.0
4.58 / 5.0
4.37 / 5.0

Since grades were not offered in our Open Access courses, these are not available for comparison.  Similarly, the small number of Flex Term students (n=5) makes quantitative comparisons untenable.  In Table 1, there are no significant differences in the proportion of A’s, B’s, or C’s across course types nor are there any differences in Student Evaluations.  Only the DFW rate between the first author’s face-to-face course and his Video On Demand course approached significance.  
Some qualitative impressions of Flex Term students hint at important insights.
EV was our first student who transferred from Open Access to our PSYC1001 Flex Term which she did on December 23, 2014 at approximately 7:00pm.  EV had never gone to university. After graduating from high school she spent some time cleaning houses before getting a job as a secretary in the psychiatry department at a local hospital.  “I’ve always wanted to do more,” said EV. “I want to help people and listen to their problems.”  EV wanted to take some psychology courses, but balancing work and care of two children while going to classes was going to be a serious challenge. Searching for something that would meet her needs, EV came across Carleton’s MOOC.  Now, at 31 years old, EV has completed her first ever university course with the intention of one day changing careers. Carleton’s MOOC and Flex Term courses gave her the confidence to try taking more university level courses. “I was really proud of myself,” said EV. “I’m proud that working full time I was still able to do one class.”  Similar stories are related by our other MOOC to Flex Term students.
MOOCs have low completion rates.  However, this is only a problem if we are concerned about formal completion or if we ignore student intentions or if we accept MOOCs as a pedagogical form in isolation.  By bending our Registrar’s Office rules somewhat, we have developed a path from MOOCs into our regular programs that we think benefits many.  While our evidence is currently only preliminary, we believe that this model can benefit many traditional and non-traditional students, particularly those who are currently underserved by our post-secondary education system.