2018 Digital Learning Innovation Awards
The 2018 class of DLIAward recipients demonstrated innovative approaches in addressing important challenges and barriers, including: improving student outcomes; altering instructional approach and pedagogy; minimizing costs of textbooks and tuition; targeting bottleneck courses; improving learning efficiency; enhancing student support; and decreasing time to course credit and degree completion.
The 2018 Award Winners
The following three institutions were awarded $100,000 for their innovative and creative programs, dedicated to accelerating the adoption of digital courseware for general education or gateway courses.
Arizona State University
Title: Transforming College Algebra: Eliminating developmental math and using adaptive courseware to enable student success
Impact: As a result of the new approach, success rates among all enrolled students increased from 55% in AY15- 16 to 75% in AY16-17, and 79% in AY17-18. Success rates for underprepared students had an even more significant improvement of 28 percentage points from the previous year’s outcomes. Given that approximately 33% of the AY16-17 students would have been enrolled in developmental math course based on math placement test results, these gains validated the efficacy of the model. Comparing student completion rates in the course for Fall 2015 and 2016, we also found that more than 800 additional students were able to complete the newly designed course on their first attempt. For those students, successfully completing College Algebra their first semester allowed them to stay on track for degree completion. This is a critical first step to improving student retention rates and, consequently, graduation rates. Moreover, when we tracked the first AY16-17 cohort of students in the new approach through their next math course, we discovered that they were 7 percentage points more likely than previous cohorts to complete it successfully on their first attempt. This indicated that innovation was paying dividends for the students well after the first course.
Salt Lake Community College
Title: Redesigning Quantitative Literacy Pathways for Student Success: Improving Learning Efficiency, Access and Outcomes in Community College Math Using OER Courseware
Impact: SLCC’s modifications to the QL completion pathway was analyzed one year after it launched. The analysis indicated the following results:
- Pass rates went down in the QL course Quantitative Reasoning (QR) but went up in college algebra. However, the lower pass rates in the QR course were still higher than those observed in college algebra.
- Higher pass rate in QR along with increased enrollment resulted in sharp increases in QL completion rates in Summer 2016 and 2017, Fall 2016 and Spring 2017, compared to historical rates. QL rates increased by about 5-7% in these semesters.
- The proportion of students retained into a higher math course the following semester was markedly higher for the new gateway course (.44 in Fall 2016) than for Intermediate Algebra – the previous gateway course (.33 in Fall 2015 and .3 in Fall 2016).
- The Math Department implemented (Fall 2017) the ‘Mulligan Project’ which intrusively assisted students who scored below 73% on the first formative assessment in the ‘Algebra for College Success’ gateway course to QL. The initial results indicate students within [60,83] range in the initial assessment successfully completed the course and changed their study habits by seeking help prior to summative assessments.
University of Central Florida
Title: Using Innovative Adaptive Courseware to Enable Student Success in Gateway Mathematics Courses
UCF’s evaluation of adaptive courseware focuses on students’ affective, behavioral, and cognitive reactions to adaptive learning. The UCF student reaction protocol is currently used by several universities (Colorado Technical University, Bay Path, UMUC, etc.). Student success has improved significantly, and Realizeit learning metrics indicate a potential for early prediction of at-risk students (Dziuban et al, 2016).
Continuous course design improvement is critical, with system analytics illustrating improvement through a gain in outcomes and a decrease in the variability of students’ performance across time in College Algebra, where success rates have increased by 15% compared to semesters prior to adoption, compared with online, non-adaptive sections. Some students even completed both the adaptive Intermediate Algebra to College Algebra sequence within a single semester, decreasing costs and time-to-degree.
Faculty-Led Team Recognition
The following faculty-led teams were awarded $10,000 for their innovative and creative programs, dedicated to accelerating the adoption of digital courseware for general education or gateway courses.
Bay Path University
Title: Scaling Adaptive Learning for a Predominantly Low-Income and Diverse Population of Adult Women Undergraduates in a Centralized Course Management Model: Capitalizing on OER Adoptions to Lower Costs and Improve Learning
Through the 4-year FIPSE grant, which concluded in September 2018, TAWC developed and currently offers its 51 adaptive courses to all students, while ensuring it can sustain the development and implementation of adaptive courses in a way that will allow TAWC to remain a low-cost tuition option for adult women undergraduates. Future content upgrades will carry reduced costs, since granular-level content can be more easily exchanged than replacing each course’s entire content.
Switching from textbooks to OER during course design projects in AY17-18 resulted in $176,000 in savings to students. Operational efficiencies in TAWC’s course development process enabled it to bring up adaptive courses for approximately $10,000 per course, significantly lower than the $50,000+ of some institutions. OER and adaptive course content embedded in TAWC’s Canvas LMS ensures that students have all of their learning materials at the course start, ten days before students’ first assignments are due. In AY18-19, TAWC has added a low-cost Day One Access fee ($40) to its adaptive courses. TAWC’s streamlined course development model, along with this modest fee, make the implementation of adaptive courses sustainable while providing affordable content to students.
TAWC’s delivery model and commitment to adaptive learning technology has been successful, with a 93% overall satisfaction rate from graduates. Additionally, students have indicated that their adaptive courses have helped them learn the course material better (82%) and that feedback from their instructor and the adaptive course helped them stay on track with the learning outcomes (75%). Overall, via TAWC’s automated outreach plans and alert rules, as well as by selective course redesigns identified via data from its adaptive platform, all of which are powered by its data warehouse, TAWC’s approach has yielded rates for retention (75%), course completion (93%), and student satisfaction (95%) that surpass national averages in the non-traditional, online space.
Bossier Parish Community College
Title: Engaging and Retaining Underprepared, Under-resourced Learners through an OER Mobile-responsive, Gamified Delivery Platform Designed to Leverage Features of Deep Game Structure, Online Retail, Social Media, and Cognitive Applications for Learning
In spring 2017, synchronous, Randomized Control Trials (RCT) were conducted to beta-test the College’s mobile-responsive design. The project’s external evaluator, Dr. Matt Giani, designed the 15-week trials to meet or exceed “What Works Clearinghouse” standards. RCTs were implemented in Beginning Algebra I (Math 098), Beginning Algebra II (Math 099) and Developmental Writing (English 099) in 34 class sections and involved 980 students both at BPCC and Northwestern State University of Louisiana.
Dr. Giani reported that use and access of the courseware significantly improved students’ outcomes for successful completion across all three courses (49). Dr. Giani reports, across the three courses, “students in the treatment group were roughly six percentage points more likely to have passed the course compared to students in the control group (61.6% vs. 55.7%).The treatment group also outperformed the control group in each of the three course numbers. The largest difference was in Math 99, where students in the treatment group were 8.9% more likely to have passed the course compared to the control group. The smallest difference was in Math 98, where treatment students were 4.3% more likely to have passed. For Engl 99, treatment students were 6.2% more likely to have passed the course” (Giani, 49).
Georgia State University
Title: Changing the Paradigm? Creating an Adaptive Learning Course to Improve Student Outcomes and Engagement in Large Section Introductory Classes
We created an adaptive course using RealizeIt, conducted a full-scale pilot in Fall 2017, and then engaged in a full-scale academic study with control groups in Spring 2018. Fall results included DFW rates down nearly 20%. A 29-question survey of roughly 800 students indicated significantly positive and statistically significant student impressions across all substantive questions. In the spring, DFW rates were again lower in adaptive courses (19.1% versus 21.4%), and student perceptive survey results were similarly highly positive and statistically significant.
Turning to student performance, across 36 common test questions given to all sections (700+ students) in the spring, adaptive sections did significantly better than non-adaptive sections, with the strongest statistical significance being in face-to-face classes (converted to a 100-point scale, adaptive students scored 83.58% correct, versus 79.61% correct for non-adaptive students). Interestingly, results were also better in online courses, but did not show statistical significance.
Ivy Tech Community College
Title: Removing Barriers to Student Success with BioBeyond
Impact: During the fall 2017 term, the first term that BioBeyond was piloted, the two sections offered saw a student success rate of 88%, which is defined as scoring a 60% or higher in the course. This is a 46% increase when compared to the average student success rate of 42% in courses not using BioBeyond. The summer 2018 term was the first term that BioBeyond was offered to all BIOL101 DE sections. Preliminary analysis shows a 28% increase in student success rates. Our summer term officially ends on August 1, at which time we will be able to review student retention rate in courses using BioBeyond versus previous courses with other course materials.
Mohave Community College
Title: Increasing Engagement and Access with BioBeyond
Mohave Community College participated in a larger study to examine the final course grades awarded to students in sections that implemented BioBeyond versus sections that did not. Students in BioBeyond-enhanced sections at MCC ended the course with a +0.26 increase on a 4.0 grade point scale compared to counterparts in traditional sections. In the same study, it was reported that online students at Arizona State University had a +0.42 grade point increase and students at Miami Dade College showed a +0.46.
Students in the action-based learning sections at MCC have since shown even greater improvements in terms of overall student success (increasing from 72% to 90%). Student scores tracked on identical exams over the course of six semesters have incrementally improved with each semester as the course format has moved further away from traditional lecture-based lessons to one built on answering the questions that engage and excite students.
Norfolk State University
Title: Learning without Barriers: Creating an Accessibility and Affordability Environment for First Generation Students
Impact: Expensive textbooks can be a barrier for all students. Most Historically Black College and Universities (HBCU) serve a large percentage of underrepresented students and low-income students. Norfolk State University, Barnes and Noble, and Cengage Learning collaborated on the computer science general education course with a specific focus on creating holistic solutions on eliminating the barrier of expensive textbook and the integration of digital courseware. The goal of the collaboration was to create learning without barriers for first-generation students and deliver all digital content directly and seamlessly to our institution’s Learning Management System (LMS) on the first day of class. With the implementation of the model, students saved between 33% and 68% on textbooks and course materials. Results have been encouraging, with pass rates increasing from 60% to 78%, homework scores jumping from 75% to 93%, and exam scores increasing from 70% to 89%.
Portland Community College
Title: Open Resources for Community College Mathematics (ORCCA): Free interactive digital course materials to improve access and affordability for underserved students in the gateway pre-college algebra sequence
PCC students taking our complete basic algebra sequence have had textbook expenses reduced by over $200. At current enrollment levels, the student body is saving roughly $700,000 annually.
Students are engaging more with ORCCA than with previous textbooks:
- ORCCA has embedded Interactive examples and video lessons.
- Students at the end of our sequence are using the same book as students at the beginning, so they are easily able to reference earlier material when needed.
- The low cost of the print book makes it something a student can feel comfortable annotating freely; there is no resale value to preserve.
- ORCCA is immediately available when the course begins and stays available throughout, regardless of financial aid status.
- The ORCCA eBook is born with functional and accessible pages, serving all users, including assistive technology and small screen users.
Faculty are free to make the textbook content match precisely what we intend to teach. There will never be a missing topic (requiring a supplement packet), an extra topic (misleading adjunct faculty to teach more than we expect), or a topic handled with a pedagogical approach we are not comfortable with. The resources that we formerly put into periodic textbook searches will now be put toward refining and enhancing the book that we have. Faculty have already collaborated on in-class assessments, activities, and summative assessments. A significant curriculum revision is underway, made more possible by the flexibility of having our own book.
Portland State University
Title: Active and Adaptive Learning in Statistics: Portland State University Addresses Inequities & Promotes Student Success
In spring 2017, the PSU Office of Academic Innovation identified 19 adaptive software vendors. In summer 2017, the committee chose Realizeit as the best fit for our statistics courses. In fall 2018, a contract was signed with Realizeit outlining a detailed timeline for training and project implementation. Key concepts were identified and organizing into a curriculum hierarchy by unit. This work continued until we launched a pilot section of STAT 241 in January 2018.
Student and faculty feedback plus insights provided by Realizeit during the winter term pilot informed decisions about changes and enhancements to the student experience. From February to March 2018, additional questions were written to augment the opportunity for students to practice and master the concepts. All STAT 241 sections used Realizeit with these changes in spring 2018. This process also revealed the need for a Faculty Guide to help with training new faculty teaching the course.
University of Florida: UF Online
Title: Using IOLab to provide access to rigorous at-home data acquisition labs for the introductory physics sequence, supporting authentic lab practices and collaborative sensemaking of “laboratory” data
The physics team created a set of 14 labs utilizing the IOLab across a two-semester laboratory sequence, which mirrors the content-specific student outcomes for residential labs. The at-home labs match the quality of the measurements taken, the analytical methods used to analyze data, and the LMS supports the communication modalities to bring students together to talk about their unique data sets and make sense of the weekly exercise. Students also explore unique features about their location and use the device to think as scientists to create answerable questions using the suite of onboard sensors and the software interface which communicates to the IOLab using a wireless USB radio transmitter.
The impact of the innovation of writing custom IOLab exercises, which are displayed within the IOLab software, provides students with a tutorial style interface to work towards reaching milestones by answering in-line analysis questions while performing the activities and making sense of the data they are collecting. So it’s not just the tool that is the innovation that removes the barrier for offering physics labs, but how the tool is used to create a coherent student experience that supports student success at achieving the outcomes, while fostering a learning community eager to discuss and provide feedback on each other’s understanding.
The impact of the IOLab is not the ability for students to complete at home data acquisition, but for students to have authentic conversations and experiences motivated by the high resolution of data that they each collect, while working collectively towards an understanding to achieve the student outcomes. Further, the skills practiced by students working with the IOLab match the traditional residential labs and conform to the American Association of Physics Teachers guidelines for outcomes of introductory labs.
University of Mississippi
Title: Addressing Access, Assessment, and College Readiness Gaps in First-Year Composition: Personalized OER Courseware Modules at the University of Mississippi
Survey results following the fall 2017 pilot of roughly 1,000 students revealed that 68.5% felt the courseware contributed to their success on major writing projects. Data also indicated that students used the courseware in various ways without being prompted, indicating the platform is versatile enough to accommodate individual student needs.
Further, this open courseware has substantially reduced textbooks costs for our students. After our success with the pilot, the courseware has been implemented program-wide for the Fall 2018 semester, and our work is being used by at least two other schools.