We Need Efficacy to Drive Outcomes

Eduventures | October 20, 2015 Last week, Eduventures attended the 21st Annual Online Learning Consortium (OLC) International Conference in Orlando, FL. The event brings together a combination of faculty members, instructional designers, administrators, and technology and service providers focused on online and distance education.

One big announcement leading up to the event was that OLC received a $2.5 million grant from the Bill & Melinda Gates Foundation. The grant will help expedite the adoption of next-generation digital courseware solutions at higher education institutions in order to improve outcomes in general education courses, particularly for disadvantaged and underserved student groups. It will accelerate the expansion of the OLC Quality Scorecard and fund a competition recognizing and rewarding exemplary institutions and faculty for the effective use of digital courseware.

On the one hand, the grant presents a compelling opportunity; the budget to purchase digital courseware is one of the biggest challenges facing institutions that are seeking to lower costs, increase access, and improve student outcomes. On the other hand, it raises questions about the definition of efficacy and effectiveness for the competition. It also raises the question of whether this approach can be the catalyst for impacting success among disadvantaged and underserved student groups.

How do we define efficacy?

While everyone talks about the need to measure the outcomes of edtech solutions, there is very little being done to truly address efficacy. While there are a few initiatives in place, mostly in K-12, they are still in the very early stages. For the OLC competition, there will likely have to be a wide range of definitions of efficacy. These may vary based on an institution’s level of maturity, the scale of the problem being addressed, and the size of the impact made. These definitions might include:

  • Successful implementation and integration with legacy systems and a new digital courseware solution.
  • Providing a more cost-effective general education course for students with financial challenges.
  • Generating insights from data created within digital courseware using predictive models to better identify at-risk students.
  • Proactively delivering adaptive content to students in general education classes to enable them to address their own success.
  • Proving that a digital courseware solution works by moving the needle on retention rates, persistence rates, and graduation rates among underserved populations.
  • Sustaining the year-over-year impact of student success by continuously improving retention rates, persistence rates, and graduation rates.

While these are all valid definitions of what efficacy could mean, only the last two definitions get to the root of the problem: outcomes. Even then, I would assert that the timeline to demonstrate these outcomes is too long and the level of measurement is still far from the validity of a randomized control trial, the gold standard for demonstrating effective practice.

CONTINUE READING to discover what it will take to address student success, the state of the solution landscape, and the problem for both providers and higher education leaders...