The Potential and Challenge of Adaptive Learning: Continuing Results from a Pilot Study

Concurrent Session 4
Streamed Session

Watch This Session

Add to My Schedule

Brief Abstract

We discuss a three-year pilot study of adaptive learning--identifying prototype adaptive learners, investigating the impact on students and faculty, and examining its potential for improving success rates of underserved populations. Also discussed are robust positive findings and possible challenges with adaptive learning and future directions for research and evaluation. 


Patsy Moskal is the Director of the Digital Learning Impact Evaluation in the Research Initiative for Teaching Effectiveness at the University of Central Florida (UCF) where she evaluates the impact of technology-enhanced learning and serves as the liaison for faculty scholarship of teaching and learning. In 2011 Dr. Moskal was named an OLC Fellow in recognition of her groundbreaking work in the assessment of the impact and efficacy of online and blended learning. She has written and co-authored numerous works on blended and online learning and is a frequent presenter on these topics. Patsy's co-authored book--Conducting Research in Online and Blended Learning: New Pedagogical Frontiers--with Dziuban, Picciano, and Graham, was published in August 2015. She currently serves on the OLC Board of Directors.
Charles Dziuban is Director of the Research Initiative for Teaching Effectiveness at the University of Central Florida (UCF) where has been a faculty member since 1970 teaching research design and statistics and is the founding director of the university’s Faculty Center for Teaching and Learning. He received his Ph.D. from the University of Wisconsin. Since 1996, he has directed the impact evaluation of UCF’s distributed learning initiative examining student and faculty outcomes as well as gauging the impact of online, blended and lecture capture courses on the university. Chuck has published in numerous journals including Multivariate Behavioral Research, The Psychological Bulletin, Educational and Psychological Measurement, the American Education Research Journal, the Phi Delta Kappan, the Internet in Higher Education, the Journal of Asynchronous Learning Networks, and the Sloan-C View. His methods for determining psychometric adequacy have been featured in both the SPSS and the SAS packages. He has received funding from several government and industrial agencies including the Ford Foundation, Centers for Disease Control, National Science Foundation and the Alfred P. Sloan Foundation. In 2000, Chuck was named UCF’s first ever Pegasus Professor for extraordinary research, teaching, and service and in 2005 received the honor of Professor Emeritus. In 2005, he received the Sloan Consortium award for Most Outstanding Achievement in Online Learning by an Individual. In 2007 he was appointed to the National Information and Communication Technology (ICT) Literacy Policy Council. In 2010, Chuck was named an inaugural Sloan-C Fellow. In 2012 the University of Central Florida initiated the Chuck D. Dziuban Award for Excellence in Online Teaching for UCF faculty members in honor of Chuck’s impact on the field of online teaching and learning. In 2017 Chuck received UCF’s inaugural Collective Excellence award for his work strengthening the university’s impact with the Tangelo Park Program and assumed the position of University Representative to the Rosen Foundation Tangelo Park and Parramore programs.

Extended Abstract


Arguably, adaptive learning is one of the most currently scrutinized innovations in higher education because it offers considerable potential for enhancing student success. At the same time, however, this approach to teaching and learning involves a number of challenges with respect to curriculum and course design, university organization, learning assessment, and faculty roles. The University of Central Florida (UCF) is in the process of conducting a three-year pilot study of adaptive learning and its impact on students and faculty, identifying prototype adaptive learners, as well as assessing its potential for improving the success rates of underserved populations. In this session, the presenters summarize robust and positive findings, in addition to sharing challenges we have found with the adaptive learning approach. We conclude with future directions for research and evaluation for this innovative educational resource.

The New is Not So New

As a learning principle, adaptiveness have been a mainstay in disciplines ranging from mathematics and science and technology, to the social sciences. Even a brief literature search will verify that many funding organizations, scientists, philosophers, psychologists and educators contend that the response cycles and feedback loops of adaptive systems’ incremental approaches enhance the prospects of successful problem solving, product development, invention, and learning (Dziuban, Moskal, Cassissi and Fawcett, 1916). John Carroll (1963) developed fundamental principles of adaptive learning when he showed that learning should be considered a function of actual time spent and time needed. He was adamant in pointing out that time spent is not defined by elapsed time. His paper—that is clearly an economic approach to learning—gave rise the often cited principle that if the time a student spends learning (actual time) is constant, then then learning will be a variable. If learning is a constant, for instance, in competency-based configurations, then the amount of time spent by the student on a topic is the variable. Essentially, this little equation presents two fundamental problems. First, it turns long held principles such a fixed school year completely upside down, and second, it challenges us to accurately determine how much time a student actually spends learning. Possibly more important, however, it focuses our attention to the fact that time is a key factor in the learning process-something that is clearly implicit but not really explicated in many educational models. Time has not always been as straightforward as we assume today. For instance, consider this quote by Steven Johnson: “If you were asked what time it was 150 years ago, you would have received at least twenty-three different answers in the state of Indiana, twenty-seven in Michigan and thirty-eight in Wisconsin” (Johnson, 2014). That problem was not solved until the development of time zones. This paradox of definition and time assessment has clear implications for education and especially adaptive learning that is in essence a time-based model. Norberg, Dziuban and Moskal (2011) suggest a rudimentary shift in the climate for both students and teachers, because the boundaries of the classroom will disintegrate where the primary source of information is not being the instructor. From a time perspective, a course can have a specified beginning but no formal ending.

Emanating from an altered view of learning time made possible by adaptive learning, several challenges arise:

  1. What do we do with semesters (or quarters) when a student completes a course early or needs additional time?
  2. What is the appropriate assessment model?
  3. Are courses necessary or can learning be module and objective-based?
  4. Is there an opportunity for insight learning in the adaptive platform?
  5. Is adaptive learning overly deterministic?
  6. What are the optimal models for learning assessment?
  7. How will faculty roles evolve?
  8. What is the future of physical space in adaptive learning?

Two exciting possibilities for adaptive learning

Mullainathan and Shafer (2013) highlight a vitally important problem in higher education for which adaptive learning can be effective. The first is scarcity. Quite simply, scarcity is having far more needs than resources. Consider the lives of students living at or near the poverty line. They may be working two jobs therefore can only take a partial course load. Health care costs stretch them even further. There may be childcare expenses in addition to all of their educational costs.  Transportation can be a problem because public transportation creates a time pressures but a car simply costs too much.

Scarcity is delicate, requiring a continual balancing act that forces the student into a situation where he or she must tunnel—concentrate on the immediate problem and ignore everything else, including their studies. However, in an adaptive learning course, with modules that encourage a students to operate on their own time, the scarcity pressure are greatly reduced.

Real-Time Predictive Analytics

Predictive analytics mechanisms have some strong correspondences to adaptive learning. They identify students who are at risk early in their courses and can offer guidance to those students. One principle is well understood: there are critical academic points in students’ lives that if successfully completed create a greater opportunity for completing a course of study. . The operational assumption is that the earlier problems can be identified the better the student’s chances become.

With the advent of adaptive learning there appears be a new opportunity for identifying students that are in danger in their courses: adaptive analytics. Most adaptive learning systems function in real time creating immediate assessment procedures, and providing instantaneous feedback while offering guidance to students. Because they begin assessing students immediately students have a much better chance of success

This Study

The presenters describe the results of three years of research on adaptive learning initially concentrating on student reaction to this learning environment including its perceived effectiveness, guidance structure, time spent, technical problems, feedback effectiveness, peer-to-peer interaction, assessment accuracy, degree of personalization, engagement, and willingness to continue with adaptive learning In addition the presenters will outline a working relationship with the research unit of the adaptive platform provider (Realizeit) to develop the underlying factors of student outcomes derived for system-derived indices. They will show models of how students navigate the adaptive learning environment and the prototypes of adaptive learners they have identified. Finally, they will discuss their findings regarding the concept of adaptive analytics and the possibility of identifying students that may be academically at risk of non-completion early on in a course.


This study was conducted to better understand adaptive learning and how it differs from and can augment other course modalities. Understanding how students function in a learning environment with expanded learning latitude is fundamental to understanding this approach.  Additionally important is discovering difficulties and needed adjustments if effectiveness is to be achieved. Can struggling students be identified? What outcomes are most informative for adaptive learning courses and how are they generally accepted? What pedagogical assumptions face challenges in adaptive learning? As with many pilot studies, issues and consequences emerge. At the underlying evaluation level, the latent trait analysis suggests that students evaluate their adaptive learning experience in much the same way that they do other course formats—design, delivery, learning facilitation, accurate assessment, understanding the rules of engagement and filtering out distraction and noise.

Given students’ control over how, when and for how long they approach learning, this modality appears to have been effective for a large majority of students—most of whom were successful. The top performing students showed learning characteristics that were very similar to the class as a whole. A number of them took the opportunity to review and revise their work and some finished the course early. Others, however, had to be reminded of pending deadlines as the course still “fits” within a traditional semester. This leads the investigators to conclude that truly progressing to adaptive learning requires a period of adjustment for students (and faculty) and may take some time as well as a change in mindset. The investigators speculate that this approach has an embedded analytics component and that assessment measures external to the course may not be necessary to predict those at risk. We discuss how this is an important consideration since a major issue in analytics involves designing effective interventions which can be facilitated by the practice opportunities provided by the adaptive learning system


Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64, 723-733.

Dziuban, C., Moskal, P., Cassisi, J., & Fawcett, A. (2016). Adaptive learning in psychology: wayfinding in the digital age. Online Learning, 20(3).

Johnson, S. (2014). How we got to now: Six innovations that made the modern world. New York, NY: RiverHead Books.

Mullainathan, S., & Shafir, E. (2013). Scarcity: Why having too little means so much. Picador.

Norberg, A., Dziuban, C. D., & Moskal, P. D. (2011). A time-based blended learning model. On the Horizon, 19(3), 207-216.