Applying best practices in the design of formative assessment for blending learning environments
Workshop Session 2
This workshop will lay out the assessment best practices for using formative assessments to increase performance in summative assessment in blended learning environments. The implementation of these best practices will be demonstrated in multiple learning management systems to enable participants to incorporate the best practices into their blended courses.
Description and Background
This workshop is based on assessment best practices and informed by prior research of a blended, international academic and professional program, its online assessment methods, and their effects on learning. That previous study had a viable data set of 25,000 students analyzed to determine the effect of formative assessment on summative assessment. The assessments included practice quizzes, chapter tests and summative course tests. The practice quizzes and chapter tests, because of their low stakes, high feedback and ability for retakes from a common pool of items, functioned as formative assessments to the summative chapter tests.
Overall, the findings showed that the positive benefits of testing effect were not evident in the data provided and with the analyses possible (and with significant limitations noted above). The highly (and negatively) skewed means for chapter tests hint at mastery of content yet the three curricula show lower summative assessments means. Since the practice quizzes and chapter test items were from the same item pool designed—with appropriate and documented rigor–for the summative, end-of-course assessment, the data show that longer term (until the end of the course) gains in retention of the material did not occur for most students. This did not mean that students did not learn the material, for their means were in the “A” and “B” letter grade ranges. It meant that the formative strategy did not explain sufficiently why they learned the material or whether they learned the material in an enduring way.
The literature is clear on assessment best practices and that study provides a cautionary tale of implementing a less-effective strategy on scale, which, in online environments, is all too common occurrence.
The negative outcome of the testing effect is easily avoidable and this workshop addresses such remedies. The research literature and that prior study from the presenting team shows that specific assessment strategies in online environments can increased student performance on summative assessments in blended learning environments. Each of these best practices will be explained in detail for the general application in blended learning environments. After establishing an understanding of these practices in a general sense the workshop will show how to apply the practices specifically for a number of the most popular learning management systems. This will include walkthroughs of each system applying the best practices with their unique setups and tools.
Applying the Best Practices in the Classroom
Participants will be able to utilize the information given to them in this workshop to incorporate the assessment best practices into their blended learning courses. They will be shown the steps necessary to enable student performance as well as the reasoning behind the practices so they will be able to fully understand the concepts and why they are applied to blended learning environments.
Participants will learn how to harness assessment best practices to blended learning. This workshop will be focused on coordinating formative, performance, and summative assessment strategies in blended learning. It will also address the practices and configurations for multiple learning management systems in order to achieve these goals.
The primary focus of this workshop is blended learning environments and the best practices for using formative assessments to increase performance on summative assessments. Those interested in blended learning will benefit by gaining insight on the effective and ineffective applications of using practice quizzes and chapter tests as formative assessments in a blended learning environment.
Goals and Objectives
The overall goal is to improve the online assessment practices of blended learning programs. The following learning objectives guide the workshop:
Participants will learn the best practices for using online quizzes and tests as a formative assessment strategy.
Participants will gain practice with designing and configuring online assessments within common Learning Management Systems (LMS).
Participants will use LMS analytics tools to monitor individual and aggregate outcomes to confirm if assessment best practices are impacting student learning.
Administrators, Design Thinkers, Faculty, and Instructional Support
Collaboration and Interactivity
The workshop structure will be as follows:
Assessment Strategies and Best Practices:
Brief description of formative and summative assessment
Discussion of testing effect potentials and pitfalls
Discussion of unique opportunities and threats to assessment as a result of online testing and scale
Introduce key variables that affect performance assessment:
Lag time between instruction and test
Item banks and content validity in pre and post tests
Leveraging content reading and assignments to testing opportunities
Determine the value of the practice tests
Inhibit last-minute test score spiking
LMS Configuration and Practices:
Four LMS platforms will be demonstrated: Merlot, Blackboard, Moodle, and Canvas. Across all platforms (and conceptually any LMS), the following topics will be addressed.
Show how to adjust lag time and date
Provide direct pointers to the reading
Enforced reading of content before assessments
Adjust grading percentages
Show how to adjust test time
Show how to adjust test frequency
The participants in this workshop will learn the assessment best practices for blended learning environments and how to specifically apply them using multiple learning management systems. The participants will be receive handouts and access to online materials that tuned for each learning management system and how the assessment best practices are applied within them.