Circuit: Rethink Peer Review

Concurrent Session 6

Brief Abstract

Peer review is an established practice that enhances students’ critical thinking skills, but the process can be difficult. With innovative sorting and scoring algorithms, Circuit by Purdue provides flexible and streamlined peer review. Circuit allows users to evaluate images, videos, audio, and text. Stop by and see Circuit in action!

 
 

Sponsored By

Presenters

Christopher Mong is a senior instructional designer at Roosevelt University and a limited-term lecturer at Purdue University. He holds a Ph.D. in Learning Design and Technology from Purdue University and his research interests include problem-based learning, purposeful use of instructional technology, and gamification in education.
Gavin Brown is an Application Developer at Purdue University. He is part of the Studio by Purdue team who has the privilege to create innovative teaching and learning applications to further and enhance college students’ learning. The team's work has been featured by the New York Times, USA Today, CNET, and The Chronicle of Higher Education; and has won awards from Wharton-QS Stars Awards 2014: Reimagine Education, Center for Digital Education, TechPoint Mira Technology Awards, and Campus Technology.

Additional Authors

Margaret Wu manages the Informatics team which creates the technology behind Studio by Purdue. The team is tasked with creating innovative solutions for gaps in the instructional technology market. She holds a Master's of Science in Education in Literacy and Language Education. He research and professional interests include purposeful use of instructional technology, educational assessment, and gamification in education.

Extended Abstract

As educational trends move toward active learning and competency based learning, there is a larger emphasis on higher-order learning and 21st century skills. In many cases, instructors want learners to take on the role of content expert and explore critical thinking skills that require an understanding of jargon, the ability to break down complex ideas to core elements, and the ability to evaluate discipline-specific content with an informed, critical eye. In such cases, peer review is a useful instructional strategy because it affords learners the opportunity to both evaluate the work of their peers and recognize the strengths and weaknesses of their own work by comparing it with exemplary work from peers (National Commission on Writing in America's Schools and Colleges, 2003).

While peer review can positively influence student perception about their learning (Ertmer et al., 2010), it can be an inefficient and confusing process, especially if it directly relates to student grades. In some cases, learners may feel uneasy about the ability of their peers to evaluate their work. This had led some instructors to implement calibrated peer review in which student peer reviewers are calibrated against known examples provided by the instructor (Ruggiero & Harbor, 2012). In such a system, scores provided by students whose scores closely align to the instructor’s examples are given more weight than scores provided by students whose scores did not align.  

 

While calibrated peer review can be used to emphasize the process and rigor of peer review, it can be an extremely complicated and time-consuming process. Purdue’s Circuit reduces some of the burden associated with calibrated peer review by guiding instructors through the process with a linear and straightforward user interface. Automation of due dates for content submission, calibration reviews, review of peers’ work, and self review allow the instructor and student to focus on the content, rather than the calibration process.  

 

Circuit also offers the flexibility to use peer review without calibration and with or without a rubric for lower stakes assignments when students may not be as concerned with the evaluation abilities of their fellow peer reviewers. With a built-in LaTeX editor and the ability for users to upload text, image, audio, and video files, Circuit offers additional flexibility in terms of the types of content instructors may have learners evaluate.

Learning Outcomes

As a results of engaging with our session, attendees will:

  • Identify the different options for Peer Review that Circuit supports

  • Understand how an application like Circuit is an improvement upon the analog peer review process in terms of speed and flexibility

  • List some ways in which peer review could be applied to their practice

Instructional designers, educational technologists, instructional technology adminstrators, and students will benefit from this session.

References

Ertmer, P. A., Richardson, J. C., Lehman, J. D., Newby, T. J., Cheng, C., Mong, C., & Sadaf, A. (2010). Peer feedback in a large undergraduate blended course: Perceptions of value and learning. Journal of Educational Computing Research, 43(1), 67-88.

 

National Commission on Writing in America's Schools and Colleges. The neglected “R”: The need for a writing revolution. New York: College Board. (2003). Available: https://eric.ed.gov/?id=ED475856

 

Ruggiero, D., & Harbor, J. (2012). Calibrated Peer Review (CPR) Effective writing assignments online with much less instructor time..... But what do the students think? In T. Amiel & B. Wilson (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2012 (pp. 981–991). Denver, Colorado, USA: Association for the Advancement of Computing in Education (AACE). Retrieved from

https://www.learntechlib.org/p/40873