Is it Scalable?: A comprehensive pilot process to evaluate new technology

Concurrent Session 4

Session Materials

Brief Abstract

Teaching and Learning Technologies at Purdue University has developed a comprehensive pilot process to evaluate new learning technologies. The pilot planning process, establishing a diverse teaching and learning support team, faculty recruitment, implementation, and assessment strategies will be shared with the participants.


Dr. Debora Steffen is an Educational Technologist at Purdue Teaching and Learning Technologies where she works to identify and pilot new technologies. Debora has a diverse background in higher education, including management of innovative teaching laboratories and stewardship of a broad variety of research, instructional, communication, and computing technologies. She earned a Ph.D. in Analytical Chemistry from Purdue University, has worked as an industrial chemist, and has higher education teaching experience.
Dr. Wanju Huang is a Clinical Assistant Professor of Learning Design and Technology at Purdue University. Prior to joining the LDT program, she was an instructional designer manager at Teaching and Learning Technologies, Purdue Online, where she led a team of instructional designers and video producers for course design and development. She received her Ph.D. in Curriculum and Instruction (with a concentration in Technology) from University of Illinois at Urbana-Champaign. For six years prior to joining Purdue in Fall 2016, she was a lecturer and an instructional designer at Eastern Kentucky University. She has taught a variety of undergraduate and graduate courses (e.g., learning theories and instructional design, educational applications of multimedia, statistics, critical thinking, and social intelligence) in face-to-face, blended, and online formats. Dr. Huang has published and presented research papers related to online learning and instructional design at national and international conferences. Her research interests include: technology for building online communities, online identities and communication, the educational use of augmented reality, and the effectiveness of faculty development programs.

Extended Abstract

Exploration and implementation of new educational technologies at large public research universities can present many challenges. To be “enterprise ready,” a new technology must be scalable across multiple dimensions, including the institutional need, course enrollment size, accessibility, LMS interoperability, and level of required support effort. This presentation will highlight the application of our process on a recent pilot of Gradescope, an online grading tool.

To explore the effectiveness of Gradescope for large-scale implementations, Purdue University launched a pilot project to evaluate the institutional need, LMS interoperability, accessibility, support effort scalability, and teaching and learning impact of the tool. Ten pilot courses had a total of 46 teaching assistants and approximately 1800 students. The effort was led by a diverse team with a goal of providing tailored support to the faculty. This team includes  educational technologists who identify faculty’s technology needs and manage the piloting of potential solutions, two educational technology consultants who provide technical training and support, and one instructional designer who assists faculty implementing instructional strategies in their courses.  The pilot has demonstrated that having a team with a variety of talents/expertise is helpful to obtain faculty adoption of the tool but also to capture more comprehensive assessment of the tool.