Using Evaluation of Online Teaching Data to Make Meaningful Change
Concurrent Session 9
You’ve completed your faculty evaluations of online teaching. Now what? Presentation goes beyond the authors’ book, Evaluating Online Teaching, to discuss how to utilize evaluation data to make meaningful changes in professional development, faculty evaluation processes, and retention of adjunct faculty teaching online.
The growth of online learning has created an opportunity to re-examine teaching practices through a scholarly lens. The review and evaluation of teaching practices in general are sometimes performed in a pro forma fashion, or only for summative reasons like promotion and tenure decisions. Donna Ellis at the University of Waterloo sees teaching-evaluation as a holistic enterprise: “teaching and its assessment should . . . be seen as scholarly activities. The review of teaching is an intentional process—one that is carefully designed, situated in context, and leads to interpreting teaching effectiveness based on multiple sources and types of evidence” (2012). Online courses offer us a rich variety of information sources from which to study and improve our teaching practices and develop our faculty.
Once colleges and universities have identified the context-specific perspectives and factors that they want to measure in their online instructors (cf. Berrett, 2017; Jacob, Stange, & De Vlieger, 2017; and Mandernach, Donnelli, Dailey, & Schulte, 2005), they design policies and instruments that shape the process of evaluating online teaching (Taylor, 2010; Tobin, Mandernach, & Taylor, 2015). This, however, is only one end of the evaluation “pipeline.”
While evaluation of teaching is widespread, effective use of evaluation data (beyond its impact on the individual faculty member being evaluated) is not. Complicating the matter is the complexity that the online format and increased reliance on geographically-remote, part-time to teach in this modality. Participants in this session will leave knowing how to
- how to identify and isolate evaluation factors that can inform more effective evaluation;
- train department chairs, deans, and other senior leaders to move beyond “I know it when I see it” assessment techniques to create more proactive and useful evaluation processes, and
- determine best practices for finding, onboarding, and retaining quality online instructors as a function of evaluation outcomes.
The session facilitators literally wrote the book on how to do this effectively. Evaluating Online Teaching was published in June, 2015, and this session provides beyond-the-book updates in three critical areas: professional development, administration support, and instructor hiring and retention.
Participants should have experience or responsibility for designing, teaching, or evaluating online courses. No specific pre-requisites are required to participate, but please be prepared for a hands-on session. Bring a laptop or other Internet-browser-capable device, since you'll be evaluating sample online courses.
Approach & Outcomes
Participants in this session should come prepared to be part of the conversation. Please bring an Internet-browser-capable device (e.g., tablet or laptop) to the session. During the session, you will
- learn strategies for making data meaningful (using evaluations of online teaching to improve teaching and learning);
- practice how to train administrators to look at online courses effectively; and
- define quality practices on online teaching to attract, select, and retain skilled online instructors.
References & Further Reading
Berrett, D. (2017, May 9). Students Don’t Always Recognize Good Teaching, Study Finds. Chronicle of Higher Education. http://www.chronicle.com/blogs/ticker/students-dont-always-recognize-good-teaching-study-finds/118274.
[Ellis, D.] (2012). Peer review of teaching: A holistic approach to the review of teaching. University of Waterloo, Centre for Teaching Excellence. https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/professional-development/reviewing-teaching/peer-review-teaching.
Jacob, B. A., Stange, K., & De Vlieger, P. (2017). Measuring up: Assessing instructor effectiveness in higher education. EducationNext 17(3). http://educationnext.org/measuring-up-assessing-instructor-effectiveness-higher-education/.
Mandernach, B. J., Donnelli, E., Dailey, A., & Schulte, M. (2005). A faculty evaluation model for online instructors: Mentoring and evaluation in the online classroom. Online Journal of Distance Learning Administration 8(3). http://www.westga.edu/~distance/ojdla/fall83/mandernach83.htm.
Ryalls, K., Benton, S., & Li, D. (2016). Response to “Zero Correlation between Evaluations and Learning.” IDEA Editorial Note #3. http://www.ideaedu.org/Portals/0/Uploads/Documents/Response_to_Zero_Correlation_Between_Evaluations_Teaching.pdf.
Shreckengost, J. (2013). Proactively guiding instructor performance through the use of a performance dashboard and real-time data. Presentation at the Sloan-C International Conference on Online Learning, Orlando, FL.
Taylor, A. H. (2010). A Peer Review Guide for Online Courses at Penn State. http://facdev.e-education.psu.edu/evaluate-revise/peerreviewonline.
[Taylor, A. H.] (2011). Faculty competencies for online teaching. Penn State Online. Faculty Engagement Subcommittee. https://www.e-education.psu.edu/files/OnlineTeachingCompetencies_FacEngagementSubcommittee.pdf.
Tobin, T. J. (2004). Best practices for administrative evaluation of online faculty. Online Journal of Distance Learning Administration 7(2). http://www.westga.edu/~distance/ojdla/summer72/tobin72.html.
Tobin, T. J., Mandernach, B. J. & Taylor, A. H. (2015). Evaluating Online Teaching: Implementing Best Practices. San Francisco: Jossey-Bass.