How to evaluate your instructional design process and make an impact on your institution.

Concurrent Session 3

Brief Abstract

Instructional design is a growing role in higher education institutions. Instructional designers often feel their role is misunderstood, prompting the question: how can an instructional designer capture the importance of this role for stakeholders?  In this session, we discuss a customizable survey created to evaluate the impact instructional design services has on your institution.

Extended Abstract

Instructional design plays an integral part in educational practices of higher education institutions, and this role is growing (Rubley, 2016). To some faculty, instructional designers and instructional design services are merely an extension of IT, whose people and programs exist to exclusively support the LMS. This presentation will focus on a small instructional design group that has been established in the last 2 years at a private, non-profit university. Since its addition, the instructional design department has served many faculty members in the institution and is striving to reach more. To understand the impact instructional design has had on the faculty since its deployment, we developed a survey that gauges current attitudes and beliefs about instructional design services and instructional designers. The survey findings will serve as the guiding force for future instructional design practices and policies in our department and we hope other instructional design departments in higher education. The presentation presents the preliminary findings of this survey and through discussion gives a framework anyone can use to conduct a custom survey for their instructional design department. 
The Chronicle of Higher Education conducted a survey in 2016 on how instructional designers and faculty members incorporate technology and design courses. The key findings suggested that colleges are using instructional designers, particularly for online and hybrid courses, and that these instructional designers are supporting a wider array of technology. Despite the growing role of instructional designers, there is confusion and many instructional designers feel institutions don’t understand what designers do (Rubley, 2016).  The survey tool was designed to capture the role of instructional designers and the instructional design process with faculty to gain a greater awareness and understanding with stakeholders of the university. The research questions we seek to answer with the survey findings are inspired by Kirkpatrick's model of evaluation, focusing on the levels of reaction and learning, with the intent of gathering some preliminary data regarding behavior and results (Kirkpatrick & Kirkpatrick, 2006).  
Research Questions based on Kirkpatrick’s Model of Evaluation
•    Level 1 Reaction: How satisfied are faculty with the instructional design process?
•    Level 2 Learning: How confident is the faculty member in their ability to develop courses and course materials after working with an instructional designer?
•    Level 3 Behavior: Is the faculty member using what they learned in working with an instructional designer in their other courses?
•    Level 4 Results: Does the faculty member believe that their online or hybrid teaching has improved due to working with an instructional designer?

Survey Design
The survey used in this study incorporates demographic questions from Bliss, Robinson, Hilton, and Wiley (2013), with a modification of Likert-type questions from a survey focused on evaluating the efficacy of a faculty development program for improving the quality of online course design by Hixon, Buckenmeyer, and Feldman (2011). Questions were carefully tailored to the unique aspects of the instructional design support provided in the private institution. The session will share lessons learned in this process and will also illcit discussion on how the survey can be tailored to other institutions.

The survey will be distributed electronically to faculty between the third and fifth week of the semester. Participants will include those who have collaborated in a substantive way with one or more instructional designers in partial or full course and/or program design. Faculty who have consulted briefly with instructional designers, but have not yet participated in the instructional design process, will be omitted at this time.

Results to Discuss
The session presents preliminary data and analyses of the results of the survey. These will include Likert-type scale item results and open-ended question results. Along with these results the implications and possible next steps for our instructional design team will be discussed with participants. 

Participants of this presentation will:
•    Learn how to craft an instructional design survey tool to adapt to their unique departmental and institutional circumstances. The survey tool will be provided to all participants.
•    Discuss and identify the survey findings and research questions that are helpful in evaluation of the instructional design process.
•    Compare and contrast the role and attitudes of instructional design within their institution and identify ways to increase understanding of the process of instructional design with stakeholders.  
Presenters of this presentation will:
•    Gather feedback from attendees for revision of the instructional design survey tool and research questions.
•    Identify ways to further develop the instructional design process of their department.

Bliss, T., Robinson, T. J., Hilton, J., & Wiley, D. (2013). An OER COUP: College teacher and student perceptions of Open Educational Resources. Journal of Interactive Media in Education, 1-25.
Hixon, E., Barczyk, C., Buckenmeyer, J., & Feldman, L. (2011). Mentoring University Faculty to Become High Quality Online Educators: A Program Evaluation. Online Journal of Distance Learning Administration, 14(4). Retrieved from
Kirkpatrick, D. & Kirkpatrick, J. (2006). Evaluating the training programs: The four levels (3rd Edition), San Francisco, CA: Berrett-Koehler. 
Rubley, J. N. (2016). Instructional designers in higher ed: Changing the course of next-generation learning. Chronicle of Higher Education. Retrieved from