Data vs. the Hunch: Letting Data Lead Innovation in Course Design
Concurrent Session 9
Is there a battle between innovation and best practices? In this session, we demonstrate how student feedback and other data helped confirm some assumptions about redesigning an engineering course, but also opened up new windows to innovation and viewing common problems like engagement and faculty time constraints from different angles.
This session focuses on using qualitative and quantitative data to create innovative redesign solutions for a traditional large lecture undergraduate-level environmental engineering course at NC State University. The project goals were to reduce instructional challenges, improve students’ learning experience, and increase engagement and excitement about environmental engineering. This session will demonstrate how we used baseline student attitudinal data and input from faculty nationwide to guide instructional design and decision making. We show how data can lead to innovation even when it might contradict assumptions grounded in course design best practices.
- Use student feedback to make pedagogically sound, data-driven design decisions
- Analyze qualitative and quantitative data to verify or reject course design assumptions
- Balance innovation with more traditional course design elements
The Redesign Project
NC State University’s Distance Education and Learning Technology Applications (DELTA) office “collaboratively applies expertise in innovative technologies and pedagogies to solve instructional challenges in an efficient, effective, and service-oriented environment, with the overarching goal of helping faculty build student success.” This course redesign project was funded by a DELTA grant, which helps faculty looking for new ideas to develop courses/materials using innovative strategies and technologies, incorporate best practices in course (re)design, and/or explore new ways to engage students with course material.
The project team redesigned a large course, Fundamentals of Environmental Engineering, from a face-to-face lecture to a partial blended learning model designed to improve students’ learning experience as well as increase student engagement and excitement about Environmental Engineering. Key instructional challenges included a diverse student population with low interest/engagement, a disconnect between homework and exams, a wide array of course topics, limited teaching assistant support, and rotating instructors. The project team, including faculty, started this redesign project with certain assumptions about how to redesign this course to accomplish these goals, mainly that a radical change to this very traditional lecture course was necessary to create a positive impact on students.
With the key challenges in mind, the team aimed to explore new technologies and learning practices to improve student engagement/support; increase interest in the field; improve faculty/TA time investments; develop students’ problem solving skills; and improve instructional consistency across semesters, among other goals. However, because DELTA aims to create a culture of inquiry, we created three primary data collection tools to gather data that could help make instructional design decisions by either verifying or rejecting our collective assumptions, and to foster creative solutions that kept in mind faculty time commitment, sustainability, and reusability. This baseline data will be compared to similar data gathered after the course redesign implementation in fall 2018, to assess its impact on student engagement and achievement.
Data-Driven Instructional Design
In addition to course evaluations, registration data, and some faculty anecdotal evidence, we utilized three tools prior to finalizing the redesign plan:
- Classroom Observations: Conducted by the instructional designer and multimedia specialist to provide multiple perspectives on faculty teaching style, student engagement with the class/materials, use of multimedia, students’ learning styles, and course activities.
- Baseline Student Survey: Administered to current students, it gathered feedback on their course experience, perceptions of the required homework and activities, and how much they think they have learned in the course.
- Colleague Survey: Administered to faculty peers nationwide, it asked those teaching a similar course about its place in the curriculum, students’ perceptions of the course, effective teaching strategies, and course redesign.
Working closely with the faculty to reexamine how to present material to students, we used these tools to create a data-driven instructional design plan. In this session, we show how the data we gathered helped confirm some assumptions (e.g., increasing active learning), but also rejected some assumptions and opened up new windows to innovation and viewing common problems (e.g., engaging diverse students) from a different angle, including finding new ways to incorporate established instructional tools. This presentation will outline how to balance traditional teaching methodology (lectures, case studies) with innovation and cutting edge technology (active learning, animation/storytelling, mobile technology).
The presentation will include a combination of crowd polling and table teams, so attendees will be able to share their views and apply what they are learning to a sample redesign context.