Fixer Upper: Applying Evidence to Renovate an Online Course Development Process

Concurrent Session 3

Session Materials

Brief Abstract

This session will present the iterative evolution and lessons learned from design, to evaluation, to renovation of a faculty online and hybrid course redesign program. A solid foundation provides the necessary framework to support program modernization.

Presenters

Lead Presenter: Thomas Penniston, University of Maryland - Baltimore County

Dr. Penniston has been involved with online and blended learning in different capacities, including as a student, instructor, builder, and administrator, for the past decade. He has extensive experience with quantitative, qualitative, and mixed methods designs. He provides consultation to faculty and staff on working with institutional data, gathering data, and conducting data analyses. Dr. Penniston also evaluates DoIT outcomes and survey feedback to inform empirically based best practices, spearheads and administers screencasting use at the university, collaborates on institutional predictive analytics projects, and supports faculty course hybridization in alignment with Quality Matters Standards.

Co-presenter: Mariann Hawken, University of Maryland - Baltimore County

As eLearning Manager, Dr. Mariann Hawken oversees several Blackboard applications at University of Maryland, Baltimore County (UMBC) and provides support for faculty course development activities. With more than 20 years of experience in educational technology, Mariann is a Blackboard MVP and a certified Peer Reviewer with Quality Matters. Past activities include distance education policy development and comprehensive faculty training programs for online/hybrid course redesign.

Co-presenter: Holly Owens , University of Maryland - Baltimore County

Holly Owens is an Instructional Technology Specialist at the University of Maryland, Baltimore County (UMBC). Holly started her education career as a K-12 educator and her love for technology led her into her current role as an Instructional Technology Specialist in higher education. She is the team’s clicker expert and also coordinates training opportunities for UMBC faculty and staff. Holly serves as the secondary point of contact for Blackboard administrative tasks. Holly has an M.Ed. in Instructional Technology and a Masters in Distance Education. Holly has a passion for online course design and helping others gain their 'tech confidence.'

Co-presenter: Sherri Braxton, University of Maryland-Baltimore County

Dr. Sherri N. Braxton is the Senior Director of Instructional Technology at UMBC where she is responsible for leading the Division of Information Technology’s (DoIT) strategy for end-user support of instructional technologies including online, hybrid, and traditional, “face-to-face” technologies. With over 20 years of experience in traditional classroom instruction and adult education strategies grounded in instructional design models, she also possesses over 17 years of experience using learning technologies in higher education settings, including the design and facilitation of online and hybrid courses. Dr. Braxton is a dynamic presenter known for her ability to engage audiences and capture their attention, even for highly complex topics. She collaborates with her staff to devise learning opportunities delivered in multiple modes that meet the varied and shifting needs of both UMBC faculty and students. Dr. Braxton is also the DoIT representative on the University System of Maryland (USM) Academic Transformation Advisory Council, a group spearheaded by the William E. Kirwan Center for Academic Innovation. Dr. Braxton has crafted a national presence through her participation in educational technology associations like EDUCAUSE, the Online Learning Consortium (OLC), and the IMS Global Learning Consortium; in addition to presenting at national, regional, and local conferences, she serves as a proposal reviewer, constituent group leader, leadership institute faculty, and both task force leader and working group participant. Dr. Braxton earned a Doctor of Science in Computer Science with Minors in Educational Leadership and Management Science from the George Washington University. She also holds a Master of Science in Computer Science with a Math Minor from North Carolina State University and a Bachelor of Science degree in Mathematics with a minor in Computer Science from Wake Forest University.

Tags

Development and Support Analytics Blended Development Faculty Faculty development Online Online course development Professional development Support Training

Extended Abstract

Goals
The goal of this panel session is to describe UMBC’s faculty online and hybrid course redesign program, discuss training and support successes, and highlight quantitative outcomes informing program redesign.
 
Description
This presentation will focus on hybrid and online course redesign training, support and evaluation. When it is rigorously evaluated and replicated, our institution has found course redesign to be one of the most scalable ways to improve student academic success.  We will share lessons learned in the iterative process of establishing, evaluating, and redesigning a faculty course design program. Specifically, we will describe UMBC’s original Alternate Delivery Program, discuss how analytics has informed effective practices, and detail how identifying, supporting, and promoting best practices within and among the faculty directed the redesign of this program.

History: Established in 2005 through a partnership between campus IT, Continuing Education and Faculty Development, the ADP is also a useful laboratory for exploring and applying learning analytics, which UMBC has pursued for several years. Since fall 2006, the Alternate Delivery Program (ADP) has provided financial, technical, and pedagogical support to 60 faculty members to redesign an existing face-to-face (F2F) course for online or hybrid (part online, part F2F) delivery during the compressed, intensive summer or winter session. The number of hybrid and online courses offered in the special sessions has grown steadily, and currently, 30 – 40% of the courses scheduled in summer or winter sessions are hybrid or online. In response to this increased demand and the need to assure design quality, program administrators have redesigned the training and support structures, beginning during the spring 2015 term.

Supporting: UMBC’s Instructional Technology Department is currently comprised four Specialists under the Director and a rotating contingency of student interns. Specialists provide administrative support for the learning management system (Blackboard), consult faculty on best practices for integrating technology into pedagogy, conduct analytics and coordinate campus-wide assessment and evaluation projects, and identify and train both faculty and staff on new and emerging technology tools and strategies.

Evaluation: The most recent program evaluation shows a statistically significant link in student academic outcomes associated with enrollment in courses instructed by teachers who participated in the ADP program.  Using a large, student-level sample (250k records) with robust controls, this session presents the positive, statistically significant (p<.001) relationship between the faculty ADP training program and student academic outcomes. The data indicate that student LMS activity, a proxy for student engagement, is higher for courses that have undergone redesign than those that have not. Additionally, using a large, student-level sample with robust controls, this presentation describes a positive, statistically significant relationship between the ADP training program and student academic outcomes (i.e., Term GPA, Cumulative GPA, Term Credits Earned, and Cumulative Credits Earned), even when controlling for student LMS activity).

Redesign: Through evaluation and consultation of the current literature outlining best practices, we determined that ADP program needed adjustments to address the varying knowledge and skill needs of our program participants in order to assure quality of the redesigned courses. In turn, the program has been redesigned to include  Skill Builder, Course Designer, and Quality Booster tracks.
The Skill Builder track offers modular training in core components of Blackboard and supporting technologies (e.g., Bb Collaborate, TechSmith Relay, Box) to help faculty better utilize the features available in Blackboard and these other tools and to prepare for the Course Designer track.

The Course Designer Track, which most closely resembles the original ADP program’s design, is available to faculty who have either completed the ADP Skill Builder track or demonstrated competency with the Skill Builder components. The Course Designer track typically requires at least a semester for course development before the redesigned course is offered in summer or winter session. It includes a two-day training workshop and an opportunity to work one-on-one with an Instructional Technology Specialist in the Division of Information Technology (DoIT). Faculty who participate in the Course Designer track should have prior experience teaching in summer or winter sessions and convert courses that have a proven enrollment track record in summer or winter sessions.

The Quality Booster track is designed for faculty who are already offering hybrid or online courses, but who have either not previously participated in formal course redesign training through the ADP or participated in ADP prior to the Quality Matters (QM) process integration. Quality Matters is an internationally recognized peer review process to assess the design quality of an online or hybrid course. The goal is to carefully evaluate the components of a hybrid or online course to ensure that it meets nationally identified quality standards.

We are developing mixed-methods strategies to identify design strategy differences between old and  new course redesign programming to further refine best practices through this iterative process.  Anyone involved in providing faculty development support that must address instructional technology and hybrid or fully online course delivery strategies should attend. Slides, handouts, and related website links will be provided during the presentation and posted on the conference web site.