A Systemic Approach to Quality Assurance for Online Courses

Concurrent Session 6

Brief Abstract

The presentation will focus on describing two quality assurance initiatives that the Teaching & Learning unit within the business school at a large private research university developed to ensure a systemic and integrative approach to quality assurance for continuous online course improvement.


Peter Ariev is an Instructional Design Specialist at the University of Maryland Baltimore County. He has previously served as the Associate Director of Teaching Development at the Johns Hopkins University Carey School of Business and as an Associate Professor of Curriculum and Instruction and Chair of the Education Specialties Department at Loyola University Maryland His scholarship addresses characteristics of teacher education programs, the role of 'performance-based' assessment in teacher development, and the use of teaching portfolios to promote professional development for higher education faculty. His work has highlighted the role of assessment in promoting student learning, the political context of teacher education reform, and the complex challenges underlying institutional change. Dr. Rennert-Ariev was the recipient of the University of Maryland’s School of Education Outstanding New Scholar award in 2009. Dr. Rennert-Ariev’s research has appeared in publications such as Teachers College Record, The Journal of Teacher Education, Teacher Education and Practice, and The Journal of Curriculum Studies and he has presented his work at various educational conferences including the American Educational Research Association, the American Association of Colleges for Teacher Education, the National Media Education Conference, and the International Conference on College Teaching and Learning. Dr. Rennert-Ariev has been involved in mentoring university faculty in developing teaching portfolios and his analysis of the role of teaching portfolios in higher education appears in the recent book The Teaching Portfolio (4th ed) , published by Jossey-Bass.

Extended Abstract

Quality Assurance (QA) has traditionally been confined in concept and practice to ensuring technical quality and viewed from a largely operational standpoint. However, this narrowed scope has limited the usefulness of QA, leaving elements of content, alignment, and overall design and usability unaddressed. This strictly technical focus, combined with a lack of an integrative approach, can lead to QA being implemented only at the end of the course development process, necessarily limiting the effectiveness and sustainability of QA and the resources required for it. Furthermore, the impact of context within all QA processes for instructional design is often not fully and carefully considered and integrated. Traditional models have tended to be more systematic than systemic as designers and behavioral scientists have adopted a minimalist view in which it is assumed that context plays a minimal role in facilitating the attainment of instructional goals. This type of “functionalist” approach to QA may account for why many instructional interventions are not maintained over time. In spite of the informal recognition by instructional designers of the role of contextual forces, there is often little formal consideration of contextual analysis or planning within QA processes.

To overcome these limitations, the Teaching & Learning unit within the business school at a large private research university developed two QA initiatives designed to ensure a systemic approach to quality assurance for continuous online course improvement based on four dimensions: 1) identification and application of the various elements of QA at different points in the online course development process, 2) augmentation of the QA process with context-sensitive dynamics in alignment with institutional goals and needs, 3) development of a comprehensive framework for conducting internal course reviews that promotes course improvements within broader institution-level changes, and 4) construction of a QA framework that defines “quality” and determines what standards and measures should be used to evaluate the quality of online courses.

The presentation will detail two initiatives that enabled the Teaching & Learning unit to take a systemic approach to QA processes and overcome the limitations of a functionalist view. The first initiative consisted of an internal review—based on Quality Matters (QM) standards—of five online foundation courses in the school’s MBA program. This review was designed to evaluate online course quality, assess alignment between online and onsite courses, and systematically develop course design processes that are collaborative and promote successful student learning outcomes. While the review team found that most courses met QM standards or were near compliance, they still found areas for significant improvement in terms of instructional design, technology, and broader institutional contexts. Considering the importance of sensitivity to institutional contexts, the team developed a comprehensive framework for conducting internal QM reviews that encourages dialogue among institutional stakeholders in ways that promote course improvements and systemic institution-level changes. The framework takes QA processes beyond focus on mere compliance and toward collaborative discovery of those context-specific factors that can have significant influence on online course quality. Among the framework’s promoting questions are: How does the institution define quality in online education? What specific metrics or measures are needed to assess quality? How can QA be integrated into course development as an iterative process? What are the quality improvement goals to meet institutional goals, vision, and mission? Does the fulfillment of the standard(s) require institutional supports and resources? What is the optimal balance between standardization and personalization in online programs? The promoting questions helped to create a broader focus beyond compliance and toward a more systemic perspective on instructional design quality.

The second initiative focused on defining quality more specifically and determining what standards and measures should be used to evaluate the quality of online courses. The Teaching & Learning unit created a comprehensive Quality Assurance Framework based on the QM Rubric, OLC Scorecard for online program administration, educational research, and best practices in online course design. The Quality Assurance Framework explicitly identifies the standards that need to be met for an online course to be approved for delivery, and incorporates standards that transcend technical considerations. It includes key course design considerations, including but not limited to: learning objectives, learning activities, assessments, content choice, technology and technical elements, accessibility and usability, branding and editorial standards, and scalability. Having created this framework, the Teaching & Learning unit will next integrate it within the online course development processes, and help instructional designers and faculty to effectively use this framework as a guide to online course design and development, and as a course quality evaluation tool once a course has been developed.

Together these initiatives will be used to show how one institution’s experience of developing a systemic approach to QA may provide a replicable conceptual model that other institutions could use to ensure that QA encompasses both technical review and examination of institutional context factors. This broader view of QA as a systemic and context-based process can help provide a justification for those institutional supports and resources that are needed to sustain online course quality. 

The presentation will include audience engagement and interaction. Attendees will have opportunities to share their insights and feedback on the QA processes at their respective institutions. In addition, the presentation will foster discussion of key context-specific elements within QA processes (including factors such as scalability of online courses, faculty development, and academic support services). Interactive technologies such as PollEverywhere will be used to elicit attendees’ insights and prompt discussion of context-specific examples.



CAST (2011). Universal Design for Learning Guidelines version 2.0. Wakefield, MA: Author.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87–105.

Online Learning Consortium. (2014). Quality scorecard 2014 handbook: Criteria for Excellence in the Administration of Online Programs. Author. Retrieved from https://clear.unt.edu/sites/default/files/Awards_and_Grants/LEG/OLC%20Score%20Card%20-%20Evaluation%20and%20Assessment%20Section.pdf

Standards from the Quality Matters Higher Education Rubric, 5th Edition. Quality Matters. Retrieved from https://www.qualitymatters.org/sites/default/files/PDFs/StandardsfromtheQMHigherEducationRubric.pdf

W3C World Wide Web Consortium Recommendation. (2008). Web Content Accessibility Guidelines (WCAG) 2.0. Retrieved from http://www.w3.org/TR/2008/REC-WCAG20-20081211/