Are you exemplary, accomplished, developing or deficient? An institution-wide implementation of the Quality Scorecard as an evaluation tool and strategic methodology for enhancing the experience of students studying online.

Concurrent Session 8

Session Materials

Brief Abstract

The Online Learning Consortium’s Quality Scorecard enables institutions to evaluate their provision of online learning across 75 indicators. This presentation explores "what happens next" - how one of the largest community colleges in US moved from assessing quality to an improvement plan mapped closely to accreditation policy and standards.


Terry Di Paolo is Executive Director of Academic Affairs and Student Success at the Dallas County Community College District where he works on major strategic projects and initiatives. He moved to Texas in 2012 having spent a decade as an academic at the UK’s Open University. Terry’s career has focused on curriculum development and learning technologies. He is a specialist in online learning and teaching, distance education with special interests in institutional assessment and the implementation of major strategic projects and activity.

Extended Abstract


This presentation reports on a comprehensive 14-month exercise focused on measuring the quality of online learning at one of the largest community college districts in the US using the Online Learning Consortium’s Quality Scorecard. It provides perspectives and insights from the project leadership team and outlines a plan attendees can use to adopt the Quality Scorecard at their own institution.

The session is organized in three parts, each respectively addressing the following outcomes:

  1. Why apply the Quality Scorecard as a tool in the development of institutional strategy associated with online learning?
  2. How to adapt the Quality Scorecard to assess the quality of online learning across separate organizational units and the institution as whole.
  3. The next steps when it comes to moving from the outcome of a Quality Scorecard assessment to strategic planning that aligns with accreditation and compliance.

Why we adopted the Quality Scorecard

The session focuses on the experience of the Dallas County Community College District (referred to from this point as the District) which comprises 7 separately accredited colleges and various service centers who in 2015-16 serviced approximately 75,000 students. One of these centers, the LeCroy Center, has historically supported the District’s online activity at the same time as making a significant contribution to distance learning across Texas and the US through the development of tele-courses.

In September 2015, a compression planning session organized by the District’s Faculty Association in conjunction with the LeCroy Center gathered approximately 30 faculty and staff to discuss the future of online learning across the District. The event concluded that ultimately there was a need for a unified vision for online learning across the District's network:  a need to determine where things were at in the present to help inform consensus building about a future for online learning across a large, diverse institution.

Online courses have been a feature of the District’s offer for over two decades. Like many institutions, the number of online courses offered by the District has steadily increased and mirrors the growth of the online curriculum at many public institutions over the last two decades. This development can be summarized as a transition from an early organic movement by pioneer faculty to what has become an increasingly common modality for both students and faculty.

In 2015-2016, 57% of students in the District took a fully online course with the DCCCD. Roughly half of those students studied only online courses, the remaining students combining online studies with hybrid and face-to-face classes. In same academic year, the District offered students a catalog of approximately 40,000 sections of which nearly 40% were taught completely online. When it comes to faculty, 34% of instructors within the District (approximately 1200 individuals across the 7 colleges) taught fully online sections in 2015-16.

Descriptions of online learning in the District, like many similar institutions, reflect a complex evolving form of instruction that has grown organically in response to the expansion of online learning systems, the changing needs of a complex student market and the increasing ubiquity of technology in 21st century life. Organic growth of online courses within the District has supported assorted design and delivery approaches: a student wishing to study an introductory psychology courses can choose not only between 7 colleges but also a range of options for studying that course online at a single college. Plurality and choice are a feature of post-modern higher education but the conclusion of the event hosted by the District’s Faculty Association and the District’s lead center for online learning was that the best interests of faculty and/or students would be served best by a more strategic approach to online learning. An approach that focused on delivery high quality online learning suited to the needs of students, the workforce and the communities served by our District in the 21st century. And that’s where the Quality Scorecard came in.

Making the Quality Scorecard meet our needs

The Quality Scorecard is a self-evaluation tool from the Online Learning Commission that lets institutions identify, measure and quantify elements of quality within associated with their online learning. Put simply, the Quality Scorecard lets institutes calibrate the quality of their online learning; but it does so not just from the perspective of what happens in an online class. Instead, the Quality Scorecard casts a much broader lens examining a host of elements that collectively contribute to an assessment of the experience of students studying online.

The Quality Scorecard was developed by Dr. Kaye Shelton and is the product of extensive research and input from a number of seasoned online learning experts and administrators. Originally published in 2010 and revised in 2014 the Scorecard has been tied to The Southern Association of Colleges and Schools Commission on Colleges (SACSCOC) Distance and Correspondence Education Policy Statement and is increasingly being adopted as a tool to aid institutional SACSCOC compliance when it comes to online and distance education.

The Scorecard (delivered as a manual) comprises 75 quality indicators organized across nine categories of institutional activity related to online learning and teaching including: technology infrastructure, student support, institutional strategy, curriculum design and evaluation. Each indicator is supported by comprehensive information designed to guide deliberation and evaluation and is rated using a detailed rubric on a three-point scale: 0= deficient, 1= developing, 2=accomplished and 3=exemplary.

The Quality Scorecard is designed to be used by full online programs but a number of institutions that are increasingly using the tool to assess the online curriculum offer across their entire institution. This was the approach adopted by the District but we are large complex system in the session we describe how we managed each of 7 seven separately accredited colleges to do this for their own college and then how we used that feedback to come up with an overall assessment for our District as a whole.

Reconstituting the Quality Scorecard – from evaluation tool to the basis of a strategic plan

Having completed a 14-month project where we worked through the Quality Scorecard the final part of the session will outline what the Quality Scorecard told us about the quality of the District’s own online learning provision – basically, we’ve got work to do. This final part of the session will also explore how the project team expanded the utility of the Quality Scorecard to schools accredited by SACSCOC by carrying out a deep dive of the way in which quality indicators map to compliance standards. Finally, we talk about a unique next step the project has taken with the Quality Scorecard – we have positioned the Scorecard as the basic of a plan that will help us deliver quality online learning and student success for our District.

Interactive elements in the session

This session details the outcomes of a completed 14 month project and the work that continues after the end of project. Hindsight is a great thing. At various points in the presentation as we detail the history of our work we stop and ask the audience “if this was you doing this at your institution now, what would you do?”