Last week, I read Dr. Fowler’s blog post, Student Learning Outcomes: A Primer,  and was immediately drawn to the ease in which he describes different types of Student Learning Outcomes (SLOs).  Dr. Fowler’s post provides an excellent primer on the types and interconnectedness of SLOs.  If Dr. Fowler’s post has you intrigued, you might consider reading Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education (Walvoord, 2010) and Assessing the Online Learner: Resources and Strategies for Faculty (Palloff & Pratt, 2009) or joining OLC in our Creating Rubrics workshop.  I encourage you to continue to read the OLC blog, OLC Insights, as well as the SNHU blog, Academically Speaking.

Student Learning Outcomes: A Primer
Dr. Gregory Fowler
Originally posted on Academically Speaking on March 2, 2016

Everyone has been in conversations or situations in which it is clearly assumed that you know what is being talked about, and you don’t but feel you should. Rather than look stupid you say nothing and wait for the opportunity to inform yourself for “the next time.”

One area in higher education in which this happens a lot is when talking about student learning outcomes. When talking to colleagues at other institutions, I often get that furrowed brow of confusion when we wander into territory about what students should know, show, do and be upon completing their courses or programs and how they are measuring across all course sections and instructors to ensure those outcomes are being met.

Eventually the assertion is made that since instructors are experts in their fields, they inherently are capable of (a) designing courses in which the majority of their students will be successful and (b) accurately assessing student performance on … wait for it … whatever outcomes they individually have designed for the course. If you are on a campus where students are still looking for the “easiest” professor for a given subject, I might be talking to you.

Critical Outcomes
In the scenario I just mentioned, the environment is focused on the instructor. And while it is quite likely that the instructor is very good in his or her field, unless there is some general standard by which all sections of a course are explicitly measured, there is no way to assess whether students across sections are succeeding and no way of systematically determining where, if the student learning environment is failing the student, such failures are occurring. Is it the student, instructor or learning resource that needs to be reviewed?

That is why outcomes are so critical. One of my mentors says, “If you aren’t measuring, you are just practicing.” Without clear outcomes it’s not even practice, since no ground rules have been established. Simply put, well-defined student learning outcomes combined with valid assessments enable an institution to ensure that regardless of who is teaching the course, all students who succeed have demonstrated the stated performances. Once this occurs students focus not on who the easiest instructor is but rather on which instructor will be most effective in helping them to meet the course and program outcomes.

SNHU’s College of Online and Continuing Education (COCE) does not explicitly use the term competencies (though many in the field would); rather we refer to clearly defined and measurable outcomes of what students have learned in the course or program. While outcomes at both levels must be clearly defined, unique and measurable, the level of granularity distinguishes them.

Program Outcomes
Program outcomes are drawn directly from the mission of the institution as well as the conceptual framework for the college that sponsors the degree program. Student mastery of program outcomes is generally evaluated through a body of evidence that accumulates over the course of a program. This is a critical element of outcomes-based education: Rather than assuming that a singular assessment at a given moment in time should determine learning mastery, COCE assesses demonstrated student success at multiple places throughout a course/program through various scenarios to ensure that rote memorization or calcified mental processes are not misinterpreted by evaluators as mastery. Often a capstone experience is also used to holistically evaluate program outcomes at the end of a program.

COCE uses this model for designing and assessing course and program requirements. Program outcomes specify the enduring knowledge, skills and dispositions that graduates will have demonstrated upon completion. Program outcomes may have anywhere from four to 10 course outcomes tracked throughout the curriculum that, as an aggregate, represent the more precise skills and abilities students will need to have learned in order to demonstrate proficiency. Program outcomes focus specifically on the knowledge, skills and dispositions that a first-day-on-the-job practitioner should possess in order to succeed in his or her chosen profession.

These outcomes are derived from a series of workshops bringing together academicians and practitioners in the given field of study to provide clear definition and valid assessment of any given outcome. At the program level a student must demonstrate proficiency as defined for the area of study in the overall course outcomes. For any given program outcome a student must demonstrate proficiency in the course outcomes aligned to the program-level outcome.

Course Outcomes
Course outcomes in a program are defined as specific, industry-relevant performances that are mapped back to the program outcomes. At the course level, outcomes have four characteristics. Course outcomes are descriptive, integrated, holistic and enduring:

• Descriptive: They describe the specific dimensions of what a student should know and be able to do, and what dispositions they should embody.
• Integrated: They integrate multiple dimensions of performance. Outcomes are not simply stated as: “student follows x procedure to create product y.” Rather, outcomes are stated in terms of complex performances acknowledging that there may be many solutions, procedures or skills students can integrate as they solve real problems.
• Holistic: Related to the above, outcomes are written to encompass the comprehensive aspects of a performance, identifying the specific criteria under which a performance is “successful.”
• Enduring: Generally speaking, the ultimate aim of any educator or educational institution is to help students develop traits or qualities that remain with them beyond graduation. Outcomes are written to apply to a variety of settings, not just to accomplish a specific task in a course.

Course outcomes are perhaps best thought of as stepping stones toward program outcomes. Students must provide an exemplary or proficient demonstration in the critical elements aligned to the course outcome to show proficiency in that course outcome. Each critical element of the course summative assessment is aligned to the course outcome to which it is explicitly related:

• As a rule, a course outcome should always be written to encompass higher-order learning, rather than a simple inventory of concepts that students should “know.”
• Course outcomes include the action to be performed, the criterion for success, and any essential context or parameters for the students’ demonstration of mastery.
• An undergraduate course should include four to seven outcomes while a graduate course might include five to eight outcomes.

Course outcomes, like outcomes in general, should include the action to be performed, the criterion for success and any essential context or parameters. The combination of these three elements helps ensure that the outcomes are context specific, measurable and more integrated.

Consider the example presented below. The first (non-example) outcome specifies the desired performance but lacks any specificity, criteria or qualification. Consequently, the outcome is context-independent, making it difficult to implement or assess. The second (example) outcome presents an action in context. It includes the performance and the criteria, and it qualifies the performance within a specific context.

Meanwhile, learning objectives within courses are prescriptive, one-dimensional and discrete. They specify a particular behavior or action that is intended to produce a desired result. Objectives indicate a path or process by which learning will occur. Only in conjunction with other learning objectives and within the context of one or more outcomes can competency be demonstrated. In this model, objectives are the stepping stones toward competency and so (a) prescribe only one construct or behavior, and (b) are not dependent on and do reference to other learning objectives.

Because of the emphasis on measurability, learning objectives should be uni-dimensional. That is, they should specify one and only one behavior or action. In the chart below the learning objective on the left specifies more than one action, while the objective on the right is uni-dimensional. In most cases in which two constructs are included in one learning objective, the higher order behavior is preferred because it will subsume the lower-order behavior.

While a learning objective prescribes a path for mastering the course outcomes, it is not the only possible path. Individual students may find success through alternative paths depending on their environment or previous experiences with the content, learning styles or individual situations. Consequently, learning objectives are also the most malleable and flexible of all of the learning goals.

Because course outcomes are broad and can be deconstructed into a nearly infinite number of possible objectives, the choice of which objectives to use is crucial. They are selected based on their criticality to mastering the course outcome and the degree to which they accurately reflect real-world situations in which students would employ those skills. COCE designs at least three to seven learning objectives to support each course outcome.

Transparency, Accountability and Success
When implemented in consultation with faculty and other stakeholders, outcomes should increase transparency and accountability for both students and instructors, as both now know what is being expected and how performance is being evaluated. Rather than restricting the instructor this should be seen as liberating, enabling them to bring all of their creativity, empathy and expertise to bear in finding ways to impact and engage the students in the classroom.

One of my favorite examples is the movie “The Karate Kid,” in which Daniel LaRusso’s eventual success depends on Mr. Miyagi’s ability to find unconventional ways to help his student ultimately meet the same outcomes as his antagonist, who has all of the conveniences of a traditional dojo. Take one look at the final scene in that movie—the freeze-frame of Mr. Miyagi glowing with pride at Daniel’s accomplishment — and you will know all you need to know about how clearly defined and measurable outcomes can help teachers and students achieve their dreams.

(The outcomes/assessment framework discussed here was created through a collaborative effort of SNHU’s College of Online and Continuing Education Academics unit.)

References
Palloff, R. & Pratt, K. (2009). Assessing the Online Learner: Resources and strategies for faculty. San Francisco, CA: Jossey-Bass.
Walvoord, B.E. (2010). Assessment Clear and Simple: A practical guide for institutions, departments, and general education. San Francisco, CA: Jossey-Bass.

About the Authors

Jill Buban, Ph.D is the Senior Director of Research and Innovation at the Online Learning Consortium. Prior to joining the Online Learning Consortium, Dr. Buban was the Assistant Provost for Research & Innovation at Post University.  In this role, Buban instituted university-wide initiatives with a forward thinking, student-centered focus.  These initiatives included, but were not limited to, the university’s transition to digital course materials, the creation of an online academy for high school students, competency-based learning initiatives, professional developing credentialing, articulation agreements, enrollment management, oversight of all academic publications, as well as a variety of teaching and learning initiatives. Prior to joining Post University, Buban worked in Academic Affairs at SUNY Empire State College.   She collaborated on a variety of online learning initiatives including the implementation of ePortfolios, open learning access and opportunities, and prior learning assessment.

 

Dr. Buban holds a Doctor of Philosophy degree in Education from the Lesley University where she wrote her dissertation on “Adult Undergraduate Students’ Experiences in Online Academic Mentoring Relationships.” She continues to study and present on topics surrounding effective technology use for adult learners in online environments. Given the opportunity, Buban continues to teach in the areas of adult and online learning.