Behind the Badge: Creating robust co-curricular competency tracking programs

Concurrent Session 2

Brief Abstract

Competency programs are becoming more common as a way of certifying previously assumed or undocumented competencies, however creating a solution that tracks competencies in meaningful, intentional, connected ways is difficult. We have worked to develop a framework for creating programs that make competencies visible and administratively facilitate long-term record keeping.

Presenters

Dr. Jessica Knott is the Learning Design Manager for MSU Information Technology and the MSU Hub for Innovation in Learning and Technology. She has worked in information technology since 1998, spanning the public and academic sectors. She has been active in collaborating with colleagues nationwide in planning conferences for the Online Learning Consortium, and she is an editor for the Hybrid Pedagogy Journal (http://www.hybridpedagogy.com). She is also on the faculty for the Online Learning Consortium Institute. Find her on Twitter @jlknott.

Extended Abstract

Competency programs are becoming more common on college campuses as a way of certifying competencies that have previously been assumed or undocumented. Like badging in higher education got off to a somewhat rough start, with many dismissing it as a passing fad or comparing it to merit badges earned in scouting, competency programs are too often dismissed as tied to models of direct instruction and memorization (Norris & Soloway, 2015). It is this image that has caused some delay in the rollout of programs as many faculty and administrators did not see the full potential of a competency-based or badging initiative.

 

Our goal, in this presentation and beyond, is create meaningful experiences for our students. We have created a competency tracking solution that aims to help guide program development, connect objectives to student work, and to assist students in better recognizing accomplishments. This presentation is not about the system itself, but about what we learned about student success and the trends in enterprise software that seem at odds with it. Creating a solution that tracks competencies in meaningful, intentional, connected ways is hard. The research phase of our design process found that competency tracking systems focused on extrinsic rewards, such as badges, are fundamentally behaviorist. This view is supported by a recent work in the digital humanities (Watters, 2016; Skallerup Bessette, 2013), and tends to drive system design decisions in directions that are at odds with modern notions of learning theory.

 

In 2010 the College of Arts and Letters at Michigan State University launched a prototype of such a system used to assist graduate students in tracking their progress toward the Graduate Certification in College Teaching. This program combines coursework with experiential opportunities and promotes reflective practice in teaching among graduate students. The system assisted both students and administrators in documenting achieved competencies for use in awarding the certification. This early system paved the way for our current work in integrating this thinking into larger programs at both the graduate and undergraduate levels. However, it was not until 2015 that we began to more critically examine competency tracking and what potential it might offer for many more of our students in terms of academic programs and preparation for the workforce. As we started to dismantle traditional notions of competency/badging programs and software we found a large gap in most programs, a tracking system for the certification process of items associated with a competency was missing. In a co-curricular program students are often producing portfolio or other pieces of evidence for review, receiving comments or approvals from reviewers, and are then granted a badge. We realized through our work that there was a missing component associated with the comments and approvals from reviewers, that of a long-term record of achievement.

 

It is this achievement record that is at the core of an alternate credentialing program. It provides the structure and scaffolding for students completing a program, while also providing a long-term record for the institution about who reviewed which pieces and when. An intended outcome of working with a system such as this is to provide students with a window into the competencies/skills they achieve in their degree programs, but which may not be as obvious to them as it is to the faculty and administrators. The competencies are often more important to employers than the specific majors the students have (Kuther, 2013), so it is essential that students are able to identify and promote the competencies they have achieved. By supporting pedagogies of reflection, the system provides students with this opportunity to identify their competencies in a meaningful way that is much more likely to benefit them in the job market. A secondary outcome of such a system is that it provides opportunities for faculty and administrators to see how student accomplishments are connected back to goals at the individual, program and institutional levels. Clear measurements of these connections are often difficult to make as goals/objectives at the various levels are rarely connected directly to student work in such a clear and available way.

 

To move from our initial prototype system to a more fully featured and extensible platform, we worked with collaborators on campus to develop the MSU Achievements System, which provides this functionality for program directors and augments our current e-portfolio and badging systems. The Achievements system at MSU is differently focused from  badging software in that we have isolated the competency tracking, thus allowing the system to focus solely on providing program goals/objectives, places for students to provide evidence, and opportunities for comments and conversation between faculty reviewers and students.

 

Students use a portfolio system to create and store reflections, projects, and other evidence of activity. The Achievements system accepts links to these items as evidence and allows the students to leave some brief comments to provide context. The system also solves issues relating to other pieces of evidence such as workshop or event attendance, and course grades. When setting up a program, directors can specify which achievements are part of the program and what evidence needs to be submitted. For example, Students attending an event can submit the event date, time and topic, along with contact information for the event leader and a reflection about attending the event. From an administrative perspective, the system allows for reports to be generated on the program at any time. Program directors can see individual, cohort, and program data with varying levels of granularity from singular achievements embedded in a larger competency all the way up to full program data.

 

Assessment of this second generation system is currently underway. Initial satisfaction measures from students and faculty have been positive, suggesting that our careful crafting of the user interface was an important piece of the process. Pilot programs are underway and we are continuing to assess both measures of satisfaction and attitude, but also how such a system is facilitating learning in more meaningful ways through its promotion of reflection and self awareness among learners. In our presentation we plan to report on current research findings as well as to demonstrate the current state of the system.