Digital Badges: The Impact of Blended Workshop on Student Engagement and Participation

Concurrent Session 8

Session Materials

Brief Abstract

Presentation demonstrates how blended learning was implemented to enhance student engagement and participation in a five-week digital badge workshop. Presenters will share findings and recommendations how to implement blended learning 21st century skills. Researchers and practitioners who are interested in blended learning and digital badges will benefit from this session. 

Presenters

Larisa A. Olesova, Ph.D., is Senior Instructional Designer and Adjunct Faculty at George Mason University. Her research focuses on distance education, specifically asynchronous online learning environments.
Birgit Debeerst works as the Associate Director of Online Initiatives for University Life, George Mason University. Her current projects include online orientation for incoming international students and digital badging.

Extended Abstract

Context: Currently, more and more employers are looking for graduates who can demonstrate 21st century skills such as collaboration, creative problem solving, and analytical thinking. To meet this need, an increasing number of universities have started offering digital badges focused on the development of those skills. This session will demonstrate how George Mason University, one of the largest eastern public universities in the United States of America and Education Design Lab, a non-profit organization in Washington, D.C., collaborated to create and conduct the digital badge program “Resilience” in spring 2016 and redesigned it in spring 2017 and fall 2017.Topics included the definition of resilience, how to measure resilience and how to cultivate resilience. The “Resilience” digital badge was implemented first - as a four-week pilot workshop in spring 2016 - and then, it was redesigned and implemented as a five-week blended workshop in spring 2017. The workshop was redesigned again in fall 2017. In spring 2016, six students (n=6) enrolled and four students (n=4) or about 66% earned the digital badge during the pilot workshop. The pilot workshop consisted of four in-person sessions. Students were required to submit a weekly reflection via email. The redesigned blended workshop in spring 2017 consisted of three in-person sessions and two online sessions. It had five weekly online discussions and students’ weekly reflections. The redesigned workshop for fall 2017 was enhanced by adding by adding a series of Adobe Captivate eLearning modules and updated discussion questions. In spring 2017, 34 students (n=34) enrolled and 22 students (n=22) or about 65% earned the digital badge. In this discovery session, presenters will discuss findings from 22 students (n=22) who completed the redesigned blended workshop in spring 2017. In addition, presenters will add more findings from fall 2017 to support the project hypotheses on student participation and engagement.  

Problem: Blended or hybrid learning is increasingly adopted by universities. Following Garrison and Vaughan (2008), we also define blended learning as “the organic integration of thoughtfully selected and complementary face-to-face and online approaches and technologies” (p.48). The benefits of blended learning have been supported by the majority of research stating that blended learning may improve outcomes and increase student participation (Halverson, Graham, Spring, Drysdale, & Henrie, 2014). In addition, according to the EDUCAUSE technology research (2016), more than 82% of students give preferences to blended learning because of a better learning experience through increased interactivity and availability of resources (Vaughan, Cleveland-Innes, & Garrison, 2013). Presenters in this session will overview and discuss students’ participation and engagement in the redesigned blended workshop when they worked on earning the “Resilience” digital badge. During this session, the audience will be able to share their own experiences how they have moved from face-to-face to blended learning environments to increase student engagement and participation.

Approach: To examine student engagement and participation in the blended workshop, we used proxy variables in online environment following the study by Kim, Park, Yoon, and Jo (2016). Proxy variables are those that alternatively used as automated prediction of students’ achievements when the direct measurement of learning is difficult (Kim et al., 2016). In this session, presenters will explain how proxy variables were used to predict undergraduate and graduate students’ participation and engagement in a blended environment.  To measure student participation, we used total time spent in Blackboard, logins to the Discussion Board and logins to content items, and number of postings in the Discussion Board (Kim et al., 2016). To analyze student engagement, we used postings’ length as the indicator of student cognitive engagement in online discussions.  The following questions were asked: Are there any relationships between total time spent in Blackboard, logins in the Discussion Board and logins in content items, and number of postings for undergraduate and graduate students? Is there any difference between engagement of undergraduate and graduate students when they participated in weekly online discussions?

Results: The findings have revealed that out of total 22 students, who earned the Resilience digital badge, 16 or 72.7% were undergraduate students and 6 or 27.3 % were graduate students. Out of 16 undergraduates, 12 or 75% attended regularly face-to-face meetings. On the contrary, all six graduate students or 100% attended all required in-person meetings.

Participation: To examine participation, we compared total time spent in Blackboard for undergraduate students and graduate students.  The findings revealed that undergraduate students spent 187.95 hours or 11.74 per student while graduates spent only 52.24 hours or 8.7 per student in Blackboard in spring 2017.  In addition, we also analyzed the students’ logins to Discussion Board and logins to content items as indicators of their active participation in the blended workshop.  The results showed that undergraduate students had 957 logins or about 59 per student in the Discussion Board and 3466 logins or about 216 per student in the content items while graduate students had 252 logins or 42 per student in the Discussion Board and 732 logins or 122 per student in the content items. Finally, we also analyzed the number of postings in the Discussion Board by undergraduate students and graduate students. The results showed that undergraduate students posted 105 messages or about seven per student in the Discussion Board while graduate students posted only 32 messages or about five per student during five weeks. These findings have revealed that undergraduate students were more active participants in asynchronous online environment in Blackboard than graduate students. However, graduate students were more active in their participation in face-to-face environment as they attended all in-person meetings regularly than undergraduate students.

Spearman’s correlation coefficients were calculated to find statistical relationships between the time spent in Blackboard by all undergraduate and graduate students, logins to the Discussion Board, logins to content items, and number of postings that all students submitted to the Discussion Board. Based on the results, the relationships between the time spent in Blackboard and logins to the Discussion Board (rṣ=.646, p= .001), content items (rṣ=.621, p=.002) and postings (rṣ=.767, p= .000) were statistically significant. This finding means that students who spent more time in Blackboard also logged more often to Discussion Board, logged more often to content items, and posted more messages to the Discussion Board. The relationships between students’ logins to the Discussion Board, logins to content items (rṣ=.739, p= .000) and number of online postings (rṣ=.750, p= .000)  were also statistically significant which means that students who regularly visited the Discussion Board and checked content items more often also posted more messages to the Discussion Board. Finally, there was also statistically positive correlation between the number of logins to content items and the number of postings to the Discussion Board (rṣ=.638, p= .001) meaning that students who regularly checked the content items also posted more messages to the Discussion Board. The results confirmed that statistically significant positive relationship between the proxy variables for participation is a strong indicator of students’ active participation in the blended workshop. The finding in this project is similar to the findings by Lara, Lizcano, Martinez, Pazos, and Riera (2014) who also noted that student participation measured by the number of online logins and interactions with resources, was positively correlated with students’ perseverance in online courses, i.e., the blended workshop in our case.

Engagement: To analyze student engagement, we analyzed their online postings length in the Discussion Board during five weekly online discussions. The diagram below shows that graduate students’ postings were longer than undergraduate students’ postings revealing that graduate students had deeper cognitive engagement in online discussions.

Long postings usually imply that students have invested a considerable amount of time in structuring and articulating their thoughts (Hewitt, Brett, & Peters, 2007). Our findings are also corroborated with previous study by Xie (2013) that the length of online postings is correlated with students’ engagement in online discussions. In addition, Ekahitanond (2013) found that students who demonstrate higher level of analytical thinking usually provide more supporting reasoning, arguing, justifying, explaining, and providing evidence during online collaborative discussions. It seems that the use of weekly online discussions helped students develop the 21st century skills such as collaboration, creative problem solving, and analytical thinking when they participated in online discussions. Overall, the blended model was well received.  One student shared “I liked how they made many of the workshops available on line and had online discussion boards. It was great to see everyone's input and progress through the whole process.” 

Diagram