Less is More - Digital Commentary Grading

Concurrent Session 5

Session Materials

Brief Abstract

This session will detail our experience training faculty to provide student feedback via video assessments. Save time and space while supporting transparency and mentoring.


Martin Mehl, a native German, is a Senior Lecturer in the Communication Studies Department at California Polytechnic State University, San Luis Obispo. He was the inaugural recipient of the 'Outstanding Educator in General Education Award,' and the founding member of CSU's Podcasting Pilot. He serves as a consultant for the Consortium to Promote Reflection in Engineering Education (CPREE) and on Cal Poly Academic Senate's Exceptional Student Service Committee (ESSC.) He draws from 20 years of consulting, management & research experience for non-profit, academic and business clients incl. Harvard Medical School, MIT, Emerson College, Sydney University, Iowa State University of Science & Technology, Apple, Bose, Audi/Volkswagen, Adobe, Eli Lily and Lionbridge. He is the co-author of the Digital Mentorship Competency Series: https://onlinelearningconsortium.org/read/olc-research-center-digital-learning-leadership/
Dr. Luanne Fose is the Lead Instructional Designer for the Center for Teaching, Learning & Technology at California Polytechnic State University in San Luis Obispo, CA. In her position, she conducts faculty technological / pedagogical workshops and assists faculty in developing flipped, hybrid, and online courses. Luanne is an Apple aficionado with expertise in Macintosh computers, iPads, iPhones, and iPods as well as their respective software/apps. As a former music professor, she has a passion for audio and video creation as it relates to teaching. Luanne holds a Doctor of Philosophy in music theory / musicology from the University of North Texas.

Extended Abstract

Destination Time: Oct 21, 2015 and the doors of the flying DeLorean just opened up on the campus of California Polytechnic State University, San Luis Obispo where the educators crawling out face a real time-space-continuum dilemma. No, we did not design the flux capacitor to solve this problem, but instead we created the next best thing: the Digital Convergence Grading Project (DCGP).

There is only so much time that faculty can give to feedback, grading, course preparation, and research collaboration. Physical space to confer is limited by both time and meeting space availability. Students demand clarity, competency and approachability from faculty, but enrollment numbers are rising and we are running out of time and space. Faculty have only so much of themselves to physically and mentally give, while administrators have infrastructural and budget limitations, and instructional designers and information technology officers are trying to make the most of existing resources. Identifying a strong national push for critical and reflective learning curricular from primary and secondary education, the college environment needs to transition to the "critical thinker training of the future."

Therefore, co-principle investigators Mr. Martin Mehl (Faculty member in Communication Studies) and Dr. Luanne Fose (Lead Instructional Designer in the Center for Teaching, Learning & Technology) have been conducting a pilot to evaluate improvement in faculty grading efficiency and assessment by means of a true paperless workflow with asynchronous oral feedback to students through video screencasting techniques, establishing a very potent technology-enhanced approach.

As a team we have invested four months in preliminary research and nearly eight months in cumulative construction for conducting and executing the DCGP. We received campus IRB approval for our pre- and post-research surveys for faculty and students after collecting baseline data. We have generated (1) tangible face-to-face training methods for instructors; (2) online training text and video resources; (3) pedagogically sound best practices; and (4) findings and trends that we will be excited to share with the conference participants.

We have collected quantitative and qualitative data to investigate perception of constructive criticism through this one-on-one process while providing the added value of instructor transparency and personally focused mentoring. Sharing trending data and learning projections will permit a system wide approach, reach and impact for this new assessment methodology and technological innovation.

After extensive research to find the instrument that would work best for all users, we have selected a free screencasting tool that works within any browser - Screencast-O-Matic for both Windows and Macintosh platforms. Screencast-O-Matic was easy for faculty to use and does not require any prior screencasting experience, nor does it require any additional equipment beyond a web cam and a computer's browser. Therefore the learning curve for technology adaption, adoption, and diffusion is minimal.

Conference Presentation Format

Our presentation will follow a reverse engineering approach:

We will focus on why & how:
1. Assessment: less time allocated to grading papers, projects,
presentations and redundancy.

We will focus on when & where:
2. Administration: less time on rubrics, scoring, space allocation, and
expectation clarification.

We will focus on what:
3. Development: establishing, enhancing and refining more
pedagogical and technological skills.

4. Impact: more competence and approachability perception, and more
clarity for better learning outcomes.

Conference participants will be asked to attend a 30-minute training session with Martin Mehl focusing on the pedagogy (its shortfalls, challenges and solutions), while Luanne Fose will identify the technological process for supporting the faculty in this endeavor (i.e., software choice, video training materials, support time required). An interactive 10-15 minute Q&A with conference participants will follow this training.

Resources Shared

We plan to share the following resources with conference participants:

ï PDF download of presentation slides that demonstrate methodology and findings of the study (to be posted at the conference website)
ï Faculty and Student testimonials, preliminary research data
ï Online handouts (PDFs) that were given to Cal Poly faculty regarding best pedagogical practices (to be posted at the conference website)
ï Links to YouTube Training Videos that were created for Cal Poly faculty

Trending and Outcomes

The findings so far have been a double win for both pedagogues (scholars) and learners: faculty have expressed delight in being able to provide higher quality feedback to students' assignments and an overall reduction in actual grading time and office hours; students have expressed appreciation in receiving a more pro-active, mentored, transparent and tailored approach of instructor feedback without standing in line for office hours.