Clustering Code Submissions for Scalable and Social Feedback

Concurrent Session 4

Brief Abstract

Large online classes often suffer from difficulty in getting students feedback on their work. For computer science classes, code submissions lend themselves to intelligent clustering. In this demonstration, we will show how such a tool can be used to author individual feedback, as well as give students social feedback.


David Joyner is the Associate Director for Student Experience in Georgia Tech's College of Computing, overseeing the administration of the college's online Master of Science in Computer Science program as well as its new online undergraduate offerings. He has developed and teaches CS6460: Educational Technology, CS6750: Human-Computer Interaction, and CS1301: Introduction to Computing, all online.

Additional Authors

Extended Abstract

In large online classes, it can be difficult to supply to students individual feedbakc on their work. This is often a major challenge in scaling equal quality online: the classes may get much larger, but unless there is a linearly growing staff to support the additional students, the feedback may suffer. This is particularly true in computer science classes, which are among the most popular courses in MOOC developments as well as on-campus offerings.

One solution to this is to use machine learning to group large numbers of submissions into smaller numbers of similar submissions to give feedback in aggregate. If, for example, a particularly assignment is most commonly answered incorrectly or suboptimally in a small number of ways, then AI may identify which incorrect solution has been attempted and pair it with a single prewritten segment of feedback. In this way, students may receive feedback individual to their particular approach, without every single student needing to be individually evaluated.

In this demonstration, we will should such a tool, called Sense. Sense operates on programming assignments to cluster them together so that an instructor may effectively give individual feedback to hundreds of students simultaneously. Furthermore, once written, this feedback can be used to give future students immediate feedback rather than wait for assignments to be graded manually, which can supply formative as well as summative feedback.

In addition to clustering for expert feedback, we will show how Sense can be used for other processes as well. For example, a major application of Sense in our class has been to inform revision of the course material itself. We have used Sense to find undesirable patterns in student answers, and then author preemptive material to prevent students from pursuing those undesirable answers in the first place. We have also begun to use Sense for social feedback: upon submission, a student will be informed of how many students attempt the solution in the same way that they selected and how many choose other approaches. They will also be specifically given alternative solutions different from their own to see the wider expanse of possible solutions.