Tapping into Student Behavior Insights tool to detect struggle in CS programming assignments

Abstract

Over the past decade, academic integrity has been an important consideration for computer science courses. Although there is vast research on mitigating cheating, tapping into student behavior that leads to cheating provides the opportunity to intervene early and has a positive impact on student learning. To further complicate this, AI is transforming the learning landscape, raising more concerns about academic integrity. With free access to tools that can easily generate solutions for programming assignments, struggling students turning to AI has become more prevalent, and detecting cheating has become even more challenging. Traditional similarity detectors are no longer sufficient, as AI-generated content can bypass these systems.

To address this, we are introducing Student Behavior Insights, a new beta feature for programming assignments in an online interactive textbook that supports autograding. This feature helps instructors identify students who may need additional support focusing on key behavior metrics. The feature provides valuable behavior data on the number of submissions, time spent, program runs, and the amount of code pasted during program development. These metrics offer a deeper understanding of how students engage with their assignments, making it easier to identify irregular behavior and struggling students so instructors can provide targeted interventions.

This paper explores various approaches to effectively utilize the Student Behavior Insights feature, providing early use cases and recommendations. It is important to note that Student Behavior Insights is not a “plug-and-play” solution for detecting cheating but is intended for use at the instructor’s discretion. Additionally, this feature can serve as a predictor of student struggles. We will present examples of how to use the feature to gain insights into: 1) a student who works earnestly, 2) a student who is utilizing an outside source for their work, 3) a student who is struggling and resorts to outside assistance to complete the work.