Analyzing the use of auto-graded labs with a built-in simulator to learn assembly programming

Abstract

Our Computer Organization and Design (COD) online interactive textbook, adapted from a leading computer organization and systems title in 2015, introduced an embedded simulator allowing students to practice assembly programming with MIPS, ARM, or RISC-V within the textbook. In addition to our built-in simulator, in summer 2021, we developed a new auto-graded lab environment to support MIPS with 10 new lab assignments that have been used in 6 different courses. To evaluate both the platform and the content of the labs, we analyzed these labs attempted by 28-84 students across all 6 courses. We summarize the average completion rate and the average time spent on each lab. Our analysis also shows how using the simulator impacts student struggle on homework assignments embedded in the textbook. Finally, we share our best practices for authoring similar auto-graded assembly problems.