Posts

Autograding Primer for CS – a 2023 zyBooks Guide

Autograding is a must for computer science education, and for good reason. 

For instructors, it dramatically reduces the time spent manually grading assignments, so they can focus on analyzing their student’s performance and giving them meaningful feedback. For students, it helps them learn as they go, at exactly the right moment (see below). And autograding does all of this at scale. Think 400+ student CS1 course.

In this post, I’ll help you supercharge your autograding efforts. I’ll share insights on how to use autograding to improve how you teach and how your students learn. And I’ll walk you through the three most important autograding methodologies and how to use them. 

Let’s get to it. 

In this post

Autograding and the data challenge

Autograding benefits for students

When to use autograding? 

When not to use autograding?

Types of autograding

Compare output tests

Unit tests

Custom script tests

Advanced zyLabs

Autograding in action (video) 

Autograding and the data challenge

I think we can agree that the issue with data is not that we don’t have enough of it. We test the heck out of students. We give them tons of assessments. We give them tons of tasks to complete. 

The challenge is making sense of this growing mountain of data, quickly, so you can have actionable information right now, not tomorrow or next week. This is where autograding comes in. 

When you auto-grade assessments you automatically see scores, analytics, even how much time your students are spending on tasks. Where are students succeeding? Where are they struggling? You can see all this instantly as you’re getting ready for class.

Point of intervention is… now

The formative assessment data you gather with autograding gives you the power to adjust your teaching at the ideal point of intervention. Maybe you’ve covered functions, but you see the class is still struggling with it. So you can go over functions again in your next class. This way you’ll avoid some unpleasant summative surprises. 

Autograding also lets you drill down to the individual student level. You can see, hey, these two students are really falling behind, and this student never turned in their work. So you can also intervene with individual students right now, before they fall behind.

Autograding benefits for students

Autograding isn’t just the gift that keeps on giving for instructors. It’s a game-changer for students, helping them improve their understanding of the material as they learn. Actually, this is my favorite part of autograding. 

For many students, computer science is a brand new subject. From kindergarten to their university experience, they’ve been writing, they’ve been doing math. So they have a grounding in those subjects, even when encountering new stuff. But CS? It’s often a new and exciting adventure. 

Instant feedback confidence builder

Auto-graded assessments to the rescue! 

Autograding provides students with instant feedback on their work as they’re doing it. They can check if a piece of code is correct, or not. The auto-grader can also give students actionable instructions as they’re tackling an assignment. That’s the best kind of instant gratification – the one that helps them succeed in class. 

Instant feedback builds confidence as students progress. They don’t have to wait for you to return a grade to know if they’re understanding the material. In fact, for formative assessment pieces, instructors will often allow students to submit as many times as they need to, to get the code right. By reworking their submissions, students really learn how to solve problems. 

Students will gain confidence about the work they’re doing, the work they’re submitting, and the grade they’re going to get in the course. And students will learn they can persist and push through anything you assign them. 

When to use autograding? 

Okay, we’ve established how autograding helps both instructors and students. So when to use autograding? (And when not to use autograding? More on that shortly.) 

You definitely want to replace tedious and time-consuming tasks with autograding. You can and should automate the checking of all reproducible values, returned values and predictable functionality of your students’ code. The key to success here is providing clear instructions with detailed expectations about what your students’ code should accomplish. Make sure they follow reproducibility guidelines that you set, so their code isn’t fighting with the defined test cases. 

When not to use autograding? 

Autograding isn’t for everything. And it certainly doesn’t replace the instructor. You’ll need to manually review at least some aspects of a coding assignment, like checking for good variable names, readability and design. And more advanced creative coding projects (like those in upper level CS courses) may have too much variability from student to student to auto-grade them completely. 

That said, however, most projects have at least some reproducible elements that can in fact be auto-graded. Consider allocating some points of an assignment to automatic checks, and others to manual checks. This way, you and your students will get the best of both worlds – you’ll save time, and they’ll receive meaningful feedback instantly while they’re working on the assignment. 

One final note: With the hypersonic advances in generative AI tools like ChatGPT, we may soon be able to give students full automated feedback on even the advanced creative coding projects. Stay tuned! 

Types of autograding

Now that we have a handle on the “why” of autograding, let’s dive into the “how.” Here are the three primary methodologies: 


Three types of autograding

  • Compare output tests
  • Unit tests
  • Custom script tests

Compare output tests

The most basic autograding approach is to use a simple compare output test. Compare output tests run a student’s code and compare the code’s output to an expected output you provide. Typically, results are shown to students in the form of a “diff” so they can see the differences in their code’s output versus the expected output. 

Example of a compare output test

Of course, you may not always want to compare student code output exactly with the expected output. Whitespace and case, for example, are usually not important factors for student code correctness. So most autograding solutions have ways for you to customize how the comparison is done. With many applications, you can even provide your own regular expression to match the outputs.

Customizing compare methods

Some programs may wait for user input, so typically autograding systems will have a way for you to provide lines of input to feed to the student’s code when it is run. It’s common to create different test cases with different input values to check that the student’s code is producing the correct output based on the input it is given.

Unit tests

Output comparisons are a great first step in autograding student code, but to test more advanced projects you need to do unit tests. Unit tests help you test “units,” or modular components of code, to “assert” that they meet certain conditions. For example, you can unit test specific functions/methods, classes, files, or other modular components of a project, but not the project as a whole. There are specific unit testing libraries designed to help you do this in different programming languages. Python has a built-in testing library called unittest, while Java has a popular framework called JUnit.

Example of a Python unittest

A typical unit test allows you to define code you want to run before and after each test (setUp and tearDown, seen in Python example above), as well as the code you want to run to perform the test. Each test needs to end with an assertion – a condition that determines if the test passes or fails.

Let me give you a simple Python example. Imagine students need to write a function named add_five() that takes an integer as a parameter and returns 5 plus the parameter value. 

This might be the student’s solution:

We can test the student’s solution using a unit test like this:

Custom script tests

If you need even more control and power when testing student code, you can always design your own shell script.  The compare output tests and unit tests described above help you easily create logic to determine if a student’s code passes or fails your test. By creating your own shell script, you have full control over the logic; determining what is a pass/failure; how many points to assign students; and the feedback you want to give. 

Bash script test

In the image above, the script writes to different files to display debugging information, display feedback, and determine if a test passes or not. For example, you write “p” to the RESULT file to identify if the student’s code passes tests. You can also assign points, for example, two points, by writing “p;2” to the RESULT file. 

You can also bundle additional files with the test, like unit test code, data files, or executables. This allows you to add the files you need and fully build the test case you want.

Advanced zyLabs

Here at zyBooks, I’m excited to share that we just launched our brand new Advanced zyLabs for beta release. Advanced zyLabs offers a fully professional cloud IDE environment that runs on any browser and device, and much more. The three autograding approaches we covered above are also fully integrated into the lab, giving you powerful options. 

If you’re interested in becoming a beta user of Advanced zyLabs click here.

Final thoughts

I hope you found this autograding primer helpful! I’m curious to know how you’re using autograding in your own classroom. I’d welcome your thoughts, comments and insights on that, or anything else related to autograding! Email me at officehours@zybooks.com.


Autograding in Action

In this video, zyBooks’ Lead Product Developer and autograding authority Joe Mazzone demonstrates the three autograding approaches using our Advanced zyLabs application:


Six Best Practices for Teaching IT Security – a 2023 zyBooks Guide

In a dozen-plus years of teaching IT Security I’ve relied on these best practices to keep my students engaged, excited and, most importantly, retaining what they’re learning. These practices have been invaluable to me. I hope you find them useful, too.

In this post

  1. Emphasize foundational principles
  2. Connect theory with practice
  3. Assign hands-on lab exercises
  4. Develop a learning community
  5. Implement a flipped classroom model
  6. Avoid the “expert blind spot”

1. Emphasize foundational principles

Although the security threat landscape is continuously changing, the foundational principles of information security remain the same. The goal of information security is to preserve the “CIA triad,” that is, the principles of: 

  • Confidentiality
  • Integrity
  • Availability of information

Nearly every topic in a security course relates to the CIA triad, and you should present course topics in the context of each security principle. 

CIA triad examples

For teaching encryption concepts, emphasize the role of encryption in preserving the confidentiality principle. Similarly, access control lists are aimed at preventing unauthorized information access to preserve the integrity principle, while load balancers are designed to preserve the availability principle. 

2. Connect theory with practice

IT is a practice-based discipline. I always keep this front-of-mind when I’m teaching.

Students come to IT security courses with diverse backgrounds and varying knowledge levels. Theory is boring to most students, and focusing only on technical information can discourage students without a deep IT background. 

Use real-world examples 

IT security is about the real world, and it’s critical that you leverage real-world events and issues to teach security concepts.

For example, at the start of each lecture, you can present a recent security breach, and then guide a class discussion on how the breach could have been prevented by applying one of the security controls covered in the course.

3. Assign hands-on lab exercises

Your students need to gain hands-on experience with the systems and tools they’ll be using in their future careers. That’s where IT labs come in, to help them master key course concepts through hands-on training.

Live virtual-machines, like those used by zyBooks, are the gold standard in IT education, exposing students to the complex challenges of real systems. Use VM-based lab assignments so students can apply the theoretical concepts taught in the class to real hardware, operating systems, applications and tools. 

Hands-on labs are also a critical form of active learning, engaging students in the learning process while at the same time preparing them for the real world.

4. Develop a learning community

In the real-world, of course, IT security professionals collaborate closely to deploy, maintain and troubleshoot security systems. So it makes sense for your students to practice working together in group assignments to learn more effectively. 

Assign group projects that develop your students’ collaboration skills by enabling them to contribute to a more involved assignment that demands engagement, as well as compromise and decision-making skills. Use classroom and online discussion forums and group projects to promote active student participation and peer-to-peer interaction. 

Remember the individual student, too. Individual contributions to group projects should be taken into account when assigning grades to reward group members who took on more responsibility in the project.

5. Implement a flipped classroom model

I’m a huge proponent of the flipped classroom model to study IT security. With flipped learning, you assign the reading prior to class and focus on more active learning during your lesson.

Use class time to engage students in class discussions, group work and practice, instead of just lecturing to convey content. With the flipped model, you’ll be helping your students develop independent learning skills and build a deeper understanding of the course material.

Assign reading ahead of your lesson 

For example, assign a reading on vulnerability assessment prior to class and use lecture time to demonstrate the use of a vulnerability scanner to assess a computer, network, or application for weaknesses.

6. Avoid the “expert blind spot”

IT instructors with deep expertise in security topics sometimes underestimate the difficulties students experience when engaging with the material for the first time. That’s the expert blind spot. 

IT students are often career changers with limited or no background in technical fields. So they may need more time and substantial cognitive energy to understand the security concepts you’re teaching. Important to be sensitive to this. 

Low-stakes quizzes

One way to find out whether students are keeping up with the course concepts is to introduce low-stakes quizzes. Based on the quiz results, you can plan to spend some lecture time reviewing material students are struggling to understand. 

Final Thought

Final thought: My colleagues on the zyBooks content team are all veteran professors, so if you have your own best practices for teaching IT security, we’d love to know! Please email us at officehours@zybooks.com. We’ll share them with the community. Thank you.