A case for unit tests

There’s a lot of chat going around on what is the role of unit tests in this AI age. Are they still useful? are they an artifact from the past where the code was more human-driven and we needed those guardrails to make sure we don’t break pre-existing behavior? or are they really useful for more than testing the written code?

So… this is my take and why I do think not only they are necessary but more important than ever, which is a bit connected to all the advantages I see outside of the popular/most commonly referred one.

Act as first safety net

This is in my opinion the classic and most commonly referred as the “main reason” why we add unit tests. We want to have a first pass through all our unit tests to make sure we didn’t “break anything else” while doing new development, acting as some sort of “soft” regression. ¹

So, the main question here should be… does this still happen? are agents so powerful they have total and complete visibility of the code they are changing and my answer is definitely no, at least not on real software development projects. ²

Of course, It clearly depends a lot on the size of the solution, but same happens with us humans, and it almost happens for the same reasons which is the context of the problem (in our case that context is how much can we predict ahead of us, in “ai agents” is more about how much context memory they can load)

Summary: Still useful but very dependent on the type and size of project. Clearly not necessary if we’re doing a small proof of concept or a bunch of scripts.

They lock down behavior

Apart from testing, if the unit at test fulfills the promise, we should also lock that behavior down.

By “locking the behavior down” I refer to having a unit test that unless the expected behavior is accomplished it won’t pass, blocking the build from happening. If we want to achieve this the test must cover what the method receives as an input and what the resulting value must be.

Last but not least, it’s important the test covers the important scenarios and edge cases or else not only we have a partially covered unit but we’re not really locking down the behavior.

Mutation Testing sounds like an effective way to identify if the unit test actually locks down behavior. To be honest, I need to investigate a bit about tools and what their capabilities are.

Summary: Depending on the size and intent of the actual solution I see is worth paying attention to this. I also see it could be useful on scenarios with shared contracts between different apps.

They document behavior

Very connected with the concept of locking down behavior (previous point). Before AI it was important because, if the unit test was complete and well-done ³, you didn’t have to go on a crazy rabbit-hole searching for documentation. You could use a well done unit test to understand what was the code trying to do, and use experience and good judgment to understand the whole context.

With AI it can change the outcome of what an agent do. I’ve seen agents change their understanding after a unit test confirmed or denied a particular behavior so tests are one of the few ways to hand an agent the context it doesn’t have. They certainly act as a feedback loop.

Summary: After seeing how some agents behave, I think today this could be one of the most important factors of having well written unit tests. A wrong unit test can confuse an agent into thinking the method’s intent is different.

Conclusion

A unit test records not “what does the code do” but “what is it supposed to do” and sometimes the reasons for that logic are business rules.

Not only I feel unit tests are relevant nowadays, but I feel they are almost a necessity. Obviously, this isn’t a one-size-fits all, there are different projects sizes and things, but the overall idea is that an agent that run tests can check its own work and fix its mistakes without interaction.

Without tests, it says “done” and you’re the test suite.