Nothing crashed. No test failed. The answer was still wrong.
The short version of our PNSQC 2026 paper. The rest is in Portland on Tuesday, October 13.
Software testing used to be rather simple. Known input, expected output, pass or fail. AI changed that.
What gets less attention is what happens when you go from one model to many.
One application talking to one model provides containment. There is a single place where the answer can vary, and you can put a rubric around it. Swa is not that. A request gets routed to an agent, which calls a model, which calls a tool, which hands its output to another agent… Every hop is a new place to sound confident while being completely wrong.
So here is the failure our tests could not see. A request goes to the wrong agent. The wrong agent answers confidently. Nothing is on fire.
We had thousands of passing unit tests while that was happening. They were never going to catch it. Unit tests prove the code ran.
Two years of working that problem produced four questions, asked in order:
- Did the code execute?
- Did the right agent answer?
- Did it do the “right” thing?
- Was the output any good?
Each one is harder to automate than the one before it, and our own coverage says so plainly: 7,612 tests answer the first question, and 39 assertions answer the second. That is the argument in two numbers. The testing pyramid comes out upside down, automation thick at the base, human judgment heaviest at the top. We stopped pretending we’d automate the top layer away. This is also the premise behind the human-in-the-loop foundation of Swa.
The paper has the rest of the story. What our AI review agent checks on every pull request before a change is allowed to advance. Why the safety layers fail closed. And the half we are still building, starting with assessing judgment at the speed of AI.
“Testing Intelligence: A Journey of Quality for a Multi-Modal AI Orchestration Platform” is being presented at the 44th Pacific Northwest Software Quality Conference in Portland, Oregon. This year’s theme is Quality in the Age of Autonomy.
Our session is Tuesday, October 13. All three of us are presenting it, Mike Sirchuk, Ben Pickett and Rahul Ravel. The conference itself runs October 12 to 14, and the paper publishes in the proceedings.
If you are going, come find us and let’s chat!
-Mike