AI vs Testers
A model fails without complaining: well-formed, confident, wrong, and invisible to your monitoring. Why every capability we hand to AI raises the demand for testers, and what the job turns into.
Introduction
Every week someone asks me a version of the same question. An agent writes test cases now. A model reviews pull requests. Tools generate whole suites from a user story. So: is there anything left for a tester to do?
I have been testing for sixteen years, and I now build AI systems at FedEx. From both sides of that, my answer is not a comforting one — it is an observation. Every capability we hand to a model creates new ways for a product to fail, and almost all of them fail quietly. Someone has to go find them.
That someone is a tester. The demand goes up, not down.
What you will learn
By the end of this lesson you will be able to:
- Recognise the three ways AI systems fail that traditional testing never had to handle
- Explain why a green pipeline tells you almost nothing about an AI feature
- Know which parts of your job are being absorbed, and which are becoming more valuable
Failure 1: the model fails without complaining
Traditional software fails loudly. A null reference throws. A failed request returns a 500. The failure announces itself, and half your job is already done.
A model does not do that. Ask it something it does not know and it answers anyway, in the same confident tone it uses when it is right. There is no exception, no stack trace, no red in the logs. The output is well-formed and wrong.
In a checkout flow, that is an annoyed customer. In a credit decision, a triage system, or anything touching money or health, it is considerably worse — and the organisation finds out from its users, or from a regulator, rather than from its own pipeline.
Failure 2: the system can be talked into things
Then there is the input side. Prompt injection sits at the top of the OWASP Top 10 for LLM applications, and it is not a theoretical entry. If your model reads a document, a support ticket, a web page, or an email, then whoever wrote that text gets to put words in front of your system.
This breaks an assumption most of us have carried for years: that data is data and instructions are instructions. In an LLM application they arrive in the same channel. A sentence buried in an uploaded PDF can tell the assistant to ignore its rules, and the assistant has no reliable way to know it should not listen.
That is a security mindset applied to natural language, and no automated suite covers it for you — because the attack surface is written in prose, and prose is infinite.
Failure 3: someone has to say whether the answer is good
The hardest question about a model's output is not "did it return something". It is "was this the right thing to return".
Correct is not the same as appropriate. Appropriate is not the same as fair. A recommendation can be technically accurate and still wrong for the person receiving it, and no assertion in a test file will tell you which.
A practical example: Netflix
The model is genuinely strong. Now consider the failures it can produce while every service stays healthy:
- An eight-year-old's profile gets recommendations for horror and adult titles
- Someone who only watches comedy is served week after week of bleak drama
- An English-only viewer is offered untranslated Korean and Japanese content
- The same five titles appear on the home row every single day
Nothing crashed. No alert fired. Every one of those is a user quietly deciding the product is not for them — and the recommendation engine is not a feature of that product, it is most of it.
What the job turns into
Two parts of testing are genuinely being absorbed, and I would not fight to keep either: writing regression cases from a spec, and executing the same script by hand every release. That work is repetitive, it is where humans are weakest, and machines are good at it now.
What is left is harder and more valuable.
1. Testing the AI itself
Companies now ship chatbots, recommendation engines, fraud detection, document extraction, agents that take actions. Every one of those is a system whose behaviour is statistical rather than deterministic, and testing it requires knowing how models fail — hallucination, drift, bias, prompt injection, brittle context. That is a skill you can learn, and demand for it is far ahead of supply.
2. Exploration
A machine will run ten thousand cases you already thought of. It will not ask the question nobody in the room has asked yet. Finding a bug that is not in any specification is a creative act, and it is still the highest-leverage thing a tester does.
3. Standing in for the user
You are the person in the process who knows what frustration feels like. A model has read descriptions of frustration; it has never been late, confused, or charged twice. That distance is not a gap you can prompt your way across.
Lesson summary
| Traditional software | AI systems |
|---|---|
| Fails loudly — exceptions, 500s | Fails silently — confident and wrong |
| Data and instructions are separate | Both arrive in the same channel |
| Correct is defined by the spec | Correct, appropriate and fair are three things |
| Green pipeline means healthy | Green pipeline means almost nothing |
| Bugs are found by tests | Bugs are found by people who look |
The most important point:
AI is taking the boring, repetitive half of testing, and that is good news. What it leaves you is the half that was always the real work: exploring, thinking critically, protecting users, and deciding what "good" means.
That is exactly what makes a tester valuable — and it is in shorter supply than ever.
Try this before the next lesson
Take one AI feature you already have in production and write down ten ways it could produce a well-formed, confidently wrong answer. Not crashes — wrong answers.
Then go check whether anything in your pipeline would notice. That exercise usually settles the question.