Software Testing Is Due for a Rethink. Here's What That Actually Means.

There is a conversation happening on X right now — among QA engineers, CTOs, and the occasional contrarian developer — about whether the traditional model of software testing is structurally broken. The QA Financial piece that surfaced it is just the most recent prompt. The underlying pressure has been building for years.

The short version: AI-generated code is shipping faster than QA teams can review it, testing pipelines built around human-written feature specs are struggling to keep up with systems whose behavior is probabilistic, and the role of the QA specialist is being questioned in places where it has never been questioned before. Some of this is legitimate signal. Some of it is hype wearing a lab coat. The goal here is to separate the two.

What "Rethinking Testing" Actually Means

The phrase "software testing needs a rethink" has been recycled enough times that it risks meaning nothing. So let's be specific about what is actually changing.

Three forces are converging at the same time:

1. AI-generated code volumes are outpacing manual review capacity. Teams using tools like GitHub Copilot, Cursor, or Claude Code are shipping more lines of code per engineer per week than they were two years ago. That is not a guess — GitHub reported in 2024 that Copilot was responsible for roughly 40% of code in repositories where it was enabled. More code, same QA headcount. The math breaks eventually.

2. AI-driven features are non-deterministic by design. A traditional test either passes or fails. An LLM-powered feature can return a technically correct response that is contextually wrong, subtly biased, or confidently hallucinated. The concept of a binary pass/fail test does not cleanly apply to a system that produces variable outputs. Validating an AI feature means something different than validating a login form, and most test infrastructure was not designed with that distinction in mind.

3. Release cycles are compressing further. Continuous deployment is standard now. The window between code merge and production is measured in minutes in many organizations. Traditional QA gates — multi-day regression cycles, formal test plans, sign-off workflows — were designed for a world that no longer exists at most software companies.

These three forces together are what the rethink conversation is actually about. Not whether testing matters (it clearly does), but whether the organizational and technical models built around it are still fit for purpose.

What X Gets Right — and Wrong

The discourse on this topic runs the full spectrum, as it always does. Some of what is being said is worth taking seriously.

The criticism that QA is often structurally siloed from development — treated as a gate at the end of a pipeline rather than an integrated function — is legitimate. When testing is an afterthought baked into the final sprint, you get exactly the problems people describe: bugs found too late, test suites that don't reflect real user behavior, and QA engineers who spend more time filing tickets than building coverage.

The argument that automated testing has reduced the need for manual QA across the board is more complicated. Automation handles regression coverage well. It does not handle exploratory testing, edge-case discovery in complex user flows, or the kind of judgment call that experienced QA engineers make when something feels wrong even if no test is failing. Replacing all manual QA with automated pipelines and calling it progress is a cost-cutting story with a QA vocabulary grafted onto it.

The most irresponsible version of the "rethink" narrative is the one that suggests testing is becoming irrelevant because AI will catch bugs automatically. This conflates two things: AI-assisted testing tooling (which is real and useful) and the idea that AI makes the need for deliberate quality assurance obsolete (which is not supported by evidence from any production system running at scale).

Shipping faster does not reduce the cost of a production failure. It increases the frequency with which you can incur one.

The QA Function Has Already Been Evolving

Here is something the discourse often misses: QA teams that have been paying attention have been adapting for years.

The shift-left movement — moving testing earlier into the development cycle — was already a response to compressed release timelines. Test-driven development, behavior-driven development, and the widespread adoption of CI/CD pipelines with automated test gates were all attempts to reconcile quality assurance with faster delivery. None of these are new ideas. They are just not universally adopted.

The combination of automated and manual testing has also proven its value repeatedly. Automated suites catch regressions fast and run continuously. Manual testers catch what automated suites miss: usability problems, inconsistent UI behavior across devices, failure modes that only appear under real-world usage patterns. Teams that have fully automated their QA and eliminated manual coverage routinely discover this the hard way when a subtle but significant UX issue reaches a large user base before anyone flags it.

The QA function today is not just running test scripts. In organizations doing this properly, QA specialists are involved in requirement reviews, they are defining acceptance criteria before development starts, and they are building test infrastructure that development teams use. That is a materially different role than the end-of-pipeline checker that critics of traditional QA are usually describing.

What Actually Needs to Change

Acknowledging that QA is evolving is not the same as defending the status quo. Some things genuinely need to change, and the engineering teams that are ahead of this are already moving.

Testing AI-Driven Features Requires New Methods

Validating a feature backed by an LLM is not the same as validating a deterministic function. You need:

• Output distribution testing — running the same prompt or input hundreds of times to characterize the range of outputs, not just checking one response.
• Red-teaming — deliberately attempting to produce harmful, incorrect, or policy-violating outputs through adversarial inputs.
• Behavioral regression testing — checking that a model update has not shifted outputs in ways that break downstream functionality, even if no traditional test fails.
• Human evaluation pipelines — some percentage of LLM outputs need human review, particularly in high-stakes domains, because no automated metric fully captures whether a response is actually good.

None of these map cleanly onto a traditional test suite. Teams building AI features and pretending their existing QA process covers them are accumulating technical and quality debt they will pay back in production incidents.

Test Coverage Needs to Reflect Actual Risk

One of the more useful things AI-assisted tooling does is help identify where test coverage is thin relative to code complexity and change frequency. Tools like Codium, Diffblue, or even LLM-assisted test generation can surface gaps faster than manual coverage analysis. The output still requires review — generated tests can be syntactically valid but semantically shallow — but the signal is useful.

The more important shift is cultural: test coverage needs to be prioritized by risk, not by line count. A codebase with 90% line coverage that has zero tests on its payment processing flow is not well-tested. Coverage metrics without coverage quality analysis are a false confidence generator.

QA Cannot Stay Siloed

The structural criticism holds. When QA is a separate team that receives code at the end of a sprint, you get late feedback, high rework costs, and a dynamic where developers and testers are adversarial rather than collaborative. Engineering organizations that have moved toward embedded QA — where quality engineers work inside feature teams from the start — report fewer late-stage defects and faster release cycles, not slower ones.

This is not a new observation, but it is one that many organizations still have not acted on, particularly in legacy enterprise structures where QA departments have been running the same process for a decade.

The Accountability Question

There is something important underneath the rethink conversation that does not get named explicitly enough: as AI-generated code becomes a larger fraction of what ships, the question of who owns quality becomes less obvious.

If a developer uses an AI coding agent to write a feature and an AI test generator to write the tests, and a bug ships — who is accountable? The developer who did not review the generated output carefully enough? The AI vendor? Nobody?

The answer has to be the engineering team. Tools do not own outcomes. The human and organizational structures around the tools do. This is exactly where having a disciplined QA function matters more, not less, in an AI-heavy development environment. Automation and AI assistance reduce the cost of producing code and tests. They do not reduce the cost of a production failure or the reputational damage of shipping something broken to users.

This is a point worth making loudly in the current environment, where some teams are treating AI-assisted development as a reason to reduce QA investment. The logic runs backwards. When output volume increases and some of that output is AI-generated — with all the hallucination and context-loss risks that entails — the case for rigorous, independent validation gets stronger, not weaker.

A Practical Take for Engineering Leaders

If you are running an engineering organization and trying to figure out what to actually do with this conversation, here is a practical frame:

• Audit your current test coverage against risk, not just line count. Where are your highest-traffic, highest-revenue, or most complex code paths? Are they well-covered? If your payment, auth, or data pipeline code is not thoroughly tested, that is the starting point — not any philosophical debate about the future of QA.

• Invest in test infrastructure, not just test count. Flaky tests, slow pipelines, and test environments that do not reflect production are more damaging than gaps in coverage because they erode trust in the test suite. Engineers who do not trust the tests stop paying attention to them.

• Treat AI feature testing as a distinct problem. If you are shipping LLM-powered features, document what "correct behavior" means for those features, build evaluation pipelines that reflect that definition, and plan for behavioral regression when you update models. Do not assume your existing test suite covers it.

• Move QA earlier in the cycle. If your QA function is still operating as a gate at the end of development, that is the structural problem worth fixing. Embedding QA earlier reduces rework costs and produces better test coverage because testers are involved when the feature is still being defined, not after it is already built.

• Do not confuse tooling adoption with quality improvement. Adding AI-assisted test generation tools is useful. Deciding that the tools mean you need fewer QA engineers is a different decision, with different consequences. Make that tradeoff consciously and with realistic expectations about what automated tooling actually catches.

Where This Goes

The rethink is real — but it is not a rethink of whether testing matters. It is a rethink of what testing looks like when the code it is covering is being generated faster, by more capable tools, in systems whose behavior is increasingly probabilistic.

The QA engineers who are building skills in LLM evaluation, test infrastructure engineering, and risk-based coverage design are not worried about this moment. They are the people doing the work that the conversation is struggling to describe.

The organizations that will struggle are the ones treating the current noise as permission to cut QA budgets while simultaneously shipping more AI-generated code at higher velocity. That combination does not end in a better product. It ends in a production incident that was entirely predictable in hindsight.

Testing is not going away. The failure modes are just getting more interesting.