Heise published the iX Developer Testing Guide 2026 on 6 November 2026, describing software quality assurance with AI as a double task: teams must check AI-generated code and tests and also test AI as a non-deterministic component. Use AI as an accelerator under human final responsibility, with classical testing techniques, repeatable validation and auditable traceability.
The core of the guide is not that AI makes testing faster, but that quality has to be checked in two directions. On the one hand, teams must assess the output of AI: are the assertions correct, do the tests cover the relevant scenarios, is the code maintainable, secure and accessible, and do the tests actually find faults? On the other hand, an AI component is itself a test object that behaves non-deterministically, meaning that the same request can produce different outcomes. That calls for checks on variation, regressions, traceability and safe boundaries. This double task fits within broader questions about AI in professional practice and quality assurance.
What does the iX Developer Testing Guide 2026 say about AI and quality assurance?
Heise's guide addresses classical testing methods alongside AI-generated code, AI-driven test automation and the testing of systems that run on large language models. The official product page of the iX Developer Testing Guide 2026 confirms that the guide covers testing AI systems, AI-generated code and testing with AI; our analysis is that non-deterministic AI behaviour makes additional testing approaches necessary.
The difference from earlier testing literature lies in that second layer. Where classical software quality assurance assumes a system that gives the same answer to the same input, an AI component has to be assessed against a bandwidth of outcomes. A test that passes once then proves little; only repeated runs on representative and high-risk cases show how stable the behaviour is. That aligns with what we described earlier about bias and output variation on repeated runs.
Do AI-generated tests demonstrably deliver better quality?
Not without qualification. An empirical study in the digital library of the Association for Computing Machinery analysed 2,232 test-related commits and found that AI wrote 16.4 per cent of the test-adding commits, with coverage contributions comparable to those of human tests and delivering positive coverage gains in various projects. That same ACM study on test generation by AI agents reports a higher assertion density. The study therefore offers no proof that coverage or assertion density on their own safeguard long-term quality, test smells or maintainability; further research is needed for those long-term effects.
In our assessment, that is the most important misunderstanding to avoid: coverage and more assertions measure how much code a test touches, not whether the test finds the faults that matter. A practical study on arXiv, in which 64 testing professionals were surveyed, confirms that concern: respondents saw time savings, but also reported missed scenarios, erroneous assertions and dependence on prompts. The authors of that survey of testing professionals on generative AI stress that they measure perceptions, not proven quality gains.
What gaps do the current studies on AI in quality assurance reveal?
The scientific literature focuses mainly on the technical side and far less on the process around it. A systematic mapping study in Frontiers in Computer Science examined 37 primary studies on agentic AI in quality assurance and found that they concentrate on test generation, product assurance and defect management.
The mapping study on quality assurance in the era of agentic AI identifies recurring limitations including validation problems, limited transparency, scalability and generalisability. The authors argue that human and organisational support remains important, while planning and process assurance are precisely the areas that are underexposed. Our reading: the technology to generate tests is ahead of the agreements on who assesses the output and how that is recorded. That shifts the centre of gravity from test automation towards governance and auditable AI verification.
What does this mean for directors, lawyers and CISOs who allow AI in development?
Our analysis: because the iX guide frames software quality assurance as a double task, an organisation that steers only on coverage may believe it is in control while high-risk scenarios remain untested; that is why the responsible manager must record before deployment, per use case, which fault types are unacceptable and which testing techniques cover them. Because the ACM study shows that AI tests have a higher assertion density without proof of better fault detection, the maintenance burden grows quietly; that is why the development team must measure and report mutation testing, fault types, false positives and false negatives and regressions alongside coverage. Because AI components are non-deterministic, a released test may turn out differently on a subsequent run; that is why the CISO must require that model, prompt, tool and test-set versions are pinned and that uncertainty and abstention remain visible. And because the practitioner survey names prompt dependence and erroneous output as real risks, liability in the event of a fault may become unclear; that is why the lawyer must have it recorded who assessed and released the AI output, so that this check is reconstructable later. The same documentation logic applies more broadly to third-party risk management around external AI models.
As a summary of that analysis, we hold to this line:
- Define per use case which faults are unacceptable and what coverage that requires, before AI is deployed.
- Retain classical testing techniques and independent human review alongside AI-generated tests.
- Test AI output repeatedly on representative and high-risk scenarios rather than on a single successful run.
- Measure mutation testing, fault types, false positives and false negatives, maintenance burden and regressions alongside coverage.
- Pin model, prompt, tool and test-set versions and make uncertainty and abstention visible.
- Record who assessed and released which AI output.
The open question that all four sources leave unanswered is what AI-generated tests do to maintainability and test smells over the long term. As long as that is unanswered, the sensible conclusion remains that AI is an accelerator under enduring human quality responsibility, not a replacement for that responsibility.
Sources and references
- iX Developer Testing-Guide 2026
- Testing with AI Agents: An Empirical Study of Test Generation Frequency, Quality, and Coverage
- Software quality assurance in the era of Agentic AI: a systematic mapping study
- Generative AI in Software Testing: Practitioners' Perspectives on the Benefits, Challenges, Opportunities, and Risks
Sources: The article relies on Heise's iX Developer Testing Guide 2026 and on peer-reviewed and preprint studies from the ACM, Frontiers in Computer Science and arXiv.