Temperature 0 reduces variation in AI-generated code but does not eliminate it

No. Temperature 0 reduces the variation in AI-generated code but does not eliminate it: even with greedy decoding, variation persists through inference hardware and provider changes. Reproducibility is a property of the whole pinned pipeline, not of one sampling setting, and requires repeated runs, tests, static analysis and explicit reporting of variation.

Many teams treat temperature 0 (the setting where a language model always chooses the most probable next word) as proof that a model works "deterministically" and therefore produces reproducible output. A study of 24 October 2026 on temperature control in so-called LLM-as-judge safety evaluations, published on arXiv, shows that this assumption does not hold. Temperature 0 lowers the noise but leaves borderline cases and divergent assessment outcomes in place. The study also reports that some newer models no longer support sampling parameters, so the setting you intend to control may not be available for every model.

For anyone deploying AI to write or assess code, that is a practical signal. Reproducibility does not mean "setting one dial to zero", but recording the whole chain and measuring repeatedly. Below is what the research establishes and what that concretely requires of decision-makers.

What exactly does the research show about temperature 0 and reproducibility?

The arXiv study reports that identical evaluations under unset sampling parameters can vary, that temperature 0 reduces the variation but does not remove it, and that residual differences even occur with top_k=1 (a setting that restricts the choice to the highest-ranked candidate word). A single pass/fail run of an AI assessor can hide that noise. The researchers therefore recommend recording temperature, seed, model identity, repeated runs and disagreements between assessors.

A preprint additionally analyses non-determinism at the token level. The results indicate that GPU execution and small numerical differences can cause token probabilities to vary, with larger effects for mid-range probabilities. This supports the narrower conclusion that temperature 0 is not, by itself, proof of bit-identical execution; infrastructure details should therefore be recorded when reproducibility matters. In short: temperature 0 is no guarantee of bit-identical execution.

Why does a higher temperature in code generation also produce more vulnerable code?

A higher temperature increases the variation in the output. That helps in finding functionally working solutions, but also surfaces more insecure variants. The peer-reviewed study Multi-SALLM, published on 26 August 2026, examined generated Python, Java and C++ code and found that temperature and the number of samples can increase the chance of correct code, but that security@k (the chance that a generated solution is secure within k attempts) actually falls in many tested configurations as temperature and k rise.

That is a concrete trade-off, not a detail. In the tested configurations, broader sampling and higher temperature were associated in many cases with more vulnerable variants alongside functionally correct candidates; this is why a working-solution criterion alone is insufficient for security assurance. In our assessment, this means that functional acceptance tests alone are insufficient: a piece of code that passes the tests can still contain a security flaw that only becomes visible under static analysis or review.

How do you measure and report reproducibility of AI code when the infrastructure causes the variation?

By treating reproducibility as a property of the whole pinned pipeline, not of the model. According to the supplied description of Skyt, reproducibility is treated as a property of a pinned chain and assessed using both byte-identical output and agreement with a validated reference implementation. The core: pin what you can pin, and measure and report what you cannot pin.

A workable sequence, based in part on these three studies and our own assessment:

• Record the chain. Prompt contract, model identity and version, API configuration, seed where supported, execution environment and dependencies.
• Define in advance what must be reproducible. Byte-identical, schema-conformant, test-equivalent or security-conformant. These are different requirements with different measurement methods.
• Perform multiple runs. One successful run proves nothing; repetition makes the spread visible.
• Validate the output. Functional tests and static security analysis, because working code is not automatically secure code.
• Preserve raw output and effective configuration and report the variation explicitly: spread, failed validations and divergent outcomes, instead of only a successful end result.

The important distinction that is rarely made explicit: deterministic computation (the same bits), a reproducible pipeline (the same validated outcome via a fixed chain) and semantically equivalent code (functionally equal, but written differently) are three different things. Which one you need depends on the use.

What does this mean for directors, lawyers and CISOs who permit AI code generation?

Our analysis: because temperature 0 does not remove the variation, a supplier statement that a model runs "deterministically" provides no verifiable certainty; this is why, in our analysis, a CISO should lay down in the acceptance policy that every AI code contribution is run repeatedly and validated with tests and static analysis before it ends up in a production repository. Because Multi-SALLM found more vulnerable variants in many tested configurations as temperature and k increased, our analysis is that development teams may face security risks that functional tests do not catch. In our analysis, this is why a static security scan belongs as a blocking step in the pipeline, not as optional reference material afterwards. Because newer models may phase out sampling parameters, a model update can silently change the behaviour of existing workflows; this is why, in our analysis, a lawyer should consider including in the supplier contract a notification duty for model and parameter changes, tied to a reassessment period. And because infrastructure non-determinism means variation occurs even without sampling, "the model got it right last time" is not defensible proof; this is why a director must require that the effective configuration, the raw output and the reported spread are preserved per delivery, so that a decision can be reconstructed afterwards. Further guidance is in the topic hub on AI verification and verifiability.

Whoever follows this logic through sees the same pitfall as elsewhere in AI governance: a single successful outcome is used as proof where repeated, documented control is needed. We described that same pattern earlier in the question why unanimous AI models do not prove independent fact-checking and in the explanation why a single reliability score falls short for AI output. For recording what you preserve, these concrete examples of audit evidence and how to document them offer practical guidance.

Sources: The article relies on peer-reviewed and preprint research published via arXiv, Springer (Multi-SALLM), ACM (Skyt) and IEEE Computer Society.