Choose open-weight or closed AI models on demonstrable control, not on the label

Choose not on the open or closed label, but on the question of which control you can prove yourself. The Australian Cyber Security Centre warns that open-weight models give local control but place provenance, safeguards, monitoring and incident response with your organisation; closed models centralise those controls with the supplier, at the cost of independent inspection.

On 5 November 2026 the Australian Cyber Security Centre published the guidance Open-weight AI models: Opportunities and cyber security risks. The core of the comparison between open-weight versus closed: whoever self-hosts an open-weight model gains a say over version management, hosting location and inspection, but in doing so also takes on responsibility for provenance, supply-chain security, protective measures, monitoring, maintenance and incident response. With a closed model, many operational controls remain with the supplier, which can reduce the organisation’s direct management burden but limits independent verification and increases reliance on supplier transparency.

For directors, lawyers and CISOs who work with confidential information, the central question therefore shifts. Not: which model scores highest? But: for which architecture can I demonstrably control the full confidentiality, security and audit chain? In our analysis, applicable obligations under the EU AI Act may be an additional reason to record responsibilities, controls and evidence per deployment; the precise obligations depend on the role, the system and the risk category.

What is the difference between closed, open-weight and genuinely open-source models?

The Australian Cyber Security Centre draws an explicit distinction between three forms. With closed models, the weights and central safeguards remain with the supplier. With open-weight models, the model weights are available for you to host and inspect yourself. Genuinely open-source goes beyond publishing model weights and may provide access to relevant code, documentation and, depending on the model and licence, additional training or development information. The guidance stresses that publishing the weights does not mean that training data, recipes or complete audit information are available.

• Closed: the supplier manages weights and safeguards; you lean on its statements.
• Open-weight: you can host and inspect the weights, but documentation and provenance are often incomplete.
• Open-source: weights plus data, code and documentation, which enables the highest reproducibility.

This distinction is not semantics. An open-weight model gives you insight into behaviour and configuration, but leaves the provenance question open. Our reading: the term "open" tempts the assumption that everything is verifiable, whereas verifiability varies greatly per layer.

Which controls shift to my organisation with an open-weight model?

Self-hosting moves security, privacy, safeguards, monitoring and maintenance to your organisation. NVIDIA Research describes in Evaluating Risk in Open-Weight Models that a deployment must be assessed as a complete system: model, harness, retrieval, tools, sandboxing and runtime enforcement. A benchmark result says something only about the conditions tested.

Concretely, with a self-hosted open-weight model you must record and monitor yourself:

• model identity, version and hashes of the model files;
• fine-tuning, configuration and the serving version running in production;
• prompts, datasets and the evaluation method used;
• supply-chain checks on provenance and model artefacts;
• protection against prompt injection and data leaks;
• logs, updates and an incident-response process.

Our analysis of the consequence the sources do not state: whoever considers open-weight must build internal capacity for version management, monitoring and audit chains, because those tasks no longer sit with a supplier. We explained earlier why running locally is no free pass in our analysis of why running locally does not automatically protect sensitive prompts.

Why does a single performance or safety score say too little about the model choice?

Because capability, cyber risk, robustness and confidentiality are separate assessment dimensions. The National Institute of Standards and Technology reported in its CAISI assessment of GLM-5.3 that, at the time of assessment, this was the most cyber-capable publicly released open-weight model, but that it performed below current US frontier models on four cyber benchmarks. That test measures cyber capability, not general safety or confidentiality.

A peer-reviewed benchmark of open-source and commercial models in financial scenarios, carried out by Marko Pribisalić and Sanda Martinčić-Ipšić, reinforces this picture. No single model dominated across all dimensions, and the measured financial capability did not reliably correlate with security robustness in the sample (Spearman rho 0.095; p=0.823). The authors stress that the results depend on deployment and benchmark and do not measure full production security.

Our connection: higher inspectability can coincide with relevant dual-use capability. Open availability is therefore not a synonym for safe. Always assess a model separately on task performance, security and confidentiality, as we also argue when comparing which subprocessors and model routes your AI system actually uses.

What does this trade-off mean for directors, lawyers and CISOs?

Our analysis: because open-weight models place provenance, safeguards and incident response with your organisation, CISOs gain a permanent management task; this is why you determine before selection whether you can genuinely staff version management, monitoring and an audit chain for model files, and record when a model is not admitted because that evidence is lacking. Because closed models limit independent inspection and increase dependence on supplier claims, the lawyer must enforce concrete inspection evidence in the contract; this is why you include clauses on retention, training use, subprocessors, model versions, region and incident notification, as we set out in setting out an enforceable audit right with concrete inspection evidence. Because the NIST and financial benchmarks show that capability and security do not coincide, the director may not base a decision on a single ranking; this is why you require a separate task, security and privacy test for each intended deployment before you give approval.

In summary, a simple selection procedure follows from this:

1. Classify the data and determine the required confidentiality boundary.
2. For open-weight, determine whether you can staff the full management and audit chain.
3. For closed services, check retention, training, subprocessors, version, region and audit options.
4. Test models separately on task performance, security and privacy.
5. Record when a model is rejected because evidence is lacking.

You will find more depth on model control and the whole chain in our topic hub on AI security and model control. The guidance from the Australian Cyber Security Centre is official but not universally binding; the NVIDIA piece is primary supplier research, and the financial benchmark examined specific models, datasets and configurations.

Sources: The article draws on guidance from the Australian Cyber Security Centre, a CAISI assessment by NIST, research by NVIDIA Research and a peer-reviewed financial benchmark in FinTech (MDPI).