No, not as a standalone basis. A peer-reviewed study shows that AI summaries of legal documents drop load-bearing arguments, with issues and reasons lost most of all; use synthesis for navigation and preparation, but base a decision on an independent review of the full, traceable source.
With the evaluation framework Argument Representation Coverage (ARC), researchers presented a method that distinguishes between facts that are missing and facts that are misrepresented, and that measures whether a summary preserves the three load-bearing parts of a legal argument: the issue, the reason and the conclusion. Across 1,049 legal rulings, argumentative passages made up 7.66% of the input but 66.51% of the reference summaries. For legal documents, the language models tested achieved markedly lower coverage than human summaries. The study on the faithfulness of legal summaries reports that omission was the dominant error category and that models retained conclusions better than issues and reasons.
What did the ARC study establish about AI summaries of legal documents?
The ARC study measures whether a summary preserves the argument structure of a legal document. The core finding: arguments are sparsely distributed across the source text, models retain conclusions relatively well and more often miss precisely the issues and reasons that support a conclusion. Omission, not distortion, was the most common error.
A supplementary ACL study investigates a technical mechanism that can contribute to source drift in long-context summarisation. In long-context summarisation under cache compression, the model can gradually pay less attention to the source text during generation and more to its own already-produced text; in their experiments, aggressive compression of the context memory sharply increased the degree of drift from the source. In our assessment, one nuance deserves emphasis: those figures are bound to specific models, datasets and methods and do not translate directly to every application or infrastructure. The mechanism is relevant; the exact figure is not, outside that setup.
Why does omitting issues and reasons change the meaning of a file?
Our analysis is that a legal or policy conclusion can lose its meaning when the issue, conditions, exceptions or reasons that support it are missing. Omitting those can give a factually incorrect picture, even when individual sentences in the summary are correct in themselves. A summary that renders the conclusion neatly but omits the condition, the exception or the counter-argument thus gives a distorted picture of the file.
That is the treacherous thing about omission as the dominant error: there is no visible untruth to detect. A reviewer who reads only the summary misses precisely what is not in the summary. This touches directly on the broader practice of AI in professional practice and the limits of support: with consequential files in particular, the missing passage is often the passage that would have tipped the outcome.
When does an AI summary stay supportive and when does it become the load-bearing basis for a decision?
Two government sources draw the same line here. The Australian Department of Industry, Science and Resources states that it uses AI for summarising, analysis and retrieving information, but that staff review the output before use and treat it as a draft or starting point, not as a standalone basis for a decision. Responsibility for advice and decision remains with the staff member.
The U.S. Office of Personnel Management guidance on AI in federal hiring goes a step further and makes the line testable. An AI summary is only not the decisive basis when a competent official independently reviews the full underlying documentation, tests the outcome against it and bases the decision on identifiable source material. The guidance explicitly warns that a formal signature is insufficient when the official in practice merely adopts the AI reasoning; then the AI output still provides the load-bearing basis.
That exposes a question neither source answers: how do you establish whether someone genuinely reviewed the source independently, and did not tacitly adopt the summary? Our editorial inference: without explicit recording of the source passages that support a decision, it cannot be established afterwards whether the review was independent. The traceability of the load-bearing passages is therefore not an administrative formality, but an important way to make independent review demonstrable and verifiable after the fact.
What does this mean for directors, lawyers and CISOs who have files summarised?
Our analysis: because models systematically omit issues and reasons, any lawyer or director who decides solely on a summary runs the risk of missing a load-bearing condition of the file. Our recommendation is therefore to block a consequential decision while the reviewer can see only the summary. Because omission does not stand out when reading the summary itself, textual checking helps insufficiently; a second reviewer should go back to the source text on a sampling basis to look specifically for omissions. Because long-context processing can create risks to source fidelity in specific setups, our recommendation is also to check the middle of large files explicitly; the studies supplied, however, do not demonstrate that the risk is necessarily greatest there.
As a summary of that analysis, a concrete protocol:
- Keep the original documents and versions alongside every synthesis.
- Have the synthesis reference, per claim or argument role, the source passage on which it rests.
- Mark uncertainty, missing pieces and counter-arguments explicitly in the synthesis.
- Test the middle of long documents explicitly for omitted issues and reasons.
- Have a second reviewer check for omission on a sampling basis, not only for distortion.
- Block consequential decisions when the reviewer sees only the summary or the sources are not traceable.
This line is also consistent with the broader discussion of verification duties and human oversight in professional AI use, including the verification duty for AI citations in legal filings and the testable requirements for human oversight of AI decisions. Our recommendation: teams using AI for contract and file analysis incorporate this way of working into their validation processes, so that the summary remains a navigation aid and the source carries the decision. Our analysis builds on practical guidance for validating AI contract analysis in legal teams.
Sources and references
Sources: The article relies on the ARC study on arXiv, an ACL study on long-context summarisation and official guidance from the Australian Department of Industry, Science and Resources and the U.S. Office of Personnel Management.