Legal AI vendors should share the verification burden

When legal AI gets something wrong, everyone looks at the lawyer. Fair enough. Lawyers remain responsible for the work they file, present and rely on.

But the conversation often stops there.

A lawyer may fail to check an answer properly. A vendor may also have designed a product that makes proper checking slow, opaque or unrealistic. If the system produces a polished conclusion, hides the limits of the search and leaves the user to reconstruct the underlying work, verification has been pushed back onto the lawyer without much help from the technology.

That matters because AI is already part of ordinary legal practice. Research from the Victorian Legal Services Board and Commissioner found that 36.7 per cent of surveyed Victorian lawyers were using AI, with more than half of those users doing so daily or weekly.

The professional position is also becoming clearer. In 2025, the Victorian regulator varied a practitioner’s practising certificate after inaccurate AI-generated authorities and summaries were put before the Federal Circuit and Family Court. In April 2026, the Federal Court issued a national practice note confirming that users remain responsible for AI-assisted material and may be required to explain what tool was used, how it was used and for what purpose.

I agree with that allocation of professional responsibility. A vendor cannot take the place of a lawyer’s judgment or duty to the court. But vendors should carry a separate responsibility for how verification is designed.

The phrase ‘human review’ is too often treated as the answer

Vendors regularly say that a human remains in the loop. It sounds reassuring, but it tells us very little.

What does the human actually see? Can they open a material proposition to the source page and read the surrounding passage? Can they see which documents were supplied, which were processed, which were excluded and what expected material was not found? Do conflicts remain visible, or are they flattened into a confident sentence?

If the lawyer has to reopen the full record, recreate the search and work out what the system may have missed, the vendor has not really designed a verification workflow. All it has done is transferred the verification cost to the customer.

A 2026 survey of 850 senior legal professionals found that 67 per cent were concerned the cost of human verification and oversight could outweigh AI’s efficiency benefits. Almost half said people always or often materially changed AI output before use.

AI can still save substantial time. The point is that vendors should be judged on the time it takes for a lawyer to arrive at having usable work, instead of simply providing a first draft (of varying quality).

What vendors should be responsible for

A legal AI vendor should be able to explain how its product makes a claim checkable.

The source should be visible in context, that is table stakes these days. The scope of the work needs to be visible too. A source link tells the lawyer what supports a proposition. It doesn’t tell them what the system never searched, could not read or failed to find.

The checking process also needs to be capable of reaching a different result. Asking the same model to approve its own answer is not much of a control if it repeats the same assumptions. When the model fails to read or understand information, the product should have a clear response. It could block, warn or escalate, but whatever it does needs to stay attached to the work.

Vendors should also be careful about the claims they make. A benchmark score is not the same as a lawyer being able to inspect a specific output in the time available. ‘Grounded’ is not enough if the user cannot see the source, the search scope and the unresolved issues.

These are design choices. Vendors control them, and they should be willing to stand behind them.

A better way to test legal AI

The most useful test is not a polished demonstration on a matter chosen by the vendor. It is a matter the firm already knows.

Use a closed, de-identified or internally approved matter. Keep the known gaps and conflicts private. Ask the product to produce work the team can check against its own knowledge. Then look at what it found, what it missed, what remained unresolved and how long it took to reach work the lawyers were prepared to use.

A vendor that is confident in its product should welcome that test.

Lawyers will continue to carry professional responsibility for the work they adopt. That should not change. But vendors should carry responsibility for whether their products make that work practical to verify.

The lawyer should own the judgment. The vendor should own the design that makes the judgment possible.

Mary Technology builds the factual record for litigation across the whole matter, keeps contradictions and gaps visible, and makes that record available across the tools firms choose.