AI in litigation: Why the reality inside law firms isn’t matching the hype

Nowhere has that promise been louder than in litigation, where AI tools are being positioned to automate document review, streamline case preparation, and deliver faster, cheaper, and radically more efficient outcomes.

But inside law firms, the reality is proving far more nuanced than the narrative suggests.

Speaking on Lawyers Weekly’s recent AI Innovate live stream, Joseph Rayment, the CEO of Automatise, flagged that AI has not yet delivered the sweeping transformation many in the legal sector anticipated, instead emerging as a slower, more measured evolution defined by experimentation and cautious adoption.

While this may not align with the hype the profession has been hearing, Rayment explained that the disconnect between expectation and reality stems from a fundamental misunderstanding of how AI actually creates value within litigation workflows.

Experimentation without transformation

Across the market, Rayment said he has observed a consistent pattern of firms actively experimenting with AI tools, while also noting a common challenge, as many are still “trying to work out where it best can fit” within their structures.

VIEW ALL

Rayment identified that this experimentation has largely taken two distinct forms, characterising the first as a “broad but shallow” model in which firms provide widespread access to AI tools, enabling practitioners to experiment with and apply them as they see fit.

“The first one is broad but shallow, giving everyone access to certain tools that can assist us, as some of the commercially available AI tools can, but within a private enterprise setting,” he said.

While he acknowledged that this approach has “certainly improved AI literacy across the board”, he noted its limitations, pointing out that it has only driven “some level of adoption” among legal practitioners.

In contrast, Rayment identified a second model he described as delivering the “highest value return on investment from AI”, characterised by a “narrow but deep” approach to implementation across a firm’s workflows.

While firms face growing pressure to adopt AI quickly to stay ahead of competitors, Rayment cautioned that even if the technology were to plateau, there are still four to five years of development ahead in how existing tools can be further embedded into legal workflows.

“Right now, assuming the technology doesn’t actually improve any more than it does at the moment, if it doesn’t get any better, there’s still four years, five years worth of development that you can throw into the existing tools of how firms can further integrate this currently available technology into their workflows and expose more tasks to automation,” he said.

The complexity of AI in discovery

Nowhere are these challenges more evident than in discovery, where the demands of accuracy, transparency, and defensibility place the use of AI under particularly intense scrutiny, making its integration both highly valuable and inherently complex.

Rayment identified the clear difficulty the litigators face in ensuring that AI-assisted processes remain reliable and explainable when used to determine which documents should be included in a discoverable production set.

“Into discovery, you need to be able to understand why every single one of the documents that has been placed into the discoverable production set has been referenced as such,” he said.

However, the challenge lies not only in identifying relevant documents, but equally in being able to justify exclusions.

He explained that when firms rely on keyword searches or AI-assisted filtering, they must still ensure sensitive material is not improperly disclosed, while actively screening out documents that should not be shared.

“You also need to be able to understand the ones that should not be included if you’re not doing a full manual review for every one of the documents involved in them,” he said.

“If you’re using keyword searches and AIs on top of it to produce a list of discoverable documents. You also need to screen the ones that should not be shared.

“That means potentially personally identifiable information, privileged information, confidential and trademark, confidential information and trade secrets.”

A profession divided on readiness

Another caution that Rayment identified is that the profession possesses a clear divide in the pace of this adoption, something that he explained is shaped by differing attitudes and still marked by a degree of scepticism.

Rayment shared how, on one hand, some firms are leading the way in AI adoption, investing heavily in education to ensure their lawyers not only understand how the technology works but can use it effectively in practice, ultimately enabling them to generate faster and more tangible returns on investment when adopting new tools.

“I’ve seen some firms that have been at the forefront of this, and their workforce is highly educated on how this technology works and when they decide to invest in a tool, they’re able to get perhaps a faster return on that investment,” he said.

However, he also observed a markedly different approach among some firms, which are deliberately holding back from the technology, operating on the view that there is no need to become “so immersed in AI today”.

This perspective, shared by many across the profession, reflects a pragmatic recognition of the challenges surrounding adoption, particularly the fact that prompt engineering is not a skill most lawyers currently possess.

Rayment acknowledged this as a key barrier to lawyers adopting the technology, noting that these skills are not traditionally taught in legal education.

“Prompt engineering was not taught in any university. There’s not a single person who has graduated from university before 2023 who would know anything about it, unless you’re in a computer science field, a highly specialised field,” he said.

As a result of this reality, he expressed that expecting an entire legal workforce to rapidly acquire these skills and learn how to utilise the technology is, for now, not a reasonable expectation.

“So, to all of a sudden expect an entire workforce to adopt a new series of technologies and be forced to learn how to work in these particular tools, we think is not all that reasonable,” he said.

So how do firms safely deploy AI?

While many practitioners remain focused on workflow integration and output reliability, AI’s deployment in legal settings raises significant privacy and regulatory concerns, forcing firms to proceed cautiously as they grapple with how to implement the technology safely.

Rayment offered a stark warning in this area, urging firms to exercise extreme caution when using commercially available AI tools with client data, referencing ongoing legal action brought by The New York Times against OpenAI and Microsoft.

“The good rule of thumb is stay the heck away from the commercially available tools if you’re going to use client data in your AI practices,” he said.

“There’s a New York Times lawsuit against OpenAI and Microsoft, where, supposedly, private conversations that were advised as being subject to being deleted are currently being held by court order and segregated and for the purposes of the Times, reviewing if any of those particular conversations might have breached their copyright facilities.

“So, first of all, be aware of where your data is being located.”

He also stressed that firms must strictly adhere to internal policies governing the use of AI systems and conduct thorough due diligence when selecting AI providers.

“Secondly, if you’ve got a firm with various different policies on what systems can be used, adhere to those policies. If you’re in the position of setting those policies for firms, work with your trusted providers,” he said.

“There are a few of them here in Australia that guarantee that the data will not be used to train models, that it will not be shared with third parties, and that it can be accessed securely and safely from behind your firm’s firewall.”

With the deployment of AI in litigation increasingly shaped by an evolving regulatory landscape and courts establishing clear boundaries around its use, Rayment stressed the importance of being fully aware of these requirements and constraints.

“There’s the privacy and security aspect to it, but there’s also a regulatory aspect to it. There are practice notes that specifically apply in NSW, which I believe are very sensible. It’s absolutely the court’s right and responsibility, and they’ve taken this responsibility very seriously to impose limitations on how AI can and should be used,” he said.

“It’s every legal professional’s obligation to be aware of those constraints, up to and including not using this for the generation of affidavits, for not using this in the construction, for understanding where it’s been used in the construction of expert reports, and absolutely when it comes to checking your citations that you’re using, if you’re using AI to help you assist in research.”