AI outperforms top corporate lawyers in accuracy — and is 100 times faster

legal startup challenged 20 expert lawyers to test their skills against its AI-powered algorithm. The lawyers generally performed more poorly than the machine. The AI was found to have a level of accuracy on par with the very best lawyers that participated in the challenge — however, it performed the job 100 times faster.

Artificial intelligence (AI) is no longer a figment of our imagination — what’s more, it’s already pretty mainstream. A lot of people are already using voice assistants like Google Home or Amazon Alexa, whose suggestions are powered by artificial intelligence algorithms that tap into vasts amounts of data. Actually, if you’re using the internet, you’re already interacting with AI tech one way or another. Search results, social news feeds, Netflix recommendations — these are all delivered by AI.

As AI becomes more prevalent, researchers estimate that millions of jobs will become displaced in the coming years. The more repetitive the task, the more likely a robot overlord will take over. If you’re employed as a truck or taxi driver, teller, cashier or even as a cook, you run at risk of being sacked in favor of a machine. Some would think creative jobs like writing, painting or creating music are exempted from such trends because there’s the impression you need inherently human qualities to deliver — but that’s just wishful thinking. For instance, AIs can now write scripts for movies, novels, and even classical music like a human composer. Sure, they might not be as good as work produced by humans, but these are all proofs of concept, foretelling grander things to come.

Compared to music or creative writing, AI-algorithms for legal tasks sounds like a breeze. While much legal work requires an actual person — such as in court or when briefing clients — attorneys and legal staff also spend much of their workday analyzing complicated ‘legalese’. But it’s precisely because the language is very rigorous and well defined that a machine might be more fitting for some legal tasks.

Case in point, when LawGeex pitted 20 corporate lawyers against an AI algorithm, the humans were outperformed. The task that both humans and the machine had to complete involved reviewing risks contained in five non-disclosure agreements (NDAs). The participants were experienced lawyers working for important companies such as Goldman Sachs, Cisco and Alston & Bird.

Credit: LawGeex.Credit: LawGeex.
The humans had an average accuracy score of 85%, with the top-performing lawyers achieving 94% and the worst performer scoring 67%. The AI matched the top-performing lawyers, recording 94% accuracy for the task. However, it took only 26 seconds to review all five documents, compared to 51 minutes for the speediest lawyer or 156 minutes for the slowest.

According to We Forum, about 23% of legal work can be safely automated, and LawGeex’s recent demonstration serves as a prime example of this. However, it’s anyone’s guess how far you can take this. For instance, some have proposed that an AI might make for the fairest judge. In 2016, an artificial intelligence system had correctly predicted the verdicts of cases heard at the European Court of Human Rights, with a 79% accuracy.

At this moment, researchers estimate that 71% of all hours spent on labor are performed by humans, and 39% by machines. In 2025, the ratio is expected to flip to 48% human labor and 52% machine.