AI's Legal Skills Leap: Are Lawyers About to Get Some Serious Competition?
Remember that piece I wrote about Mercor's AI benchmark? The one where we all collectively breathed a sigh of relief because AI couldn't even come close to replacing lawyers? Yeah, well, scratch that.
Things in the AI world move faster than you can say "amicus curiae." Anthropic dropped their new Opus 4.6 model, and it's turning heads. While last month's scores for AI on professional tasks were, let's just say, underwhelming (think sub-25% from the big players), this new release is a game-changer.
This new model nearly hit 30% on the benchmark in a single attempt. But here's where it gets interesting: give it a few more tries, and that number jumps to an average of 45%! That's a massive leap, folks. It seems those new "agentic features," including the intriguing "agent swarms," are seriously boosting AI's problem-solving skills.
Mercor's CEO, Brendan Foody, pretty much summed it up when he said that going from 18.4% to almost 30% in just a few months is crazy. And I agree. While the AI isn't taking over the legal world tomorrow, it can affect how lawyers perform their tasks.
What Does This Mean for the Future?
Look, 30% (or even 45%) is still a far cry from a perfect score. So, lawyers aren't going to be replaced anytime soon. They shouldn't be as comfortable as they were before. This rapid progress shows that the development of these models is not slowing down. In the future, AI might become a partner. It could help with researching, writing briefs, and more. Therefore, lawyers who adopt these technologies could gain a big advantage.
And honestly, isn't that the point of technology? To make our lives easier and more efficient? Maybe instead of fearing the rise of the machines, we should be figuring out how to work with them.
So, keep an eye on this space. The AI revolution is happening, and it's going to be fascinating (and maybe a little scary) to watch it unfold.
1 Image of AI agents:
Source: TechCrunch