AI Raised the Floor. Judgment Still Raises the Ceiling.
A randomised trial of 133 patent lawyers found AI improved everyone's drafting. Only senior lawyers got better at unaided review. What that means for how firms train juniors.
Give lawyers an AI drafting tool for three months and their drafts get better. Take the tool away and ask them to review a draft alone, and only the seniors have improved.
That is the finding of a randomised field trial with practising patent lawyers. It should change how every firm trains its juniors.
The short answer
- 133 lawyers at 11 IP firms took part in a three-month randomised trial. Two-thirds got early access to an AI patent-drafting tool. The rest got AI training but no tool.
- AI improved drafting for everyone, by 0.34 standard deviations at 10 days and 0.38 at 90 days.
- On unaided review, only seniors improved. Lawyers with seven or more years outperformed the control group by 0.45 standard deviations. Juniors showed no discernible improvement.
- Juniors split apart. Their unaided scores showed more very low results, fewer mediocre ones, more good ones, and no more excellent ones.
How the study worked
The paper is NBER Working Paper 35720, by David Autor, Tanya Rodchenko and co-authors. It is a working paper, so it has not been through journal peer review yet.
The setup:
| Item | Detail |
|---|---|
| Participants | 133 lawyers at 11 intellectual property firms that do regular, non-exclusive business with Google |
| Duration | Three months |
| Treatment | Two-thirds of lawyers at each firm got early access to an AI patent-writing tool |
| Control | AI training, but no tool access until the study ended |
| Tasks | Drafting with AI at 10 and 90 days; an unaided redlining task at 90 days |
Read the design with one caveat in mind. The firms work with Google, and Google Research published the summary. That does not make the result wrong. It does mean you should want a second study before treating it as settled.
What it found
| Task | Who | Effect |
|---|---|---|
| Drafting with AI, day 10 | All | +0.34 SD |
| Drafting with AI, day 90 | All | +0.38 SD |
| Unaided redline, day 90 | All | +0.32 SD |
| Unaided redline, day 90 | Seniors (7+ years) | +0.45 SD |
| Unaided redline, day 90 | Juniors (under 7 years) | No discernible improvement |
Source: Google Research summary of NBER w35720.
The drafting result is what vendors will quote. The redline result is the one that matters for a firm.
Three months with AI made seniors better reviewers even without the tool. They were learning from it. Juniors were not. On average they stood still, and underneath that average some got better and some got much worse.
Why this happens
The study measures outcomes, not causes. But the pattern fits what every partner already suspects.
A senior lawyer reading an AI draft compares it with a mental model built over years. Each disagreement is a lesson. A junior has no such model yet. When the draft looks fluent, there is nothing to push against, so there is nothing to learn.
That is why "AI raises the floor" and "AI flattens training" can both be true.
What regulators are saying
The Law Society of England and Wales published a new AI practice note in October 2026. It says AI must not come at the expense of junior lawyers' professional development, and that solicitors must keep the ability to carry out and scrutinise legal work without AI assistance. Its president, Brett Dixon, warned that "overreliance on AI systems may result in a catastrophic gap of future legal skills and knowledge."
Some firms are already redesigning junior time. Ropes & Gray asked first-year associates to spend around 20% of their billable time, roughly 400 of a 1,900-hour target, experimenting and training with AI tools.
The MENA angle
Most Arab bars admit lawyers through a supervised traineeship under a practising advocate. That traineeship is exactly where this study says the risk sits.
Two things make it sharper in the region:
- Juniors carry the language work. In many Gulf and Levant firms, juniors translate between Arabic and English and do first-pass research. These are the tasks AI now does fastest, and the ones where juniors used to build judgment.
- Supervisors are stretched. A partner reviewing three languages of output has less time to explain why a clause is wrong. The study suggests that explanation is the part that turns a junior into a senior.
How to train juniors now
- Keep unaided reps. Every week, juniors review or draft something without AI and get it marked. The trial's key measure was unaided work. Make it yours.
- Make AI drafts the exercise, not the answer. Ask juniors to redline an AI draft and write one line per change explaining why.
- Review reasoning, not only output. A correct clause with the wrong reason is a future mistake.
- Track the split. The study's juniors divided into strong and weak. Look at the spread across your juniors, not only the average.
Where HAQQ Legal AI fits
HAQQ Legal AI shows the sources behind each answer, so a junior can check the reasoning instead of accepting the result. It also gives supervisors one place to see the work. It does not replace the marked, unaided exercise. Nothing should.
Frequently asked questions
Does AI make junior lawyers worse?
Not on average in this study. Juniors did not improve on unaided review, and their results spread out: more very low scores and more good ones.
Is this study peer-reviewed?
Not yet. It is an NBER working paper.
Does it apply outside patent law?
It hasn't been tested outside patent drafting. The mechanism (learning by comparing AI output with your own model) should apply to any drafting-heavy practice, but that is an inference.
Should juniors draft first and use AI second?
For training, yes, at least part of the time. The unaided skill is the one the study found juniors did not build.
What does the Law Society expect?
That AI does not stunt junior development, and that solicitors can still do and scrutinise the work without it.
Sources and further reading
- Google Research, "Does better work always mean better workers?", summary of NBER Working Paper 35720 (Autor, Rodchenko et al.)
- NBER Working Paper 35720 (PDF)
- Legal Cheek, "AI is no substitute for junior lawyers, Law Society warns", 22 October 2026
- Legal Cheek, "Ropes & Gray encourages juniors to spend 20% of billable hours on AI", 19 November 2025
FAQ
Does AI make junior lawyers worse?
Not on average in this study. Juniors did not improve on unaided review, and their results spread out: more very low scores and more good ones.
Is this study peer-reviewed?
Not yet. It is an NBER working paper.
Does it apply outside patent law?
It hasn't been tested outside patent drafting. The mechanism (learning by comparing AI output with your own model) should apply to any drafting-heavy practice, but that is an inference.
Should juniors draft first and use AI second?
For training, yes, at least part of the time. The unaided skill is the one the study found juniors did not build.
What does the Law Society expect?
That AI does not stunt junior development, and that solicitors can still do and scrutinise the work without it.