← Back to HAQQ Blog

Can AI Give Legal Advice? We Tested It on 100 Real Questions

By Nicolas Dawalibi · · Updated · 8 min read · Guides

Under US unauthorized-practice rules and OpenAI's own policy, AI cannot give legal advice. What our 2026 tests show it gets right on legal questions, and where it fails.

Key facts

In the United States, no. Every state restricts the practice of law to licensed lawyers. The comment to ABA Model Rule 5.5 notes that the definition of the practice of law "is established by law and varies from one jurisdiction to another". Where the line falls differs by state, but applying law to one person's specific facts, which is what people want when they ask an AI about their landlord or employer, sits at the core of most definitions.

The model providers say the same in their own terms. Since 29 October 2025, OpenAI's usage policies prohibit tailored advice that requires a license, including legal advice, without appropriate involvement by a licensed professional. ChatGPT still answers legal questions; what the policy disclaims is personalised advice, the regulated kind.

Enforcement is arriving from three directions:

Outside the US the rules differ. Many civil-law and Gulf jurisdictions reserve representation and certain advisory work to admitted lawyers without the same broad unauthorized-practice doctrine, so the question there turns on local bar law. Our country-by-country guide covers what lawyers themselves may do with AI.

Legal information is general: what a statute says, how eviction normally works in your state, what at-will employment means. Legal advice is applied: whether you should sign, what you should file, what is likely to happen in your case. Information educates. Advice directs action, invites reliance and carries accountability: a lawyer who gets it wrong faces malpractice claims and bar discipline, and a chatbot faces nothing.

Lawyers using AI have their own rules, from ABA Formal Opinion 512 to the EU AI Act. Consumers have none of that protection, so the caveats have to come from the tool, and caveats are where raw models score worst.

Good at the general picture, unreliable on the specifics that decide a case. Three of our tests measure different parts of that.

TestModelsWhat it measuredResult
100 real r/legaladvice questions (May 2026)Claude Sonnet 4, GPT-4o, Gemini 2.5 FlashAccuracy, completeness, reasoning, practical value, caveatsPassed 88, 87 and 78 of 100; caveats lowest for all three (3.0 to 3.15 of 5)
300 commercial legal tasks (published 5 Jun 2026)10 frontier models, 3,000 answersWhether cited law says what the model claims24% of answers cited or applied law that does not say what was claimed
Five-question MENA trap test (30 Sep 2026)Claude Opus 5.5, GPT-6 Astra, DeepSeek V4 Pro, Gemini 3.1 ProTraps, invented authority, jurisdiction3.30, 3.00, 2.20 and 2.10 of 4; two models named 16 judgments that did not check out

The May run used models that are now two generations old, and newer models are better at reasoning: every model in the September test spotted every trap we set. The failure that survived is invented authority. Asked to support a claim with court judgments, two current frontier models produced citations that matched nothing public as cited. That is the same failure behind most of the 2,202 decisions in Charlotin's database, and 1,267 of those decisions involve people representing themselves.

The second failure is missing caveats. In May every model dove into detailed analysis, often correctly, without saying that a local lawyer should check jurisdiction-specific facts. Gemini 2.5 Flash had the highest accuracy score (4.30 of 5) and the lowest pass rate, because some answers were truncated and others skipped the disclaimer. For legal questions the floor matters more than the ceiling.

Are AI chats protected by attorney-client privilege?

No. Privilege protects confidential communications with a lawyer for legal advice. A chatbot is not a lawyer. In United States v. Heppner (S.D.N.Y., 17 February 2026), Judge Rakoff held that about 31 documents a criminal defendant prepared with a consumer AI tool, on his own initiative and before sharing them with counsel, were protected by neither privilege nor work product. Commentators read the ruling narrowly (the tool's privacy terms and the absence of counsel's direction mattered), but for a consumer the lesson is plain: treat what you type as discoverable.

When do you need a lawyer instead of AI?

Sources

FAQ

Can ChatGPT give legal advice?

Not under its own rules. Since 29 October 2025, OpenAI's usage policies prohibit tailored advice that requires a license, including legal advice, without appropriate involvement by a licensed professional. ChatGPT still answers legal questions, but what it gives is legal information, with no license, no liability and no privilege behind it.

Is it legal to ask AI legal questions?

Yes. Unauthorized-practice rules restrict who may give legal advice and represent people, not who may try to understand the law. The legal risk sits with products that present themselves as lawyers, as the FTC's 2025 DoNotPay order and the Nippon Life v. OpenAI lawsuit filed in March 2026 show.

How accurate is AI on legal questions?

In our May 2026 test of 100 real consumer legal questions, Claude Sonnet 4 passed 88, GPT-4o 87 and Gemini 2.5 Flash 78, with caveats the weakest dimension for all three. Newer models reason better: in our September 2026 trap test every model spotted every trap, but two of four still named court judgments that did not check out.

What is the difference between legal information and legal advice?

Legal information explains the law in general, such as what a statute says or how a process works. Legal advice applies the law to your facts and tells you what to do. Advice is regulated and reserved for licensed lawyers in every US state, and it carries professional accountability.

Are conversations with AI chatbots privileged?

No. In United States v. Heppner (S.D.N.Y., February 2026), the court held that documents a defendant made with a consumer AI tool on his own initiative were protected by neither attorney-client privilege nor work product. Treat anything you type into a general-purpose chatbot as potentially discoverable.

When should I stop using AI and hire a lawyer?

When you are sued, charged or served; when a deadline is running; when the outcome is hard to reverse; when the other side has a lawyer; or when someone must sign, file or appear for you. Before that point, use AI to understand your situation so the lawyer's time goes to judgment.