Large language models — GPT-5, Claude Opus, Gemini, and others — are no longer experimental curiosities. They are reshaping how lawyers draft contracts, analyze case law, conduct due diligence, and communicate with clients. Yet most attorneys still lack a clear understanding of what these tools actually are, how they work, and where they fail.
Key facts
- Research shows hallucination rates of 69–88% for legal queries on general-purpose models (EXTERNAL-CITE: academic research cited in-article).
- One page of text equals roughly 375–400 tokens.
- Targeted follow-up questions improve LLM output quality by ~20%; vague feedback degrades it (EXTERNAL-CITE: NeurIPS research cited in-article).
This guide bridges that gap. It is written for practicing lawyers who want to use LLMs effectively without needing a computer science degree. We cover the fundamentals, the practical applications, the real risks, and the prompting techniques that separate productive use from dangerous overreliance.
The single most important rule: NEVER rely on case citations provided by any LLM — including those offered by legal-specific tools — unless you have personally verified that the cited case exists and says exactly what you are citing it for.
What Is a Large Language Model?
An LLM is a type of artificial intelligence trained on massive amounts of text — books, articles, websites, court filings, and legal documents. Instead of storing facts like a database, it learns statistical patterns in how language is used. When you type a prompt, the model predicts the most likely next word, one word at a time, based on the patterns it has absorbed.
Think of it less like a search engine and more like an extraordinarily well-read associate. It has encountered virtually every public legal document, treatise, and case commentary ever published. But it does not retrieve stored information — it generates responses based on learned patterns. This fundamental distinction explains both its remarkable capabilities and its dangerous failure modes.
Popular LLMs include OpenAI's GPT-5 (powering ChatGPT), Anthropic's Claude Opus, Google's Gemini 2.5 Pro, Meta's LLaMA 4, and Mistral's Medium 3. Each has different strengths: Claude excels at tone and long-document analysis, GPT-5 at structured reasoning, and Gemini at handling very large context windows.
How LLMs Actually Work: The Mechanics Lawyers Should Understand
Tokenization: Breaking Language Into Pieces
Before an LLM can process your prompt, it breaks the text into smaller units called tokens. A token can be a word, part of a word, or punctuation. For example, the phrase 'liquidated damages' might be processed as two tokens or one, depending on the model's training. One page of text equals roughly 375–400 tokens.
Understanding tokens matters because LLMs have strict limits on how many tokens they can process at once. GPT-5's context window is approximately 128,000 tokens (~300 pages). Exceed that limit and the model starts dropping information — usually from the middle of your document, not the beginning or end.
The Attention Mechanism: How Models Find What Matters
Unlike a human who reads sequentially, an LLM examines all tokens in your prompt simultaneously using an 'attention mechanism.' This allows the model to weigh the importance of every word against every other word. When it encounters 'bank' in your prompt, attention helps it determine whether you mean a financial institution or a riverbank by looking at surrounding context like 'savings account' or 'river.'
For lawyers, this has a critical practical implication: the way you frame your prompt — which words you emphasize, what context you provide, how you structure the question — directly shapes the quality of the response. The model is not just reading your words; it is weighing them against each other.
Training, Fine-Tuning, and RLHF
LLMs go through three stages of development. Pre-training exposes the model to billions of tokens of text, teaching it the patterns of language. Fine-tuning then narrows the model for specific domains — a legal AI platform might fine-tune on court opinions, contracts, and regulatory filings. Finally, Reinforcement Learning from Human Feedback (RLHF) uses human evaluators to rank the model's outputs, teaching it to produce responses that are accurate, professional, and appropriately structured.
This is why a purpose-built legal AI tool like HAQQ consistently outperforms generic ChatGPT for legal tasks: it combines the base model's broad language understanding with domain-specific fine-tuning and feedback from legal professionals.
The Hallucination Problem: Why LLMs Fabricate
Hallucination is not a bug — it is an inherent feature of how LLMs generate text. Because the model predicts the next most likely word based on patterns rather than retrieving verified facts, it can produce responses that sound authoritative but are entirely fabricated. Invented case citations, non-existent statutes, and misquoted holdings are common.
Research shows hallucination rates of 69–88% for legal queries on general-purpose models. Even when you provide the actual case text to the model and ask it to summarize, it may still misquote passages because it generates text from patterns rather than copying from sources. Some studies show models can even 'double down' when challenged, confidently reasserting fabricated citations.
The consequences are real. In Mata v. Avianca (2023), an attorney submitted a brief containing six fabricated case citations generated by ChatGPT. The court sanctioned both the attorney and the law firm. Multiple bar associations have since issued ethics opinions requiring lawyers to verify all AI-generated citations.
LLMs do not know they are hallucinating. The model generates the same kind of confident, well-structured prose whether it is stating a verified fact or inventing a case from whole cloth. Your professional judgment — not the model's confidence level — is the only safeguard.
Why LLMs Give Different Answers to the Same Question
If you ask an LLM the same question twice, you will often get different responses. This is by design. The model introduces controlled randomness (governed by a 'temperature' parameter) when selecting which word to predict next. Lower temperature produces more predictable, focused responses. Higher temperature produces more varied and creative output.
For legal work requiring precision — contract analysis, regulatory compliance — you want low temperature. For brainstorming trial strategies or generating creative arguments, higher temperature may be useful. Most legal AI platforms handle this automatically, but understanding the mechanism helps explain why results vary.
Practical Prompting: The Associate Analogy
The most productive mental model for working with LLMs is to treat them like an extraordinarily knowledgeable but context-deprived junior associate. They have read everything, but they know nothing about your specific case, your client's priorities, or your firm's standards. The quality of their work product is directly proportional to the quality of your assignment memo.
Rule 1: Be Absurdly Specific
Specificity is ranked as the single highest-impact prompting technique by both Anthropic and OpenAI. Never say 'Review this NDA.' Instead: 'Identify all non-compete, non-solicitation, and non-disclosure obligations. For each, specify the restricted activity, geographic scope, duration, and carve-outs. Flag any provision that would survive a change of control.'
The difference in output quality is dramatic. A vague prompt produces a generic summary you could have written yourself. A specific prompt produces section-by-section analysis with citations to exact provisions — work product that actually advances your case.
Rule 2: Use the IRAC Framework
You already know how to structure legal analysis. Apply the same framework to your prompts: Issue (state the task), Rule (set the criteria), Application (point to the facts), Conclusion (define the output format). This structure — which Anthropic and OpenAI call 'structured framework prompting' — consistently produces superior results because it mirrors how the model's attention mechanism processes information.
Rule 3: Curate Your Exhibits
Stanford research confirms a significant performance drop when models process queries buried in long context. The model attends most strongly to the beginning and end of your input. Instead of uploading a 200-page agreement and saying 'tell me everything,' extract the relevant sections and definitions. Feed the model curated, focused input — the same way you would prepare exhibits for a brief.
HAQQ AI kostenlos testen
Erleben Sie KI-gestützte juristische Recherche und Entwurf
Rule 4: Show a Work Product Example
When you train a new associate, you show them a good memo. LLMs work identically. Providing 2–3 high-quality examples of the output format you want (what researchers call 'few-shot prompting') dramatically improves results. Beyond 3 examples, returns diminish. Poor examples actively degrade output — the model pays attention to bad examples just as much as good ones.
Rule 5: Never Accept the First Draft
Research from NeurIPS shows targeted follow-up questions improve output quality by approximately 20%. But vague feedback like 'make it better' or 'try again' actually degrades quality. Instead, challenge the model on specific points: 'What about the affiliate transfer carve-out in Section 3.2(b)? Does that create a gap in the anti-assignment protection?'
Rule 6: The AI Won't Tell You It's the Wrong Question
LLMs will confidently and competently answer whatever you ask. They will never tell you that you asked the wrong question. Issue-spotting remains the lawyer's exclusive domain. The model answers; you must identify what needs answering.
What LLMs Can Do for Legal Teams Today
- Contract review and redlining: Scan agreements, identify deviations from standard templates, and flag risky clauses
- Legal research and summarization: Synthesize large volumes of case law, statutes, and regulatory material into structured memos
- Due diligence: Extract and condense information from large document sets, identifying patterns across hundreds of agreements
- Drafting: Generate first drafts of pleadings, briefs, memos, client letters, and discovery requests
- Compliance checking: Compare contract terms against regulatory requirements like GDPR, AML, ESG, or DORA
- E-discovery: Analyze document sets using semantic understanding rather than just keyword matching
- Search term generation: Convert natural language descriptions into Boolean queries and refine search strategies
- Template creation: Generate jurisdiction-specific templates that adapt to contract type and party details
Privacy, Security, and Ethical Considerations
As lawyers, we are entrusted with sensitive, privileged, and private information. Using general-purpose LLMs like ChatGPT for client work raises serious ethical obligations. Most public models retain conversation data and may use it for training unless you explicitly opt out. This creates a potential breach of client confidentiality.
Key Privacy Steps
- Disable training data sharing: In ChatGPT settings, toggle off 'Improve the model for everyone'
- Avoid sharing links to conversations containing privileged information
- Consider using enterprise-grade legal AI platforms with built-in data protection, anonymization, and on-premise hosting options
- Establish firm-wide AI usage policies that comply with your bar association's ethics guidelines
- Never upload unredacted client documents to public AI tools without informed client consent
Purpose-built legal AI platforms like HAQQ address these concerns by providing enterprise-grade security, data anonymization, and compliance with legal professional standards — without requiring you to configure privacy settings on consumer tools.
Questions to Ask AI Vendors
When evaluating legal AI tools, these five questions separate serious platforms from flashy demos:
- What data is the model trained on, and is it jurisdiction-specific?
- Where is the model hosted, and does it meet your data sovereignty requirements?
- How is sensitive information handled before it reaches the model?
- Are prompts and outputs stored? Are they ever used to improve the model?
- Can outputs be traced and audited for compliance and accountability?
How LLMs Are Changing the Practice of Law
For junior lawyers, the shift means developing new skills: refining AI-generated outputs, spotting gaps in model analysis, translating AI summaries into actionable next steps, and handling exceptions that fall outside learned patterns. For senior lawyers, the challenge is strategic: deciding where LLMs add value, setting review standards, and training teams to critically evaluate AI output.
The firms that adapt fastest will not be those that use the most AI — they will be those that use it most intelligently. That means understanding the technology's capabilities and limitations, establishing governance frameworks, and choosing platforms built specifically for legal work rather than repurposing consumer tools.
The Bottom Line
LLMs are the most significant technology shift in legal practice since electronic discovery. They will not replace lawyers — but lawyers who understand and use them effectively will replace those who do not. The key is informed adoption: knowing what LLMs can do, understanding where they fail, and implementing them within proper professional and ethical guardrails.
HAQQ combines the power of state-of-the-art language models with legal-specific fine-tuning, enterprise security, and purpose-built workflows — giving lawyers the benefits of LLMs without the risks of consumer tools. Whether you are reviewing contracts, drafting pleadings, or conducting research, HAQQ's Justinian engine delivers the accuracy and reliability that legal work demands.
