How Legal AI Works: LLMs, NLP, and Machine Learning in Law
The technology behind modern legal AI tools rests on three building blocks working together: natural language processing (NLP) lets a machine interpret legal language, machine learning (ML) lets a model learn patterns from case data, and large language models (LLMs) generate and summarize text on demand. Together they turn a document search into something closer to a research assistant.
Understanding that distinction matters because the stakes are asymmetric. Legal AI adoption reached 79% of lawyers in 2025, according to Clio’s Legal Trends Report, and the same systems that save lawyers hours of review time every week can also invent case citations that never existed if their limits go unchecked.
What Is Legal AI?
Legal AI is the umbrella term for generative AI, machine learning, and NLP systems purpose-built for legal work rather than general use. The label has academic roots too — a 2020 ACL survey by Zhong et al. formalized “legal AI” as its own research field years before ChatGPT made the term mainstream. Clio’s 2025 Legal Trends Report found 82% of lawyers who already use legal AI technology plan to expand its use, and among adopters, 65% report better work quality, 54% report more capacity, and 36% report higher profitability.
Legal AI vs. traditional legal software
Older legal software works on deterministic rules and keyword search: type “breach of contract” and it returns documents containing that exact phrase. Legal AI tools differ from general-purpose chatbots and from that older software in a specific way — they search for meaning and intent, not string matches, according to Thomson Reuters. A query about “backing out of a deal” can surface documents discussing termination clauses even without a single matching word.
That shift traces back to how the systems are built. Rule-based programs needed lawyers and engineers to hand-write every pattern they could recognize. Modern legal-specific NLP models instead learn patterns directly from large corpora of contracts, briefs, and case law — no manual rule-writing required.
Why “AI” here means three technologies, not one
Ask what legal AI is and how it works, and vendor documentation from Clio, Thomson Reuters, and Onit all converge on the same answer: it’s three technologies stacked together, not one.
- NLP reads and interprets the language.
- Machine learning finds patterns across thousands of documents.
- LLMs generate fluent text — drafts, summaries, explanations — from what the other two layers surface.
The rest of this article walks through each one.
NLP: How Machines Read Legal Language
Legal language is arguably a worst-case scenario for machines to parse. It mixes archaic phrasing, heavily defined terms, dense cross-references, and words that carry a legal meaning nothing like their everyday one — “consideration” in contract law has nothing to do with being considerate, and “shall” carries obligation that “may” does not. NLP is the layer that enables computers to interpret legal language despite that friction, and a 2021 American Bar Association piece on the technology cited market research projecting the broader NLP market to reach $27.6 billion by 2026 as adoption accelerated.
From keyword search to understanding meaning
Inside a legal AI system, NLP handles a specific set of tasks: named-entity recognition (pulling out parties, dates, dollar amounts), clause classification, extracting relationships between terms, and distinguishing a court’s binding holding from its non-binding dicta. It also has to recognize synonymy that a keyword search would miss entirely — a system that understands legal meaning knows that a “force majeure clause” and an “impossibility provision” are addressing the same underlying risk, per Irys’ legal AI glossary.
None of that is trivial for a machine. A contract might define “Affiliate” in section 1 and then use the term two hundred pages later with no reminder of what it means, and a keyword-matching system has no way to connect the two. Legal-specific NLP models are trained specifically to hold that kind of long-range structure in mind, which is what separates them from general-purpose text search.
The transformer revolution in legal NLP
Legal NLP evolved from rule-based systems to transformer models over roughly two decades: hand-crafted grammar rules gave way to statistical machine learning in the 2000s, which gave way to transformer architectures trained on millions of legal documents — Legal-BERT, introduced by Chalkidis et al. in 2020, is the best-known example. A separate, more deterministic branch called natural language understanding (NLU) extracts meaning with explainability built in, in contrast to the probabilistic, harder-to-audit output of generative models, as the ABA’s SciTech Lawyer noted in its 2026 analysis of cognitive AI in law.
Early proof that this direction worked came in 2016, when a computational study published in PeerJ Computer Science showed an NLP model predicting European Court of Human Rights case outcomes with 79% average accuracy using only case text — years before transformer architectures existed. Legal-BERT and its successors built on that same insight at a much larger scale, training on millions of documents instead of hundreds.
Machine Learning: How Legal AI Learns From Data
Machine learning is a type of statistical modeling that improves at a task by training on examples rather than following pre-written rules, and it is what powers predictive coding in e-discovery. Nearly half of legal professionals report saving one to five hours a week on tasks like document review after adopting generative AI, according to Everlaw’s 2025 Ediscovery Innovation Report, largely because ML-driven review replaces manual document-by-document reading with prioritized, probability-ranked review.
Supervised learning and probability scores
Supervised ML in legal tools works from labeled examples: a lawyer marks a set of documents as relevant or privileged, and the model learns the pattern well enough to score new, unseen documents against the same categories — assigning each one a probability rather than a flat yes/no. The more the system is used and corrected, the more accurate its scoring becomes, since each correction feeds back into the model.
This is a fundamentally different mechanism from an LLM generating text. There is no next-word prediction here — the model is producing a single number per document, and that number is only as reliable as the labeled examples it learned from. A poorly chosen seed set produces a model that is confidently wrong at scale, which is why lawyers stay involved in reviewing the early rounds of scoring.
Predictive coding and TAR in e-discovery
The clearest real-world example is technology-assisted review (TAR) in e-discovery. Lawyers manually code a “seed set” of documents, the model uses those probability scores to rank the entire document population, and the review team then reads only the highest-ranked slice — typically 10-15% of the total volume — rather than every file. Quality is measured by recall: the Electronic Discovery Reference Model recommends recall above 75%, and in practice many TAR workflows target 80% or higher.
That recall threshold is negotiated, not fixed — opposing parties in litigation often agree on a target recall rate before review begins, and the agreed number becomes part of the discovery protocol. Falling short of it can force a firm to redo part of the review, which is why recall tracking runs continuously rather than as a one-time check at the end.
| Review method | Documents reviewed | Ranking basis |
|---|---|---|
| Manual linear review | ~100% | None — read in order |
| Predictive coding / TAR | ~10-15% | ML probability score |
| Keyword search only | Varies, often incomplete | Exact term match |
LLMs: How Generative AI Writes Legal Text
Large language models are a type of machine learning model trained on billions of data points using reinforcement learning from human feedback (RLHF), according to Onit’s overview of LLMs in legal work. Unlike NLP’s classification tasks or ML’s scoring tasks, an LLM’s job is to generate text token by token — producing the drafts, summaries, and explanations lawyers actually read.
Tokens, attention, and next-word prediction
Text entering an LLM is first broken into tokens — roughly 4-5 characters each, not whole words — and the model predicts the next token by weighing every other token in the input through a mechanism called attention. That process repeats one token at a time until a full response is generated. An LLM does not “know” the law; it models the statistical likelihood of language, which is exactly why it can draft fluent prose and why it can also generate a confident, fabricated case citation with equal fluency.
Attention is what lets the model connect a defined term on page one of a contract to its use on page forty, or tie a pronoun back to the party it refers to several sentences earlier. That connective ability is what makes LLM output read as coherent legal prose rather than disjointed fragments — but it is still a statistical weighting, not a legal analysis, which is why a fluent-sounding sentence can still be substantively wrong.
Why the same prompt gives different answers
A setting called temperature controls how much randomness the model injects when selecting the next token, so identical prompts can yield different phrasing — or different substance — run to run. A second issue compounds this on long documents: context drop-off, where clauses near the end of a lengthy contract get less weight in the model’s attention than clauses near the beginning.
The practical takeaway is that LLM output is unstable by design, which is precisely why legal AI vs. general-purpose chatbots is a meaningful distinction — see Legal AI vs. ChatGPT for how specialized tools address that instability with guardrails a consumer chatbot lacks. In legal work, an LLM draft is a starting point, never a final filing.
The Legal AI Stack: From Raw Documents to Verified Answers
A production legal AI stack consists of four layers working in sequence: a data layer, a reasoning layer, a retrieval and citation layer, and a workflow integration layer, as Clio’s technology explainer breaks it down. That layered design is also what separates a reliable enterprise tool from a lawyer typing a question directly into a general-purpose chatbot.
The four layers
- Legal data layer — verified corpora of case law, statutes, and contracts, not the open internet.
- AI reasoning layer — the NLP, ML, and LLM technologies covered above, working together to interpret and generate.
- Retrieval and citation systems — retrieval-augmented generation (RAG) that grounds answers in real sources, plus automated citation verification.
- Workflow integration — connections into document management systems, intake forms, and calendaring so the output reaches a usable place.
| Stack layer | What it does | Example |
|---|---|---|
| Legal data layer | Supplies verified source documents | Case law and statute databases |
| AI reasoning layer | Interprets and generates text | NLP + ML + LLM combined |
| Retrieval & citation | Grounds answers, checks citations | RAG pipeline |
| Workflow integration | Delivers output into daily tools | DMS, intake, calendaring |
One legal AI platform, CoCounsel, has gone as far as achieving ISO 42001 certification — an international standard for AI management systems — as a way to signal that its retrieval and citation layer is independently audited.
How RAG cuts hallucinated citations
Retrieval-augmented generation reduces hallucinated citations by forcing the model to first retrieve real source documents and then generate its answer only from that retrieved material, rather than from memory alone. A verification step follows, cross-checking each citation the model produces against the actual source.
That combination lowers the hallucination rate substantially — it does not eliminate it, which is why human review is required for AI-generated legal output at every stage, not just the drafting stage. A retrieval step is only as good as the corpus behind it, so a RAG system pointed at an incomplete or outdated case database will still confidently cite the wrong thing, just less often than a model with no grounding at all.
How Law Firms Actually Use Legal AI
Clio groups real-world legal AI use into six workflow categories:
- Litigation prep
- Contract drafting and review
- Legal research
- eDiscovery
- Client intake
- Firm operations
The efficiency gains vary by task but are consistently large — in a widely cited LawGeex study, AI reviewed five NDAs for legal risk in 26 seconds at 94% accuracy, while 20 experienced corporate lawyers averaged 92 minutes and 85% accuracy on the identical task.
Research, drafting, and contract review
Legal research now runs on semantic search — matching meaning and legal concept rather than exact keyword strings, so a query surfaces relevant precedent even when the phrasing differs entirely.
Drafting tools generate first-pass memos, client letters, and standard filings from templates combined with generative output, giving associates a starting draft instead of a blank page.
Contract review tools classify clauses automatically, flag deviations from a firm’s negotiation playbook, and suggest redlines — Thomson Reuters notes that AI can pull structured data out of thousands of documents in seconds, a task that would take a paralegal team days.
eDiscovery, intake, and operations
The remaining workflows lean on the same underlying technology in less visible ways: TAR-driven document ranking in eDiscovery, AI chatbots handling initial client intake questions, and automation applied to billing and calendar management.
The thread running through all six categories is the same: AI absorbs volume, and the lawyer keeps the judgment calls. The specific task varies by practice area, but the division between what the software does and what the human is accountable for does not.
Hallucinations, Bias, and Other Risks
The best-documented failure mode in legal AI is the hallucinated citation — a confidently written case reference that does not exist. The canonical example is Mata v. Avianca, a June 2023 case in the Southern District of New York in which two attorneys were sanctioned $5,000 after submitting a brief citing six cases that ChatGPT had fabricated entirely.
Many harms flow from the submission of fake opinions. The opposing party wastes time and money in exposing the deception.
Judge P. Kevin Castel, sanctions opinion in Mata v. Avianca, Inc.
The scale of the underlying problem goes well beyond one case. A January 2024 study from Stanford’s RegLab and Institute for Human-Centered AI found that general-purpose LLMs — including ChatGPT (GPT-3.5), Llama 2, and PaLM 2 — hallucinated on specific legal queries at rates ranging from 69% to 88%. A follow-up Stanford RegLab study published that May tested purpose-built legal AI research tools instead: it found lower but still meaningful hallucination rates of 17% for Lexis+ AI and 33% for Westlaw AI-Assisted Research, both well below the 43% rate measured for general-purpose GPT-4 on the same legal queries.
The hallucination problem in legal research
LLMs hallucinate because they generate the statistically likely next token, not because they check a fact against a ground-truth database — there is no built-in model of what is true unless retrieval and verification are added on top. Mata v. Avianca is the case everyone cites, but Stanford’s data shows it was never an outlier — it was an early, public example of a failure mode that the underlying statistics say happens most of the time a general-purpose model is asked a specific legal question.
That is the core distinction covered in legal AI agents: the workflow layer sitting above raw generation is what turns a plausible-sounding draft into a checked one. A model with no retrieval step and no citation verifier is, by design, the riskiest possible configuration for legal work.
Bias, confidentiality, and inconsistency
Three further risks round out the picture, as Onit’s overview details.
Bias creeps in because a model trained on historical legal data can absorb and reproduce the patterns — including inequities — present in that data. Inconsistency follows from the same temperature-driven variability described earlier, meaning outputs are not fully reproducible run to run.
Confidentiality is the most operationally sensitive of the three: prompts sent to a public, consumer-facing chatbot may be retained or used for further model training, which is exactly why firms handling privileged client information generally move to closed-loop, enterprise legal AI deployments with contractual data-use guarantees instead.
Ethics and Human Oversight: The Lawyer Stays in Charge
Technology competence in legal AI use is not optional — it is a professional obligation. Comment 8 to ABA Model Rule 1.1 requires lawyers to understand “the benefits and risks associated with relevant technology,” a duty the American Bar Association reinforced specifically for generative AI in its Formal Opinion 512, issued in July 2024.
What Model Rule 1.1 means for AI use
Rule 1.1 doesn’t require a lawyer to become an engineer, but it does require understanding roughly what this article covers: how the tool generates its output and where that output is most likely to break. Verifying every citation against its actual source before filing anything is not a best practice — under the competence rule, it is close to a baseline requirement.
State bar associations have started disciplining lawyers specifically for skipping that verification step, not for using AI itself. The distinction matters: the rule does not penalize adoption, it penalizes submitting unchecked output to a court.
A practical division of labor
The ABA’s SciTech Lawyer described a three-tier division of labor in 2026 that lines up with how the technology actually works: deterministic NLU handles extraction tasks with explainable output, LLMs produce drafts and first passes, and the lawyer supplies judgment and takes professional responsibility for the final work product.
Every vendor source referenced in this article converges on the same conclusion — legal AI is built to augment a lawyer’s capacity, not replace their judgment. The technology changes what a first draft costs to produce; it does not change who is accountable for what gets filed.
How to Evaluate Legal AI Tools
Clio’s evaluation framework for legal AI tools breaks down into five steps: examine what data the tool is trained and grounded on, confirm what accuracy safeguards exist, verify security and compliance posture, assess integration with existing systems, and test the tool on a real workflow before committing to it.
Five questions to ask any vendor
- What data was the model trained on, and what does it retrieve from when answering? Vague answers here are a red flag.
- How are citations verified before they reach the user? A retrieval-and-verification step should be visible, not implied.
- Where does client data live, and who can access it? Data residency and sovereignty matter for privileged information.
- Are prompts stored or reused to train future models? This determines whether the tool is safe for confidential matters.
- Can every output be traced back to a source document? Traceability is what separates an auditable tool from a black box.
Beyond the questionnaire, LegalFly and other vendors recommend running a pilot against a real matter before signing, and checking whether the tool has been benchmarked against standard legal NLP datasets such as LegalBench, CUAD, or ContractNLI. On the security side, look for SOC 2 Type II certification, AES-256 encryption at rest, and TLS 1.2 or higher in transit — the baseline expected of any enterprise legal AI vendor handling client data.
