Citations
Cases and statutes that do not exist, or do not say what the summary claims. Courts have sanctioned lawyers who filed AI-invented citations.
Cited case not found in any reporter.
Legal AI writes fluent summaries, citations and redlines that still miss the clause that matters. Lasker supplies review by admitted lawyers: evals for legal-AI teams, and review inside the firm when client work cannot leave.
For teams building contract review, legal research and discovery tools, and for firms deploying them.
Note from Daniel ReyesReviewer: the cap in section 9.2 excludes indemnity claims, so the $2M cap does not limit them.
The agent said the cap covers all claims. Marked incorrect. Counterparty Castle Freight Inc.Party B.
Daniel ReyesReviewer · NY bar · dreyes@halbrook.comemail +1 212 555 0143phone
Illustrative example. Names, places and numbers are made up.
✓ Cleared to leave · rights on recordA fluent answer is the easy part. These are the places a partner or a court finds the problem.
Cases and statutes that do not exist, or do not say what the summary claims. Courts have sanctioned lawyers who filed AI-invented citations.
Cited case not found in any reporter.
The clause that matters is often the exception to the clause the model summarised: carve-outs, caps, survival.
Cap in 9.2 excludes indemnity claims.
Governing law and state rules change the answer. California, for one, generally voids employee non-competes.
Non-compete analysis ignores California law.
A privileged document produced by mistake can put protection at risk. Review has to catch it before production.
Email to outside counsel marked responsive, not privileged.
Response periods depend on the court’s rules, the method of service and holidays.
Deadline counted from the wrong trigger date.
Defined terms, cross-references and conforming changes across a long document.
“Effective Date” defined twice.
Lawyers must keep client information confidential when they use AI, and generally need the client’s informed consent to share it. So the work that teaches legal AI the most rarely leaves the firm.
Benchmarks test whether a model knows the law. A firm needs to know whether it handled this clause, in this deal, under this law.
A lawyer stays responsible for what goes out under their name, so every AI draft still needs a lawyer’s review.
For teams whose legal AI produces work a lawyer has to stand behind.
Review notes on real outputs: what was wrong, why, and the fix.
Source: admitted lawyers reviewing your tool’s outputs
Each authority found, read and matched to the claim it supports.
Source: citation checks by lawyers
Answers checked by a lawyer admitted where the law applies.
Source: reviewers matched to the jurisdiction
Privilege calls on realistic document sets, with the reason.
Source: reviewers with discovery experience
Two drafts for one task, ranked by a lawyer, with the reason.
Source: preference data from reviewers
Review on live matters, inside the firm’s systems.
Source: on-site review; only findings leave
Evals for legal-AI teams use your tasks and documents without client data. Review on live matters happens inside the firm.
| Dataset | What is in a record | Used for | Format |
|---|---|---|---|
| Expert evals | Tool output scored against a written rubric by an admitted lawyer | Evals, release gates | Scores + rationale, JSONL |
| Citation checks | Every authority in a draft: found, accurate, supports the claim | Hallucination tests | JSONL per citation |
| Redline review | Clause-level findings on a contract review, with the correct position | Contract review training | JSONL + marked-up document |
| Preference data | Two drafts for one task, ranked, with the reason | Reward models | JSONL |
| Workflow traces | A lawyer doing the task step by step on a public or synthetic matter | Demonstrations, fine-tuning | JSONL + source documents |
| Review inside the firm | Lawyers review the tool on live matters in the firm’s environment | Deployment evals | Findings the firm approves |
A lawyer’s eval of a contract summary, and a citation check on a research memo.
Illustrative. Field names are our draft schema, and we adapt it to your pipeline in the pilot.
{
"record_id": "lg-0233",
"task": "Summarize limitation of liability, MSA",
"jurisdiction": "New York",
"reviewer": {
"credential": "NY bar",
"verified": true
},
"scores": {
"accuracy": 2,
"authority": 5,
"completeness": 3
},
"finding": "Cap in 9.2 excludes indemnity claims.",
"verdict": "incorrect",
"rights": {
"source": "vendor task, no client data",
"personal_data_removed": true
}
}{
"record_id": "ct-0091",
"draft": "research memo, 14 citations",
"jurisdiction": "California",
"checked": 14,
"issues": [
{ "cite": 6, "type": "not_found" },
{ "cite": 11, "type": "misstated_holding" }
],
"reviewer": {
"credential": "California bar",
"verified": true
}
}The questions a reviewing partner asks, written down before scoring so two reviewers can agree.
| Dimension | What the reviewer checks | Evidence per finding |
|---|---|---|
| Legal accuracy | The statement of the law and the facts is right | Rule or clause cited |
| Authority | Every citation exists and supports the point | Citation-by-citation result |
| Jurisdiction | The right law is applied | Governing law noted |
| Completeness | The issues a lawyer would raise are raised | Missing issues listed |
| Usability | Could go out after light edits | Edits needed |
Reviewers are admitted in the jurisdiction they score, checked with the admitting body. Items with disagreement go to a second reviewer.
Pick the task where your tool is weakest: contract review, research or discovery. The pilot answers whether this review moves it.
One task, one jurisdiction and the failure modes you care about.
Week one
A set of outputs scored by admitted lawyers against a rubric you approve.
You test it in your pipeline
Full set, ongoing review or stop. Price is set after the sample.
No commitment before
Lawyers admitted in a US state, solicitors in England and Wales and barristers, checked with the admitting body and matched to the jurisdiction of the task.
Not for evals: they use your tasks and documents with no client data. Review on live matters happens inside the firm’s environment, and only the findings the firm approves leave.
No. Reviewers evaluate AI output for your team. It is not legal advice to you or to your clients.
US states and England and Wales first. The law applied is recorded on every record.
The license scope is written per dataset: training, evaluation or both, and whether a model trained on it can be used commercially.
Per batch of tasks or per engagement, after the sample. We do not publish ranges until we have closed deals to base them on.
Run a firm? Have lawyers review your AI tools inside the firm, on your systems. Only the findings you approve leave.
Email as@lasker.ai →Send the task, the jurisdiction and the volume. We reply with a rubric draft and what a pilot would include.