Expert evals
Agent output scored against a written rubric by credentialed reviewers.
How we source, what a delivery looks like, what it costs and what you agree to. Ready to ask? The request builder is on the home page.
Each maps to a stage of your training loop.
Agent output scored against a written rubric by credentialed reviewers.
Step-by-step expert work with the reasoning kept alongside each step.
Experts rank outputs and write the rubrics that define good work.
Records from data owners, cleared by five rights checks before delivery.
Delivered as JSONL, CSV, documents or your own schema.
Every step ends with something you can review. Nothing is billed before the pilot is accepted.
We confirm the capability, the schema, the volume and what your license must allow.
Share task and rubric
Write the spec
Signed-off spec
Experts for evals and traces. Data owners for licensed sets, found through our data partnerships team.
Nothing needed
Match and recruit
Sourcing plan
Rights record per dataset. Two credentialed reviewers per item, with agreement reported.
Review the record
Five checks, QC
Pilot sample
Full set in your format, under the license you agreed, with a delivery log.
Accept the pilot
Deliver in batches
Dataset and license
Each item carries its reviewers, their agreement and the rights record it came from. Your pipeline reads the fields. Your counsel reads the record.
{ "id": "ev_000184", "domain": "accounting.close", "task": "Reconcile bank statement to GL, March", "rubric": { "completeness": 4, "accuracy": 3, "controls": 4 }, "score": 0.73, "reviewers": [ { "credential": "CPA", "verified": true }, { "credential": "CPA", "verified": true } ], "agreement": 0.86, "rights_record": "RR-SAMPLE-0001", "license": "non-exclusive / 24m / train+eval" }
The detail behind the six steps on the data owners page. Every phase ends with a document you keep. You can stop after any phase until the license is signed.
We map what you hold and where it comes from.
Name one owner
Run the inventory
Data inventory
Title, consent, PII, license terms and sanctions, checked and recorded.
Confirm title and consents
Run five checks
Draft rights record
A small de-identified sample, reviewed before anyone else sees it.
Approve the sample
De-identify and review
Approved sample and scan log
Your counsel reviews the license and sets the buyer rules.
Counsel review
Draft and negotiate
Signed license
Transfer, payment, buyer audits and deletion at term end.
Release approval
Deliver and monitor
Delivery log, deletion certificate
Price is set after scoping and written into the order. The pilot comes first.
Expert evals, priced by credential, rubric depth and number of reviewers.
Evals
Recorded workflows, priced by length and the expertise needed.
Workflow traces
Licensed datasets, priced by scope, term and exclusivity.
Licensed data
We do not publish rates until we have closed deals to base them on.
If you already run agents or staff on professional workflows, their work is data too. Each option is separate and optional.
Credentialed reviewers score where your agent fails on real tasks, and why. You get the findings and a rights record for every item.
Every agent task leaves a trail of steps. With your approval, qualifying trails can be licensed as workflow traces.
Your accountants or analysts record how they do a task, step by step, as a project your company approves, under written terms.
Clean rights run both ways. The seller's terms protect them; yours protect you.
Documented in the rights record and the order.
Standard for every licensed dataset.
We run pilots in finance and accounting now and are sourcing legal. For anything else we reply with a sourcing plan or a clear no. We do not take work we cannot source with clean rights.
A small sample produced through the full process: spec, sourcing, rights check, two reviewers per item. You accept it before the full order starts.
Credentials are verified with the issuer before paid work. Each item gets two reviewers and we report their agreement. Items below the agreed threshold are re-reviewed.
Yes. We deliver in your schema and score against your rubric, or help you write one.
For licensed datasets, yes, for a defined term. For evals and traces written for you, the work is yours under the order.
The company is in setup and no dataset has been licensed yet. Send a request and we will tell you plainly what we can deliver now.