Design schemas that extract well.
Your json_schema is a contract, not a suggestion — the agent must return data that validates against it, or the job fails. This chapter is about writing schemas the agent can actually satisfy.
You should be comfortable creating jobs and reading their lifecycle first — 02 · Your first extraction covers that. Here we focus on the two inputs that decide extraction quality: the json_schema and the description.
1
How your schema is enforced
Two checks at create time, two more at run time. Knowing where each one bites tells you what a schema can and can't get away with.
At create time, the API checks the schema's shape: it must be a non-empty JSON object with "type": "object" at the top level. A bare array, a primitive, null, an empty {}, or any other top-level type is rejected with 400 before a job is created. That's the only schema validation the create endpoint does — it does not deep-check every keyword you used.
At run time, the schema does double duty:
- It becomes the agent's output contract — the agent's final answer must conform to it.
- After the run, the result is validated against the schema again. If the output doesn't conform, the job fails with
result failed schema validation— malformed or non-conforming data is never delivered as a success.
Run time is also where unsupported constructs surface: the schema is converted mechanically before the run, and a keyword the converter can't handle fails the job with invalid json_schema: … — after the create call already returned 201. Stick to plain type / properties / required / items / enum shapes and you stay well inside the supported set.
One more standing rule that shapes everything below: the agent is explicitly instructed that if a value cannot be found, it should use null rather than invent it. Your schema decides whether null is a legal answer — and that single decision is the difference between a graceful partial result and a failed job.
2
The top level is always an object
Want a list? Wrap it. A bare top-level array is rejected at create time.
Rejected — 400 at create time
{
"type": "array",
"items": { "type": "string" }
}Accepted — the same list, wrapped in an object property
{
"type": "object",
"properties": {
"items": {
"type": "array",
"items": { "type": "string" }
}
},
"required": ["items"]
}The wrapper costs you nothing and buys you room to grow — when you later want a count, a source URL, or a confidence note alongside the list, they slot in as sibling properties instead of forcing a breaking reshape.
For the array items themselves, prefer objects with named fields over parallel primitive arrays. One array of { name, price } objects keeps each record's fields together; two parallel arrays of names and prices ask the agent to keep indexes aligned across pages for you.
3
Names carry meaning — pair them with a good description
The description tells the agent what to find and roughly where; the schema tells it the shape. Self-explanatory field names do a surprising amount of the work.
Schemas extract well when field names and enums are self-explanatory. Sparse, anonymous shapes extract poorly:
Extracts poorly — the shape says nothing
{
"type": "object",
"properties": {
"a": { "type": "string" },
"b": { "type": "number" }
}
}Extracts well — every name answers “what goes here?”
{
"type": "object",
"properties": {
"product_name": { "type": "string" },
"monthly_price_usd": { "type": "number" }
}
}The description field (required, up to 2,000 characters) is the other half of the contract. It's what actually drives the agent — be specific about what to extract, and give it a where-ish when you can (“from the pricing page”, “from the company's about page or press coverage”). A precise description plus descriptive field names beats either one alone: the description scopes the hunt, the names disambiguate each slot at fill time.
Naming convention: use snake_case property names, as every example in these docs does — it matches the snake_case wire convention used across the API.
4
Required, nullable, and data that isn't there
The web doesn't always have your field. Decide up front what the agent should do about that — because the default outcome of an unfindable required field is a failed job.
The agent won't invent values — it's instructed to use null for anything it can't find. So a field that is both required and non-nullable paints the agent into a corner when the value isn't findable: it either keeps burning budget looking (ending in extraction did not complete within the step/time budget) or returns something that can't validate (ending in result failed schema validation). Either way you get a failure instead of a useful partial result.
Two ways to make missing data representable:
- Nullable and required —
"type": ["string", "null"]with the field kept inrequired. The key must be present in the result, so the agent answers every field explicitly —nullmeans “looked, couldn't find it”. This is the right default for most fields. - Dropped from
required— the field may be omitted entirely. Fine for genuinely nice-to-have extras, but an absent key is ambiguous (not found? not looked for?), so prefer nullable-required when you care about the distinction.
Reserve strict required + non-nullable for the fields the job is pointless without — the anchor the rest hangs off, like the entity's name. If that can't be found, failing is the correct outcome.
Note that a schema-valid payload isn't automatically a success: if the agent finishes but judges that it couldn't actually satisfy the request, the job fails with agent could not complete the extraction even when the data would have validated. Nullable fields help here too — they let the agent legitimately complete with a partial answer instead of declaring defeat.
5
Constraints: enums help, tight formats hurt
Validation is strict, so every constraint you add is enforced. Use that power where it clarifies, not where it gambles.
enum and numeric types are honored — a constrained field genuinely comes back constrained. An enum is also a naming device: the allowed values tell the agent exactly how to classify what it reads.
{
"type": "object",
"properties": {
"availability": {
"type": "string",
"enum": ["in_stock", "out_of_stock", "preorder"]
},
"price_usd": { "type": ["number", "null"] }
},
"required": ["availability", "price_usd"]
}The flip side: because validation is strict, over-constraining raises the failure rate. Tight regex patterns and exact formats are the usual culprits — a date the page renders as “March 2021” can't satisfy a ^\d{4}-\d{2}-\d{2}$ pattern, and the whole job fails schema validation over one field. Prefer a looser type (string, or an integer year) and normalize on your side.
And remember from section 1: exotic keywords beyond the plain type / properties / required / items / enum set risk failing the job at run time with invalid json_schema: … — the create call won't warn you.
6
Size the schema to the budget
Every field you ask for is work the agent has to do inside the job's time budget. Big schemas need big budgets — or fewer pages.
- Cross-page aggregation is fully supported — the agent carries findings across navigations, so one job can legitimately combine data from several sources.
- But every extra page means extra agent steps, and steps cost time. The default
timeout_sof 120 seconds is fine for a job that reads one to three pages; research-style jobs that aggregate across many sources should raise it — up to the 3,600-second maximum. suggested_urlstrade budget for precision: without seeds the agent spends steps searching for sources; with good seeds it spends them extracting.- If a schema keeps failing on budget, split it: two focused jobs with five fields each often succeed where one ten-field job times out — and you can run them concurrently.
7
Worked example: naive to robust
A company-research job, iterated three times. Same goal each round — the schema is what changes.
Goal: given a company name, pull founding year, headquarters, headcount, and total funding from the public web.
Attempt 1 — rejected at create time. “Just give me the facts as a list” feels natural, but a bare top-level array never gets past the POST:
{
"type": "array",
"items": { "type": "string" }
}
// → 400: json_schema must be a JSON Schema with `type: "object"`Attempt 2 — accepted, fails at run time. An object now, but every field is required and non-nullable, and the founding date demands an exact format most sources don't publish:
{
"type": "object",
"properties": {
"name": { "type": "string" },
"founded": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$" },
"employees": { "type": "integer" },
"funding_usd": { "type": "number" }
},
"required": ["name", "founded", "employees", "funding_usd"]
}The web knows this company was “founded in 2019” — not on which day. The agent can't satisfy the pattern and won't invent a date, so the job ends in result failed schema validation or burns its budget hunting for a precision that doesn't exist. The strict required list has the same effect on employees for a private company that never discloses headcount.
Attempt 3 — robust. Only the anchor field stays strictly required; everything hard-to-find is nullable but still required, so the agent must answer each one explicitly; the date constraint relaxes to a year; names say exactly what goes where:
{
"type": "object",
"properties": {
"company_name": { "type": "string" },
"founded_year": { "type": ["integer", "null"] },
"headquarters_city": { "type": ["string", "null"] },
"employee_count_estimate": { "type": ["integer", "null"] },
"total_funding_usd": { "type": ["number", "null"] },
"source_urls": {
"type": "array",
"items": { "type": "string" }
}
},
"required": [
"company_name",
"founded_year",
"headquarters_city",
"employee_count_estimate",
"total_funding_usd",
"source_urls"
]
}Submit it with a description that scopes the hunt, and a budget sized for multi-source research. No suggested_urls here — the agent searches the web for sources itself when you don't seed it:
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs \
-H 'authorization: Bearer pcft_live_...' \
-H 'content-type: application/json' \
-H 'idempotency-key: company-research-acme-2026-07-10' \
-d '{
"description": "Research the company Acme Robotics (industrial robotics, acmerobotics.example). Find its founding year, headquarters city, an employee-count estimate, and total funding raised in USD. Use the company site, press coverage, or funding databases. Record the URLs you relied on in source_urls.",
"json_schema": {
"type": "object",
"properties": {
"company_name": { "type": "string" },
"founded_year": { "type": ["integer", "null"] },
"headquarters_city": { "type": ["string", "null"] },
"employee_count_estimate": { "type": ["integer", "null"] },
"total_funding_usd": { "type": ["number", "null"] },
"source_urls": { "type": "array", "items": { "type": "string" } }
},
"required": ["company_name", "founded_year", "headquarters_city", "employee_count_estimate", "total_funding_usd", "source_urls"]
},
"timeout_s": 600
}'A typical result on the succeeded job
{
"company_name": "Acme Robotics",
"founded_year": 2019,
"headquarters_city": "Rotterdam",
"employee_count_estimate": null,
"total_funding_usd": 42000000,
"source_urls": [
"https://acmerobotics.example/about",
"https://example-news.com/acme-robotics-series-b"
]
}That null on employee_count_estimate is the schema doing its job: the agent looked, didn't find a number it could stand behind, and said so explicitly — instead of failing the whole job or making one up. Every other field landed. That's what a robust schema buys you.
Checklist
Before you submit a schema
- Top level is a non-empty object with
"type": "object"; lists are wrapped in a property. - Property names are descriptive
snake_case; no anonymousa/b/cslots. - Every hard-to-find field is nullable (or out of
required); only the anchor fields are strictly required. - Constraints are enums and basic types — no tight
patterns or exact formats you could normalize client-side instead. - Keywords stay in the plain
type/properties/required/items/enumset. timeout_smatches the page count the description implies; thedescriptionsays what to find and where to look.
With results flowing, the next step is getting them pushed to you instead of polling — 04 · Webhooks sets up signed delivery. Field-level details for every endpoint live in the API reference.