Quickstart.
First extraction in four steps.
By the end of this guide you'll have submitted an extraction job, watched it run, and read back a schema-conforming JSON result. Plan ten minutes.
Prerequisites
- A ProductCraft workspace (create one at
console.productcraft.co) - Any HTTP client (curl, fetch, Postman, …), or the typed Node client:
npm install @productcraft/trawlBoth surfaces are shown side by side below. @productcraft/trawl is generated from the same OpenAPI spec, so paths and field names are identical.
Step 1
Enable Trawl and mint an API key
Trawl is workspace-scoped. Enable the trawl service on your workspace at console.productcraft.co, then mint a workspace API key (prefixed pcft_live_) with the trawl.create and trawl.read actions.
Every request carries the key as a bearer token:
Authorization: Bearer pcft_live_...If the workspace hasn't enabled the service, every route returns 403 with code SERVICE_NOT_ENABLED before the route runs. If the key lacks the required trawl.* action you get a 403 policy denial instead. Full detail on workspaces, keys, and permissions is in Guide 01 · Foundations.
Step 2
Create a job
A job takes two required inputs: a JSON Schema describing the shape you want back, and a natural-language description of what to extract. Seed URLs are optional — without them the agent searches the web itself. With them, it starts from your pages.
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs \
-H 'authorization: Bearer pcft_live_...' \
-H 'content-type: application/json' \
-d '{
"description": "Extract the product name and the pricing plans (name and monthly USD price) from this pricing page.",
"json_schema": {
"type": "object",
"properties": {
"product_name": { "type": "string" },
"plans": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"monthly_price_usd": { "type": ["number", "null"] }
},
"required": ["name"]
}
}
},
"required": ["product_name", "plans"]
},
"suggested_urls": ["https://example.com/pricing"]
}'import { Trawl } from "@productcraft/trawl";
// Reads PRODUCTCRAFT_API_KEY from the environment
const trawl = new Trawl();
const job = await trawl.jobs.create({
description:
"Extract the product name and the pricing plans " +
"(name and monthly USD price) from this pricing page.",
json_schema: {
type: "object",
properties: {
product_name: { type: "string" },
plans: {
type: "array",
items: {
type: "object",
properties: {
name: { type: "string" },
monthly_price_usd: { type: ["number", "null"] },
},
required: ["name"],
},
},
},
required: ["product_name", "plans"],
},
suggested_urls: ["https://example.com/pricing"],
});
// job.status === "queued". The SDK sent an Idempotency-Key for you,
// so a retried create never queues a second job.Response — 201
{
"id": "3f2b6c1a-9d4e-4b7f-8a21-...",
"workspace_id": "<workspace_id>",
"status": "queued",
"json_schema": { "type": "object", "...": "..." },
"description": "Extract the product name and the pricing plans...",
"suggested_urls": ["https://example.com/pricing"],
"max_steps": 10,
"timeout_s": 120,
"model_tier": "default",
"webhook_id": null,
"result": null,
"error": null,
"attempts": 0,
"enqueued_at": "2026-07-10T09:00:00.000Z",
"started_at": null,
"completed_at": null,
"created_at": "2026-07-10T09:00:00.000Z"
}Save the id — that's what you poll in the next step. Two things worth knowing before you write your own schema:
- The top level of
json_schemamust be a non-empty object with"type": "object". To extract a list, wrap it in an object property (asplansdoes above) — a bare top-level array is rejected with a400. - Make hard-to-find fields nullable, like
monthly_price_usdabove. The agent is instructed to usenullrather than invent a value it can't find.
description can be up to 2,000 characters; suggested_urls takes up to 50 http(s) URLs. The default wall-clock budget is 120 seconds — raise it with timeout_s (1–3,600) for jobs that need to visit more pages. Retrying the create safely? Add an optional Idempotency-Key header — replays with the same key and body return the original response for 24 hours. The Node SDK sends one on every create and reuses it when it retries.
Step 3
Poll until the job finishes
Jobs run asynchronously: queued → running → succeeded or failed. The result is stored on the job row, so polling GET is a first-class way to collect it — no webhook required.
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs/<job_id> \
-H 'authorization: Bearer pcft_live_...'// Retrieves until status is succeeded, failed or cancelled,
// honouring Retry-After. Throws APITimeoutError after timeoutMs.
const finished = await trawl.jobs.poll(job.id, {
intervalMs: 5_000,
timeoutMs: 600_000,
});
console.log(finished.status, finished.result, finished.error);status moves through queued (accepted, waiting for a worker) → running (a worker has picked it up — started_at is stamped) → one of the terminal states: succeeded, failed, or cancelled. Terminal states are final. A few seconds between polls is plenty — but budget for queue time as well as run time: timeout_s only covers execution once a worker picks the job up, so a busy workspace can leave a job queued for a while first.
Step 4
Read the extracted data
On success, the result field holds the extracted data — a JSON object validated against your schema before it's stored. Malformed or non-conforming output is never delivered.
Response — status succeeded
{
"id": "3f2b6c1a-9d4e-4b7f-8a21-...",
"status": "succeeded",
"result": {
"product_name": "Example",
"plans": [
{ "name": "Starter", "monthly_price_usd": 9 },
{ "name": "Team", "monthly_price_usd": 29 },
{ "name": "Enterprise", "monthly_price_usd": null }
]
},
"error": null,
"started_at": "2026-07-10T09:00:02.000Z",
"completed_at": "2026-07-10T09:00:41.000Z",
"...": "..."
}That's the whole loop: schema in, validated JSON out. If the job ends failed instead, result is null and error carries a human-readable reason — for example extraction did not complete within the step/time budget when the sources didn't yield the data in time. Adjust the description, seeds, or timeout_s and submit a new job.
On Node, trawl.jobs.createAndPoll() does steps 2 to 4 in one call and types result as the shape you pass:
import { Trawl } from "@productcraft/trawl";
const trawl = new Trawl();
type Pricing = {
product_name: string;
plans: { name: string; monthly_price_usd: number | null }[];
};
const job = await trawl.jobs.createAndPoll<Pricing>({
description: "Extract the product name and the pricing plans.",
json_schema: {
type: "object",
properties: {
product_name: { type: "string" },
plans: { type: "array", items: { type: "object" } },
},
required: ["product_name", "plans"],
},
suggested_urls: ["https://example.com/pricing"],
});
if (job.status === "succeeded") {
// result is typed as Pricing (a type assertion: not validated)
for (const plan of job.result?.plans ?? []) console.log(plan.name);
} else {
console.log(job.status, job.error);
}Where to go next
- Guide 04 · Webhooks — replace polling with signed push notifications when jobs finish.
- Guide 03 · Schema design — schemas that extract well: nullability, descriptive field names, and sizing the budget.