02 · Your first extraction

Your first extraction, in depth.

The quickstart got you a result; this chapter explains the machinery. Every job field, the full status lifecycle, poll patterns that behave well, pagination, and cancellation.

You'll need a workspace with the trawl service enabled and an API key with the relevant trawl.* actions — Chapter 1 covers both. If you haven't set those up, start with 01 · Foundations.

Create the job

One POST enqueues an extraction. Two fields are required — the schema and the description — and everything else tunes how the job runs. Requires the trawl.create action.

POST /v1/workspaces/:workspace_id/jobs

curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs \
  -H 'authorization: Bearer pcft_live_...' \
  -H 'content-type: application/json' \
  -H 'idempotency-key: pricing-crawl-2026-07-10' \
  -d '{
    "description": "Extract the product name and the pricing plans (plan name and monthly USD price) from this pricing page.",
    "json_schema": {
      "type": "object",
      "properties": {
        "product_name": { "type": "string" },
        "plans": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": { "type": "string" },
              "monthly_price_usd": { "type": ["number", "null"] }
            },
            "required": ["name"]
          }
        }
      },
      "required": ["product_name", "plans"]
    },
    "suggested_urls": ["https://example.com/pricing"],
    "timeout_s": 180
  }'

Field by field:

json_schema (required) — a JSON Schema describing the shape you want back. The top level must be a non-empty object with "type": "object"; a bare array, a primitive, or an empty {} is rejected with 400. To extract a list, wrap it in an object property, as plans does above. Full design advice lives in 03 · Schema design.
description (required) — a natural-language description of what to extract, up to 2,000 characters, non-empty. This is what drives the agent; the schema only shapes the output. Be specific about what to find and roughly where.
suggested_urls (optional) — up to 50 seed URLs, each http or https. Without seeds, the agent searches the web to find sources itself; with seeds, it starts from your pages. URLs pointing at private or reserved addresses are rejected with 400.
webhook_id (optional, ≤ 64 characters) — the id of a webhook you've already created in this workspace; the job's terminal event is delivered to it. An id that doesn't belong to your workspace returns 404 before the job is created. Skip it for now — polling works without one, and 04 · Webhooks covers push delivery properly.
timeout_s (optional, 1–3,600; default 120) — the wall-clock budget for the extraction, in seconds. The default is fine for one to three pages; raise it for research-style jobs that aggregate across many sources.
max_steps (optional, 1–1,000; default 10) — accepted and stored on the job record. The service manages the agent's effective step budget itself today, so treat timeout_s as your budget lever.
model_tier (optional, ≤ 32 characters; default "default") — leave it unset unless you have a specific reason not to.

The optional Idempotency-Key header (1–256 characters from A–Z a–z 0–9 _ - :) makes retries safe: replaying the same key with the same body within 24 hours returns the original response with the Idempotent-Replay: true header instead of creating a second job. The same key with a different body is rejected with 409 (IDEMPOTENCY_KEY_REUSE).

Two more wire facts worth internalizing now: everything is snake_case on the wire, in both directions, and unknown body fields are silently dropped rather than rejected — so a typo like timeout_seconds won't error, it just won't apply. Double-check field names against the API reference.

Read the response

A successful create returns 201 with the full job resource. The same shape comes back from every job read — learn it once.

Response — 201

{
  "id": "3f2b6c1a-9d4e-4b7f-8a21-...",
  "workspace_id": "<workspace_id>",
  "status": "queued",
  "json_schema": { "type": "object", "...": "..." },
  "description": "Extract the product name and the pricing plans...",
  "suggested_urls": ["https://example.com/pricing"],
  "max_steps": 10,
  "timeout_s": 180,
  "model_tier": "default",
  "webhook_id": null,
  "result": null,
  "error": null,
  "attempts": 0,
  "enqueued_at": "2026-07-10T09:00:00.000Z",
  "started_at": null,
  "completed_at": null,
  "created_at": "2026-07-10T09:00:00.000Z"
}

The first block echoes your inputs (with defaults filled in). The fields that change over the job's life:

status — where the job is in its lifecycle. Section 3 walks through every value.
result — null until the job succeeds; then the extracted data as a JSON object validated against your schema.
error — null unless the job failed; then a human-readable reason (section 7).
started_at / completed_at — stamped when a worker picks the job up and when it reaches a terminal status. completed_at − started_at is your actual extraction time — useful for calibrating timeout_s.
attempts — an informational counter, incremented when a terminal outcome is recorded. Not a retry knob.

What each status means

Five statuses, one direction of travel. Terminal states are frozen — a job never leaves succeeded, failed, or cancelled.

queued — accepted and waiting for a worker. Normally brief. A job that's never picked up is failed automatically after 30 minutes, so nothing waits forever.
running — a worker has claimed the job (started_at stamped) and the agent is browsing. The job stays here until it finishes, hits its budget, or you cancel it.
succeeded — the agent returned data that validated against your schema. result is populated; error is null. Non-conforming output is never delivered as a success.
failed — the extraction didn't produce a valid result. error says why; result is null. There are no automatic re-runs — retrying means submitting a new job.
cancelled — you cancelled it while it was still queued or running (section 6).

Overrunning jobs are cleaned up server-side: a running job that exceeds its timeout_s is failed by a periodic sweep with error set to job exceeded its time budget (reaped). Budget enforcement isn't instantaneous — expect a stuck job to reach failed within roughly timeout_s plus a one-minute grace period plus up to five minutes of sweep cadence. Design pipelines around “terminal eventually,” not “terminal at exactly timeout_s.”

Poll to completion

The result lives on the job row, so GET-until-terminal is a first-class way to collect it — no webhook required. Requires the trawl.read action.

GET /v1/workspaces/:workspace_id/jobs/:id

curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs/<job_id> \
  -H 'authorization: Bearer pcft_live_...'

Pattern notes:

Interval: a few seconds is the right order of magnitude — 5 seconds against a 120-second default budget means at most ~24 requests per job. For long timeout_s jobs, widen the interval accordingly; sub-second polling buys you nothing.
Stop on all three terminal statuses (succeeded, failed, cancelled), not just succeeded — otherwise a failed job spins your loop forever.
Bound the loop in production: give up after timeout_s plus several minutes of headroom, since budget enforcement itself has slack (section 3).
404 means “not yours or not there”: an unknown job id — including one that belongs to another workspace — returns 404 with Job not found. A non-UUID id returns 400.

Once you're running real volume, replace the poll loop with a signed webhook — 04 · Webhooks — and keep GET as the reconciliation path.

List jobs, with pagination

Newest first, cursor-paginated, filterable by status. Requires the trawl.list action.

GET /v1/workspaces/:workspace_id/jobs

curl 'https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs?status=failed&limit=50' \
  -H 'authorization: Bearer pcft_live_...'

Response — 200

{
  "data": [
    { "id": "…", "status": "failed", "error": "…", "...": "..." },
    { "id": "…", "status": "failed", "error": "…", "...": "..." }
  ],
  "pagination": {
    "next_cursor": "eyJjcmVhdGVkX2F0IjoiMjAyNi0wNy0xMFQwOTowMDo…",
    "has_more": true
  }
}

Query parameters:

status (optional) — one of queued, running, succeeded, failed, cancelled.
limit (optional) — 1–100, default 20.
cursor (optional) — the opaque next_cursor from the previous page. Never parse or construct it.

Keep fetching while has_more is true:

list-all-failed.ts

async function listJobs(workspaceId: string, status?: string) {
  const jobs = [];
  let cursor: string | null = null;

  do {
    const qs = new URLSearchParams({ limit: '100' });
    if (status) qs.set('status', status);
    if (cursor) qs.set('cursor', cursor);

    const res = await fetch(
      `${BASE}/v1/workspaces/${workspaceId}/jobs?${qs}`,
      { headers: { authorization: `Bearer ${process.env.PCFT_KEY}` } },
    );
    const page = await res.json();

    jobs.push(...page.data);
    cursor = page.pagination.has_more ? page.pagination.next_cursor : null;
  } while (cursor);

  return jobs;
}

const failed = await listJobs(process.env.WORKSPACE_ID!, 'failed');

Ordering is stable keyset pagination on creation time — new jobs created while you paginate won't shuffle the pages you've already read.

Cancel a job

Changed your mind, or submitted with the wrong schema? Cancel while the job is still queued or running. Requires the trawl.cancel action.

POST /v1/workspaces/:workspace_id/jobs/:id/cancel

curl -X POST https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs/<job_id>/cancel \
  -H 'authorization: Bearer pcft_live_...'

Response — 200

{
  "id": "3f2b6c1a-9d4e-4b7f-8a21-...",
  "status": "cancelled",
  "result": null,
  "error": null,
  "completed_at": "2026-07-10T09:00:12.000Z",
  "...": "..."
}

Only queued and running jobs can be cancelled. A job that has already reached a terminal status is rejected — check status before cancelling if you're racing a fast job.
Cancellation is a bookkeeping operation: an extraction already in flight isn't interrupted mid-browse, but its late result is discarded — a cancelled job never flips to succeeded afterwards.
No webhook event fires for a cancellation. Webhooks deliver job.succeeded and job.failed only.
Unknown id (or another workspace's job) → 404.

When jobs fail

Failures come with a human-readable error string on the job row. The common ones, and what to change before resubmitting:

extraction did not complete within the step/time budget — the agent ran out of time or steps before finishing. Raise timeout_s, provide better suggested_urls so it doesn't spend the budget searching, or narrow the description to fewer facts.
agent could not complete the extraction — the agent finished but judged that it couldn't actually satisfy the request; a “couldn't-do-it” is never shipped as a success. Rework the description, or make hard-to-find fields nullable so a partial answer is representable.
result failed schema validation — the agent's output didn't conform to your schema. Usually an over-constrained schema; loosen strict required lists and tight formats. See 03 · Schema design.
invalid json_schema: … — the stored schema uses a construct the runtime converter can't handle. Stick to plain type / properties / required / items / enum shapes.
seed URL rejected: … — a suggested URL resolved to a private or reserved address at run time. Remove it; Trawl extracts from the public web only.
job exceeded its time budget (reaped) — the job was cleaned up by the server-side sweep: it overran its budget, or was never picked up within 30 minutes. Safe to resubmit as-is; if it recurs, treat it like the budget failure above.

In every case the job row is the debugging record: error tells you what happened, started_at / completed_at tell you how long it ran, and resubmitting with adjusted inputs is the retry mechanism — there is no automatic re-run. Chapter 5 covers operating this at volume: 05 · Operations.

Next: Schema design →