Run Trawl in production.

timeout_s is the budget you control. Set it per job at create time: 1 to 3,600 seconds, default 120. It bounds the extraction run itself — an agent that hasn't finished inside the budget ends as failed with extraction did not complete within the step/time budget. (There is also a server-side step budget — each agent step is roughly one model call — so timeout_s and page count are the levers you size; see 03 · Schema design for sizing guidance.)

The reaper is the safety net. A background sweep runs every 5 minutes and force-fails jobs that can no longer produce a result:

• A running job is failed once started_at is older than timeout_s plus a 60-second grace period — the grace covers a result that's in flight or a worker restart, so a job finishing right at the buzzer isn't clobbered.
• A queued job that no worker ever picked up is failed after 30 minutes.

Reaped jobs come out looking like any other failure: status failed, completed_at stamped, and a distinctive error string —

If the job references a webhook, a reap fires job.failed to it like any other failure — your receiver doesn't need a special case.

Worst-case time to a terminal state (useful for setting your own alerting thresholds): for a running job whose worker died, timeout_s + 60 s grace + up to 5 minutes of reaper cadence. For a queued job that was never picked up, 30 minutes + up to 5 minutes. If a job is still non-terminal well past those bounds, that's worth a support ticket — it should be impossible.

• Extraction: one run per job. A job is executed once; there are no automatic re-runs of a failed extraction. The attempts field on the job is an informational counter, not a retry knob. To try again, create a new job — ideally with the failure's lesson applied (looser schema, better seeds, bigger timeout_s).
• Transient model-provider errors: retried in place. Inside a run, transient 429/5xx errors from the model provider are retried up to 4 times with backoff. You never see these; they just consume budget.
• Webhook delivery: exactly one attempt per event. There is no redelivery backoff schedule. A failed delivery is recorded in the deliveries log (section 5) and counts toward auto-disable — but it is not re-sent. This is safe because the result lives on the job row: a missed webhook is a missed notification, never lost data. Poll GET .../jobs/:id to recover.
• Messaging-layer redelivery: rare duplicates possible. Delivery is at-least-once end to end — if our side crashes mid-dispatch, the event re-delivers and your endpoint can see the same event twice. Dedupe on X-Trawl-Event-Id.
• Your own retries of the create call: use Idempotency-Key. Replaying POST .../jobs with the same key and body returns the original response (with header Idempotent-Replay: true) for 24 hours instead of enqueuing a second job; the same key with a different body is rejected with 409.

There are no request rate limits on the Trawl API today — no per-workspace quotas, no 429s to handle. Design your integration to be polite anyway (poll on a sensible interval rather than hot-looping on job status); usage-based limits may arrive later, and well-behaved clients won't notice when they do.

The list is newest-first with cursor pagination (limit 1–100, default 20; pass cursor from pagination.next_cursor while pagination.has_more is true). The status filter accepts queued, running, succeeded, failed, and cancelled. Listing needs the trawl.list action; reading a single job needs trawl.read.

On a failed job, the error string is the diagnosis. The common ones — and what to do about each — are in the troubleshooting matrix below.

Returns the 50 most recent attempts, newest first (requires trawl.webhook.read). How to read a row:

• status_code is null when the request never reached the wire — DNS failure, TLS failure, the 10-second delivery timeout, or a URL our delivery guard refused (delivery requires an HTTPS URL resolving to a public address; private, loopback, link-local, and cloud-metadata targets are rejected). In those cases error_message says why.
• response_body_truncated holds up to the first 1,024 bytes of your endpoint's response — enough to see your own error message.
• Success is any 2xx; anything else is a recorded failure.

Auto-disable. Failures accumulate per webhook: after 20 consecutive failures, or once a failure streak has lasted 24 hours, the webhook is disabled — active flips to false and auto_disabled_at is stamped on the webhook resource. Events for a disabled webhook are dropped, not queued. Any 2xx along the way resets the streak.

Recovery is one call — re-activate after fixing your endpoint:

Setting active: true clears the auto-disable state — auto_disabled_at, the failure count, and the streak start all reset (requires trawl.webhook.manage). Then exercise the pipe end to end with the test endpoint: POST .../webhooks/:id/test returns 202 { "accepted": true } and fires a synthetic job.succeeded delivery you can watch land in the deliveries log. Events missed while disabled aren't replayed — but the results are still on the job rows, so reconcile by listing jobs that completed during the outage.

• Requires the trawl.cancel action. Only queued and running jobs can be cancelled; a job that has already reached succeeded, failed, or cancelled is immutable and the cancel is rejected. An unknown id (or a job in another workspace) returns 404.
• On success you get 200 with the job back: status cancelled, completed_at stamped.
• Cancellation doesn't interrupt a running extraction. It marks the job terminal; a worker already browsing keeps going until its own budget, and its late result is discarded — terminal states are frozen, so a cancelled job never flips to succeeded afterwards. Practical upshot: cancelling a running job stops you from seeing the result, not from the work happening.
• Cancellation fires no webhook event. Only job.succeeded and job.failed exist — if your pipeline tracks cancelled jobs, track them from the API response or by polling, not from webhooks.

• Job stuck in queued. Workers pull jobs from a queue; under load, queued time is normal. A job that is never picked up is failed by the reaper after 30 minutes with job exceeded its time budget (reaped) — so “stuck forever” can't happen. If you see queued jobs regularly aging toward that ceiling, that's a platform-side capacity signal — contact support rather than re-submitting the same job repeatedly.
• Failed with extraction did not complete within the step/time budget. The agent ran out of time or steps. Unreachable pages are a common cause — the agent sees a navigation error and tries other sources, which burns budget; if every source is dead or the information genuinely isn't findable, the budget runs out. Fix: verify your suggested_urls resolve, raise timeout_s for multi-page jobs, or narrow the ask.
• Failed with agent could not complete the extraction. The agent finished but judged that it couldn't actually satisfy the request — it never ships a “successful” result it doesn't stand behind. Usually the data isn't on the public web in the shape you asked for. Make hard-to-find fields nullable (03 · Schema design) so a partial answer is a legal answer.
• Blocked or bot-walled pages. Trawl doesn't do anything special to defeat anti-bot walls. A blocked page behaves like any other unreachable page: the agent typically routes around it to other sources, or the job ends in one of the budget failures above. If the only source for your data aggressively blocks automation, Trawl is the wrong tool for that source. Note also that pages behind a login are out of scope — Trawl reads the public web only.
• Failed with result failed schema validation or invalid json_schema: …. Schema problems, not operations problems: the first means the agent's output couldn't satisfy your constraints; the second means the stored schema used a construct the converter can't handle. 03 · Schema design covers both, including the keyword set to stay inside.
• “Empty” extraction — success, but fields are null. Working as designed: the agent is instructed to return null rather than invent a value it couldn't find. Lots of nulls means the description didn't point at where the data lives, or the data isn't public. Sharpen the description, add suggested_urls, and treat persistent nulls on a field as a signal to drop it.
• Webhook silent while jobs are completing. Check the webhook resource first: if auto_disabled_at is set, it was auto-disabled after a failure streak and events are being dropped. Fix your endpoint, PATCH { "active": true }, then POST .../test to confirm — see section 5.
• 403 on every call: trawl is not enabled on this workspace. The workspace doesn't have the trawl service enabled — enable it at console.productcraft.co. A 403 naming a specific action (e.g. Policy denies trawl.create on *) means your key or role lacks that permission instead.

• Duration scales with pages: every extra source the agent visits costs extra steps, and each step is roughly one model call. The default 120-second budget suits a job that reads one to three pages; research-style jobs that aggregate across many sources need a larger timeout_s (up to 3,600).
• Good suggested_urls shorten runs — without seeds the agent spends steps searching for sources before it extracts anything.
• Measure your own workloads: enqueued_at, started_at, and completed_at are on every job row, so queue wait and run duration are one subtraction each. Alert on the worst-case bounds from section 1, not on averages.

From here:

• API reference for every endpoint, field, and status code.
• Concepts for the mental model behind jobs, schemas, and webhooks.
• Revisit 04 · Webhooks when you harden your receiver — signature verification and event dedupe live there.