Trawl guides
04 · Webhooks

Push delivery, done properly.

Polling works, but at volume you want results pushed to you. This chapter covers the whole webhook surface: endpoints, the signed delivery, an exact verification recipe, testing, secret rotation, and the delivery log.

You'll need a workspace with the trawl service enabled and an API key with the trawl.webhook.read and trawl.webhook.manage actions — 01 · Foundations covers both. If you haven't run a job yet, do 02 · Your first extraction first.

1

The delivery model

Webhooks are a workspace-level resource. You create an endpoint once, then reference it from any number of jobs via webhook_id.

  • Two events, no subscription matrix. A webhook receives job.succeeded and job.failed — nothing else. It fires for exactly the jobs that reference it via webhook_id; the event name tells you which way the job ended. A job that times out and is reaped server-side fires job.failed. A cancellation fires nothing.
  • The webhook is a notification, not the record. The extracted data also lives on the job row (result), readable any time with GET /v1/workspaces/:workspace_id/jobs/:id. A missed delivery is never lost data — you can always reconcile by polling.
  • Every delivery is HMAC-signed with a per-webhook secret, in the same t=…,v1=… format as other ProductCraft products' webhooks. Section 5 has the exact recipe.

There's no Trawl SDK yet — the HTTP API is the integration surface today, and generated SDKs will follow the pattern of the other products later. Everything below is curl and plain fetch.

2

Create an endpoint

One POST registers a URL and mints a signing secret. Requires the trawl.webhook.manage action.

POST /v1/workspaces/:workspace_id/webhooks
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks \
  -H 'authorization: Bearer pcft_live_...' \
  -H 'content-type: application/json' \
  -H 'idempotency-key: trawl-hook-prod-2026-07-10' \
  -d '{
    "url": "https://hooks.acme.com/webhooks/trawl"
  }'

Response — 201

{
  "id": "9c1e5b7d-2f4a-4c8e-b3d6-...",
  "workspace_id": "<workspace_id>",
  "url": "https://hooks.acme.com/webhooks/trawl",
  "active": true,
  "failure_count": 0,
  "last_status_code": null,
  "last_attempt_at": null,
  "auto_disabled_at": null,
  "first_failure_at": null,
  "created_at": "2026-07-10T09:00:00.000Z",
  "signing_secret": "f3a91c…64 lowercase hex characters…b04e"
}
  • url (required) — up to 2,048 characters. Register an https:// URL on a publicly resolvable host. Validation will accept http://, but delivery enforces HTTPS and rejects private, loopback, link-local, and cloud-metadata targets — an http:// or internal URL saves fine and then every delivery to it fails.
  • active (optional, default true) — an inactive webhook receives nothing.
  • signing_secret — 64 lowercase hex characters (256 bits, generated server-side), returned in plaintext exactly once: here and on a secret rotation. It's stored encrypted and never shown again — reads expose "secret_set": true instead. Put it straight into your secret store.

The Idempotency-Key header matters more here than anywhere else: a retried create without it mints a second webhook with a second secret. With it, replaying the same key and body within 24 hours returns the original response (including the original secret) with Idempotent-Replay: true; the same key with a different body is rejected with 409.

3

Attach it to jobs

Reference the webhook from job creation. One endpoint can serve every job in the workspace.

POST /v1/workspaces/:workspace_id/jobs
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/jobs \
  -H 'authorization: Bearer pcft_live_...' \
  -H 'content-type: application/json' \
  -d '{
    "description": "Extract the product name and pricing plans from this page.",
    "json_schema": { "type": "object", "...": "..." },
    "suggested_urls": ["https://example.com/pricing"],
    "webhook_id": "9c1e5b7d-2f4a-4c8e-b3d6-..."
  }'

The webhook_id must belong to this workspace — an unknown or foreign id returns 404 before the job is created, so you can't enqueue a job whose notification would go nowhere. When the job reaches succeeded or failed, the endpoint gets one signed POST.

4

Anatomy of a delivery

A delivery is a single HTTPS POST with a JSON body and five Trawl headers. Your endpoint has 10 seconds to answer; any 2xx counts as delivered.

Request headers
POST /webhooks/trawl HTTP/1.1
Content-Type: application/json
User-Agent: trawl-webhook/1.0
X-Trawl-Event: job.succeeded
X-Trawl-Event-Id: 7a4f2c91-8e3b-4d6a-9c05-...
X-Trawl-Delivery-Attempt: 1
X-Trawl-Signature: t=1783069351,v1=5f8a3c…hex hmac…d92e
Body — job.succeeded
{
  "event": "job.succeeded",
  "occurred_at": "2026-07-10T09:02:31.000Z",
  "workspace_id": "<workspace_id>",
  "job": {
    "id": "3f2b6c1a-9d4e-4b7f-8a21-...",
    "status": "succeeded",
    "result": {
      "product_name": "Acme",
      "plans": [{ "name": "Starter", "monthly_price_usd": 29 }]
    },
    "error": null
  }
}
Body — job.failed
{
  "event": "job.failed",
  "occurred_at": "2026-07-10T09:04:05.000Z",
  "workspace_id": "<workspace_id>",
  "job": {
    "id": "b81d0f42-6c3e-4a95-b7d0-...",
    "status": "failed",
    "result": null,
    "error": "extraction did not complete within the step/time budget"
  }
}
  • job.result is the full extracted data, validated against your schema — on success you don't need a follow-up GET to fetch it. On job.failed it's null and job.error carries the reason.
  • X-Trawl-Event-Id is unique per event — treat it as your idempotency key (section 7 explains why).
  • X-Trawl-Delivery-Attempt is 1 today; there is no automatic redelivery (section 7).
  • Answer fast. The delivery request times out after 10 seconds. Verify, record, respond 2xx, and do heavy processing asynchronously. Your response body is stored truncated to 1,024 bytes in the delivery log.

5

Verify the signature

Never process an unverified delivery — anyone who discovers your URL can POST to it. Verification is four steps and one HMAC.

The recipe, exactly:

  1. Read the X-Trawl-Signature header. Format: t=<unix seconds>,v1=<lowercase hex>, comma-separated.
  2. Build the signed payload: <t> + "." + raw request body — the body byte-for-byte as received, not re-serialized JSON.
  3. Compute HMAC-SHA256(key = your signing secret, message = signed payload), hex-encoded.
  4. Compare it to v1 with a constant-time comparison.
  5. Reject stale timestamps: if |now − t| exceeds your tolerance, drop the delivery. We recommend about 5 minutes — this is your replay protection, enforced on your side.

It's the same recipe as the ProductCraft Mail and Waitlist webhooks, under a different header name — one verifier function covers all three. Copy-paste version:

verify-trawl-signature.ts
import { createHmac, timingSafeEqual } from 'node:crypto';

const DEFAULT_TOLERANCE_SECONDS = 5 * 60;

export function verifyTrawlSignature(
  signatureHeader: string, // the X-Trawl-Signature header value
  rawBody: string,         // request body, byte-for-byte as received
  secret: string,          // your 64-hex signing secret
  toleranceSeconds = DEFAULT_TOLERANCE_SECONDS,
): boolean {
  const parts = new Map(
    signatureHeader.split(',').map((part) => {
      const i = part.indexOf('=');
      return [part.slice(0, i), part.slice(i + 1)] as const;
    }),
  );
  const t = parts.get('t');
  const v1 = parts.get('v1');
  if (!t || !v1) return false;

  // Replay protection: reject timestamps outside the tolerance window.
  const timestamp = Number(t);
  if (!Number.isFinite(timestamp)) return false;
  if (Math.abs(Date.now() / 1000 - timestamp) > toleranceSeconds)
    return false;

  // HMAC-SHA256 over "<t>.<raw body>", hex-encoded.
  const expected = createHmac('sha256', secret)
    .update(`${t}.${rawBody}`)
    .digest();
  const given = Buffer.from(v1, 'hex');

  // Constant-time comparison.
  return (
    given.length === expected.length && timingSafeEqual(expected, given)
  );
}

And a minimal receiver. The one integration bug to avoid: verification needs the raw body. A JSON body parser that deserializes and re-serializes will change the bytes and every signature check will fail — mount a raw parser on the webhook route.

receiver.ts
import express from 'express';
import { verifyTrawlSignature } from './verify-trawl-signature';

const app = express();

app.post(
  '/webhooks/trawl',
  express.raw({ type: 'application/json' }), // raw bytes, not parsed JSON
  (req, res) => {
    const rawBody = req.body.toString('utf8');

    const ok = verifyTrawlSignature(
      req.header('x-trawl-signature') ?? '',
      rawBody,
      process.env.TRAWL_WEBHOOK_SECRET!,
    );
    if (!ok) return res.status(401).end();

    // Deliveries are at-least-once — dedupe on the event id.
    const eventId = req.header('x-trawl-event-id')!;
    if (alreadyProcessed(eventId)) return res.status(200).end();

    const payload = JSON.parse(rawBody);
    if (payload.event === 'job.succeeded') {
      recordResult(payload.job.id, payload.job.result);
    } else {
      recordFailure(payload.job.id, payload.job.error);
    }

    // Respond 2xx within 10 seconds; queue heavy work for later.
    res.status(200).end();
  },
);

There is no versioned multi-signature scheme — one v1 value, one active secret per webhook. After a rotation (section 9), signatures made with the old secret simply stop verifying.

6

Send a test delivery

Exercise the whole path — SSRF checks, TLS, signature, your handler — without running a real job.

POST /v1/workspaces/:workspace_id/webhooks/:id/test
curl -X POST https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks/<webhook_id>/test \
  -H 'authorization: Bearer pcft_live_...'

Response — 202

{ "accepted": true }

The test is asynchronous — 202 means “queued,” not “your endpoint got it.” What arrives is a job.succeeded event with a random synthetic job id and "result": { "sample": "This is a Trawl test delivery." }, signed with your real secret — so your verifier runs for real. Check your endpoint's logs, or the delivery log (section 8), for the outcome. The webhook must be active to test it — re-enable a disabled one first (section 7).

7

Delivery policy and auto-disable

One attempt per event, a failure streak counter, and an automatic circuit breaker. Design your receiver around these three facts.

  • One delivery attempt per event — no retry schedule. If your endpoint is down, slow (>10 seconds), or returns non-2xx, that delivery is recorded as failed and is not re-sent. This is safe because the webhook is a notification, not the record: the result stays on the job row, and 05 · Operations shows the poll-based reconciliation sweep that catches anything you missed.
  • At-least-once, rarely more. Under normal operation each event is delivered at most once, but an internal crash mid-dispatch can re-deliver — so duplicates are possible, and X-Trawl-Event-Id is the dedupe key. Make your handler idempotent.
  • Success = any 2xx within the 10-second window. Everything else — non-2xx status, timeout, TLS or DNS error, a URL rejected for being non-HTTPS or privately addressed — counts as a failure.
  • Auto-disable: a webhook is switched off automatically after 20 consecutive failures or when a failure streak has lasted 24 hours, whichever comes first. It gets "active": false and an auto_disabled_at timestamp, and subsequent events for it are silently dropped. Any successful (2xx) delivery resets the streak.
  • Recovery is one PATCH: setting "active": true clears the auto-disable state — auto_disabled_at, failure_count, and first_failure_at all reset (section 10). Fix the endpoint first, confirm with a test delivery, then reconcile the jobs that finished while it was down via GET /jobs.

Watch failure_count, last_status_code, and auto_disabled_at on the webhook resource in your monitoring — a climbing failure count is your early warning before the breaker trips.

8

Debug with the delivery log

Every attempt — success, failure, or never-reached-the-wire — is recorded per webhook. Requires the trawl.webhook.read action.

GET /v1/workspaces/:workspace_id/webhooks/:id/deliveries
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks/<webhook_id>/deliveries \
  -H 'authorization: Bearer pcft_live_...'

Response — 200

{
  "data": [
    {
      "id": "1d9e4a72-...",
      "webhook_id": "9c1e5b7d-...",
      "job_id": "3f2b6c1a-...",
      "event_id": "7a4f2c91-...",
      "event_type": "job.succeeded",
      "attempt_number": 1,
      "status_code": 200,
      "response_body_truncated": "",
      "error_message": null,
      "latency_ms": 184,
      "attempted_at": "2026-07-10T09:02:31.000Z"
    },
    {
      "id": "c2b70e15-...",
      "event_type": "job.failed",
      "attempt_number": 1,
      "status_code": null,
      "error_message": "request timed out after 10000ms",
      "latency_ms": null,
      "attempted_at": "2026-07-10T08:55:02.000Z",
      "...": "..."
    }
  ]
}

Returns the most recent attempts, newest first — 50 by default, up to 200 with ?limit=. How to read a row:

  • status_code — your endpoint's HTTP status, or null when the request never reached the wire (DNS failure, TLS failure, timeout, or a URL rejected by the delivery-time HTTPS/private-address checks). For those, error_message says what happened.
  • response_body_truncated — the first 1,024 bytes of your endpoint's response. Have your handler return a short reason on rejection (e.g. invalid signature) and debugging becomes self-service.
  • event_id / event_type / job_id — correlate a row back to the delivery your endpoint saw and the job that caused it.
  • latency_ms — how close you're running to the 10-second budget.

A signature that fails only in production, an expired TLS certificate, a load balancer answering 502 — all of it shows up here before you go log-spelunking on your side.

9

Rotate the signing secret

Rotate on schedule, on staff departure, or on any suspicion of leakage. The new secret is shown once, and old signatures stop verifying immediately.

POST /v1/workspaces/:workspace_id/webhooks/:id/rotate-secret
curl -X POST https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks/<webhook_id>/rotate-secret \
  -H 'authorization: Bearer pcft_live_...'

Response — 201

{
  "id": "9c1e5b7d-2f4a-4c8e-b3d6-...",
  "secret": "0b7d42…new 64 lowercase hex characters…9e1f"
}

Rotation is immediate: every delivery signed after the call uses the new secret, and there's no dual-secret window on the Trawl side. To keep the switchover gap-free, put the flexibility in your receiver:

  1. Teach your verifier to accept a list of secrets — e.g. secrets.some((s) => verifyTrawlSignature(header, rawBody, s)) over a comma-separated env var. Deploy that first.
  2. Rotate, and capture the secret from the response — like the original, it's shown this once and never again.
  3. Add the new secret to your receiver's list immediately. Deliveries dispatched between steps 2 and 3 are signed with the new secret and will fail your verification — keep that window short, and remember a rejected delivery isn't lost data: it's visible in the delivery log and the result stays on the job row.
  4. Drop the old secret once new-signed deliveries are verifying, and after your replay-tolerance window (~5 minutes) has passed.

Two side effects worth knowing: rotation resets the failure streak (failure_count and first_failure_at), but it does not re-enable an auto-disabled webhook — that still takes the PATCH { "active": true } in section 10.

10

List, update, delete

The rest of the management surface. Reads require trawl.webhook.read; mutations require trawl.webhook.manage.

GET /v1/workspaces/:workspace_id/webhooks
curl https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks \
  -H 'authorization: Bearer pcft_live_...'

Response — 200

{
  "data": [
    {
      "id": "9c1e5b7d-2f4a-4c8e-b3d6-...",
      "workspace_id": "<workspace_id>",
      "url": "https://hooks.acme.com/webhooks/trawl",
      "active": true,
      "failure_count": 0,
      "last_status_code": 200,
      "last_attempt_at": "2026-07-10T09:02:31.000Z",
      "auto_disabled_at": null,
      "first_failure_at": null,
      "created_at": "2026-07-10T09:00:00.000Z",
      "secret_set": true
    }
  ]
}

Newest first, no pagination. The secret never appears on reads — secret_set: true confirms one exists. GET …/webhooks/:id returns a single webhook in the same shape (404 for unknown ids, including another workspace's).

PATCH /v1/workspaces/:workspace_id/webhooks/:id
curl -X PATCH https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks/<webhook_id> \
  -H 'authorization: Bearer pcft_live_...' \
  -H 'content-type: application/json' \
  -d '{ "url": "https://hooks.acme.com/webhooks/trawl-v2", "active": true }'

Both fields are optional — send only what you're changing. url follows the same rules as create (≤ 2,048 characters; make it HTTPS and public). Setting "active": true is the recovery path from auto-disable: it clears auto_disabled_at and resets the failure streak. Setting "active": false mutes the webhook without deleting it — useful during receiver maintenance, since jobs keep running and their results stay pollable.

DELETE /v1/workspaces/:workspace_id/webhooks/:id
curl -X DELETE https://api.trawl.productcraft.co/v1/workspaces/<workspace_id>/webhooks/<webhook_id> \
  -H 'authorization: Bearer pcft_live_...'

Returns 204 with no body. Deletion also removes the webhook's delivery-attempt history, so export the delivery log first if you need it for an audit trail. Prefer "active": false when the pause is temporary.

Next: Operations →