BATCHING SCHEDULED WEBHOOK DELIVERIES BY SUBJECT

Batching scheduled webhook deliveries by subject

A thousand events at 9am shouldn't mean a thousand HTTP calls. Group scheduled events by subject and deliver them as a single batched webhook your receiver can process atomically.

Naive event systems have a scaling problem disguised by politeness: every event is one HTTP request. Run a job across a thousand customers at midnight and your schedule fires a thousand webhooks into an endpoint that was sized for a Tuesday. Rate limits, thundering herds, cold-start stampedes, and per-call overhead all arrive together.

Recurrence Cloud handles this with a deliberately simple mechanism: events grouped by subject. Events that share a subject on the same scheduled fire get delivered as one webhook, not many.

What “subject” buys you

Group related events under the same subject key:

{
  "type": "billing.reminder",
  "subject": "invoice-run-2025-11",
  "payload": { "invoiceId": "in_123" }
}

When the schedule fires, all billing.reminder events under invoice-run-2025-11 arrive in a single delivery:

{
  "schedule": "monthly-invoices",
  "subject": "invoice-run-2025-11",
  "events": [
    { "type": "billing.reminder", "payload": { "invoiceId": "in_123" } },
    { "type": "billing.reminder", "payload": { "invoiceId": "in_124" } },
    { "type": "billing.reminder", "payload": { "invoiceId": "in_125" } }
  ]
}

One HTTP call. One signature check. One activity-trail entry — with one place to read the receiver’s status instead of correlating a thousand rows.

What “subject” buys you

  1. Rate-limit headroom. N events → 1 call is N× fewer requests on both sides.
  2. Atomic units of work. “Process this invoice run” is one job, not a million micro-jobs scattered across a retry window.
  3. Honest observability. When a batch fails, it fails as one unit with one cause — see monitoring and reliability.
  4. Cheaper agent runs. An AI agent processing a nightly sweep gets its work in one payload, not a loop of tool wake-ups.

Choosing subject boundaries

The subject is your grouping decision — slice by what you want as a unit of work:

  • Per tenant/org: subject: "tenant-acme" — a company’s events delivered together, respecting their rate limits.
  • Per run period: subject: "eod-cleanup-2025-11-02" — a single daily batch that either completes as one transactional unit or retries as one.
  • Per entity type: subject: "customer-sync" — all sync payloads for one receiver in one pass.

Receiver-side notes

Batched deliveries reward the same discipline as single events, with a couple of additions:

  1. Process per-event with an idempotency key so a partially processed batch can resume.
  2. Return success only if the batch is safely committed (or checkpointed) — a 2xx means “all events durable.”
  3. Treat oversize batches as their own unit of work if payloads can grow large; chunk internally rather than expanding the schedule.

Where this matters most

  • Daily digest / report jobs — one receiver, many content sources.
  • Invoice and payroll runs — legally meaningful “did the whole run happen?” semantics.
  • Tenant-graded SaaS — fan-out per customer without fan-out per request.
  • AI/MCP agents — batched tool invocations; see the scheduled MCP post.

Batching keeps delivery counts sane; signed verification keeps the endpoint itself sane. Or go straight to the guide and set one up.