Clarify the task and decision
This guide turns webhooks and batch processing into a reviewable operating workflow. It connects domain decisions, ownership, evidence, and acceptance so the result continues to work in production.
Select synchronous, webhook, or batch delivery from volume, latency, retry, and operational ownership.
Practical workflow
- 1
Inventory source types, identifiers, fields, locales, owners, and publication states.
- 2
Choose the delivery pattern from volume, latency, editorial control, and failure tolerance.
- 3
Map the source record to a separate language-version record with durable linkage.
- 4
Add authentication, idempotency, retry, cache invalidation, logging, and access controls.
- 5
Test publication, source changes, unavailable results, rollback, keyboard operation, and monitoring before release.
Worked example or tool
A lifecycle diagram becomes an implementation contract for retries, idempotency, monitoring, and replay. In the tool, also record the baseline, owner, decision, evidence, open issue, and approval date. Use a real page or transaction so the team sees dependencies, exceptions, and the maintenance work that follows release.
| Decision point | Record | Acceptance criterion |
|---|---|---|
| Baseline | Observed current state | Source and date recorded |
| Decision | Selected option and rationale | Risk and audience considered |
| Evidence | Test, document, or measure | Reviewable and version-specific |
| Approval | Name, role, and date | All mandatory criteria met |
Choose delivery from latency, volume, and ownership
Use synchronous delivery for small interactive requests that normally complete within the interface timeout and can report a result immediately. Use webhooks when work is asynchronous but each result should enter the CMS as soon as it is ready. Use batch processing for large scheduled collections, controlled imports, or migrations where throughput and reproducibility matter more than immediate delivery. The patterns can coexist, but each content class should have a documented default.
Estimate daily and peak volume, item size, acceptable completion time, retry window, ordering requirement, reviewer capacity, and operational owner. A fast result has no value if the editorial queue cannot process it. Include upstream and downstream limits: CMS export, API rate, queue workers, callback endpoint, database writes, cache invalidation, and review workload. Select the simplest pattern that meets the complete service objective.
| Pattern | Use when | Essential control |
|---|---|---|
| Synchronous | Small request and short bounded latency | Timeout with safe client retry |
| Webhook | Independent jobs should arrive promptly | Signature verification and idempotent event handling |
| Batch | Large controlled set and scheduled completion | Manifest, checkpoint, reconciliation, and replay |
Implement a verifiable webhook lifecycle
Accept HTTPS POST only, verify the signature against the raw body, check the timestamp tolerance, and reject unsupported event versions. Store the event ID under a unique constraint before applying business changes. Return success after durable receipt, then process asynchronously. A repeated event returns success without repeating the side effect. Rotate signing secrets with an overlap period and restrict diagnostic output so it does not reveal signatures or content.
Model states such as received, validated, matched, applied, ignored, retrying, and failed. Match the result to job ID, source ID, source revision, locale, and mode. If the current source is newer, store the result for audit but do not open or replace the current draft. Handle out-of-order events by state transition rules rather than arrival order. Keep a replay tool that requires a reason, operator identity, and scope.
- 1
Verify transport, raw-body signature, timestamp, event type, and contract version.
- 2
Persist the unique event and acknowledge durable receipt.
- 3
Resolve job and exact source revision before changing the CMS.
- 4
Apply an idempotent state transition and create the reviewable draft.
- 5
Record completion or route the event to controlled retry and replay.
Make batches reproducible and reconcilable
Create an immutable manifest with batch ID, creation time, query or selection rule, individual item ID, source revision, locale, mode, priority, and checksum. Freeze the manifest before submission so a later CMS query cannot change what the batch means. Split it into bounded chunks and use stable idempotency keys for every item. Checkpoint completion after durable writes so workers can resume without starting over.
At the end, reconcile submitted, accepted, completed, rejected, stale, failed, and intentionally skipped items. Counts must balance to the original manifest, and every non-completed item needs a reason and next action. Replaying a subset creates a new replay manifest linked to the original. Do not alter the original counts or erase failed evidence. Publish batch outcomes only into review states, with workload limits that protect the editorial team.
The manifest fixes item identity, source revision, settings, and checksum.
Every item operation is idempotent and independently retryable.
Checkpoints resume after interruption without duplicating drafts.
Final status counts reconcile exactly to the manifest.
Replay is scoped, authorised, linked, and auditable.
Monitor queues and rehearse recovery
Monitor accepted rate, completion rate, error rate by category, queue depth, oldest-item age, processing duration percentiles, webhook verification failures, retry count, dead-letter volume, stale-result rate, and time from result to editorial approval. Alert on user impact and growing backlog rather than isolated transient failures. Dashboards separate provider processing, callback delivery, CMS application, and editorial waiting time.
The runbook identifies owners, safe pause, scaling limits, credential rotation, replay approval, dead-letter handling, provider communication, and recovery verification. Exercise lost callback, repeated event, out-of-order event, provider outage, CMS outage, schema mismatch, expired secret, and partial batch completion. Acceptance requires recovery without duplicate publication, silent loss, manual database editing, or removal of the last approved content.
Roles, evidence, and approval
Keep generation separate from publication. A successful response is a draft, not an approval. Store the source identifier and version, transformation settings, result identifier, review state, approver, and publication time. When the source changes, mark the language version for review instead of silently replacing approved content. This makes rollback and audit possible across platforms.
Operations and maintenance
The work does not end at publication. Link the language version or configuration to its source, monitor quality and service measures, and define concrete review triggers. Triggers include source changes, legal changes, new audience needs, recurring support questions, technical changes, and incidents. A named owner evaluates the trigger, opens a new revision when needed, and records renewed approval.
Release checklist
The integration uses durable source identifiers.
Credentials are stored server-side and rotated.
Timeout, retry, and rate-limit behavior are defined.
Repeated requests are idempotent.
Generated content enters a review state.
Source changes invalidate or reopen the version.
Language navigation works by keyboard and assistive technology.
Monitoring covers failures, queues, latency, and stale content.