Error Handling
Learn how the SRVANT Gateway handles upstream failures, network outages, and retries to ensure zero data loss.
Data Durability & Dead Letter Queues
Networks fail, upstreams experience outages, and rate limits are often hit. To ensure that your mission-critical logs are never lost, the SRVANT Gateway implements robust data durability mechanisms out of the box.
1. Automatic Retries
Before dropping any payload, the Gateway automatically retries the delivery up to 3 times with an exponential backoff (1s, 2s, 4s). This gracefully handles transient 502, 503, and 504 upstream errors without dropping data.
2. Local Dead Letter Queue (DLQ)
If an upstream destination is permanently down and all retries are exhausted, the Gateway writes the failed payload directly to its local filesystem into a Dead Letter Queue (DLQ). By default, these files are saved to /tmp/srvant_dlq (configurable via the DLQ_DIR environment variable).
3. Automated Recovery (DLQ Reprocessor)
The Gateway runs a background daemon that sweeps the DLQ_DIR every 5 minutes. It dynamically cross-references the orphaned payloads with your active pipeline configurations, successfully re-enqueues them for delivery, and safely deletes them from the filesystem once delivered. No manual intervention is required to recover lost data!