Agents
Learn what a Agent is and how it acts as the high-performance edge proxy for all your log data.
What is an Agent?
A Agent is a high-performance, stateless daemon that you deploy inside your own infrastructure. It acts as the edge ingress point for all of your log data, sitting between your data forwarders (like Vector, FluentBit, or Promtail) and your upstream sinks (like Datadog, Splunk, or Elasticsearch).
Agents pull their configuration down from the SRVANT Control Plane dynamically. This allows you to update routing logic and processing rules in real-time without ever needing to restart your Agent instances or drop incoming traffic.
Payload Limits & OOM Protection
To ensure high availability and protect the Agent from Out-Of-Memory (OOM) crashes, SRVANT imposes a strict 10MB limit on incoming request payloads.
Any HTTP request body exceeding 10MB will be immediately rejected with a 413 Payload Too Large response before any pipeline processing or metrics routing occurs. Please ensure your ingestion agents (e.g., Vector, FluentBit) are configured to batch events within this size constraint.
Parsing Optimization Guarantee
To ensure maximum throughput and perfect byte-for-byte fidelity, the proxy engine intelligently checks your pipeline's processors before modifying any event frames. If a pipeline only contains read-only rules (e.g. filter_drop or sampling), the agent skips the JSON or String re-encoding steps entirely. While it must parse the data to evaluate your filter rules, it forwards the exact original raw bytes untouched, saving CPU cycles on serialization and guaranteeing zero data mutation.
Data Durability (The Outbox Pattern)
Networks fail, upstreams experience outages, and rate limits are often hit. To guarantee that your mission-critical logs are never lost, the SRVANT Agent implements a highly durable Write-Ahead Log (WAL) using the Outbox pattern, backed by a lightning-fast LevelDB instance.
1. Write-Ahead Log (LevelDB)
As soon as an event enters the Agent, it is immediately persisted to a local LevelDB database (by default at /tmp/srvant_wal) before any downstream delivery is attempted. This guarantees zero data loss even if the Agent process experiences a hard crash or Out-Of-Memory (OOM) kill.
2. Asynchronous Delivery & Retries
Once safely written to disk, the Agent queues the payload into a high-throughput in-memory channel. A pool of dedicated async workers attempts to deliver the payload. If the upstream provider returns a transient error (e.g., 429, 502, 503, 504), the Agent automatically retries the delivery with exponential backoff.
3. Background Sweeper & Clean-up
When an event is successfully delivered to its Sink, the worker safely deletes it from the LevelDB WAL. If an event fails to deliver after all retries, or if the Agent restarts with orphaned items in its WAL, a background Sweeper routine continuously scans the disk and automatically re-queues them for delivery. No manual intervention is required to recover stuck data!