In the world of B2B SaaS, billing is usually an afterthought. You set up a few subscription tiers in Stripe, add a middleware to check user.is_active, and you’re done.
But in the world of high-scale intelligence platforms—where one client might trigger 10 million API calls in a weekend and another might perform only a single, deep forensic search—the static subscription model fails. To build a sustainable intelligence business, you must move to Usage-Based Billing.
Here is the engineering reality: Usage-based billing is not a financial feature; it is a Distributed Systems Problem.
If your billing system is slow, it throttles your ingestion. If it is inaccurate, it destroys your client’s trust. If it is technically brittle, it becomes a single point of failure that can take down your entire worker fleet.
1. Why Billing Breaks Systems
In an intelligence system, every capture has a cost. There is the proxy cost, the compute time, the storage fee, and the third-party API fee. To capture this accurately, your system must track usage at the Fulfillment Layer.
The naïve approach is to write a billing record to the database every time a worker completes a task:
UPDATE users SET credits = credits - 1 WHERE id = 123
At scale, this kills your database. You are creating a massive write-bottleneck on the most contentious table in your system. If 10,000 workers are hitting that same row simultaneously, your database will collapse under lock contention.
Billing should never be a synchronous write to your primary relational database. It must be an asynchronous, eventual-consistency pipeline.
2. Redis as a Global Counter: Near-Real-Time Quota Enforcement
The first requirement of a usage-based system is Quota Enforcement. You must be able to stop a client from exceeding their budget in near-real-time. We solve this using Redis.
Every client has a “Credit Bucket” in a global Redis instance. Before a worker starts a high-cost ingestion task, it performs an atomic DECR on that bucket.
- Success: The worker proceeds.
- Exhaustion: The worker rejects the task and the system alerts the client.
Because it happens in Redis, this check is sub-millisecond and can handle hundreds of thousands of concurrent requests. It provides the “Hard Ceiling” required to protect the business from runaway costs.
3. Itemized Fulfillment Logs: The Source of Truth
While Redis is great for quota enforcement, it is not a “Source of Truth.” Redis can fail, it can be cleared, and it doesn’t store the metadata required for a client’s invoice.
The second requirement is the Fulfillment Log. Every time a worker completes a task, it emits a Usage Event. This event contains:
- The Client-ID.
- The Correlation-ID of the task.
- The specific SKU (e.g., “Premium Proxy Search”).
- The exact cost in credits.
These events are streamed into a high-throughput queue (like AWS Kinesis or RabbitMQ). A separate Fulfillment Consumer reads these events and writes them to a dedicated, write-optimized “Usage History” table (often a time-series database like ClickHouse or a highly partitioned PostgreSQL table).
This log is immutable. It is the “itemized receipt” we present to the client if they dispute their bill.
4. Stripe Usage-Based Patterns: Syncing the Value
The final step is synchronization. We don’t want to build a custom billing engine; we want to use Stripe.
Stripe’s Metered Billing API allows you to report “Usage Phases.” Instead of reporting every single API call to Stripe (which would be a secondary network bottleneck), we perform Aggregated Syncs.
Every few hours, a background job calculates the total credits used by each client from our internal Fulfillment Logs and pushes that number to Stripe as a single “Increment Usage” call.
This design decouples the high-velocity frequency of the intelligence system from the lower-velocity requirements of financial systems.
5. Conclusion: Billing as a Technical Foundation
Building a usage-based infrastructure is a significant engineering investment. It requires distributed state management, idempotent streaming, and high-performance caching.
But the rewards are immense.
- Profitability: You are never “upside down” on a client’s proxy usage.
- Scalability: Your system can handle bursty clients without manual intervention.
- Trust: You can provide your clients with a transparent, per-task cost breakdown.
Do not treat billing as a “back-office” problem. Treat it as a technical constraint. If you can’t meter your system accurately, you can’t scale your system profitably. Usage-based billing is the fuel for the operator-grade intelligence engine.