Simulate your AI cost alerts before they matter
A spend alert is only useful if it fires on real spikes and stays quiet on normal days. Alert fatigue — where the threshold is set so low it fires on busy Mondays — trains people to ignore it, and the one time it matters the alert goes unread. This simulator takes your baseline daily spend and a candidate alert threshold, then projects what a 3x, 10x, and runaway spike would cost per day and shows exactly which scenarios cross the line.
How spike detection works
Most teams alert on daily spend crossing a threshold. The threshold is typically a multiple of baseline:
projected_daily_spend = baseline × spike_multiplier
fires_alert = projected_daily_spend >= thresholdThe three standard scenario multipliers and what they typically represent:
| Multiplier | Common cause |
|---|---|
| 3× | Legitimate traffic surge — product goes viral, a campaign lands, marketing runs an event |
| 10× | Likely a bug — retry loop, broken client, runaway prompt that generates enormous responses |
| 50× or more | Runaway scenario — infinite loop, malicious abuse, misconfigured batch job without a budget cap |
Your threshold should sit comfortably above normal variation (so a good Monday does not page you) but below the smallest spike you would actually care about.
The layered approach: alerts and caps
An alert tells you something is wrong. A hard spending cap is what actually stops the money flowing. These are different tools and you need both:
- Alert (soft): fires when daily spend crosses the threshold. You investigate and decide how to respond. Can be slow if checked only once a day.
- Hard cap / kill switch: stops API calls outright when a budget ceiling is hit. Set this at a level you would never legitimately exceed in a day. Different from the alert threshold.
- Rate alert (faster): an hourly burn-rate check catches a runaway within the hour rather than at end-of-day. Pair with the daily total alert for two layers of detection.
Tuning the threshold
If the 3x scenario crosses your threshold on the simulator and that kind of traffic surge is normal for your product (e.g. social media mention, newsletter drop), your threshold is too low. Raise it until only genuine incidents trip it. If the 10x scenario does not cross the threshold, raise it — a 10x spike almost always warrants a page.
A practical starting point: set the alert threshold at 3–4x your typical 90th-percentile daily spend, and the hard cap at 10–15x. Adjust from there based on your cost variability.
Common runaway patterns to guard against
- Retry storms: a misconfigured client retrying failed API calls without exponential backoff floods the API.
- Prompt amplification: a user-controlled input that generates very long prompts or triggers recursive tool calls.
- Missing rate limits on a public endpoint: direct API exposure without user authentication can be abused quickly.
- Batch job without a token budget: a data pipeline that processes an unexpectedly large dataset without a per-run cap.