01 — LLM INFRASTRUCTURE
Governor
A lightweight LLM gateway, built in Go (learning Go as I go). Real-time cost visibility and hard spend caps — atomic budget checks before a request goes out, mid-stream cancellation the instant one is hit — without standing up LiteLLM/Helicone-style infrastructure.
- Engineering
- Go
- LLM Gateway
- Cost Enforcement
The problem
LLM spend is invisible until the invoice arrives. Teams that want hard limits usually have to stand up heavyweight proxies like LiteLLM or Helicone, which is a lot of infrastructure for one question: can this request afford to run?
The approach
- A single Go gateway that sits between the app and the model provider.
- Atomic budget checks before a request is forwarded, so concurrent calls can't race past a cap.
- Streaming responses are metered as they arrive and cancelled mid-stream the moment a budget is hit.
- Real-time cost visibility per key, without a separate observability stack.
The result
A lightweight, self-hostable gateway that enforces spend caps instead of just reporting them. Actively in development, and also my vehicle for learning Go properly.