Why We Built Gandalf: A Reliable, Single Machine, Rate Limiter

When you’re building an API-driven SaaS in 2022, everyone tells you to use Redis for rate limiting. But here’s the thing nobody talks about: most startups don’t need distributed rate limiting on day one. What they need is something that works reliably on a single machine, doesn’t require managing another service, and can actually handle real-world traffic patterns.

That’s why we built Gandalf – and yes, we named it Gandalf because “You shall not pass!” jokes never get old in the context of rate limiting. The Gopher-as-Gandalf logo was too good to resist.

We’re open-sourcing it at github.com/Feedback-Frog/gandalf.

The Single Machine Reality

Let me paint you a picture of early-stage Feedback Frog: We had real customers, real traffic, and real SLAs to meet. We were running on a single beefy VM on Fly.io because, honestly, that’s all we needed. Our traffic patterns were bursty – a customer would integrate our SDK and suddenly send thousands of feedback submissions in minutes.

The conventional wisdom said: “Set up Redis, use it for rate limiting, done.” But that meant:

Another service to manage and monitor
Network latency for every rate limit check
Complexity we didn’t need yet
Another potential point of failure

We needed rate limiting that lived with our application, not in a separate service.

Enter BadgerDB: The Embedded Solution

Our first attempt used BoltDB – a solid embedded key-value store. It worked, but write performance was painful. During traffic spikes, BoltDB’s single-writer limitation became our bottleneck. We were rate limiting ourselves more than our users.

Then we discovered BadgerDB. Built by the Dgraph team, it’s an embedded key-value store optimized for SSDs with an LSM tree structure. The performance difference was staggering:

Sequential operations: 38,000+ requests/second
Concurrent operations: 3,600+ requests/second
Write performance: 10x faster than BoltDB

Suddenly, our rate limiter wasn’t the bottleneck anymore.

Building Gandalf: Simple by Design

Here’s what Gandalf looks like in practice:

// Create a rate limiter - that's it, no Redis, no config files
provider := &gandalf.StaticProvider{
    TimeInterval: 1000,
    TimeUnit:     "second",
}

limiter, _ := gandalf.NewRateLimiterWithProvider("rate_limits.db", provider)
defer limiter.Close()

// Use it anywhere in your code
result, funcErr, rateLimitErr := limiter.Limit("user-123", func() (any, error) {
    // Your actual API logic here
    return processFeedback(), nil
})

if rateLimitErr == gandalf.ErrRateLimitExceeded {
    return TooManyRequestsError
}

That’s it. No Redis connection strings. No network calls. Just a local database file that persists limits across restarts.

The Power of Not Over-Engineering

In our early days at Feedback Frog, Gandalf protected us from several near-disasters:

The Accidental Loop: A customer accidentally put their feedback submission in a tight loop. Gandalf limited them to their allocated rate, kept our API responsive, and gave us time to reach out and help them fix their integration.

The Viral Widget: One customer’s feedback widget went viral. Traffic spiked 100x in an hour. Gandalf smoothly enforced per-project limits, ensuring all our other customers stayed unaffected.

The Migration Mishap: A large customer tried to migrate years of historical feedback all at once. Gandalf’s persistent storage meant even when they hit limits, they could resume exactly where they left off after the rate limit window reset.

Growing with Your Startup

The beauty of Gandalf is that it scales with your journey:

Stage 1: Single Machine Simplicity

When you’re just starting, Gandalf gives you enterprise-grade rate limiting without enterprise-grade complexity. One VM, one binary, one embedded database. Your entire rate limiting infrastructure fits in a single Go binary.

Stage 2: Service Protection

As you grow and split into services, Gandalf becomes your SLO protector. Each service gets its own Gandalf instance, protecting itself from overwhelming upstream services:

// Payment service protects itself from the API
paymentLimiter.Limit(requestID, func() (any, error) {
    return processPayment(), nil
})

// Email service protects itself from notification storms
emailLimiter.Limit(userID, func() (any, error) {
    return sendEmail(), nil
})

Stage 3: Public API Protection

When you launch your public API, Gandalf’s database-backed providers let you set custom limits per API key:

rateLimitFetcher := func(key string) (int, string, error) {
    // Fetch from your database
    apiKey := getAPIKeyFromDB(key)
    return apiKey.RateLimit, apiKey.RateLimitUnit, nil
}

provider := gandalf.NewDataProvider(rateLimitFetcher)

Real Numbers from Production

After two years running Gandalf in production at Feedback Frog:

Zero rate limiting outages (BadgerDB is rock solid)
Sub-millisecond latency for rate limit checks
38,000+ requests/second capability on a single machine
10GB of RAM saved by not running Redis
$200/month saved on infrastructure costs

Why We’re Open Sourcing

We built Gandalf to solve our own problem, but we realized every early-stage SaaS faces the same challenge: you need rate limiting from day one, but you don’t need distributed systems complexity until much later.

The Go community gave us so much – Twirp for making protobuf fun and simple, GORM for our ORM, Fiber for high-performance HTTP. It’s time to give back.

Gandalf is our contribution to founders and engineers who are building the next generation of API-driven products. It’s for those who believe in:

Starting simple and adding complexity only when needed
Embedded solutions that reduce operational overhead
Practical engineering over resume-driven development
Solving real problems instead of imaginary scale challenges

When NOT to Use Gandalf

Let’s be honest about Gandalf’s limitations:

If you need distributed rate limiting across multiple regions, use Redis
If you’re already at massive scale, you probably have Redis anyway
If you need sliding window algorithms, Gandalf uses fixed windows
If you need rate limiting across multiple machines, each Gandalf instance is independent

But if you’re building an API-driven SaaS, running on one to a few machines, and want rate limiting that just works? Gandalf has your back.

The Philosophy: Build for Today, Prepare for Tomorrow

Too many startups die from premature optimization. They build for imaginary scale while real customers suffer from actual problems. Gandalf embodies our philosophy:

Solve today’s problem with today’s architecture
Make migration easy when tomorrow comes
Don’t carry complexity you don’t need
Performance matters, but simplicity matters more

Getting Started

Installing Gandalf takes one line:

go get github.com/feedback-frog/gandalf

Implementing basic rate limiting takes ten lines. Customizing it for your needs takes maybe fifty. Compare that to setting up Redis, managing connections, handling failures, and monitoring another service.

A Rate Limiter for Builders

Gandalf isn’t trying to be the ultimate rate limiting solution. It’s trying to be the right solution for right now – for builders who have customers to serve, features to ship, and don’t have time for unnecessary complexity.

We’ve been using it in production for two years. It’s protected our API through viral spikes, accidental loops, and deliberate attacks. It’s saved us from running Redis when we didn’t need it. Most importantly, it’s let us focus on building Feedback Frog instead of managing infrastructure.

Now it’s yours. Use it to build something awesome. And when you eventually outgrow it and need distributed rate limiting? That’s a success story, not a failure.

Build for today. Ship for customers. Scale when you need to.

Gandalf is open source and available at github.com/Feedback-Frog/gandalf. We’d love to hear how you use it in your projects. Reach out at hello@feedbackfrog.com or open an issue on GitHub.