Smart Rate Limiting Strategies to Protect Your API Quotas

Rate limiting is your first line of defense against API abuse, unexpected costs, and service degradation. Let's explore strategies that go beyond simple request counting to create a robust protection system.

Why Rate Limiting Matters More Than Ever

With the rise of AI APIs charging per token and traditional APIs billing per request, a single abusive user or compromised key can result in thousands of dollars in unexpected charges. Rate limiting isn't just about fairness—it's about survival.

Understanding Rate Limiting Strategies

1. Fixed Window

The simplest approach: X requests per time window.

// Example: 100 requests per hour
if (requestsThisHour >= 100) {
  return rateLimitError();
}

Pros: Simple to implement and understand Cons: Burst traffic at window boundaries

2. Sliding Window

A more sophisticated approach that prevents burst abuse.

// Tracks requests over a rolling time period
const requests = getRequestsInLastHour();
if (requests.length >= 100) {
  return rateLimitError();
}

Pros: Smoother rate limiting, no burst issues Cons: More complex, higher memory usage

3. Token Bucket

Allows for controlled bursts while maintaining overall limits.

// Tokens replenish at a fixed rate
if (bucket.tokens > 0) {
  bucket.tokens--;
  processRequest();
} else {
  return rateLimitError();
}

Pros: Flexible, allows short bursts Cons: Can be confusing for users

Implementing Multi-Tier Rate Limiting

Global vs. Per-User Limits

Combine both for comprehensive protection:

// Configure in Conduit Link dashboard
// Global rate limit: Applies to all requests
Rate Limit: 10000 requests
Duration: hour
// Per-user rate limit: Requires JWT verification
Rate Limit per User: 100 requests
Duration: hour
JWT Verification: Enabled (uses JWT 'sub' claim)

Tiered Limits by User Type

Different limits for different user segments:

const rateLimits = {
  free: { requests: 100, window: '1h' },
  pro: { requests: 1000, window: '1h' },
  enterprise: { requests: 10000, window: '1h' }
};
function getUserLimit(userType) {
  return rateLimits[userType] || rateLimits.free;
}

Advanced Protection Strategies

1. Adaptive Rate Limiting

Adjust limits based on current load:

function getAdaptiveLimit() {
  const systemLoad = getCurrentSystemLoad();
  const baseLimit = 1000;
  if (systemLoad > 0.8) {
    return baseLimit * 0.5; // Reduce by 50% under high load
  } else if (systemLoad < 0.3) {
    return baseLimit * 1.5; // Increase by 50% under low load
  }
  return baseLimit;
}

2. Cost-Based Rate Limiting

For expensive operations, limit by cost rather than count:

// Track API costs instead of just requests
const costLimits = {
  'gpt-4': 0.03,        // $0.03 per request
  'gpt-3.5': 0.002,     // $0.002 per request
  'embeddings': 0.0001  // $0.0001 per request
};
function checkCostLimit(user, endpoint) {
  const userSpending = getUserSpendingThisMonth(user);
  const requestCost = costLimits[endpoint];
  if (userSpending + requestCost > user.monthlyLimit) {
    return rateLimitError('Monthly spending limit reached');
  }
}

3. Behavioral Analysis

Detect and block suspicious patterns:

function detectAbusePatterns(userId) {
  const recentRequests = getRecentRequests(userId, '5m');
  // Check for suspicious patterns
  if (allRequestsIdentical(recentRequests)) {
    return blockUser(userId, 'Suspicious activity: identical requests');
  }
  if (requestsFromMultipleIPs(recentRequests) > 10) {
    return blockUser(userId, 'Suspicious activity: distributed access');
  }
}

Implementing with Conduit Link

Basic Configuration

Set up rate limiting in your Conduit Link dashboard:

• Global Rate Limit: Protect your API quota
- Set maximum requests (e.g., 1000) - Choose duration: second, minute, or hour - Applies to all requests to your link
• Per-User Rate Limit: Fair usage across users
- Enable JWT verification first - Set per-user limit (e.g., 100 requests/hour) - Uses JWT 'sub' claim to identify users
• Security Settings:
- Allowed paths: Restrict to specific endpoints (e.g., /v1/chat/completions) - Allowed methods: Limit HTTP methods (GET, POST, etc.) - CORS origins: Configure allowed domains

Advanced JWT-Based Limiting

// Generate JWT with user information
const jwt = sign({
  sub: userId,  // Required: User ID for per-user rate limiting
  email: userEmail,
  tier: 'pro',  // Your custom claims
  exp: Math.floor(Date.now() / 1000) + (60 * 60) // 1 hour expiry
}, process.env.JWT_SECRET);
// Include JWT in your API requests
fetch(https://proxy.conduit.link/${LINK_ID}/api/data, {
  headers: {
    'X-Access-Token': 'cl_xxxxx',
    'Authorization': Bearer ${jwt}
  }
})

Handling Rate Limit Errors Gracefully

Client-Side Retry Logic

async function requestWithRetry(fn, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (error.status === 429) {
        const retryAfter = error.headers['X-RateLimit-Reset'];
        const delay = calculateDelay(retryAfter, i);
        console.log(Rate limited. Retrying in ${delay}ms...);
        await sleep(delay);
      } else {
        throw error;
      }
    }
  }
  throw new Error('Max retries exceeded');
}

User-Friendly Error Messages

function formatRateLimitError(error) {
  const remaining = error.headers['X-RateLimit-Remaining'];
  const reset = new Date(error.headers['X-RateLimit-Reset'] * 1000);
  return {
    message: 'You've exceeded your API limit.',
    details: You have ${remaining} requests remaining. Limits reset at ${reset.toLocaleTimeString()}.,
    upgradeLink: '/pricing'
  };
}

Monitoring and Analytics

Key Metrics to Track

• Rate Limit Hit Ratio: How often users hit limits
• Top Limited Users: Identify power users or abusers
• Limit Effectiveness: Are limits too strict or lenient?
• Cost Protection: Money saved by rate limiting

Setting Up Alerts

// Alert when users frequently hit limits
if (user.rateLimitHits > 10 && user.tier === 'pro') {
  sendAlert({
    type: 'user-needs-upgrade',
    user: user.id,
    message: 'Pro user frequently hitting limits'
  });
}
// Alert on suspicious activity
if (globalRequestRate > normalRate * 3) {
  sendAlert({
    type: 'possible-ddos',
    rate: globalRequestRate,
    message: 'Abnormal request spike detected'
  });
}

Best Practices

1. Start Conservative

• Begin with strict limits
• Gradually increase based on legitimate usage
• Monitor for abuse patterns

2. Communicate Clearly

• Show remaining quota in responses
• Provide clear upgrade paths
• Document limits thoroughly

3. Plan for Growth

• Design limits that scale with your business
• Implement tiered pricing early
• Automate limit adjustments

4. Consider User Experience

• Allow bursts for better UX
• Provide grace periods
• Implement soft limits with warnings

Common Pitfalls to Avoid

❌ Setting limits too high initially

• Start low and increase based on data

❌ Ignoring time zone differences

• Use UTC for consistency

❌ Not considering retry storms

• Implement exponential backoff

❌ Forgetting about webhooks

• Rate limit incoming webhooks too

❌ Hard-coding limits

• Make limits configurable

Conclusion

Effective rate limiting is a balance between protection and user experience. By implementing smart strategies like tiered limits, cost-based restrictions, and behavioral analysis, you can protect your API quotas while providing a great experience for legitimate users.

With Conduit Link, these strategies are just a configuration away. Start protecting your APIs today before it's too late.

Configure Rate Limiting with Conduit Link →