Building a Parameter-Aware Rate Limiter SDK with Redis, Lua, and MongoDB
A technical deep-dive into designing a distributed, parameter-aware rate limiting SDK that handles multi-tenant workloads with heterogeneous payload contracts. The Problem: Rate Limiting by Request Payload, Not Just Endpoints At Sprinklr’s AI team, multiple product subteams were building ML services, each with different rate limiting requirements. Some needed to throttle by document_id in the request […]