Request Layer
Three-tier cache, coalescer, and dollar-aware rate limits for the agent fleet.
Request Layer sits between Kong and the Nasiko agent fleet. It adds L1 exact, L2 semantic, and L3 router-decision caching, collapses 200 concurrent duplicates into one origin call with an in-flight coalescer, and gates traffic with a per-agent token-bucket, rolling dollar-per-minute meter, and three-lane priority queue. Observable via admin REST and SSE with Phoenix cache.hit spans.
Stack
- Python
- Redis
- Kong
- FastAPI
- Phoenix
