Skip to content

Request Layer

Request Layer

Three-tier cache, coalescer, and dollar-aware rate limits for the agent fleet.

Request Layer sits between Kong and the Nasiko agent fleet. It adds L1 exact, L2 semantic, and L3 router-decision caching, collapses 200 concurrent duplicates into one origin call with an in-flight coalescer, and gates traffic with a per-agent token-bucket, rolling dollar-per-minute meter, and three-lane priority queue. Observable via admin REST and SSE with Phoenix cache.hit spans.

Stack

  • Python
  • Redis
  • Kong
  • FastAPI
  • Phoenix