Serve a GNN Score Inside a 100 ms Authorisation Budget

Your fraud model must return a decision within 100 ms at the 99th percentile, of which 40 ms is already spent by the rest of the pipeline. A prototype 2-layer GraphSAGE with fan-out (15, 10) and 64-dimensional float32 hidden state beats the GBM by 0.03 PR-AUC offline. The prototype fetches the neighbourhood at request time from a key-value store.

Assumptions: each sequential key-value round trip is 1 ms at p50 and 6 ms at p99, and traffic is 2,000 requests per second.

1. Estimate the latency and the tail behaviour of the prototype's request path (neighbour list, neighbours' neighbour lists, features).
2. Estimate the bandwidth per request and per second.
3. Propose a serving design that keeps the graph off the critical path and say what you give up.
4. Say what you would monitor and how you handle new accounts.