Advanced
Open
Pro
On-Device or Server? Decide, Ship, Version and A/B Test a 20 MB Model
Part of the AI Engineer Interview path →
Part of the MLOps & Production ML path →
Part of the Deep Learning Foundations path →
A camera app wants real-time scene classification at 30 frames per second. Options: call a server model (accurate, 25M parameters) or ship a model on the phone. Constraints: p95 under 30 ms, 20 MB shipped model, accuracy within 1.5 points of the server model, a fleet with flagship and four-year-old mid-range phones, strict privacy requirements.
- Apply a decision matrix and recommend an architecture (device, server or hybrid). Justify each factor.
- Plan how you get a 20 MB model that meets the accuracy target: architecture, distillation, quantization, runtime format. Do the byte arithmetic for how many parameters fit.
- Describe the pitfalls of NPU acceleration and how you verify that the accelerator is really used.
- Design delivery and versioning: how the model ships, how you roll it out, how you roll back, and how you A/B test it on device.
Share this question