Advanced
Open
Pro
Serving Per-Customer Fine-Tunes Without Per-Customer Deployments
Your product lets enterprise customers customize the assistant's writing style to match their brand voice. Sales wants this available to any customer on the enterprise tier — today that's 40 customers, growing toward several hundred within a year. An engineer proposes full-parameter fine-tuning a separate model per customer.
- Explain what's wrong with that proposal as it scales, in terms of both training and serving cost.
- Propose the alternative architecture, and describe concretely how a single request gets served under it.
- What data would each customer's fine-tune need, and what's the evaluation gate before enabling a new customer's adapter in production?
Share this question