Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Choosing an Architecture Shape for a New Product Feature

Your company is building a document-translation feature: users upload a document (up to ~20k tokens) in one language and get back a full, faithful translation in another. A teammate proposes using your existing decoder-only chat LLM (the one already deployed for the product's chat assistant) with a "translate the following document:" prompt, since "it's already a great LLM and reusing it means no new infra."

  1. Is that a reasonable default, or does the task's shape argue for something else? Walk through the reasoning, not just the conclusion.
  2. If you did consider an encoder-decoder model for this feature, what would you actually be trading off against the "just prompt the LLM you already have" approach?
  3. What would make you say "no, decoder-only prompting is clearly the right call here, don't overthink it"?

Share this question

← Back to Transformers for AI Engineers practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.