Match a job Paths Subjects Questions Quizzes Pricing
Advanced Open Pro

A Model Card That Missed Its Own Limitation

A hiring-screening model's model card states: "Intended use: rank candidate resumes by predicted interview-to-offer likelihood for software engineering roles. Overall AUC: 0.81." Six months after launch, a recruiter notices the model consistently ranks candidates who took a nonstandard career path (career changers, bootcamp graduates) far lower than their eventual interview performance would suggest is fair. When you dig into the original evaluation report (which was never included in the model card), you find the offline evaluation was done per-role but never broken down by education/career background, and the training data was 6 years of the company's own historical hiring decisions.

  1. Identify which sections of a proper model card were missing or inadequate here, and explain how each omission contributed to the problem going undetected for six months.
  2. What does "training data was 6 years of historical hiring decisions" imply about a likely source of the bias, independent of any model architecture choice?
  3. Rewrite the relevant sections of the model card (briefly) as they should have been written, including what would have made this limitation visible before launch rather than after.

Share this question

← Back to MLOps Model Lifecycle & Governance practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.