Match a job Paths Subjects Questions Quizzes Pricing
Intermediate Open Free

Choose an anomaly detection method for four different scenarios

Your team has four unlabelled anomaly detection problems on the table. For each one, name the method you would start with, one reason, and the failure mode you would test first.

  1. Refund amounts. 2 million refunds a month in USD, heavily right-skewed (median 12, a long tail into the thousands). Finance wants a short daily list of refunds to review.
  2. Plant sensors. 10 temperature and pressure sensors on one machine that normally move together. Faults show up as one sensor drifting against the others while every sensor stays inside its normal range.
  3. Daily signups. One metric with a strong weekly pattern and a slow upward trend. A marketing team wants to know about unusual days, including a Saturday that looks like a Wednesday.
  4. Customer profiles. 80 numeric and categorical features per customer, millions of rows, and the fraud team will review at most 25 customers a day.
Solution

1. Refund amounts: transform, then a robust score, then a budget. The raw robust z is the wrong tool on a heavy right tail: on a lognormal sample in the simulation it flagged 773 of 10,000 points (753 false) while taking the log first flagged 26 (6 false), and ranking by value and taking the top 20 caught all 20 far anomalies and 18 of 20 moderate ones (verified). So: log-transform (or fit the distribution), score with a robust z on the log amount, ideally per merchant segment or customer segment, and take the top-k that the review capacity allows instead of a fixed cut-off. The failure mode to test: the base rate of flagged refunds per segment when the segment is small (MAD of a segment with few refunds is unstable), and whether one customer's repeated refunds are each flagged separately (group them).

2. Plant sensors: a correlation-aware method, PCA residual (SPE) or Mahalanobis. The anomalies keep their marginals, so per-sensor rules cannot see them. In the 10-D simulation with 3 latent factors, the PCA residual (SPE) and full Mahalanobis scored AUROC 1.000, the maximum per-sensor z only 0.690 and Hotelling's T^2 in the PCA space 0.246 (verified). Choose the number of components on normal data (SPE AUROC was 0.696, 0.839, 1.000, 1.000, 0.990 for 1, 2, 3, 5 and 8 components, verified), use a robust covariance if the training window may contain faults, and then check the contamination failure: a coherent fault mode in 3 percent of the training rows rotated the PCA subspace and dropped AUROC from 0.981 to 0.547 (verified). Isolation Forest is a poor first choice: with correlation 0.95 it scored AUROC 0.957 against 1.000 for Mahalanobis on off-diagonal points (verified).

3. Daily signups: STL residual plus MAD, or seasonal-hybrid ESD, with context made explicit. A raw 3-sigma rule found 1 of 25 anomalous days on the simulated series because the weekly pattern and trend are larger than the anomalies; the three "Saturday looks like Wednesday" days were inside the raw band yet had a robust z near 8 on the STL residual (verified). Start with a robust STL fit (periodic seasonality, a moderately stiff trend), score the residual with a robust z of about 3.5, and use generalized ESD to cap the number of flags if a budget exists. Failure modes to test: the smoother settings (the default smoothers gave 46 false alarms and a tuned set gave 4 on the same data, verified), the first and last points of the window, and level shifts that the trend absorbs and a CUSUM would find.

4. Customer profiles: Isolation Forest (plus feature selection) with a review budget. It scales to millions of rows with a small subsample per tree and handles mixed numeric features without a distance metric. The failure modes: dilution by irrelevant features (with 3 informative dimensions plus 100 noise ones, AUROC fell from 0.998 to 0.790 and AP from 0.933 to 0.077, verified), so select features by domain knowledge first; and contamination='auto' (it flagged 479 rows for 50 true anomalies in the test), so set the threshold from the 25-a-day budget: the quantile 1 - 25/N of scores on a held-out week. Expect recall to be capped by the budget; compare with kNN or LOF on a scaled subset if local density matters.

In every case, the same two checks apply: the threshold comes from the alert budget on unseen data, and precision@k comes from a reviewed sample.

Share this question

← Back to Anomaly Detection for Tabular Data & Time Series practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.