ML interview follow‑ups: three worked examples.

1. A metric needs a decision.

Question

You are evaluating a classifier that sends transactions for manual fraud review. Which metric would you use?

Incomplete answer

F1, because fraud is an imbalanced classification problem.

Follow-up

The review team can inspect only 200 transactions per day, and missed fraud varies in cost. What does F1 leave out?

Stronger explanation

I would start from the review decision. At the actual daily review capacity, I would measure how much fraud we catch and how many reviews are useful, then assess missed-loss and review costs. I would choose the decision threshold on validation data, preserve a separate final evaluation, and compare the policy with the existing one. F1 alone does not encode the queue limit or unequal financial consequences.

What remains uncertain: this still needs reliable labels, an agreed cost model and evaluation data that represents the intended deployment. No single metric settles those choices.

Try another follow-up: What changes if fraud prevalence shifts after launch but the review capacity stays fixed?

Technical reference: scikit-learn on model evaluation and scoring.

2. Preprocessing can leak information.

Question

You standardize the full dataset, then run cross-validation. Is that a valid evaluation procedure?

Incomplete answer

Yes. The scaler never sees the target labels.

Follow-up

Does each validation fold influence the mean and variance used to transform its training fold?

Stronger explanation

Yes, so the training procedure has used information from held-out features. I would fit the scaler inside each training fold and use those fitted parameters to transform that fold’s validation data. A pipeline keeps the scaler and model together during cross-validation. I would apply the same boundary to imputation, feature selection and other learned preprocessing.

What remains uncertain: a pipeline prevents this preprocessing mistake when used correctly. It cannot repair features recorded after the outcome, duplicated entities across folds or features built from information that would not be available at prediction time.

Try another follow-up: A feature is “number of support tickets in the next seven days.” Can a pipeline make it safe for a prediction made today?

Technical reference: scikit-learn on data leakage and pipelines.

3. A random split answers a particular question.

Question

Your dataset has many weekly observations from each customer. Can you randomly split rows to estimate performance on new customers?

Incomplete answer

Yes, if the train and test sets have the same class balance.

Follow-up

What happens if the model sees earlier observations from a test customer during training?

Stronger explanation

The test may reward familiarity with customers rather than generalization to unseen ones. For a new-customer objective, I would keep each customer entirely within one side of the split. If the objective is future predictions for existing customers, I would use a time-based design and verify feature availability. If deployment requires both unseen customers and future periods, I would construct a split that respects both constraints.

What remains uncertain: a group split alone does not simulate future drift; a chronological split alone does not guarantee unseen customers. The evaluation design has to match the claim you want to make.

Try another follow-up: Labels summarize the following month. How could label windows overlap the boundary of a time split?

Technical reference: scikit-learn on grouped and time-dependent cross-validation.

Review the assumption you skipped.

For each answer, name the decision, the assumption it depends on, the evidence you would collect, and what could change your mind. Then change one condition in the question and try again without reading your previous answer.

Use the illustrative sample report to see how an unsupported assumption becomes a concrete practice step. Or explore the scope of an ML engineer mock interview before starting a timed attempt.