Practise AI engineer interview follow‑ups.

What you can practise

LLM fundamentals and transformer internals, embeddings and retrieval, retrieval-augmented generation, prompting and structured output, evaluation, agents and tool use, context engineering, guardrails and security, fine-tuning, and inference and serving. The ML foundations behind them are in scope too: supervised learning, evaluation metrics and ML system design. Questions are selected for your role and seniority; a short sitting samples that scope. It does not test every topic.

Choose two to four questions and review the time budget before starting. The timer runs without hints or pauses. This is written technical reasoning practice. Coding, whiteboarding and behavioral rounds need separate preparation, and company context does not select an employer-specific question set.

Eight follow-ups worth preparing for

Each example follows the shape of a depth round: a starting question, an answer that is common and incomplete, the follow-up an interviewer uses to test it, and how a stronger candidate reasons. These are original teaching examples. They are not product transcripts or a promise that a mock will contain these questions. Read the question, write your own answer, then pause at the follow-up before reading on.

1. Identical outputs are not the requirement.

Starting question

A product team reports that the assistant gives different answers to the same prompt. What would you change?

Incomplete answer

Set temperature to 0. Decoding becomes greedy: the model always takes the most likely token, so the same prompt returns the same answer.

Follow-up

They set it to 0 and a hundred runs of one prompt still produce four distinct outputs, two of them wrong. Where does the variation come from, and what is the requirement you would write instead?

Stronger explanation

I would first hash the fully rendered request, history and retrieved context included, and confirm the hundred calls were byte-identical. If so, the variation is inside the serving stack. Greedy decoding takes the largest logit, but the logits are not fixed: a request shares a batch with whatever else arrives, the kernels choose a reduction order by batch shape, and floating-point addition is not associative. When two tokens are nearly tied, the last bits decide and the continuation diverges. Mixture-of-experts routing adds another batch-dependent choice, and the provider can change the stack behind the same name. If we serve the model ourselves, batch-invariant kernels remove that source. I would write the requirement as a number: at production settings, sample each evaluation prompt a hundred times and require a validator pass rate and, for structured fields, agreement on the parsed value. Constrained decoding plus validation and retry enforce it per request.

What remains uncertain: A pinned model version and a seed still do not give bitwise repeatability across provider hardware and batching, so one passing sample says little about the next. Agreement is not correctness: a hundred identical outputs can all be wrong.

Try another follow-up: The same model miscounts the letters in a long word and makes mistakes adding a twelve-digit number to a nine-digit one. Does tokenization explain both failures, or only one?

Technical reference: Thinking Machines Lab, Defeating Nondeterminism in LLM Inference.

2. Locate the failure before changing the model.

Starting question

A retrieval-augmented support assistant tells a customer the wrong refund window. What would you change first?

Incomplete answer

I would tighten the system prompt to answer only from the retrieved passages and require a citation, and try a stronger model if that does not hold. This reads like hallucination.

Follow-up

The larger model, given the same passages-only prompt, now returns a fluent, well-cited quote of the wrong refund window. What did that establish, and what would you measure next?

Stronger explanation

I would first separate retrieval failure from generation failure. The model can only be faithful to what it is given. A larger model would likely repeat the same wrong window with more confidence. I would pull the trace and check whether the current document is indexed and passes the access, recency, region and plan filters. I would also check whether it ranks in the top k and survives reranking into the final prompt. Where both policy versions are retrieved, the fix is version handling, not the model: mark the superseded document archived at ingestion. I would measure retrieval by recall at k, labelled at the version level. I would measure generation by faithfulness to the supplied passages, and the outcome by correctness against a reviewed answer key. I would touch generation only where the correct passage was retrieved but the answer still ignored it.

What remains uncertain: Both measurements depend on a reviewed set with labelled passages, which may miss the questions customers actually ask and goes stale each time the policy changes. Neither stage metric can show whether the policy document itself is wrong; only end-to-end correctness against a verified answer key exposes that.

Try another follow-up: The current policy is indexed and ranks 12th; k is 8. Do you raise k or change the ranker, and what does each cost at generation time?

Technical reference: Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System (arXiv).

3. An embedding encodes meaning, not identity.

Starting question

Users report that search over a product catalogue returns the wrong part. Logs show the failures cluster on queries that contain a part number. What would you change?

Incomplete answer

I would switch to a stronger embedding model. The current one was trained on general text, so it does not know our catalogue vocabulary.

Follow-up

A newer embedding model still gives SKU-4471 and SKU-4417 a cosine similarity of 0.98. What property of dense embeddings causes that, and what would you change?

Stronger explanation

I would first name the mechanism. SKU-4471 and SKU-4417 share nearly the same subwords, so their pooled vectors sit close together. A digit-swapped identifier almost never appears as a training negative that would separate them. An identifier query has exactly one correct answer, so it does not belong in a ranking problem. I would detect identifier-shaped queries and resolve them by exact lookup against a normalised keyword field before any retriever runs. Descriptive and mixed queries go through dense retrieval fused with BM25 using reciprocal rank fusion. The analyser skips stemming and normalises case and hyphen variants. A lookup only works once both sides are normalised. An empty filter falls back to the fused list, and a mixed query gets the identifier boosted rather than filtered away. I would measure rank-one accuracy on the identifier slice and recall at k on the descriptive slice.

What remains uncertain: The identifier detector is a pattern match over known SKU formats, so a new supplier with a different numbering scheme falls through to the fused list silently. Nothing in the offline evaluation catches that until the slice is relabelled.

Try another follow-up: Your detector misses a query like "gasket for 4471" and falls through to the fused list, where dense and lexical search disagree on which SKU ranks first. How would you close that gap instead of leaving it to fusion?

Technical reference: Elasticsearch documentation on reciprocal rank fusion.

4. A judge is a model and needs its own evaluation.

Starting question

A colleague has rewritten the prompt behind a summarization feature. How would you decide whether the new prompt is better than the current one?

Incomplete answer

I would run both prompts over a sample of documents and ask a strong model to rate each summary from 1 to 10. Then I would compare the average scores.

Follow-up

That judge tends to prefer longer outputs, and the new prompt produces summaries about a third longer. With a week and no labelled data, what is the cheapest check you would run first?

Stronger explanation

I would first check length. If summaries have a target length, the new prompt already breaks it. The cheapest check is a length counterfactual: ask the old prompt for summaries as long as the new ones and re-run the judge. If the gap closes, length was the driver. I would then run the judge pairwise, in both orders, and keep only swap-consistent wins. Position bias applies even though self-preference does not when one generator wrote both. In that week I would have people label a few hundred pairs, a preference plus claim-level faithfulness, and report the judge's agreement with them. Roughly eighty percent is about the human-human ceiling. Human labels decide this change. The judge is calibrated against them so later changes skip a labelling round. I would ship only once a held-out slice clears a bootstrap win-rate margin, faithfulness holds, and an A/B on edit and regenerate rates agrees.

What remains uncertain: Judge agreement on a few hundred labels holds for that distribution only. It cannot show that the rubric captures what users want, or that the preference transfers to input types the set does not cover.

Try another follow-up: Your two human labellers disagree on a third of the faithfulness labels. What does that do to your judge calibration?

Technical reference: Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv).

5. A prompt cannot make a retry safe.

Starting question

Your agent files support tickets through a create-ticket tool. Some tickets are filed twice. How would you stop the duplicates?

Incomplete answer

I would add a system prompt rule: search for an existing ticket first, and only call create-ticket if none matches.

Follow-up

The duplicates appear when the create call times out and the runtime retries it, after the server has already filed the ticket. What does your prompt rule guarantee on that path?

Stronger explanation

Nothing on that path. The retry runs below the model, so no instruction reaches it. I would mint one key per tool invocation, before the first attempt, and reuse it on every retry. The server claims that key atomically before writing, then stores the request. On repeat it returns the stored result, an in-progress state, or a mismatch for a different payload. That is the guarantee. A pre-check is convenience, not protection. If the model itself re-issues the call after an error, that is a new key. Create-ticket must then return an existing match, backstopped by a unique constraint on the ticket table. To prove it, I would inject the fault: drop the response after the write commits, then assert the retry still returns one ticket id.

What remains uncertain: Idempotency covers only writes that honour the key, within its retention window. A third-party ticket system that does not support keys needs a wrapper in front of it holding that table, with retention outlasting the longest retry. Fault injection proves the failure modes I chose to inject, not the ones I did not.

Try another follow-up: Now the model itself, not the runtime, re-issues create-ticket with slightly reworded text after seeing the error. What in the tool contract decides whether that is a retry or a second ticket, and what do you do when it cannot?

Technical reference: Amazon Builders' Library, Making retries safe with idempotent APIs.

6. Abstention is a threshold, not a prompt.

Starting question

Users report that your retrieval-augmented assistant invents citations to documents that do not exist. What would you change first?

Incomplete answer

I would add an instruction to the system prompt: answer only from the provided context and cite the passage used. The model fabricates because it falls back on its own memory, so restricting it to the context removes that.

Follow-up

After that change the assistant still invents a source when the context holds no answer, and it now refuses some questions it could answer. How would you measure that, and what would you fix?

Stronger explanation

I would first separate three numbers. I would build a labelled set with unanswerable items topically adjacent to the corpus, not obviously off-topic. On it I would track the answer rate on unanswerable items, where any answer is a failure. I would also track the false-refusal rate on answerable items and the invalid-citation share. The prompt edit moved the first two in opposite directions. I would stop the model writing citation text. It would select an id from the retrieved set instead. A citation to a nonexistent document now fails a simple membership check. That check also runs at serving time: an invalid id gets retried once, then dropped and counted. I would then gate generation on an answerability classifier, not the uncalibrated retriever score. The model would return a structured abstain field rather than prose. That threshold, not the prompt, is the knob I show product owners.

What remains uncertain: The identifier check proves a cited chunk was retrieved, not that the chunk supports the claim made about it. That support judgement, whether the chunk actually entails the claim, is a separate measurement it does not make. The evaluation set also only covers the unanswerable cases someone thought to write.

Try another follow-up: The retrieved chunk is the right document and the citation id validates, but the chunk states an exception the answer drops. Your invalid-citation rate is zero and your false-refusal rate is unchanged. Which number should have moved, and what would you add to the eval set?

Technical reference: Rajpurkar et al., Know What You Don't Know: Unanswerable Questions for SQuAD (arXiv).

7. Latency has two phases before it has a fix.

Starting question

The p95 latency of a chat feature is nine seconds, measured from request to complete answer. How would you reduce it?

Incomplete answer

Switch to a smaller model. Fewer parameters mean less compute per token, so every request gets faster and cheaper.

Follow-up

Profiling shows a two-second median but a nine-second p95 on long prompts, where the smaller model fails the quality bar, and the product team wants the personalised profile pinned to the top of every prompt. What do you do?

Stronger explanation

I would first ask whether we serve the model ourselves or call a vendor API: that decides which levers exist. Either way, I would split latency into first-token time and decode. Then I would split first-token time into queue wait, retrieval or rerank calls, and prefill compute. If queue wait or a pre-model call dominates, the fix is capacity or admission control, not the prompt. If prefill dominates, the prompt is the first lever. I would put shared system content first, so it forms one cacheable prefix, and put the personalised profile after it. Pinning the profile ahead of the shared prefix breaks cache reuse for every other user. I would also cap retrieved context to a token budget, re-checked against the quality bar. If self-hosted, chunked prefill or separate prefill and decode replicas stop long prefills stalling the batch. Quantisation and speculative decoding speed decode and come later.

What remains uncertain: It still cannot say what p95 the changes will reach, since prefix-cache hits need a byte-identical prefix that survives between requests, and vendor TTLs or peak KV memory pressure can both break that. It also cannot say whether the tail is prefill compute or queue wait behind other long prefills, a distinction that decides a prompt-budget fix versus a capacity fix.

Try another follow-up: After enabling prefix caching, the hit rate is 30 percent even though every request shares the same system prompt. What would you look for?

Technical reference: vLLM documentation on automatic prefix caching.

8. Access control belongs in retrieval, not the model.

Starting question

Design an assistant that answers support questions from internal documents. Outline the main components.

Incomplete answer

I would split the documents into chunks, embed them and store the vectors. For each question I would embed it and retrieve the five most similar chunks. Then I would send those chunks with the question to a large model, which answers from that context.

Follow-up

A contractor cannot read one of the documents, but its chunks still turn up among the five nearest for their question. State where this design blocks that, and say whether removing them from a group is as fast as changing the document's own permissions.

Stronger explanation

I would enforce access in retrieval, not in the prompt. Each query filters on the caller's verified identity inside the vector search, before top-k is chosen. Discarding unreadable chunks afterwards would starve the answer. The index, not the application, enforces that filter and pays its recall cost. The application must never assemble that filter from a service credential with broad read access. Each chunk stores its document's groups, never an expanded user list. A user leaving a group is resolved from the identity provider at query time, so revocation lands within one cache refresh. A document's permission change has to reach every chunk through the change feed, backed by periodic ACL reconciliation. A change inherited from a parent folder is included. Until it lands, the old permission still serves. I would watch the lag from an ACL change to its effect on results, and how many answers used stale-ACL chunks.

What remains uncertain: The design cannot choose the freshness bound; document owners have to say how long a revoked document may still be retrievable. The filter is only as correct as the permission metadata the source systems export, and the assistant cannot verify that metadata on its own.

Try another follow-up: You add a cache of final answers to cut cost and latency. What must be in the cache key for the permission check to still hold, and what can still leak even with the right key?

Technical reference: Azure AI Search documentation on document-level access control.

Leave with a specific next step.

After a mock, review the feedback and the recorded answer behind each supported finding. Separate what broke down from what stayed untested. Use the repair plan to choose a concept to study or a drill to practise, then check your reasoning again.

Read the illustrative sample report to see that chain for one answer: the assumption, a revealing follow-up, and a next exercise. Its example is a machine learning one; the report format is the same for an AI engineer mock.

Start with one completed mock free.

Create an account, choose AI Engineer and your seniority in the app, then review the sitting settings. The first completed mock includes feedback and its repair plan, with no card required. Further mocks and individual practice drills require a paid plan. Your completed report remains available.

Preparing for a machine learning engineer loop instead? See the ML engineer mock interview page and the ML follow-up question guide.