AI Engineer Interview

Transformers for AI Engineers

Covers what an AI Engineer interview actually probes about transformer internals: scaled dot-product and multi-head attention, why decoder-only architectures won, why the KV cache exists and how its memory footprint is computed, what parameter count does and doesn't predict, and positional encoding — with worked numbers for KV cache memory and prefill-vs-decode cost.

Tokenization and Context Windows

A deep, numbers-first look at subword tokenization (BPE), why tokens are not words or characters, the 'lost in the middle' effective-context problem, worked token-budget arithmetic for a real prompt, and why long context windows do not make RAG obsolete. Written as an AI Engineer interview reference with concrete worked examples and comparison tables.

Prompt Engineering

A practitioner's tour of prompt engineering as an AI Engineer interview topic: what belongs in the system prompt vs the user prompt and why, when few-shot examples help and when they stop paying off, what chain-of-thought actually buys you mechanically, structured outputs and schema-constrained decoding, treating prompts like versioned code with regression tests, and the decision framework for when a longer prompt is the wrong answer and RAG or fine-tuning is the right one.

Fine-Tuning: SFT, LoRA/QLoRA, RLHF and DPO

A deep, standalone treatment of fine-tuning for AI engineering interviews: the mechanics of SFT, LoRA/QLoRA, RLHF and DPO; realistic data requirements and where the data comes from; the sharp line between behaviour problems (fine-tune) and knowledge problems (RAG); training and serving cost order-of-magnitude; how to evaluate a fine-tune against catastrophic forgetting and a prompted baseline; and a worked decision scenario for a formatting-adherence bug in a support bot.

Multimodal LLMs and Vision

Learn how multimodal LLMs actually process images: patch tokenization, vision-encoder pretraining, and how the model distinguishes between multiple objects in a scene through attention rather than bounding-box regression. Covers the practical gap between vision-language models and classical object detectors, common failure modes (counting, fine-grained discrimination, spatial relations), and prompting techniques (referring expressions, crops, set-of-mark) that make multi-object questions reliable in production.

RAG Architecture End to End

A deep, standalone treatment of RAG architecture: the offline ingestion path and online query path as one diagram, a named failure mode at every stage, why the 'naive RAG' first pass plateaus at mediocre quality, query rewriting and multi-query fan-out, the confidence gate and the 'I don't know' path, citations and grounding checks, advanced patterns (parent-child, HyDE, agentic retrieval) and when they earn their complexity, and a RAG maturity ladder for the 'how would you improve this system' follow-up.

Chunking and Embedding Strategies

Deep dive on the two decisions that most determine RAG retrieval quality: chunking strategy (fixed-size, structure-aware, semantic, parent-child), overlap, chunk size vs recall, embedding model selection, and embedding drift when you swap models — with worked recall@k examples.

Vector Databases and Hybrid Search

A systems-level treatment of the retrieval layer for RAG: pgvector vs dedicated vector databases (Pinecone, Weaviate, Chroma, Milvus) with a comparison table and a 'when Postgres is enough' decision rule; HNSW vs IVF ANN indexing and their recall/latency trade-offs; pre-filter vs post-filter vs filtered-ANN and why naive post-filtering silently starves results; hybrid dense+BM25 search with reciprocal rank fusion (RRF) worked by hand; and cross-encoder re-ranking with its O(k) cost model.

Evaluating RAG Systems

A deep, interview-ready treatment of RAG evaluation: why retrieval and generation must be measured separately, worked recall@k/MRR/NDCG@k arithmetic, generation metrics including faithfulness and answer relevance, how to build and version a gold evaluation set, LLM-as-judge rubric design and bias calibration, RAGAS-style automated pipelines, hallucination-detection techniques, and a worked diagnostic example that separates a retrieval problem from a generation problem.

Tool and Function Calling

Deep dive on tool/function calling: the request-execute-return loop that turns a text generator into something that can act, why schema and description quality is the real bottleneck on tool-selection accuracy, tool-choice modes (auto/forced/none), parallel tool calls and their latency payoff, error handling and argument validation, where the permission boundary actually lives, keeping results compact, and a side-by-side of Anthropic tool use and OpenAI function calling.

Agent Architectures and the Agentic Loop

A precise, vendor-neutral treatment of the agentic loop for AI engineering interviews: the observe-decide-act loop itself, the ReAct pattern and why explicit reasoning traces help tool selection, the plan-act-observe-reflect variant and when its extra LLM calls are worth it, single-agent vs multi-agent at a glance, termination and budget controls, failure modes with mitigations (loops, tool hallucination, runaway cost, context poisoning, analysis paralysis), a worked compounding-error-rate calculation, and a worked workflow-vs-agent decision.

Memory Systems for LLM Applications

A working taxonomy for one of the most conflated terms in AI engineering: the conversation window (short-term), persistent facts across sessions (long-term), what-happened-last-time (episodic), and externalized scratchpads (working memory). Covers durability criteria for what to persist, eager-load-vs-retrieval-triggered design, the failure modes — staleness, injection, bloat, conflicting facts — and a concrete walkthrough of how CLAUDE.md and context compaction implement exactly this taxonomy in a real product.

Multi-Agent Orchestration

A vendor-neutral treatment of multi-agent LLM systems: the supervisor/worker pattern, handoffs versus shared state, pipeline-vs-barrier synchronization for parallel fan-out, a worked cost-multiplication example, and the honest heuristic for when a single stronger model beats a fleet of coordinating agents. Cross-links `mcp-and-subagents` as the concrete Claude Code implementation and `plan-and-loop-modes` for the pipeline and adversarial-verification mechanics.

MCP and Tool Integration Protocols

The concept and design-interview layer above MCP: why a standard client-server protocol replaced bespoke per-app tool integrations, the server/client/resource/tool abstractions at an architectural level, what MCP adds on top of plain function calling (and when bespoke calling is still the right answer), the trust boundary a third-party server introduces, and how to frame the 'build a server vs write a function' decision when an interviewer asks.

Guardrails and Prompt-Injection Defense

A deep dive on the topic every AI Engineer interview probes: guardrails as a pipeline wrapped around the model, not a sentence in the prompt. Covers input filtering (PII redaction, injection classifiers, topic filters), prompt injection as the defining LLM threat and why it has no clean code/data separation, defense-in-depth layers with worked examples, output filtering (schema, groundedness, PII, refusals), the hard limits of any classifier against an adaptive attacker, the latency/cost budget for a guardrail pipeline, and a full worked trace of an attack through every layer.

Cost and Latency Engineering for LLM Apps

The reusable toolkit behind every 'make it cheaper and faster' interview question: why input tokens dominate cost and output tokens dominate latency, how prompt-prefix caching and semantic caching work and where each breaks, model routing and cascades, streaming and parallel tool calls as latency levers, the TTFT-plus-decode model of p95 latency, and the cost-per-1,000-conversations math that ties every lever to a number leadership will ask for.

LLM Observability and Evaluation

Interview-ready coverage of running LLM systems in production: what a full request trace must capture, token/cost/latency dashboards as product metrics, the precise line between offline evals and online monitoring, LLM-as-judge as a general technique with its biases and calibration, human review workflows, CI regression suites that treat prompts and models as code, canarying prompt and model changes, and a worked dashboard-diagnosis example.

Frameworks Landscape: LangChain, LlamaIndex, CrewAI, AutoGen, Semantic Kernel, Assistants API

A trade-off-first tour of the LLM application framework landscape — LangChain, LlamaIndex, CrewAI, AutoGen, Semantic Kernel and hosted Assistants-API-style platforms — organised around what each abstraction gives you, what it hides, and when the honest answer is to roll your own thin orchestration instead.

Case Study: Design a Customer-Support Assistant

Model interview answer for designing an LLM customer-support assistant: requirements and non-goals, workflow-vs-agent decision, tenant-safe RAG over a help centre, tool calls for account lookups and refunds with confirmation, conversation state, escalation and the 'I don't know' path, layered guardrails, offline and online evaluation, cost and latency numbers with levers, and a rollout plan.

Case Study: Design a Coding Agent

Model interview answer for designing a terminal coding agent that reads, edits, runs and verifies code in a repository: requirements and threat model, the agentic loop, a minimal tool set with output caps, a permission model that separates read from write from execute, context management for a finite window (truncation, compaction, subagents, cached prefixes), layered memory, cost and step budgets, prompt-injection defence, evaluation on task suites, and observability.