Applied GenAI: RAG, agents & tools
Build LLM systems that are grounded, bounded and debuggable, not just impressive in a demo.
Roadmap: AI foundations & first builds, Production GenAI & integration
Why it matters
Most FDE roles today are about deploying AI. The hard part isn't calling a model. It's choosing the simplest design that works, keeping the model away from things it shouldn't decide, and making failures visible. Interviewers actively mark down candidates who reach for agents when a workflow would do.
What it's made of
- LLM fundamentals: tokens, embeddings, context windows
- Structured outputs and writing prompts as specs
- Production RAG: ingestion, chunking, hybrid search, reranking, citations
- Choosing between rules, RAG, tools, fine-tuning and agents
- Workflow patterns (chain, route, parallelise) before agents
- Tool calling with schemas, allow-lists and step limits
- Human approval for irreversible actions
- MCP and agent-to-agent protocols, at a working level
Pick the right tool for each job
Before choosing an architecture, sort the problem's parts:
- Explaining with citations from documents → retrieval (RAG).
- Live data, calculations or actions → tool calls to real systems.
- Eligibility, thresholds, permissions → plain deterministic code.
- Anything irreversible, regulated or financial → human approval.
- A stable, repeated style or classification at high volume → consider fine-tuning, last.
Workflows before agents
Most production systems are workflows: a fixed sequence of steps where the model does one judgment task each time. They're predictable, testable and cheap. Reach for an agent, where the model chooses its next step, only when the path genuinely can't be known in advance. Even then, give it explicit state, a step limit, a cost ceiling and an allow-list of tools.
Autonomy is a dial, not a switch
Grant the model authority in steps, and only move up when evals and trust support it:
- Suggest: the model drafts, a human does everything.
- Prepare: the model fills in the action, a human clicks send.
- Auto-run low-risk, reversible actions, with an audit trail.
- Human-approved for high-impact actions.
- Prohibited: some actions never get automated.
When an answer is wrong, find which layer failed
Debug in order. Was the right document retrieved at all? Was it retrieved but cut from the context? Was it in context but the answer ignored it? Was the answer grounded but unhelpful? Each has a different fix (ingestion, ranking, context assembly, or prompt), and editing the prompt only fixes the last one.
Mid-level looks like
Builds a working RAG pipeline with sensible chunking and prompts.
Senior looks like
Separates where the model reasons from where code controls execution, sizes autonomy to risk, and can explain exactly what RAG won't fix.
Common pitfalls
- "Let the model decide what to do next" with no state, limits or approvals.
- Calling something an agent when it's really a fixed workflow (that's fine; just call it one).
- Fixing every bug by editing the prompt instead of the data or retrieval.
- Applying permission filters after reranking, so restricted text still influences answers.
Practice drills
Drill 1
Build a ticket classifier with structured output (an enum of categories). Then harden it for tickets with two intents and tickets that are too vague to classify.
A strong answer: Schema-validated output, a multi-label or primary/secondary design, an explicit "needs human" class, and measured agreement on samples.
Drill 2
A finance team wants an agent that reads dispute emails, checks the ERP and issues credit notes. Design it.
A strong answer: Draft-only above a threshold, an idempotent credit-note tool, email treated as untrusted input, an audit trail, a kill switch and stop conditions.
Drill 3
Take an "agent" design and rewrite it as a deterministic workflow. Justify the one step that keeps model judgment.
A strong answer: Clear reasoning about where uncertainty actually lives, and fewer moving parts without losing capability.
Best resources for this skill
Repos, courses, articles and podcasts, ranked by how much they teach.
Hugging Face Agents Course
Hands-on, free course on building and deploying AI agents, with a certificate of completion.
calmrocks/ai-engineer-notebooks
Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.
Anthropic Academy
Free courses on building with Claude: API, tool use, MCP, and agent patterns. Core to AI-era FDE work.
mchittineni/ultimate-ai-engineering-guide
100+ AI, Gen AI, LLM, Agentic AI, Forward Deployed Engineer (FDE), AI Systems Architect, Applied AI, LLMOps, AI Platform engineering interview questions across 10 topics - short answer, detail, worked example, and interview tips for each.
bicced/ai-engineer-interview-handbook
Free, open-source AI engineer interview questions & handbook — Applied AI Engineer / Forward Deployed Engineer (FDE), LLM, RAG, AI agents, evals, fine-tuning, AI system design (2026).
DeepLearning.AI Short Courses
One-to-two hour courses on RAG, agents, evals, and fine-tuning. Fast ramp for customer-facing AI builds.
Read next
AI Engineering
Chip Huyen
The most complete practical guide to building products on foundation models: evaluation, prompting, RAG, agents, fine-tuning, latency and cost. Close to a syllabus for the technical side of an AI-focused FDE role.
Designing Machine Learning Systems
Chip Huyen
Focuses on ML in production: data pipelines, feature engineering, deployment, monitoring and distribution shift. Still relevant for the classic ML pieces of customer deployments and for thinking about model drift.
Hands-On Large Language Models
Jay Alammar, Maarten Grootendorst
A visual, code-first walkthrough of embeddings, semantic search, RAG, classification and fine-tuning. Good for building intuition about what is happening under the API calls you wire into customer systems.
Prompt Engineering for LLMs
John Berryman, Albert Ziegler
Written by engineers who built GitHub Copilot, it treats prompts as engineered artifacts: context assembly, few-shot design, tool use and workflow design. Directly useful when tuning an LLM feature against a customer's real data.