Exam¶
Modern AI Engineer Roadmap
Exam Center
Select a stage, part, and question type. A full part exam uses 30 levelled problems: 10 answer, 10 blank, and 10 apply.
Choose filters and start.
Part Exam Directory
Open any part exam directly. Every part has one standalone HTML exam with 30 levelled problems.
Stage 0: Orientation - 5 part exams
Stage 0: Orientation
0.1 AI Engineering Mental Model
Build a mental model of AI products as systems around foundation models, not as isolated prompts. This exam has 30 problems across 10 levels.
Stage 0: Orientation
0.2 Tooling and Learning Environment
Create a reliable local workspace so every later project can be reproduced, debugged, and shared. This exam has 30 problems across 10 levels.
Stage 0: Orientation
0.3 Use Case Judgment
Choose AI projects that have clear users, data, success criteria, and acceptable risk. This exam has 30 problems across 10 levels.
Stage 0: Orientation
0.4 Learning Operating System
Turn the roadmap into a repeatable loop of learning, building, measuring, and reflection. This exam has 30 problems across 10 levels.
Stage 0: Orientation
0.5 First Portfolio Skeleton
Start documenting work from day one so every stage leaves behind usable evidence. This exam has 30 problems across 10 levels.
Stage 1: Foundations - 6 part exams
Stage 1: Foundations
1.1 Python Software Craft
Write Python that can grow from notebooks into maintainable AI project code. This exam has 30 problems across 10 levels.
Stage 1: Foundations
1.2 Math for AI Engineers
Learn the practical math that explains representations, uncertainty, optimization, and metrics. This exam has 30 problems across 10 levels.
Stage 1: Foundations
1.3 Data Handling
Turn messy raw data into inspected, documented, and validated inputs. This exam has 30 problems across 10 levels.
Stage 1: Foundations
1.4 Databases and Storage
Understand where application, training, retrieval, and observability data live. This exam has 30 problems across 10 levels.
Stage 1: Foundations
1.5 Web and API Basics
Learn the service contracts used by LLM APIs, agent tools, dashboards, and deployed products. This exam has 30 problems across 10 levels.
Stage 1: Foundations
1.6 Systems Thinking Basics
Build enough runtime intuition to reason about memory, concurrency, queues, containers, and deployment. This exam has 30 problems across 10 levels.
Stage 2: Machine Learning - 5 part exams
Stage 2: Machine Learning
2.1 Problem Framing
Turn product goals into learnable tasks with clear targets, labels, features, and constraints. This exam has 30 problems across 10 levels.
Stage 2: Machine Learning
2.2 Splits and Leakage
Design evaluation splits that represent future use instead of memorized training data. This exam has 30 problems across 10 levels.
Stage 2: Machine Learning
2.3 Supervised Models
Use simple and strong supervised models before neural complexity. This exam has 30 problems across 10 levels.
Stage 2: Machine Learning
2.4 Metrics and Error Analysis
Understand model behavior through metrics, thresholds, slices, and concrete failures. This exam has 30 problems across 10 levels.
Stage 2: Machine Learning
2.5 Unsupervised Representations
Prepare for embeddings, clustering, retrieval, and topic discovery. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning - 6 part exams
Stage 3: Deep Learning
3.1 Neural Network Core
Connect tensors, layers, losses, gradients, and optimizers to learning behavior. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning
3.2 Training Loop Engineering
Build training code that handles data, devices, checkpoints, validation, and experiment records. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning
3.3 Optimization Dynamics
Tune training behavior with optimizers, schedules, initialization, regularization, and ablations. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning
3.4 Architectures and Modalities
Read and adapt model structures for tabular, vision, text, sequence, and multimodal tasks. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning
3.5 Debugging Neural Systems
Diagnose the ordinary reasons neural networks fail before blaming the architecture. This exam has 30 problems across 10 levels.
Stage 3: Deep Learning
3.6 Hardware Aware Training Preview
Understand the first layer of GPU utilization, memory pressure, and data bottlenecks. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models - 7 part exams
Stage 4: Large Language Models
4.1 Token and Context Mechanics
Understand how language becomes tokens and why context is a scarce engineering resource. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.2 Transformer Mental Model
Learn the architecture concepts that explain modern LLM behavior. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.3 Generation Controls
Control probabilistic text generation and make outputs fit software contracts. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.4 Model Landscape
Choose among model families, sizes, licenses, providers, and hosting patterns. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.5 Prompting and In-Context Learning
Use instructions, examples, constraints, and decomposition before heavier adaptation. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.6 Fine-Tuning and Dataset Engineering
Know when model weights should change and how data quality drives the result. This exam has 30 problems across 10 levels.
Stage 4: Large Language Models
4.7 LLM Evaluation Methodology
Evaluate open-ended model behavior with exact checks, rubrics, judges, and comparative tests. This exam has 30 problems across 10 levels.
Stage 5: AI Applications - 7 part exams
Stage 5: AI Applications
5.1 AI Product Interface
Design user flows that expose uncertainty, citations, correction, and escalation clearly. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.2 Prompt System Engineering
Treat prompts as versioned product assets with tests and release discipline. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.3 RAG Ingestion
Turn documents into clean, retrievable, inspectable knowledge assets. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.4 Retrieval and Reranking
Retrieve the right evidence before asking the model to answer. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.5 Grounded Generation
Assemble context so answers are faithful, cite evidence, and admit when evidence is missing. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.6 Application Evaluation and Feedback
Measure the whole application through golden sets, rubrics, judges, human review, and user feedback. This exam has 30 problems across 10 levels.
Stage 5: AI Applications
5.7 Guardrails and Release Architecture
Wrap model calls with gateways, routers, validators, caches, monitoring, and incident response. This exam has 30 problems across 10 levels.
Stage 6: AI Agents - 8 part exams
Stage 6: AI Agents
6.1 Agent Loop Fundamentals
Build the observe-think-act-observe-finish loop explicitly before using frameworks. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.2 Reasoning and Planning Patterns
Use planning, decomposition, reflection, and routing only when the task needs them. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.3 Tool Design
Give agents safe, typed, observable interfaces to useful software capabilities. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.4 MCP and Tool Ecosystems
Use Model Context Protocol concepts to expose tools and resources with clear boundaries. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.5 Memory and Agentic RAG
Manage short-term state, long-term memory, retrieval, summarization, and forgetting. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.6 Multi-Agent Systems
Coordinate specialized agents only when the extra communication improves outcomes. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.7 Agent Evaluation and Observability
Make runs inspectable through task evals, tool tests, traces, metrics, and replay. This exam has 30 problems across 10 levels.
Stage 6: AI Agents
6.8 Agent Security and Safety
Defend against prompt injection, tool abuse, secret leaks, and excessive agency. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure - 6 part exams
Stage 7: Model Infrastructure
7.1 Data Pipeline Architecture
Move raw data through repeatable ingestion, cleaning, validation, and lineage steps. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure
7.2 RAG and Feature Infrastructure
Operate embeddings, vector indexes, feature artifacts, and refresh jobs as production assets. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure
7.3 Training and Adaptation Pipelines
Automate dataset generation, fine-tuning, evaluation, and artifact export. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure
7.4 Serving and Deployment
Expose model behavior through reliable APIs, containers, streaming, batching, and deployment environments. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure
7.5 Observability and Quality Operations
Instrument AI systems across prompts, retrieval, model calls, tools, traces, and quality metrics. This exam has 30 problems across 10 levels.
Stage 7: Model Infrastructure
7.6 Reliability and Cost Control
Prevent dependency failures, runaway retries, quota blowups, and silent quality regressions. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration - 7 part exams
Stage 8: Optimization and Hardware Acceleration
8.1 Inference Performance Model
Use the measurement vocabulary and workload model behind optimization. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.2 Transformer Inference Internals
Connect transformer computation to memory bandwidth, attention, cache layout, and sampling. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.3 Model Optimization
Change or approximate computation while measuring quality risk. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.4 Serving Engines
Understand how runtimes schedule, batch, stream, and manage model memory. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.5 Distributed Inference
Scale inference across replicas or devices when one process is not enough. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.6 GPU and Kernel Basics
Build the hardware intuition needed to read profiles and understand bottlenecks. This exam has 30 problems across 10 levels.
Stage 8: Optimization and Hardware Acceleration
8.7 Edge and Accelerator Co-Design
Connect workloads to Jetson, mobile NPUs, FPGA prototypes, compilers, and future chips. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML - 6 part exams
Stage 9: AI Security, Blockchain, and ZKML
9.1 LLM and Agent Security
Protect systems where models read untrusted content and call tools. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML
9.2 Secure AI Application Architecture
Apply ordinary AppSec and privacy controls to AI services and data flows. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML
9.3 Governance and Responsible AI
Map risks to controls, monitoring, human oversight, documentation, and review processes. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML
9.4 Blockchain Fundamentals
Understand wallets, signatures, transactions, gas, state, and finality before AI systems touch chain state. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML
9.5 Smart Contract Security
Recognize common contract vulnerabilities and design AI-to-chain permissions safely. This exam has 30 problems across 10 levels.
Stage 9: AI Security, Blockchain, and ZKML
9.6 ZK and Verifiable AI
Use cryptographic verification for trust, privacy, and constrained model claims. This exam has 30 problems across 10 levels.
Stage 10: Mastery - 5 part exams
Stage 10: Mastery
10.1 Capstone Problem and Architecture
Choose a narrow but real system and design it before coding. This exam has 30 problems across 10 levels.
Stage 10: Mastery
10.2 Capstone Build Execution
Build the capstone in vertical slices that integrate product, model, data, and deployment early. This exam has 30 problems across 10 levels.
Stage 10: Mastery
10.3 Portfolio Communication
Explain your work through READMEs, diagrams, eval reports, demos, and failure analysis. This exam has 30 problems across 10 levels.
Stage 10: Mastery
10.4 Interview and Collaboration Readiness
Turn project experience into clear debugging stories, design reviews, and collaboration habits. This exam has 30 problems across 10 levels.
Stage 10: Mastery
10.5 Specialization and Research Frontiers
Choose a deeper direction after the core architecture is complete. This exam has 30 problems across 10 levels.