
Program lead and theory instructor
Yasemin Arslan, MSc.
Responsible for curriculum design, theory sessions, engineering methodology and the evaluation approach. Founder of LLM Turkey.
LinkedIn ↗New professional program · 2027
Building, Securing and Evaluating Agentic AI Systems with Claude Code
Build real agentic AI systems with Claude Code. Learn MCP, RAG and agent orchestration. Measure the accuracy, safety and performance of what you build with LLM & Agent Evaluation.
About the program
The program covers AI systems for real business problems end to end: designing the architecture, building with Claude Code, wiring up tool and data integrations, applying security controls and measuring system quality with LLM and Agent Evaluation.
This is not a Claude Code tutorial. Claude Code is the program's main development tool; the engineering principles taught are not tied to any particular model or provider.
Who is it for?
Prerequisites
This is not a beginner program and is not suitable for participants without coding experience.
Four engineering competencies
The program treats the four competencies as one continuous engineering lifecycle rather than separate courses. Every week closes at least one part of the loop with a concrete deliverable.
System design
Software and agent development
LLM & Agent Evaluation
Operations and continuous improvement
Learning outcomes
14-week curriculum
Each week has a 2-hour theory session and a 3-hour practice session. Every week ends with a piece added to the final project.
Theory
Agentic systems, how agents differ from LLMs, harnesses, the agentic loop and system architecture.
Practice
Setting up Claude Code, analysing a repository and laying out the final project's architecture.
Outcome
Tells the components of an agentic system apart; defines the project skeleton and its first evaluation criteria.
Theory
Tool calling, agent state, tool execution and task management.
Practice
A first working agent and its tool integrations.
Outcome
Builds a working agent that calls tools and manages state, with tool calls logged.
Theory
MCP architecture: client–server, tools, resources; integration security and access control.
Practice
A custom MCP server and an enterprise API integration.
Outcome
Writes an MCP server with defined permission boundaries and connects it to the agent.
Theory
Retrieval, context management, embeddings, grounding and source verification.
Practice
Source-grounded retrieval over enterprise documents.
Outcome
Builds a RAG layer that cites sources and starts measuring answer groundedness.
Theory
Workflow orchestration, state, memory, retries, fallbacks and error handling.
Practice
Running multi-step enterprise tasks.
Outcome
Builds a traceable multi-step workflow that recovers from failures.
Theory
Delegation, orchestration, handoffs and multi-agent architecture decisions.
Practice
Comparing single-agent and multi-agent approaches on the same task.
Outcome
Compares single- and multi-agent designs on cost, complexity and success, with justification.
Theory
Structured outputs, error handling, reliability and test strategy.
Practice
Unit and integration tests, failure scenarios and fallback behaviour.
Outcome
Covers the system's critical paths with tests and documents failure scenarios.
Theory
Prompt injection, tool permissions, unauthorized access, sandboxing and data security.
Practice
Controlled security scenarios and permission-boundary tests.
Outcome
Produces a threat model for the agent and adds prompt-injection and privilege tests.
Theory
Ground truth, rubrics, metrics, human evaluation and LLM-as-a-Judge; the nine quality dimensions.
Practice
Building an evaluation dataset and scoring model outputs.
Outcome
Prepares an evaluation dataset with rubrics and ground truth, and scores outputs with justification.
Theory
Task success, tool-call accuracy, trajectory evaluation and failure analysis.
Practice
Multi-step evaluation of agent behaviour.
Outcome
Evaluates an agent on the path it took and the tools it used, not just its final answer.
Theory
Rater calibration, dataset versioning, evaluation automation and regression.
Practice
Building a repeatable evaluation pipeline.
Outcome
Sets up a versioned evaluation pipeline that re-runs on every change.
Theory
Observability, tracing, CI/CD, quality gates; token and latency measurement.
Practice
Wiring automated quality and evaluation checks into the development process.
Outcome
Uses evaluation results as a CI/CD quality gate and tracks cost and latency.
Theory
System quality criteria, integrated test strategy and production readiness.
Practice
System testing of the final project; performance and security improvements.
Outcome
Runs the final project through integrated tests and improves it based on the findings.
Theory
Technical evaluation methodology, system reporting and results analysis.
Practice
Final project presentations, independent tests and technical assessment.
Outcome
Defends the system, its evaluation report and engineering decisions in a technical presentation.
How it works
EvalOps integration
LLM Turkey's EvalOps evaluation approach is now part of this program. All nine quality dimensions are taught; hands-on work goes deep on a selected set of scenarios.
9 quality dimensions
Agent evaluation
Operational metrics
Final project
Participants build an agentic AI system that handles enterprise operations requests. It uses API and MCP tools, retrieves enterprise documents through RAG, runs multi-step tasks, asks for human approval at critical steps and ships with its own evaluation infrastructure.
What the system does
Deliverables
The goal is a working system, tested under realistic conditions with its limits documented — not a claim of full autonomy or zero errors.
Assessment
Instructors and program management

Program lead and theory instructor
Responsible for curriculum design, theory sessions, engineering methodology and the evaluation approach. Founder of LLM Turkey.
LinkedIn ↗
Practice instructor
AI ENGINEER
Responsible for the 42 hours of live practice sessions and technical labs.
Pre-registration
We contact pre-registered people once cohort dates and enrollment terms are confirmed. Pre-registration is free and non-binding.
No. The program is intermediate to advanced. You are expected to have basic software development experience, working knowledge of Git and the terminal, and familiarity with APIs and backend concepts. If you are looking for evaluation training that does not require coding, write to us.
No. Claude Code is the program's main development tool; the subject is designing, building, securing and evaluating agentic AI systems. The engineering principles are not tied to a particular model or provider.
Two live sessions a week: 2 hours of theory and 3 hours of practice, 70 hours in total. Session days and times are announced on this page when the cohort opens.
EvalOps Specialist focused on evaluating model outputs and did not require coding. This program also covers building agentic systems; EvalOps' nine quality dimensions and evaluation methods are integrated into its Evaluate and Operate stages.
By credit card through iyzico secure payments, or by bank transfer.
They are set out in the Refund and Cancellation Terms and the Distance Sales Agreement, which you accept during registration.
14 weeks, 70 live hours, and a working system you can defend at the end.