advanced

Testing AI Applications

A rigorous QA methodology for LLM-powered and AI-driven products — evaluating non-deterministic outputs, hallucination and safety testing, RAG system validation, adversarial red-teaming, and regression testing as models change under you.

+340 XP9h (205m total)17 lessons0 enrolled

What you'll learn

Non-deterministic test strategy
LLM output evaluation
Hallucination testing
Prompt injection & red-teaming
RAG testing
AI agent testing

ai-testing llm prompt-testing red-teaming evaluation rag

Curriculum— 5 modules, 17 lessons

Non-Determinism — Designing Tests for Systems That Don't Give the Same Answer TwiceFREE10m The New Test Pyramid for AI ApplicationsFREE10m

Defining "Correct" — Rubrics, Golden Sets & Acceptance Criteria12m

LLM-as-Judge — Using AI to Grade AI12m

Hallucination and Factuality Testing12m

Building a Golden Dataset for Regression Testing12m

Exercise — Write an Evaluation Rubric for a Real AI Feature15m

Prompt Injection — What It Is and How to Test for It12m

Red-Teaming AI Applications12m

Testing Guardrails, Content Filters & Refusal Behavior10m

How RAG Pipelines Work (and Where They Break)12m

Testing Retrieval Quality vs. Generation Quality Separately12m

Testing AI Agents and Tool-Use Behavior12m

Exercise — Design a Test Plan for a RAG-Powered Support Bot15m

Regression Testing Across Model/Prompt Version Changes12m

Cost, Latency, and Performance Testing for AI APIs10m

Checkpoint — Testing AI Applications Review15m