AI Risk Assessment
AI risk assessment is the structured process of identifying what could go wrong with an AI feature before it ships — harmful or biased outputs, privacy leakage, security exposure via prompt injection, over-reliance by users — and rating each risk's likelihood and severity so testing effort is directed at the highest-impact failure modes first.
Traditional risk assessment asks what could break about deterministic functionality; AI risk assessment has to additionally ask what could this model say — a much larger and fuzzier surface, since a generative model's output space isn't fully enumerable the way a form's valid inputs are. That's why AI risk assessment usually happens as its own step before test planning, not folded into a generic risk matrix.
A useful assessment covers several risk categories separately: output harm (toxic, biased, or factually wrong content), security (prompt injection, data exfiltration through the model), privacy (PII leakage from training data or user context), and reliability (hallucination, non-determinism breaking downstream logic) — each needs a different kind of test to actually validate.
The output of a good risk assessment is a prioritized test plan, not just a document — the highest-severity, highest-likelihood risks, like a customer-facing chatbot leaking another user's data, get dedicated test suites and release gates; lower-risk ones get spot-checked.
