Course description
Continuous Security Testing for LLM Apps teaches security engineers, AppSec and platform teams, ML engineers and security leaders to test the LLM applications they own on every change, not once. You will learn what the OWASP GenAI Red Teaming Guide, the OWASP AI Testing Guide, NIST AI 600-1, the NIST AI RMF, MITRE ATLAS and the joint guidance Deploying AI Systems Securely ask of evaluation, then write a test charter and rules of engagement for authorized self-testing. In four labs on a local sample app with a stub model you configure a garak scan and read its report, score canary requests with PyRIT, build your own regression suite of canary, marker, policy and benign control tests, and wire a GitHub Actions job that fails above a threshold. You will choose scorers, thresholds and trial counts with the arithmetic behind them, handle flaky results, triage a failing gate, monitor controls in production and keep the suite current. The capstone is a continuous testing plan with a CI gate for one of your own applications. Last verified 2026-11-03.
About this course
A red-team result describes one version of an LLM application. Change the prompt, the model, a data source or a tool, and the result no longer applies. This course teaches security, AppSec, platform and ML engineering teams to make security testing continuous: scoped and authorized, repeatable, measured as rates against thresholds, and enforced by a CI gate that can stop a change from shipping. It is grounded in the OWASP GenAI Red Teaming Guide, the OWASP AI Testing Guide, the OWASP Top 10 for LLM Applications 2025, NIST AI 600-1, the NIST AI Risk Management Framework, MITRE ATLAS and the joint guidance Deploying AI Systems Securely. Every lab runs against a small sample app with a stub model on your own machine, and the only attack-style input is a harmless canary string. The principles are vendor-neutral; where the course shows a control on ibl.ai, it links the live documentation.
What you will be able to do
- Explain why security testing of an LLM application must run continuously, and map a test program to the OWASP GenAI Red Teaming Guide, the OWASP AI Testing Guide, NIST AI 600-1, the NIST AI RMF MEASURE function, MITRE ATLAS and the joint guidance Deploying AI Systems Securely.
- Write a test charter and rules of engagement for authorized self-testing of your own application, covering scope, test accounts, synthetic data, limits, evidence handling, escalation and stop conditions.
- Configure open-source frameworks (NVIDIA garak and Microsoft PyRIT) against a local application, read their reports, check hits by manual review and file findings that a team can act on.
- Design a regression suite of harmless canary, marker, policy and benign control tests that verifies each control you rely on, and prove that every case can fail against a known-bad build.
- Choose scorers, thresholds and trial counts with the arithmetic behind them, handle flaky results without loosening the gate, and wire a CI job that blocks a merge when the failure rate exceeds the threshold.
- Triage a failing gate, monitor the controls in production, and keep the suite current as models, prompts, tools and guidance change.
Who this course is for
Roles: security engineers, AppSec and platform teams, ML engineers, and the security leaders who approve releases of LLM applications and agents.
Organizations: any sector that builds or runs LLM applications, from customer support assistants to internal agents with tools.
Before you start: comfort with a terminal, basic Python and a CI system. For the labs you need Python 3.12, about 2.5 GB of disk space for the two tools, and, for the CI lab, a GitHub repository you own. No API keys are needed. All lab data is synthetic and supplied in the course.
Format
Time: about 10 to 12 hours, self-paced. Level: intermediate. Hands-on: four labs (a garak scan, PyRIT canary scoring, a custom regression suite and a GitHub Actions gate), with an AI tutor that coaches each step from the output you paste.
Structure: a Start here section, four content sections of readings, worked examples, templates and hands-on labs, each ending in an AI assessment, and a capstone in which you write a continuous testing plan with a CI gate for one of your own applications, followed by a reading list.
Grading: the four section AI assessments count for 60 percent of the grade and the capstone assessment for 40 percent. The pass mark is 70 percent.
Currency
Last verified 2026-11-03 against the OWASP GenAI Red Teaming Guide (version 1.0, January 2025), the OWASP AI Testing Guide (version 1, November 2025), the OWASP Top 10 for LLM Applications 2025, NIST AI 100-1, NIST AI 600-1, NIST AI 100-2e2025, MITRE ATLAS release 2026.10, the joint guidance Deploying AI Systems Securely (April 2024), garak 0.17.1, PyRIT 1.1.0, the GitHub Actions documentation and the ibl.ai documentation.
This field changes quickly. OWASP, NIST and MITRE revise their guidance often, and new tool versions can change probes, scoring and results. Check the current versions before relying on any page or template, and test only systems you own or are authorized in writing to test. This course is training, not a security assessment of your systems.
Outline
Section 0 · Start here
- 0.1 Welcome and orientation
Section 1 · Plan authorized, continuous security testing
- 1.1 Why LLM apps need continuous security testing
- 1.2 Scope, rules of engagement and data handling
- 1.3 Section 1 assessment
Section 2 · Scan your own app with garak and PyRIT
- 2.1 The lab app and the frameworks
- 2.2 Run and read scans
- 2.3 Section 2 assessment
Section 3 · Build a regression suite you can trust
- 3.1 Designing the suite
- 3.2 Scoring, thresholds and flaky results
- 3.3 Section 3 assessment
Section 4 · Gate merges in CI and keep testing in production
- 4.1 The CI gate
- 4.2 Triage, monitoring and keeping the suite current
- 4.3 Section 4 assessment
Section 5 · Capstone and wrap-up
- 5.1 Capstone: a continuous testing plan with a CI gate for your application
- 5.2 Capstone assessment
- 5.3 Reading list and next steps