AI agents are shipping into production faster than teams can test them. They're connected to tools, databases, customer data, and external APIs - and the safety guardrails they rely on were never designed for adversarial conditions. Most agents deployed today have never been tested for attacks.
This session is about changing that.
HackAgent is an open-source red-team toolkit that automates the discovery of security vulnerabilities in AI agents. It implements the attack techniques like prompt injection, jailbreaking, goal hijacking, tool misuse and turns them into a repeatable, scriptable pipeline you can run against any agent before it reaches users.
The first half of this session is offence. I'll walk through the attack classes that matter most in agent deployments today, demonstrate each one against real agent targets, and explain why traditional API security testing doesn't catch any of them:
Prompt Injection: malicious inputs that hijack agent behaviour mid-task, overriding instructions the developer thought were fixed
Jailbreaking: bypassing safety guardrails and content filters using iterative refinement, character-level obfuscation, and persuasion-based paraphrasing
Goal Hijacking: manipulating an agent into pursuing objectives the developer never intended, without triggering any safety classifier
Tool Misuse: exploiting the agent's own capabilities (file access, code execution, API calls) to perform unauthorised actions the user never requested
For each class I'll show the specific HackAgent attack technique that exposes it (PAIR for adaptive black-box jailbreaking, TAP for query-efficient tree search, FlipAttack for deterministic obfuscation, PAP for social-engineering-style persuasion, AdvPrefix for deep prefix optimization) and explain why these techniques succeed where static prompt templates fail.
The second half is defence. I'll cover how to take the output of a HackAgent red-team run and turn it into concrete security improvements:
Establishing a vulnerability baseline: using Baseline template attacks and BoN stochastic augmentation for fast first-pass scans that tell you where you stand before investing in deeper analysis
Iterative guardrail hardening: feeding PAIR and TAP results back into system prompt design, safety classifier tuning, and input/output filtering to close the gaps each attack exposed
Continuous red-teaming in CI/CD: integrating HackAgent's CLI and SDK into your deployment pipeline so every agent update gets tested against the full attack surface before it ships
Measuring and reporting robustness: using HackAgent's judge framework and local/cloud dashboards to track attack success rates over time, compare model versions, and generate audit-ready reports
The session closes with framework-specific patterns: how to configure HackAgent against Google ADK agents, OpenAI SDK agents, LangChain chains, and LiteLLM-backed deployments and what each framework's architecture means for the attack surface you need to cover.
