The Application Security Podcast
The Application Security Podcast is a practitioner-led show for anyone building or securing modern software—and now AI-powered applications. Hosts Chris Romeo and Robert Hurlbut talk with the people shaping application security about AI and LLM security, threat modeling, secure development, OWASP, cloud, DevSecOps, security champions, and building programs that help engineering teams move faster and safer. With more than 300 episodes, the show turns real-world experience into practical guidance for developers, architects, AppSec professionals, and security leaders.
The Application Security Podcast
Javan Rasokat and Andra Lezza -- When Chatbots Go Rogue - Lessons Learned from Building and Defending LLM Applications
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
What happens when teams add large language models to real applications and discover that familiar AppSec controls are no longer enough? Andra Lezza and Javan Rasokat share lessons from building, breaking, and defending LLM-enabled systems at Sage and presenting their findings at DEF CON. They compare prompt injection with SQL injection, explain AI red teaming, and unpack hallucinations, retrieval-augmented generation, grounding, and model safeguards. The conversation also examines corporate data leaking through prompts, the limits of trusting model providers, and how the OWASP Top 10 for LLM Applications complements issues observed in production. Andra and Javan close with practical advice for developers, data scientists, and security teams: treat AI systems as a new attack surface, establish clear data boundaries, and test controls against realistic abuse cases.
The Application Security Podcast is brought to you by Security Journey.
About Security Journey
Security Journey provides application security education for developers and everyone in the software development lifecycle.
→ Learn more about Security Journey
Connect with Andra Lezza and Javan Rasokat:
→ Andra Lezza on LinkedIn
→ Javan Rasokat on LinkedIn
→ Javan Rasokat
→ AppSec Village
Mentioned in this episode:
→ Adversarial Misuse of Generative AI (Javan's blog article)
→ TLDR newsletter
→ The Cuckoo's Egg by Cliff Stoll
→ AppSec Village
→ DEF CON
→ ChatGPT
→ DeepSeek
→ NIST AI Risk Management Framework
→ OWASP Top Ten for LLM Applications project homepage
Follow the Application Security Podcast:
➜ Home
➜ X
➜ LinkedIn
➜ YouTube
➜ Instagram
➜ Facebook
Chapters:
00:00 When Chatbots Go Rogue with Andra Lezza and Javan Rasokat
01:49 Andra’s path into application security
02:56 Javan’s path into application security
04:38 Lessons from building and defending LLM applications
08:22 Prompt injection compared with SQL injection
11:46 Critical vulnerabilities found in real AI systems
12:19 What AI red teaming actually means
13:46 Hallucinations, RAG, and grounding
19:51 Model safeguards and harmful requests
20:35 Common AI development and deployment mistakes
23:20 Corporate data exposure through prompts
29:59 OWASP Top 10 for LLMs versus real-world findings
32:02 Practical security advice for AI developers
34:36 Bringing security practices to data scientists
45:37 Key takeaways for defending LLM applications
Follow the Application Security Podcast:
➜ Home: appsecpodcast.com
➜ X: @AppSecPodcast
➜ LinkedIn: The Application Security Podcast
➜ YouTube: @ApplicationSecurityPodcast
➜ Instagram: @appsecpodcast
➜ Facebook: Application Security Podcast
People on this episode
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
The Security Table
Izar Tarandach, Matt Coles, and Chris Romeo