TestGuild Automation Testing Podcast
TestGuild Automation Podcast is one of the longest-running software testing podcasts for SDETs, QA engineers, software testers, and automation architects. Hosted by Joe Colantonio, the show has delivered 600+ episodes featuring the biggest names in software testing, covering test automation, AI testing, Playwright, Selenium, Cypress, API testing, performance testing, DevOps, CI/CD, and quality engineering. Each week, you'll hear practical conversations with industry experts, tool creators, and testing practitioners who share real-world lessons, automation strategies, and insights you can apply immediately. Whether you're building automation frameworks, exploring AI-powered testing, improving software quality, or advancing your career, TestGuild helps you test smarter, release faster, and stay ahead of what's next.
info_outline
Testing AI: Engineering Confidence in Non-Deterministic Systems with Jason Arbon
08/12/2026
Testing AI: Engineering Confidence in Non-Deterministic Systems with Jason Arbon
In this episode 600 of the TestGuild Automation Podcast, Joe Colantonio talks with Jason Arbon, founder of Testers.ai, Jank.AI and IcebergQA and author of the new book Testing AI: Engineering Confidence in Non-Deterministic Systems. Take Our 2027 Survey Now: Jason makes a case most testers have not heard yet. Coding is being absorbed by AI. Specification work is thinning out. Product, development, and test roles are converging into one. And when the music stops, the only seat left belongs to the person who can look at what the machine produced and make an evidence backed call on whether it ships. He calls that confidence engineering, and he argues it is not a rebrand of QA. It is what QA was always supposed to be. Along the way, Joe and Jason get into the containment problem and why alignment, not lockdown, is now the real safety goal. They dig into why testing cost scales quadratically, meaning ten times more generated code creates roughly a hundred times more testing demand. Jason also pushes back hard on skeptics of agentic testing, pointing out that almost nobody has run the obvious experiment of testing a site themselves for a week and comparing their results against what AI finds. You will also hear Jason's most practical piece of advice in the whole conversation. If you are not running the same suite five times against the same build and looking at the actual results, not just flake, you are not testing seriously in an AI world. Plus a detour into grokking, the Chinese Room, and Geoffrey Hinton, because it would not be a Jason Arbon episode without one. Listen up!
/episode/index/show/testtalks/id/42398730
info_outline
Playwright With AI: How to Automate Tests Without Shipping AI Slop with Andrew Knight
08/04/2026
Playwright With AI: How to Automate Tests Without Shipping AI Slop with Andrew Knight
Your developers just got supercharged by AI coding agents. Your test coverage did not. So how do you keep quality high when product code is shipping faster than any test team can follow? In this episode Andrew Knight, the Automation Panda and Senior Director of Product and Engineering at Cycle Labs, shares how his six person team is running the biggest release quarter in company history using AI coding agents, spec driven development, and Playwright. You will discover: How Playwright turned itself into an AI automation platform with the MCP server, the planner, generator, and healer agents, and the new CLI and skills approach that cuts your token usage way down. Why Andy uses Spec Kit to codify his testing strategy once, in markdown, so quality standards get baked into every single pull request instead of being caught in review. How to decide which AI generated tests are actually worth running when compute time and budget are finite. What AI slop looks like from a manager's seat, and how to build a team culture that catches it before it ships. Why Andy believes AI coding tools are the new compiler and markdown is the new programming language, plus my pushback on what that means for everything testers were trained to care about. What Andy really thinks about token costs, subscription tiers, and what happens when the AI subsidies run out. Plus the one piece of advice he gives to any tester still sitting on the sidelines of the AI shift. Whether you are an automation engineer, a QA lead, or an engineering manager trying to figure out where testing fits in an AI first workflow, this episode gives you a practical playbook you can start using this week.
/episode/index/show/testtalks/id/42315310
info_outline
Test Automation Won't Save Your QA Career, but These Skills Will with Keith Klain
07/28/2026
Test Automation Won't Save Your QA Career, but These Skills Will with Keith Klain
Keith Klain has spent 25+ years leading enterprise quality programs in financial services and is one of the testing industry's most respected voices. In this episode, he joins Joe to discuss his BrowserStack talk, "The Death of Test Engineering," why testing isn't dying—but outdated testing roles are—and how AI is reshaping the future of software quality. In this episode: Why "The Death of Test Engineering" isn't what you think The difference between checking and testing Will AI replace software testers? Why confidence engineering matters How to communicate testing in business terms Where automation adds real value Why agentic AI testing is overhyped Notable Quotes "If you look at AI as a competitor, you're screwed. If you look at it as just another tool... it's not a threat." "Uncertainty in testing is a feature, not a bug." "If an organization doesn't value testing, you're doing it wrong." "You can be incredibly confident about something and incredibly wrong." One Thing to Try This Week Ask yourself after every critical defect: "What would the business impact have been if this had gone live?" It's a simple way to shift conversations from test cases to business risk. Resources Mentioned Rethinking Expertise — Harry Collins How Complex Systems Fail — Richard Cook Measuring and Managing Performance in Organizations — Robert Austin Emily Bender Alex Hanna Timnit Gebru Dagmar Monett Olivia Gambelin Ed Zitron Joe's Recommendation: Prophecy
/episode/index/show/testtalks/id/42246415
info_outline
AI Testing Strategy: Stop Being a Cost Center, Start Protecting Revenue with Nandini Srinivasan
07/22/2026
AI Testing Strategy: Stop Being a Cost Center, Start Protecting Revenue with Nandini Srinivasan
Nandini Srinivasan has spent 25 years in the quality industry and leads a global QA organization of over 150 engineers across the US, Canada, India, and Pakistan. In this episode, she breaks down exactly how she built a QA AI acceleration charter, ran a train-the-trainer model, and used a phased proof-of-concept approach to separate real AI from what she calls "powerful automation dressed up as AI. We get into: How she frames quality metrics for executives Using language around revenue protection Risk mitigation Feature velocity instead of test coverage percentages. She talks about the four pillars she uses to present her team's value: quality, scalability, performance, and availability. Nandini also shares her take on the future of QA hiring, why the judgment layer will always require a human, and what skills testers need to stay relevant as AI agents take over the more mechanical parts of automation. She is also writing a five-part LinkedIn series called "The Voice of QA in the AI Era" if you want to follow along.
/episode/index/show/testtalks/id/42182585
info_outline
How to Move from Prompt Engineering to Harness Engineering in Testing with Matt Wynne
07/14/2026
How to Move from Prompt Engineering to Harness Engineering in Testing with Matt Wynne
Matt Wynne, co-creator of Cucumber and BDD practitioner, joins Joe for the first time in over a decade to talk about what two years inside a Silicon Valley AI startup taught him about the future of software testing. Matt spent time at Mechanical Orchard working alongside experienced XP practitioners to modernize legacy COBOL mainframes using LLMs, and then spent a week with the team that coined the term "software factory," where the rule was simple: humans never write the code, never read the code. In this episode, Matt breaks down what harness engineering actually means, why shared understanding is still the real bottleneck even in an agentic world, and how testers can use multiple LLMs to review AI-generated pull requests without reading every line. He also gets honest about the grief that comes with realizing you can encode years of hard-won expertise into a Markdown file, and why that does not mean your skills are worthless. If you are working in a brownfield codebase, wondering how to handle the flood of agentic PRs, or trying to figure out where testers fit in a world where agents write the code, this conversation is worth your time. Find Matt at: mattwynne.net leansoftware.ai Also check out his course: Build a Software Factory: Hands-off agentic coding for experienced engineers
/episode/index/show/testtalks/id/42070265
info_outline
Agentic Engineering for Testers: How to Automate Your Way to the Top with Amit Rawat
07/07/2026
Agentic Engineering for Testers: How to Automate Your Way to the Top with Amit Rawat
Amit Rawat is an agentic engineer who spent two decades in QA before shifting fully into building AI agents. He's the creator of PromptWright, a desktop tool that turns natural language prompts into automated Playwright browser tests, complete with screen recording, Gherkin scenario generation, and self-healing locators. In this episode, Amit and Joe get into what it actually takes to work with AI agents at a high level, starting with why the planning phase matters more than the prompt itself. Amit breaks down his own workflow, brainstorming with AI, building a detailed plan in HTML before ever executing, and why curiosity and technical depth still matter even as AI gets more capable. They also cover why Amit believes QA professionals, more than developers or DevOps engineers, are best positioned to thrive in the agentic era, how he tracks the ROI on his $200-a-month Claude subscription, the "Chief of Staff," "Chief Health Officer," and "Chief Financial Officer" AI agents he's built to help manage different aspects of his personal life, and how he uses a memory layer so those agents understand his preferences and become more useful over time. If you're a tester, automation engineer, or QA leader trying to figure out where AI agents fit into your workflow and your career, this conversation is a practical look at what's already working today.
/episode/index/show/testtalks/id/41993740
info_outline
How to Test Any API Without Documentation with Liudas Jankauskas
06/30/2026
How to Test Any API Without Documentation with Liudas Jankauskas
Most API testing stops at the happy path. The problem is that the bugs that actually hurt you in production are sitting in everything many testers skip, like the boundary values, the oversized payloads, the missing tokens, the security headers, the inputs that make no sense at all. In this episode, Joe sits down with Liudas Jankauskas, who has spent almost twenty years breaking software and testing APIs since 2008. Liudas demonstrates Rentgen, his free and open-source API testing tool, live on screen. You'll watch him take a single request from a real app, map it in seconds, and generate dozens of tests covering security, boundaries, performance, and load—all from one click. You'll learn: How to discover APIs hiding under the hood of any application, even when there is zero documentation Why happy path testing leaves you exposed How to run a fast hygiene check before your real automation ever starts Liudas also explains why Rentgen runs completely locally with no server and no data leaving your machine, making it safe for banking, healthcare, and other regulated environments. Plus, he demonstrates the killer Copy Bug Report feature that drops a standards-based ticket straight into Jira or Trello. In This Episode You'll Discover How to find and test undocumented internal APIs using the browser DevTools Network tab Why happy path-only testing misses the bugs that matter most How Rentgen turns one request into security, boundary, performance, and load tests automatically Where Rentgen fits in your workflow as a pre-automation hygiene layer—not a Postman replacement How to use it for regression by comparing results across environments The one piece of advice Liudas gives every tester to level up their API testing Try Rentgen, free and open source, at Rentgen.io. Connect with Liudas Jankauskas on LinkedIn:
/episode/index/show/testtalks/id/41904600
info_outline
Your AI Code Review Is Lying to You (Here's the Fix) with Evan Marshall
06/23/2026
Your AI Code Review Is Lying to You (Here's the Fix) with Evan Marshall
Your AI code review tools read the diff. They stare at your code. But they never actually run it. So the bugs that only show up at runtime, the broken user flows, the bad query plan, the duplicate submission, sail right past review and land in front of your customers. In this episode, Joe Colantonio sits down with Evan Marshall, founder of Ito and a fifteen year engineer who spent five years in applied cryptography securing hundreds of millions of dollars for millions of people. Evan is taking that ship fast without breaking things discipline and pointing it straight at testing. Ito is an agentic QA platform that builds and runs your actual app on every pull request, navigates it like a real user, exercises the frontend and backend as one system, and brings back real runtime evidence: video replays, logs, the exact lines responsible, and steps to reproduce, posted right in your PR. You will learn: Why static code review misses the bugs that cause real production incidents How Ito spins up ephemeral environments and tests across UI, API, and database Why QA is not disappearing, it is leveling up into a manager and quality strategist role How to keep your test layer separate from your code generation so your signal stays honest The skills testers and engineers need as AI writes more of the code If you are shipping AI generated code at high velocity and your QA cannot keep up, this one is for you. Try Ito on your own code. Your first ten pull requests are reviewed free, no credit card required. Check it out at now. And as Joe always says, seeing is believing.
/episode/index/show/testtalks/id/41744120
info_outline
AI QA Agents Explained: How Amikoo Helps Testers with Ivan Barajas Vargas
06/16/2026
AI QA Agents Explained: How Amikoo Helps Testers with Ivan Barajas Vargas
What happens to QA when AI is writing ten times more code than your team can test? That is the exact problem Ivan Barajas Vargas set out to solve with Amikoo, a purpose-built AI QA agent designed to help testers, SDETs, and even developers move faster without sacrificing coverage or quality. Ivan is no stranger to AI in testing. Before generative AI became mainstream, he co-founded MuukTest, a test automation platform built on symbolic reasoning and expert systems. After six years and thousands of customer conversations, he went back to first principles to build Amikoo from scratch, this time with a harness of 12 specialized agents and 43 tools trained specifically for testing workflows. In this episode, Ivan and Joe dig into the real-world gap between AI code generation and AI-powered testing, why the QA role is being elevated rather than replaced, how Amikoo uses Playwright and page object model patterns under the hood, and where human judgment still has to stay in the loop. Ivan also shares practical advice on what skills QA engineers should be building right now and which test scenarios should never be fully delegated to an agent. If you are trying to figure out where testing fits in an agentic development world, this episode gives you a clear picture of what is possible today and what is coming next. Visit to try the freemium account, and mention you heard this on TestGuild to unlock double the free usage.
/episode/index/show/testtalks/id/41660775
info_outline
The AI Illusion: Why Testing Still Needs Humans with Joe Colantonio
06/09/2026
The AI Illusion: Why Testing Still Needs Humans with Joe Colantonio
Everyone is talking about AI replacing testers, writing tests, and transforming software quality. But what if we're asking the wrong question? In this solo episode, Joe Colantonio shares a growing concern he's seen while traveling across the country for TestGuild IRL events: a decline in testing fundamentals at the exact moment AI hype is reaching a fever pitch. Drawing insights from Carissa Véliz's book Prophecy: Prediction, Power, and the Fight for the Future, Wayne Roseberry's work on AI and meaning, and Tariq King's concept of Human Experience Testing, Joe explores why AI systems may be far less intelligent than many believe, and why human testers remain more important than ever. You'll discover: ✅ Why large language models generate plausible answers without understanding truth ✅ The difference between prediction, correlation, and genuine understanding ✅ Why AI can test software but cannot experience software ✅ What "Everything is tested, but nothing is experienced" really means ✅ How AI hype may be distracting teams from critical testing fundamentals ✅ Why empathy, context, and human judgment are becoming competitive advantages for testers Whether you're excited about AI, skeptical of it, or somewhere in between, this episode will challenge you to think more deeply about the future of testing and your role in it. Resources Mentioned 📖 Prophecy: Prediction, Power, and the Fight for the Future by Carissa Véliz 📖 Work and presentations by Wayne Roseberry 🎓 Free course: thebullshitmachines.com 🎤 Learn more about TestGuild IRL events: TestGuild.com/irl If you enjoy this episode, be sure to subscribe, leave a review, and share it with a fellow tester who's trying to navigate the AI era without losing sight of the fundamentals. #SoftwareTesting #AI #QualityEngineering #TestAutomation #SoftwareQuality #HumanExperienceTesting #ArtificialIntelligence #TestGuild #QA #TechPodcast
/episode/index/show/testtalks/id/41586345
info_outline
AI Agents in QA: How to Keep Up with AI-Driven Dev Velocity with Vilhelm von Ehrenheim
06/02/2026
AI Agents in QA: How to Keep Up with AI-Driven Dev Velocity with Vilhelm von Ehrenheim
AI coding tools promised to make development faster — and they delivered. But here's the problem nobody talks about enough: when you speed up coding, you don't eliminate the bottleneck in the SDLC. You just move it. And for most teams, it lands squarely in QA. In this episode, Joe sits down with Vilhelm von Ehrenheim, Co-founder and Chief AI Officer of QA.tech, to dig into how agentic AI is reshaping software testing from the ground up. Vilhelm brings serious ML credibility, he helped build Motherbrain, one of the earliest production LLM systems in venture capital, and he's now applying that experience to one of the hardest problems in software delivery: testing at AI development velocity. You'll learn how QA.tech's behavioral knowledge graph gives AI agents the context they need to actually understand your application, why validating user intent beats checking element identifiers every time, how autonomous agents can review PRs, reproduce bugs from Slack messages, and generate targeted tests without a single line of test code ,and what the tester's role actually looks like when agents do the heavy lifting. If you're wondering whether your QA practice can survive the pace of AI-driven development, this one's required listening. 🔗 Book a demo now: https://testgld.link/qatechdemo
/episode/index/show/testtalks/id/41510625
info_outline
Maestro MCP, AI Mobile Testing That Fixes Its Own Tests with Leland Takamine
05/19/2026
Maestro MCP, AI Mobile Testing That Fixes Its Own Tests with Leland Takamine
What happens when AI agents can not only write mobile app code, but also validate their own work automatically? In this episode, I sit down with Co-founder and CEO Leland Takamine to explore one of the biggest shifts happening in software testing right now: agentic mobile testing. Leland shares how his team went from solving mobile performance testing challenges to building one of the fastest-growing mobile automation frameworks used by companies like Microsoft, Meta, Amazon, and DoorDash. We dive deep into: How AI coding agents are changing mobile testing workflows What “closing the agentic feedback loop” actually means Why deterministic testing still matters in the age of AI How Maestro MCP lets AI agents validate mobile app changes automatically Why mobile test maintenance may finally become manageable The future role of testers as AI-generated code explodes Leland also gives a live demo showing an AI agent building, validating, debugging, and generating a reusable mobile test completely autonomously. If you care about AI testing, mobile automation, MCP servers, or the future of QA engineering, this episode will likely change how you think about testing workflows over the next few years. Try it out now for yourself: Maestro Studio: Maestro MCP docs:
/episode/index/show/testtalks/id/41352195
info_outline
AI Testing Costs, How to Prevent Runaway Token Bills with Arthur Hicken
05/12/2026
AI Testing Costs, How to Prevent Runaway Token Bills with Arthur Hicken
AI-powered testing tools are exploding across software engineering teams… but so are the hidden costs. In this episode, Joe sits down with Arthur Hicken to unpack the growing problem of runaway AI token usage, unexpected LLM billing, and the operational risks of deploying AI agents into testing and DevOps pipelines. Inspired by Arthur’s article on the emerging “Token Tax,” this conversation explores why many teams are underestimating the true cost of AI automation. You’ll learn: Why AI-generated testing can create unexpected scaling costs How runaway AI agents and infinite loops happen Real-world examples of massive AI billing surprises Why deterministic problems shouldn’t always use LLMs The hidden risks of “vibe testing” and autonomous AI remediation How QA teams can monitor, test, and control token usage Why performance testing and service virtualization matter more than ever in AI systems Practical strategies to avoid expensive AI deployment mistakes Whether you’re a software tester, automation engineer, QA leader, or DevOps practitioner, this episode will help you think more strategically about AI testing before costs spiral out of control.
/episode/index/show/testtalks/id/41271765
info_outline
Stop Rewriting Tests: How to Add AI to Selenium and Playwright Without Starting Over with Alex Rodionov
05/05/2026
Stop Rewriting Tests: How to Add AI to Selenium and Playwright Without Starting Over with Alex Rodionov
AI-powered testing tools promise faster automation and less maintenance, but most require teams to abandon their existing frameworks. In this episode, we explore Alumnium, an opensource AI-native end-to-end testing solution created by Alex Rodionov, an engineer at Airbnb and a tech lead on the Selenium project. Instead of replacing tools like Playwright or Selenium, Alumnium adds an AI layer on top, helping teams: Reduce test maintenance by removing brittle locators Build more resilient, self-healing tests Write less code while increasing coverage Run tests across web and mobile using intent-based steps We also go beyond the hype and break down what actually matters for real teams: Why AI-driven tests can still become flaky The performance and cost tradeoffs of LLM-based execution What “context rot” is—and how it impacts long test runs How to safely introduce AI into existing test suites without rewriting everything Check it out now: https://testguild.me/alumAI
/episode/index/show/testtalks/id/41169305
info_outline
Claude AI Mobile Testing, Run Real Device Tests with AI with Frank Moyer and Chris Faulhaber
04/28/2026
Claude AI Mobile Testing, Run Real Device Tests with AI with Frank Moyer and Chris Faulhaber
AI is changing how we build and test software, but most teams are still struggling to turn AI-generated tests into real production value. Use code TESTGUILD3 try for yourself free now for 3 months: In this episode, we break down what actually works when it comes to AI-powered mobile test automation, especially when running tests on real devices not simulators from Claude. You’ll learn: How teams are generating and running Appium tests using natural language in minutes Why AI-generated tests often fail—and how to avoid costly false positives The real impact of AI on test automation roles and responsibilities How real device testing exposes issues AI alone can’t catch Practical ways to reduce test maintenance while increasing coverage We also explore a major shift happening right now: AI is making it easier to create tests—but dramatically increasing the volume of code and risk that needs to be validated. That means one thing: Testing isn’t going away—it’s becoming more critical than ever. If you’re a QA engineer, automation engineer, or DevOps leader trying to keep up with AI-driven development, this episode will give you a clear, practical perspective on what to focus on next.
/episode/index/show/testtalks/id/41049355
info_outline
AI Testing Is Breaking Your Pipeline. Fix Quality Before It’s Too Late with Eric Minick
04/15/2026
AI Testing Is Breaking Your Pipeline. Fix Quality Before It’s Too Late with Eric Minick
AI coding tools are helping teams move faster than ever, but there’s a hidden cost. In this episode, we break down new insights from a DevOps industry report revealing a growing “velocity paradox”: teams are shipping more code, but experiencing more failures, rollbacks, and burnout. You’ll discover why AI adoption is heavily skewed toward coding, but not testing, pipelines, or observability, and how that imbalance is creating fragile systems that break under pressure. More importantly, you’ll learn what high-performing teams are doing differently to maintain quality while scaling speed. What You’ll Discover: ✔️ Why AI is increasing deployment failures (and how to stop it) ✔️ The “velocity vs quality” trap hurting modern DevOps teams ✔️ How to reduce flaky tests and pipeline instability ✔️ Why observability and feature flags are now critical, not optional ✔️ Practical ways to improve your CI/CD pipeline for AI-driven development ✔️ The role of QA engineers in the age of AI (and why it’s growing, not shrinking) If you're a tester, automation engineer, or DevOps leader trying to keep up
/episode/index/show/testtalks/id/40887420
info_outline
Scaling Quality Engineering: How to Deliver Faster Across Global Teams with Sunita McCoy
04/07/2026
Scaling Quality Engineering: How to Deliver Faster Across Global Teams with Sunita McCoy
AI is changing how we build and test software, but most teams are struggling to turn that promise into real results. In this episode, we break down what it actually takes to scale quality engineering across global teams without creating bottlenecks, burnout, or broken processes. You’ll learn: why most test automation and transformation initiatives fail how to separate AI hype from reality what high-performing teams are doing differently to ship faster with confidence Today’s expert, Sunita McCoy, a Global Engineering Leader and Transformation Specialist, shares practical insights from leading large-scale engineering transformations, including: how to build a culture that supports AI adoption why “quality as a phase” is dead how to shift toward treating quality as a product If you're a QA leader, automation engineer, or DevOps professional trying to improve reliability, reduce risk, and future-proof your skills in the age of AI, this episode gives you a clear path forward.
/episode/index/show/testtalks/id/40761085
info_outline
Mobile Test Automation is Broken. Here’s How QApilot Fixes It with Aditya Challa
03/31/2026
Mobile Test Automation is Broken. Here’s How QApilot Fixes It with Aditya Challa
Mobile test automation is still one of the biggest bottlenecks in modern software delivery. In this interview, QApilot’s Co-founder Aditya Challa explains why most AI testing approaches fail and how to fix them. Learn more about QApilot: If your mobile tests are flaky, slow, or hard to trust, you’re not alone. Most teams are trying to apply LLM-based AI to problems that actually require deterministic reliability—and that’s where things break down. In this video, you’ll learn: Why mobile test automation breaks at scale The real issue with “99% accurate” AI in testing LLMs vs deterministic AI (and why it matters for mobile apps) How flaky tests destroy confidence in your pipeline How QApilot approaches mobile testing differently What reliable, scalable mobile automation should look like What this means for you: Fewer false positives, faster releases, and mobile tests you can actually trust. 00:00 Why Mobile Test Automation Is Still Broken 01:10 QApilot Overview 01:51 Why Mobile Testing Tools Fail 03:13 Why Appium Isn’t Enough 05:09 QApilot’s Approach to Mobile Testing 07:10 Scaling Mobile Testing Across Devices 08:02 Autonomous Testing + Human in the Loop 10:55 How QApilot Works (Architecture + Agents) 13:45 Real Example: Mobile App Crawling in Action 16:31 Finding Bugs Automatically (Performance + Accessibility) 18:52 Device Farms & Real Device Testing 21:50 Future of Mobile Testing (SRE + AI + Quality Layer) 27:06 Real Customer Results & Case Study 31:02 Why QApilot Focuses Only on Mobile 34:04 Where QApilot Fits in CI/CD 36:00 How to Try QApilot + Final Advice
/episode/index/show/testtalks/id/40682595
info_outline
AI Testing: How Solo Testers Stay Confident in Releases with Christine Pinto
03/25/2026
AI Testing: How Solo Testers Stay Confident in Releases with Christine Pinto
Are you the only tester on your team—and expected to ensure quality across everything? In this episode, we break down the growing challenge of solo QA testing in the age of AI-driven development—where code is generated faster than ever, but confidence hasn’t caught up. Christine Pinto shares real-world insights from her experience as a solo tester and now as a founder building tools designed to help testers reduce risk, collaborate better, and make smarter release decisions. You’ll learn: Why “all tests passing” doesn’t mean your product is safe The hidden risks of AI-generated code and test automation How to shift from test coverage to risk-based testing Practical ways solo testers can avoid burnout and isolation How to bring collaboration back into QA—even if you’re the only tester Why better requirements still matter more than better AI
/episode/index/show/testtalks/id/40615755
info_outline
AI Testing from Production Logs: Generate Smarter Regression Tests with Tanvi Mittal
03/17/2026
AI Testing from Production Logs: Generate Smarter Regression Tests with Tanvi Mittal
What if your production logs could automatically generate new test cases? In this episode, Joe Colantonio sits down with Tanvi Mittal to break down how AI-powered log mining is changing the way teams approach software testing, quality engineering, and DevOps. Most teams ignore production logs or use them only for debugging. But those logs contain real user behavior, real failures, and real edge cases—the exact scenarios your test suite is probably missing. 👉 Learn how to: Convert production logs into automated regression tests Use AI to detect real-world failure patterns Apply shift-right testing to catch bugs earlier (and smarter) Handle the challenge of testing non-deterministic AI systems Reduce flaky tests and automation debt with real data If you're working with Playwright, Selenium, Cypress, or AI-driven testing tools, this episode will give you a completely new way to think about test coverage.
/episode/index/show/testtalks/id/40518175
info_outline
AI Testing: How to Ensure Quality in Non-Deterministic Systems with Adam Sandman
03/10/2026
AI Testing: How to Ensure Quality in Non-Deterministic Systems with Adam Sandman
How do you ensure software quality when the system you're testing doesn't give the same output twice? Go to and start your free 30-day trial, no credit card, no contract required. That's the core challenge facing every QA team building or testing AI-powered applications today and it's breaking all the rules we've relied on for decades. In this episode of the TestGuild Automation Podcast, I sit down with Adam Sandman, co-founder of Inflectra, to get into what non-deterministic AI testing actually means in practice, why traditional pass/fail testing no longer cuts it, and what quality professionals need to do differently right now. We cover: Why AI-generated code is raising the stakes for QA teams while budgets stay flat The fundamental difference between deterministic and non-deterministic systems — and why it changes everything about how you test How to set acceptable risk thresholds for AI systems (hint: it depends on whether you're building an e-commerce chatbot or an air traffic control system) Why testers who embrace AI as a tool — not a threat — will be the ones leading their organizations forward How a live demo failure at a conference inspired Inflectra's new non-deterministic testing tool, SureWire If you're a tester, QA manager, or automation engineer trying to figure out how to keep up with AI-driven development without losing your mind — or your job — this one's for you.
/episode/index/show/testtalks/id/40390120
info_outline
Test Automation Tools That Scale: From Zero to 1.6M Users with Sanjay Kumar
03/03/2026
Test Automation Tools That Scale: From Zero to 1.6M Users with Sanjay Kumar
What does it really take to build a test automation tool that millions of testers rely on, without venture capital, paid ads, or a massive team? In this episode, we explore how SelectorsHub grew into one of the most widely used productivity tools in software testing, reaching over 1.6 million testers worldwide. You’ll discover: How to build test automation tools that solve real QA pain Why community-driven development beats chasing funding How to prioritize features when you have thousands of users Whether AI testing tools will replace selector-based automation How to choose between Playwright vs Selenium using automation analysis What founders and QA leaders can learn from scaling without VC If you’re an automation engineer, QA lead, DevOps professional, or tool builder looking to scale smarter, this episode delivers real-world insight without hype. Whether you're building frameworks internally or launching your own automation product, you’ll walk away with a clearer strategy for solving problems testers actually care about.
/episode/index/show/testtalks/id/40303970
info_outline
AI Test Automation: Ship Twice as Fast with 10x Coverage with Karim Jouini
02/24/2026
AI Test Automation: Ship Twice as Fast with 10x Coverage with Karim Jouini
AI test automation is evolving fast — but most tools still generate brittle code that breaks with every UI change. See it for yourself now: In this episode of the TestGuild Podcast, Joe Colantonio sits down with Karim Jouini, founder of Thunders, to explore a radically different approach to AI testing: executing test automation in plain English without generating Selenium or Playwright code. Instead of “auto-healing selectors,” Thunders interprets natural language directly — allowing teams to: Ship twice as fast Achieve 10x test coverage with the same resources Reduce regression cycles from weeks to days Eliminate massive automation maintenance overhead Karim shares real-world case studies, including: A European bank that reduced a 3-year core banking upgrade testing effort to 4 months A SaaS company that transitioned from a traditional QA team to AI-assisted product-led testing We also discuss: Whether AI test agents replace QA roles How QA managers must shift from individual contributors to AI managers The risks of adopting AI without a defined success metric The future of shift-left testing in the AI era If you’re a software tester, automation engineer, QA lead, or DevOps leader trying to understand what’s hype versus real ROI in AI testing — this episode breaks it down. Try it for yourself and see how AI testing fits into your pipeline. Get personal demo:
/episode/index/show/testtalks/id/40221620
info_outline
Performance Testing with AI w/ Akash Thakur
02/17/2026
Performance Testing with AI w/ Akash Thakur
Is traditional performance testing becoming obsolete? In this episode, performance engineering expert Akash Thakur shares why AI is fundamentally transforming load testing, scripting, observability, and shift-left strategies. With 17 years of real-world enterprise experience, Akash explains how AI-augmented tools are already reducing scripting time by 30%, improving analysis speed, and helping teams move from reactive performance testing to predictive intelligence. You’ll learn: How AI is accelerating performance scripting and analysis Why shift-left performance testing is finally becoming realistic The role of structured data in predictive QA models How to test AI applications (LLMs, GPUs, inference throughput) differently than traditional web apps What the future role of performance engineers looks like — architect, not script writer If you’re a performance tester, SRE, QA leader, or DevOps engineer wondering how AI will impact your role — this episode gives you practical, actionable insights you can apply immediately.
/episode/index/show/testtalks/id/40139000
info_outline
Spec2TestAI: Stop Defects Before They Reach Production with Missy Trumpler
01/27/2026
Spec2TestAI: Stop Defects Before They Reach Production with Missy Trumpler
Most teams find defects after the damage is done — during regression, late-stage testing, or production incidents. That’s expensive, stressful, and completely avoidable. Try Spec2Test AI now: In this episode, Joe Colantonio sits down with Missy Trumpler, CEO of AgileAILabs, to explore how Spec2TestAI helps teams prevent defects before code ships by applying AI directly to requirements. You’ll learn: Why traditional test automation still misses critical risk How predictive, requirements-based AI testing works in practice What “shift-left” actually looks like beyond the buzzword How to reduce escaped defects without writing more tests Why secure, explainable AI matters for QA and enterprise teams This conversation is especially valuable for software testers, automation engineers, and QA leaders who want earlier visibility into risk, faster feedback, and higher confidence releases. Don't miss Automation Guild 2026 - Register Now:
/episode/index/show/testtalks/id/39902690
info_outline
Locust Performance Testing with AI and Observability with Lars Holmberg
01/13/2026
Locust Performance Testing with AI and Observability with Lars Holmberg
Performance testing often fails for one simple reason: teams can’t see where the slowdown actually happens. In this episode, we explore Locust load testing and why Python-based performance testing is becoming the go-to choice for modern DevOps, QA, and SRE teams. You’ll learn how Locust enables highly realistic user behavior, massive concurrency, and distributed load testing — without the overhead of traditional enterprise tools. We also dive into: Why Python works so well for AI-assisted load testing How Locust fits naturally into CI/CD and GitHub Actions The real difference between load testing vs performance testing How observability and end-to-end tracing eliminate guesswork Common performance testing mistakes even experienced teams make Whether you’re a software tester, automation engineer, or QA leader looking to shift-left performance testing, this conversation will help you design smarter tests and catch scalability issues before your users do.
/episode/index/show/testtalks/id/39720365
info_outline
Top 8 Automation Testing Trends for 2026 with Joe Colantonio
01/06/2026
Top 8 Automation Testing Trends for 2026 with Joe Colantonio
AI testing is everywhere — but clarity isn’t. In this episode, Joe Colantonio breaks down the real test automation trends for 2026, based on data from 40,000+ testers, 510 live Q&A questions, and 50+ interviews with industry leaders. This isn’t vendor hype or futuristic speculation. It’s what working testers are actually worried about — and what they’re doing next. You’ll learn: Why 72.8% of testers prioritize AI, yet don’t trust it alone The real reason AI testing feels harder instead of easier How integration chaos is blocking automation success Why “AI auditor” and “quality strategist” are emerging career paths What agentic AI, MCPs, and vibe testing really mean in practice How compliance, accessibility, and security will redefine QA in 2026 If you’re a tester, automation engineer, or QA leader trying to stay relevant — this episode gives you the signal through the noise, and a clear path forward. If you’re a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out what’s coming — and how to stay connected. Discount code: 100GUILDCOIN (https://testguild.me/podag26)
/episode/index/show/testtalks/id/39634490
info_outline
Automation Testing Podcast 2026: New Schedule, Events, Discounts with Joe Colantonio
12/28/2025
Automation Testing Podcast 2026: New Schedule, Events, Discounts with Joe Colantonio
This is a special end-of-year episode of the Automation Testing Podcast. With family in town and a busy holiday season, Joe didn’t want to skip a week without checking in and saying thank you to the TestGuild community. In this short episode, Joe shares: A huge milestone as the podcast approaches its 13-year anniversary Why the Automation Testing Podcast is moving from Sundays to Tuesdays starting in 2026 How loyal listeners can still get $100 off a full 5-day Automation Guild 2026 pass A sneak peek at TestGuild IRL — live, in-person events coming next year Gratitude for the listeners, YouTube community, and sponsors who make TestGuild possible If you’re a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out what’s coming — and how to stay connected. Discount code: 100GUILDCOIN () Questions or ideas? Email Joe directly at As always — test everything, and keep the good.
/episode/index/show/testtalks/id/39545935
info_outline
AI Testing LLMs & RAG: What Testers Must Validate with Imran Ali
12/21/2025
AI Testing LLMs & RAG: What Testers Must Validate with Imran Ali
AI is transforming how software is built, but testing AI systems requires an entirely new mindset. Don't miss AutomationGuild 2026 - Register Now: Use code TestGuildPod20 to get 20% off your ticket. In this episode, Joe Colantonio sits down with Imran Ali to break down what AI testing really looks like when you’re dealing with LLMs, RAG pipelines, and autonomous QA workflows. You’ll learn: Why traditional pass/fail testing breaks down with LLMs How to test non-deterministic AI outputs for consistency and accuracy Practical techniques for detecting hallucinations, grounding issues, and prompt injection risks How RAG systems change the way testers validate AI-powered applications Where AI delivers quick wins today—and where human validation still matters This conversation goes beyond hype and gets into real-world AI testing strategies QA teams are using right now to keep up with AI-generated code, faster release cycles, and DevOps velocity. If you’re a tester, automation engineer, or QA leader wondering how AI changes your role,not replaces it,this episode is your roadmap.
/episode/index/show/testtalks/id/39485570
info_outline
AI Codebase Discovery for Testers with Ben Fellows
12/14/2025
AI Codebase Discovery for Testers with Ben Fellows
What if understanding your codebase was no longer a blocker for great testing? Most testers were trained to work around the code — clicking through UIs, guessing selectors, and relying on outdated docs or developer explanations. In this episode, Playwright expert Ben Fellows flip that model on its head. Using AI tools like Cursor, testers can now explore the codebase directly — asking questions, uncovering APIs, understanding data relationships, and spotting risk before a single test is written. This isn’t about becoming a developer. It’s about using AI to finally see how the system really works — and using that insight to test smarter, earlier, and with far more confidence. If you’ve ever joined a new team, inherited a legacy app, or struggled to understand what really changed in a release, this episode is for you. Registration for Automation Guild 2026 Now: https://testguild.me/podag26
/episode/index/show/testtalks/id/39404170