On AI Evals, TAR vs. LLMs and What Will be Different Next Year - Aron Ahmadia (VP Applied Science, Relativity)
Technically Legal - A Legal Technology and Innovation Podcast
Release Date: 09/17/2026
On AI Evals, TAR vs. LLMs and What Will be Different Next Year - Aron Ahmadia (VP Applied Science, Relativity)
Technically Legal - A Legal Technology and Innovation Podcast
In advance of this year’s in Chicago, , joins the podcast to discuss AI evaluation, AI benchmarking and the transition from traditional Technology Assisted Review (TAR) to generative AI in legal document review and eDiscovery. The conversation covers the daily operations of a global AI team, the concept of "evaluation as a product," and the methodology behind testing large language models (LLMs) for legal tasks. Aron also discusses a recent study that compared the accuracy and efficiency of TAR and human document review against Relativity's generative AI tool, aiR for Review. Aron talks...
info_outline
Moving Beyond the Billable Hour: Pricing Legal Work in the AI Age (Emily Logan Stedman - Husch Blackwell)
Technically Legal - A Legal Technology and Innovation Podcast
, commercial litigator at Husch Blackwell, discusses moving beyond the billable hour not just to modernize legal billing but to also prioritize lawyer well-being. The conversation explores the practical steps required to transition traditional litigation models toward Alternative Fee Arrangements (AFAs). Emily details her methodical approach to utilizing secure legal AI tools to analyze historical time entries, allowing her to offer phased flat fees, strategic counsel subscriptions, and predictable contract review pricing. The discussion also covers strategies for maintaining mental health in...
info_outline
AI Regulation: Where it is and Where it Might (or Should) Go (Justin Wales CLO @ Crypto.com & Author of AI Legal Handbook)
Technically Legal - A Legal Technology and Innovation Podcast
In this episode, makes a second visit to the podcast. This time around, he discusses his new book, The AI Legal Handbook, the emerging regulatory landscape, liability frameworks, and practical applications of artificial intelligence. Justin outlines how attorneys and regulators should approach modern AI models, the mechanics behind AI, and the effect of generative AI on legal education. The discussion covers key regulatory principles, the balance between strict liability and negligence frameworks, copyright issues in machine learning training, and the governance challenges of artificial...
info_outline
From Managing Partner to Intel CLO: Career Lessons and Thoughts on Modern Legal Practice (April Miller Boise)
Technically Legal - A Legal Technology and Innovation Podcast
In this episode April Miller Boise, with stints as Chief Legal Officer at Intel and managing partner at Thompson Hine, shares insights from her career. The conversation explores the shifting dynamics between corporate legal departments and outside counsel and the necessity of embracing legal technology. April also provides advice for junior lawyers on developing long-term expertise and trusted relationships in an AI-driven environment. Things we Talk About in this Episode Career progression: From Wall Street M&A partner and law firm managing partner to Chief Legal Officer. The...
info_outline
When Machines Have the Answers, Build Better Humans With AI Augmentation Not Automation (Vivienne Ming, Scientist, Professor, Author)
Technically Legal - A Legal Technology and Innovation Podcast
In this episode, discusses her new book, . The conversation examines the intersection of artificial intelligence and knowledge work, exploring how professionals can adapt to an increasingly automated economy. The discussion contrasts the limitations of basic AI task automation with the advantages of human-AI collaboration—the "cyborg" model—for solving complex, ill-posed problems. Ming highlights her research on forecasting market outcomes, the critical role of endogenous motivation, and how the legal industry and other elite professions must rethink entry-level training. Key Topics...
info_outline
Inside Microsoft Legal: A Look at AI Use Built for Scale (Rebecca Anderson - Head of Legal EMEA Microsoft)
Technically Legal - A Legal Technology and Innovation Podcast
This episode features a discussion between host Chad Main and , regarding the integration of artificial intelligence in corporate legal departments. The conversation covers Rebecca's career progression from a law firm associate to an international in-house counsel role. The core of the discussion examines Microsoft's internal AI deployment strategies, including the shift from personal productivity to intent engineering. Additionally, the episode addresses changing expectations for outside counsel regarding AI adoption and outlines the necessary skills for future legal professionals. Episode...
info_outline
Cost, Speed and Quality - One Law Firm’s Quest to Deliver All Three (Laurence Brown, Partner EIP and Head of Visser AI)
Technically Legal - A Legal Technology and Innovation Podcast
In this episode of Technically Legal, veteran patent lawyer Laurence Brown and Partner at the UK-based intellectual property firm EIP discusses the evolving concept of the "AI-native" law firm and how EIP is actively navigating this transition through their hybrid intelligence service, Visser AI. Laurence shares insights from his 25-year career, discussing how EIP uniquely integrates litigators and patent attorneys to provide comprehensive IP services. The core of the conversation centers on , named after the renowned European patent attorney Derk Visser, which leverages specialist...
info_outline
Affordable & Accessible: The Democratization of Legal Tech (Tyler Foreman VP of AI - Rocket Lawyer)
Technically Legal - A Legal Technology and Innovation Podcast
at , joins the show to discuss the intersection of artificial intelligence and the legal industry. Foreman shares his untraditional legal tech career path, spanning engineering at Intel, drone data analytics, and ultimately making the move to legal via contract lifecycle management (CLM) at DocuSign, before diving into his current work at Rocket Lawyer helping to provide legal resources for small-to-medium businesses and individuals. The conversation focuses on how modern generative AI and Large Language Models (LLMs) act as a legal operating system to simplify contract reviews, document...
info_outline
Building Software for Less: The Death of Expensive SaaS in the AI Era? (Nathan Wenzel - LegalOperatorAI)
Technically Legal - A Legal Technology and Innovation Podcast
We catch up with legal tech entrepreneur Nathan Wenzel to discuss his journey from founding and exiting to launching his newest venture, (formerly Lexiomatic). Nathan shares insights on the evolution of private equity (PE) structures, the shifting economics of the enterprise software market, and how artificial intelligence is disrupting traditional software development and corporate legal operations. Key Takeaways The Reality of Private Equity: Nathan breaks down the differences between growth PE, transition PE, and dividend-focused PE, highlighting his experience with operational scaling...
info_outline
The Rise of the Legal Quants (Jamie Tso & Raymond Sun)
Technically Legal - A Legal Technology and Innovation Podcast
Jamie Tso and Ray Sun, the founders of , discuss a fundamental shift in the legal profession: the transition from legal engineering to "legal quant" work. The conversation explores how technically fluent lawyers are using vibe coding, frontier AI models, and first-principles thinking to move beyond mere efficiency and toward the complete redesign of legal services. Jamie and Ray share their backgrounds in Big Law and their journeys into building custom legal tech tools. They delve into the origins of the LegalQuants community—an exclusive, invitation-only network of "builders"—and discuss...
info_outlineIn advance of this year’s RelFest in Chicago, Aron Ahmadia, Vice President of Applied Science at Relativity, joins the podcast to discuss AI evaluation, AI benchmarking and the transition from traditional Technology Assisted Review (TAR) to generative AI in legal document review and eDiscovery. The conversation covers the daily operations of a global AI team, the concept of "evaluation as a product," and the methodology behind testing large language models (LLMs) for legal tasks. Aron also discusses a recent study that compared the accuracy and efficiency of TAR and human document review against Relativity's generative AI tool, aiR for Review.
Aron talks about:
-
A day in the life of the VP of Applied Science at Relativity.
-
Navigating the rapid pace of AI and predicting what remains constant in legal tech.
-
Aron's background: From early computer networking in a law firm to data science at DARPA and Capital One.
-
Defining "evaluation as a product" and the importance of maintaining private benchmarks for LLMs.
-
The trade-offs between human evaluations and automated AI evaluations.
-
TAR (Predictive Coding) versus Generative AI: How prompt-based reasoning differs from label-based classification.
Episode Credits
Editing and Production: Grant Blackstock
Theme Music: Home Base (Instrumental Version) by TA2MI