Homegrown Intelligence: AI Features for On-Prem MySQL Enterprise

leFred and Scott sit down with Gaurav Chadha to explore MySQL AI, a new solution that brings advanced AI features available in HeatWave to organizations running MySQL Enterprise Edition on-premises. Discover how MySQL AI bridges the gap between cloud innovation and on-premise infrastructure, making transformative AI capabilities more accessible, secure, and efficient for teams that rely on MySQL Enterprise Edition wherever their databases reside.

--------------------------------------------------------------

Episode Transcript:

00:00.000 --> 00:25.000
Welcome to Inside MySQL: Sakila Speaks, a podcast dedicated to all things MySQL. We bring you the latest news from the MySQL team, MySQL product updates and insightful interviews with members of the MySQL community.

00:25.000 --> 00:32.000
Sit back and enjoy as your hosts bring you the latest updates on your favorite open source database. Let's get started.

00:32.000 --> 00:37.000
Welcome back to another episode of Inside MySQL: Sakila Speaks. Hi, I'm LeFred.

00:37.000 --> 00:38.000
And I'm Scott Stroz.

00:38.000 --> 00:41.000
Today, we are thrilled to have Guarav Chadha joining us.

00:41.000 --> 00:51.000
Guarav is a Senior Development Manager leading development of MySQL HeatWave Lakehouse with a keen interest in systems, machine learning and computer architecture.

00:51.000 --> 01:10.000
Guarav brings a multifaceted expertise to database technology. Following the completion of his PhD from the University of Michigan, Ann Arbor, Guarav started at Oracle Labs in 2016, working on a research project which eventually graduated into MySQL HeatWave.

01:10.000 --> 01:16.000
But today we will talk with him about MySQL and AI on premise. Welcome Guarav.

01:16.000 --> 01:17.000
Thanks, Fred. Hi, Scott.

01:17.000 --> 01:18.000
Hi, Guarav. How are you?

01:18.000 --> 01:19.000
Doing good.

01:19.000 --> 01:32.000
So we're going to dive right in. And AI, we see AI is taking over the world. It's being touted for the solution to everything.

01:32.000 --> 01:41.000
How do you see AI transforming traditional on-premise database environments, especially in enterprise setups?

01:41.000 --> 01:54.000
Yes, Scott. So, I completely agree. AI is a transformational technology, and it has the potential to improve everything that we see around us.

01:54.000 --> 02:07.000
So, with regards to traditional on-premise database environments, especially in enterprise setups, I see multiple categories here. So, AI is a technology and a toolset.

02:07.000 --> 02:32.000
And like many other operators in databases, it can help with more and different data analysis. So, think of AI as a new set of SQL operators, which can tease out or analyze data and derive insights that are hard to do it with other operators, with other analysis tools.

02:32.000 --> 02:45.000
And hard for folks to call up. And hard for folks to code up. And that's where I think AI enhances it very easily enters into the database environments.

02:45.000 --> 02:56.000
What I mean by that is examples are recommendation systems, anomaly detection, so on and so forth.

02:56.000 --> 03:02.000
The other category is what I would say user assistance.

03:02.000 --> 03:15.000
So, not everyone is a SQL expert. And we want database technology and databases to be accessible to more people who may or may not come from a traditional database background.

03:15.000 --> 03:22.000
And SQL is a very powerful language and where it can be daunting to start with.

03:22.000 --> 03:35.000
So, again, this is a general category where maybe folks who are not very familiar with a specific programming language like SQL could write things out in just plain natural text.

03:35.000 --> 03:42.000
And AI tools could translate this into a programmatic interface or programmatic language or SQL directly.

03:42.000 --> 03:50.000
And that's another facet where I think AI can make database systems more approachable to a larger category of folks.

03:50.000 --> 03:57.000
It can also give you more user friendly responses, like instead of saying, oh, here's the error code, something went wrong.

03:57.000 --> 04:00.000
It can give you more information, more user friendly responses.

04:00.000 --> 04:06.000
So those are some examples of where I would say the second category, user assistance.

04:06.000 --> 04:12.000
The third category of where AI could help is database management.

04:12.000 --> 04:21.000
So databases are systems of record, the sources of truth and have a very high bar of staying up and being available.

04:21.000 --> 04:30.000
AI can help schedule maintenance at the right time where maybe the workload is low.

04:30.000 --> 04:35.000
They can predict things that might get slow.

04:35.000 --> 04:43.000
We have a whole area called predictive maintenance and make databases more highly available, more easily approachable.

04:43.000 --> 04:44.000
Thank you.

04:44.000 --> 04:46.000
This sounds very interesting.

04:46.000 --> 04:50.000
And because we are talking about MySQL on-prem, right?

04:50.000 --> 05:04.000
So from these categories, what features could we expect then one day to see in MySQL enterprise with AI or for AI?

05:04.000 --> 05:07.000
So what can you tell us about that?

05:07.000 --> 05:16.000
So for MySQL AI, we are bringing a whole host of AI features to on-premise MySQL deployments.

05:16.000 --> 05:22.000
And we will lean heavily with this first version on the first category, which is data analysis.

05:22.000 --> 05:25.000
How can AI help with data analysis?

05:25.000 --> 05:34.000
And within this, I would focus on, I would say, a few subcategories.

05:34.000 --> 05:48.000
The first is, with AI and generative AI specifically, it has brought the industry a new tool set to search through and understand documents.

05:48.000 --> 05:58.000
And not just structured data or relational data, just plain documents, which is true for a lot of enterprise companies.

05:58.000 --> 06:09.000
Companies have years and years worth of information stored in documents, in PDF documents, in HTML documents, and not really put into a database necessarily.

06:09.000 --> 06:12.000
And this has always been hard to search.

06:12.000 --> 06:23.000
It has been very manual, it has been very hard to bring to a database and perform a very fast and meaningful search.

06:23.000 --> 06:39.000
With generative AI and what we call vector store and vector search, you can search through unstructured data like documents, semantically, instead of just through keywords.

06:39.000 --> 06:42.000
You can search them by meaning.

06:42.000 --> 06:47.000
That's a very powerful technology that we are bringing to MySQL AI.

06:47.000 --> 07:02.000
So if users have documents in their file systems, they can ingest them into the database, and we will automatically create what we call a vector store out of it, which prepares the data in these documents to be searched semantically.

07:02.000 --> 07:13.000
Obviously, in order to this, we are adding a new operator, which does this semantic search, we call this vector distance.

07:13.000 --> 07:31.000
Additionally, I spoke about data analysis tools like recommendation systems, like anomaly detection, and these operators also being brought to MySQL AI, where you can plug them into your logs, or you can plug them into other metrics.

07:31.000 --> 07:40.000
And figure out when things can go wrong, or any other domain that is useful.

07:40.000 --> 07:46.000
An example of a domain for anomaly detection would be financial fraud, credit card fraud.

07:46.000 --> 07:49.000
So it's very useful in those scenarios.

07:49.000 --> 07:56.000
And the last category I would say among data analysis is generative AI.

07:56.000 --> 08:15.000
We're bringing LLMs to MySQL AI, and the power of LLMs really is they can generate new data and new user-friendly text from just bullet points, for instance.

08:15.000 --> 08:22.000
So not just analyzing data, but generating new data is possible through LLMs.

08:22.000 --> 08:29.000
So that I would say covers the first category.

08:29.000 --> 08:31.000
This is all data analysis.

08:31.000 --> 08:34.000
Among the second category, which is user assistance.

08:34.000 --> 08:41.000
User assistance is by bringing LLMs to on-premise MySQL AI deployments.

08:41.000 --> 08:50.000
It gives the user freedom to build more user-friendly applications or make the existing applications more user-friendly.

08:50.000 --> 08:53.000
And this is what we will start with, with version one of MySQL AI.

08:53.000 --> 09:04.000
So are there any specific features in MySQL Enterprise, like Firewall or Enterprise Audit, that support AI-enabled applications?

09:04.000 --> 09:09.000
So as we discussed, AI is an incredibly powerful set of tools and technologies.

09:09.000 --> 09:15.000
And this is our first salvo in enabling our customers to build and augment applications using AI.

09:15.000 --> 09:28.000
So we're bringing a whole tool set, we're bringing faster data analysis, more meaningful and different kinds of data analysis to help users build and augment existing applications.

09:28.000 --> 09:34.000
The door is certainly open to bringing AI to the ecosystem of products, as you mentioned, around the MySQL server.

09:34.000 --> 09:39.000
But with this first version, we are building these right into the MySQL server.

09:39.000 --> 09:54.000
With this MySQL AI, like you call it, right, is it compatible with or will it be compatible with all the architecture solutions that we also provide on-premise,

09:54.000 --> 10:01.000
like such as the InnoDB cluster, the cluster set, replica set, you know, for HA, for disaster recovery?

10:01.000 --> 10:08.000
If somebody goes in that direction, will he be able to keep deploying his MySQL the same way?

10:08.000 --> 10:15.000
So the AI feature set, the tool sets, are built right into the MySQL server.

10:15.000 --> 10:23.000
So all the architecture solutions which are deploying MySQL AI instances benefit from these features.

10:23.000 --> 10:33.000
Okay, so just to be clear, so we're not going to have two distinct products where one's MySQL EE and another one is MySQL AI.

10:33.000 --> 10:35.000
They'll all be together in the same product?

10:35.000 --> 10:41.000
So MySQL AI will be a distinct offering.

10:41.000 --> 10:48.000
It will have everything that Enterprise Edition has, plus the additional AI features we spoke about.

10:48.000 --> 10:51.000
And this is all about user choice.

10:51.000 --> 10:57.000
Customers can continue using MySQL EE if that is what they prefer.

10:57.000 --> 11:10.000
They can switch to MySQL AI and or buy new MySQL AI deployments to try out these new AI tools and get them familiar, get themselves familiar with it.

11:10.000 --> 11:17.000
The MySQL AI will have everything that Enterprise Edition has, plus the AI features.

11:17.000 --> 11:32.000
So it opens the door for users and customers to build their applications worry-free, as they've always built with MySQL EE, because the entire feature set of EE will be present in MySQL AI.

11:32.000 --> 11:37.000
But additionally, they can build more newer things with AI.

11:37.000 --> 11:43.000
In the previous episode, so the other speakers or guests, right?

11:43.000 --> 11:49.000
They extol the virtues of the cloud for AI, our cloud.

11:49.000 --> 11:52.000
Everything was nice and fast and it's good.

11:52.000 --> 12:06.000
And I would like to ask you if there are a performance trade-off between then deploying AI solutions in the cloud using MySQL HeatWave versus on-prem with this new MySQL AI.

12:06.000 --> 12:21.000
So with MySQL AI, we have brought the AI technology, we have built and deployed in the cloud to our EE customers, to on-premise environments.

12:21.000 --> 12:24.000
Cloud has some unquestionable advantages.

12:24.000 --> 12:28.000
Cloud has the benefit of scale-out, which can bring higher performance.

12:28.000 --> 12:37.000
It has GPUs, which can execute LLMs faster or larger LLMs, higher quality LLMs.

12:37.000 --> 12:48.000
So with MySQL AI, we have brought the AI technology that we built and deployed in the cloud to our on-premise customers and our Enterprise Edition customers.

12:48.000 --> 12:57.000
There are a very large, very large set of customers who are on-premise for a variety of reasons.

12:57.000 --> 13:02.000
And we want to serve them where they are.

13:02.000 --> 13:07.000
What I do want to point out is that cloud has some unquestionable advantages.

13:07.000 --> 13:14.000
It has the benefit of scaling out with more and more hardware, which can give you high performance.

13:14.000 --> 13:22.000
And like HeatWave, all these AI features are built to scale out with more resources.

13:22.000 --> 13:31.000
Cloud also has GPUs, which bring more performance for LLMs and can execute larger and higher quality LLMs.

13:31.000 --> 13:47.000
Cloud also has our HeatWave analytics engine, which provides faster analytics performance, allowing users to build combined applications with analytics, OLTP, AI.

13:47.000 --> 13:58.000
What is very important to note, all AI features we bring into MySQL AI are 100% API compatible with HeatWave in the cloud.

13:58.000 --> 14:02.000
So users can build their applications on MySQL AI.

14:02.000 --> 14:15.000
And should they feel the need for higher performance or an expanded feature set, and they want to move to the cloud, the applications will work without modifications.

14:15.000 --> 14:19.000
And we have tried very hard to make it 100% API compatible.

14:19.000 --> 14:40.000
So with MySQL AI, we have optimized inference of open-width LLMs on CPUs, right from one core all the way to 192 cores, using proprietary weight caching and quantization techniques.

14:40.000 --> 14:49.000
And this allows us to, the users to deploy MySQL AI on a range of computer infrastructure, depending on their need.

14:49.000 --> 14:51.000
It can be very small MySQL node.

14:51.000 --> 14:53.000
It can be a very beefy MySQL node.

14:53.000 --> 15:13.000
And as they improve the hardware, the performance, the latency, the quality, the concurrency we deliver increases, providing incentive to users to deploy on larger and larger hardware.

15:13.000 --> 15:25.000
Are we targeting, like, is MySQL AI targeted towards existing MySQL EE customers who are looking for more features?

15:25.000 --> 15:35.000
Or are we kind of targeting other potential customers and luring them in with AI to get them into the MySQL ecosystem?

15:35.000 --> 15:50.000
Both. We are bringing more features to MySQL EE, and we hope that more and more users, more classes of users, more classes of applications, find their home in MySQL.

15:50.000 --> 16:00.000
And absolutely, we want to allow our existing on-premise customers to be able to bring more of their workloads into MySQL.

16:00.000 --> 16:06.000
And for a number of reasons, many customers want to be on-premise.

16:06.000 --> 16:10.000
They might require deployments in the edge devices.

16:10.000 --> 16:13.000
They might require deployments in air-gapped environments for data security.

16:13.000 --> 16:21.000
So we want to enable these existing customers to bring more workloads.

16:21.000 --> 16:29.000
Of course, there are other customers who may not have ever looked at MySQL EE because they have requirements to deploy AI or to deploy generative AI.

16:29.000 --> 16:41.000
And we want to obviously give them this tool set and this enhanced feature set to bring their primary, secondary, tertiary workloads to MySQL EE.

16:41.000 --> 16:46.000
That's very nice to bring this to our on-prem customer.

16:46.000 --> 17:05.000
So correct me if I'm wrong, but to what I understood is that we modified LLMs to work on normal CPU for performance, and so we can run it.

17:05.000 --> 17:20.000
We don't need to have GPUs, but you also said that, yeah, if we really want to use a very large LLM, then it's better to use it to use the cloud than on-prem.

17:20.000 --> 17:33.000
But I wanted to also ask you, because LLMs that are evolving and not at the same speed of the MySQL releases.

17:33.000 --> 17:35.000
I have two questions in this one.

17:35.000 --> 17:42.000
It's like, oh, if there are new LLMs, will it be the possibility to the user to use it directly in GenAI?

17:42.000 --> 17:49.000
Or will it be updated at every new release of MySQL?

17:49.000 --> 18:01.000
Good question. So there's absolutely no exaggeration to state that a new LLM seemingly is released every month with its unique set of characteristics and benefits.

18:01.000 --> 18:20.000
So we will definitely bring new LLMs, which enhance the performance of all quality of results, as and when we feel that is useful to add to the existing set of LLMs we offer with future MySQL releases.

18:20.000 --> 18:31.000
So we will have we have taken the approach of building in LLMs in the package that we ship to our customers.

18:31.000 --> 18:37.000
There are many reasons for this. We have optimized these LLMs to run on CPUs.

18:37.000 --> 18:54.000
So we are able to run larger LLMs faster on CPUs, allowing our users to seamlessly use these LLMs without extra hardware or call outs to other services.

18:54.000 --> 19:02.000
So we will continue doing that with new LLMs as new ones prove to be useful for our users.

19:02.000 --> 19:14.000
What you say is that, OK, when we're going to release a new version of MySQL, if there are new LLMs that were interesting to update, they will be updated at that time, right?

19:14.000 --> 19:19.000
Build the LLMs into the package the users download and deploy.

19:19.000 --> 19:24.000
Users do not need to bring their own LLMs.

19:24.000 --> 19:51.000
Gurav, thank you for joining us today. It really was interesting for me, I can say personally, to learn about some of the advancements that are coming in MySQL AI and how it's going to be integrating with our EE version to allow people who need on premise or prefer on premise installations of MySQL to actually harness some of the AI power that we offer in HeatWave.

19:51.000 --> 19:54.000
Thank you, Scott. Thank you, Fred.

19:54.000 --> 19:55.000
Thank you very much. Bye bye.

19:55.000 --> 20:00.000
That's a wrap on this episode of Inside MySQL: Sakila Speaks. Thanks for hanging out with us.

20:00.000 --> 20:04.000
If you enjoyed listening, please click subscribe to get all the latest episodes.

20:04.000 --> 20:07.000
We would also love your reviews and ratings on your podcast app.

20:07.000 --> 20:12.000
Be sure to join us for the next episode of Inside MySQL: Sakila Speaks.