S4EP11: Inferencing and RAG at the Edge

What’s the real state of AI adoption, and how can edge computing play a crucial role in driving innovation? In this episode, Bill is joined by Nick Brackney, Senior Consultant of Generative AI Messaging at Dell. Nick has 20 years of experience across a variety of technology domains, spanning startups, tech giants and consulting work. The two discuss the true state of AI adoption and how inferencing and Retrieval Augmented Generation (RAG) will unlock new business opportunities at the edge.

---------

Key Quotes:

“I think AI is going to be synonymous with edge at some point, right? It's going to be that the only applications out there are AI-enabled applications.”

“Especially for the folks here who like the edge, RAG is the only way to do real-time, because fine tuning, there's always going to be this delay. So it's like, if I want to take my data from the edge and I want to retrain my machine, that's going to take cycles, hours of GPU time to do. But if I want to just sideload it into the generative AI using RAG, that could be instantaneous.”

--------

Timestamps:
(01:10) How Nick got started in tech

(04:09) What is the state of AI adoption?

(09:48) Understanding Retrieval Augmented Generation (RAG)

(14:11) Generative AI at the edge

(23:57) Achieving results with custom models

(24:41) Applications and benefits of synthetic data

(32:44) Advancements in chip development

(38:20) AI, PCs, and the tech refresh cycle

(40:07) Data sovereignty and edge AI

--------

Sponsor:

Over the Edge is brought to you by Dell Technologies to unlock the potential of your infrastructure with edge solutions. From hardware and software to data and operations, across your entire multi-cloud environment, we’re here to help you simplify your edge so you can generate more value. Learn more by visiting dell.com/edge for more information or click on the link in the show notes.

--------

Credits:

Over the Edge is hosted by Bill Pfeifer, and was created by Matt Trifiro and Ian Faison. Executive producers are Matt Trifiro, Ian Faison, Jon Libbey and Kyle Rusca. The show producer is Erin Stenhouse. The audio engineer is Brian Thomas. Additional production support from Elisabeth Plutko.

--------

Links:

Follow Bill on LinkedIn

Follow Nick on LinkedIn

‍