Speak With An Expert

Mail icon to contact Liqid about our composable infrastructure technology

As the AI market matures from training to inferenceworkloads, the time for partners to capitalize on Inference is now. If you’restill coming up to speed on AI terminology and go-to-market strategies, ourfriends at Cloudflare covered the differences between “Training” and“Inference” in a great blog. Here are the Cliff’s Notes from the post:

-  Training isthe first phase for an AI model. Training may involve a process of trial anderror, or a process of showing the model examples of the desired inputs andoutputs, or both.

-   Inference isthe process that follows AI training. The better trained a model is, and themore fine-tuned it is, the better its inferences will be — although they arenever guaranteed to be perfect.

More simply, Training costs money while Inference makes money.

But what does this mean forresellers/integrators and their customers? Let’s discuss.

 The most important thing to consider is that training and inference require vastly different types of infrastructure. Training, for example, leverages very fast GPUs that consumeincredible amounts of power (750w-1kw each) making it very difficult to optimize the cost of these environments, measured by “tokens/dollar” or “tokens/watt.”For background on the lingo, a “token” is a unit of measurement in AI such as a word being read or an image being processed so, in other words, “how much information can my model review for the money I am investing.”

As the last several months have shown, only the largest companies in the market (OpenAI, Tesla, Google, Meta,etc) have the resources to perform training at scale, while mere mortals simply don’t have the power (literally and figuratively) to compete. As Mark Zuckerberg said in Lex Fridman’s interview/podcast from the Metaverse last fall, power has become the limiting factor for high-performing platforms such as AI.

So, if participating in training projects is off the table for most customers, does that mean resellers and integrators are on the outside looking in for AI revenue? No. Why?  Because Inference will soon be bigger than training. In fact, as shown in the graph below, computing needs for training applications are expected tolevel off in 2024/2025 while inference starts to take off, culminating ininference consuming 90% of AI computing needs by 2030.

How do we expect this all to playout? Well, when customers are interested in training a new model, they will use cloud infrastructure, leveraging the big players who have the ability to power such environments (keep an eye on the hyper-scalers using modular nuclear power). Less power hungry, on-prem or edge data center infrastructure will beleveraged to move the model into production (i.e. inference). 

End result? OEM and channel partners need to start havingInference conversations now. Optimizing for inference – and building out the infrastructure to do it efficiently – is the next big thing in AI. This willrequire open standards with the ability to work among vendors and devices tomeet the requirements of these individual deployments. They will differ vastly depending on the specific function, but energy efficiency and effective footprint management will be key as AI moves from buzz to widescale production overthe next few years. The most successful providers will be nimble enough to identify what technology works best with the task and set their customers up to demonstrate early wins to get greater buy-in and set them up for long-term success.

Written by
Mike Richards, Director, Reseller Sales
Posted on
June 4, 2024

Would you like to learn more?

Speak with one of our sales experts to learn more about how we aim to deliver complete composability. For other inquiries, you can drop us a line. We'll get back to you as soon as possible.