Enterprises often believe AI models inherently need substantial computing power, leading them to seek out more resources.
Sasha Luccioni from Hugging Face suggests a shift in focus toward smarter AI utilization. Instead of acquiring more compute power, the goal should be improving model performance and accuracy.
Luccioni argues that the emphasis should be on smarter computing rather than increasing computational intensity.
“We currently overlook smarter methods because we’re fixated on acquiring more FLOPS, GPUs, and time,” she explained.
AI Scaling Hits Its Limits
Power constraints, token costs, and inference delays are challenging enterprise AI. Attend our exclusive salon to learn how leading teams are:
- Using energy strategically
- Developing efficient inference for real throughput gains
- Achieving competitive ROI with sustainable AI systems
Reserve your spot to stay ahead: https://bit.ly/4mwGngO
Hugging Face presents five essential strategies for enterprises to optimize AI usage efficiently.
1: Right-size the model to the task
Avoid using large, multipurpose models for every application. Smaller, task-oriented models can match or outperform larger models in accuracy for specific tasks, at a lower cost and with less energy consumption.
Luccioni observed that task-specific models consume significantly less energy. “These models focus on one task, unlike large language models meant for any task,” she explained.
Distillation is crucial; models start large and are refined. For example, DeepSeek R1 needs 8 GPUs, whereas distilled versions are smaller and require just one GPU.
Open-source models aid efficiency as they don’t need retraining, unlike a few years ago when resources were wasted searching for suitable models. Now, organizations can adapt base models.
“It allows shared innovation, avoiding the isolated, resource-intensive training on proprietary datasets,” said Luccioni.
Companies face disillusionment with instances where costs outweigh benefits. While standard uses like emailing and noting meetings are advantageous, task-specific models demand more effort, as generic models often prove inadequate and costly.
Luccioni emphasized, “Companies seek specific intelligence for specific tasks, not AGI.”
2. Make efficiency the default
Utilize “nudge theory” in system design, define reasoning budgets, and limit persistent generative features to reduce high-cost computing.
“Nudge theory” influences behavior subtly, like offering takeout cutlery only on request, reducing waste, Luccioni noted.
Default settings elevate usage and costs, with models performing unnecessary work. For instance, a default AI summary on search engines can be excessive.
Luccioni pointed out that in her experience, even for straightforward queries, some AI models resort to comprehensive reasoning.
“In complex situations, advanced reasoning is needed; for simple ones, it isn’t necessary,” she argued.
3. Optimize hardware utilization
Implement batching and adjust precision to optimize hardware energy and memory use.
Enterprises should assess when constant model operation is crucial, and when it isn’t. Periodic operation and batching can enhance efficiency.
“It’s challenging to generalize changes like distillation or precision adjustments,” Luccioni explained.
In her studies, batch size affects energy use, depending on specific hardware.
“Despite assumptions that larger batches maximize efficiency, it’s a nuanced issue requiring context,” she explained.
4. Incentivize energy transparency
Incentives can encourage energy efficiency; Hugging Face’s AI Energy Score assigns efficiency rankings, inspiring eco-friendliness.
Similar to Energy Star, this system encourages adoption among model builders, Luccioni stated.
Hugging Face maintains an updated leaderboard highlighting energy-efficient models.