If you’ve spent any time with artificial intelligence platforms you know how incredibly valuable they can be to improving your business.
You also may have found out how challenging it can be to get its costs under control.
A recent study found that only 11 percent of nearly 400 surveyed businesses were able to accurately forecast their spending on artificial intelligence. And in a third of those tested cases, which AI model you used had a big impact on the cost, with lower priced platforms often outperforming higher-priced models.
The problem in figuring out costs gets to the heart of how AI software works. Unlike traditional software it more often functions like a human worker: making decisions, taking action and sometimes making mistake -- on the clock. It’s why more and more companies are working to get a better handle on the use of tokens -- the basic measurement for AI use. Tokenmaxxing, the original tool for AI use, is being rethought so companies can better budget their AI usage. Picking the right platform has a lot to do with it.
Researchers from Stanford University, Carnegie Mellon University and the University of California, Berkeley, as well as Microsoft Research worked on this earlier this year, the Wall Street Journal recently wrote, running models through more than 6,800 tasks spanning math, programming, science and other areas. In 32 percent of cases, lower-priced models actually cost more than higher-priced models.
“While more-advanced models tend to cost more per token, they can sometimes perform tasks more efficiently, leading to lower overall costs,” the WSJ wrote. “Likewise, asking a ‘cheap’ model to do something it isn’t suited to handle could cause a token run-up.”
“The practical takeaway is clear,” said Lingjiao Chen, one of the researchers. “Price alone should not be used to infer which model is actually cheaper.”
When asked to comment, Google told the WSJ, “Some prompt-level fluctuation is inherent to AI, and our testing shows this averages out across a high volume of real-world, diverse workloads. Total costs depend on many factors for a given task, which can make it hard to forecast new and evolving technology with precision.” She said the company offers customers more control via spending caps and flexible pricing.
The best advice from the WSJ? “As more companies add AI to their daily workflows, they will also need strategies to manage their use, such as training employees on which model to use for a particular task. Otherwise, they could end up with IT bills that come out of a black box.”