Are you looking for smarter insights delivered directly to your inbox? Subscribe to our weekly newsletters for essential updates on enterprise AI, data, and security.
New Research Highlights Cost Disparities in AI Models
A recent study has unveiled that open-source artificial intelligence models require considerably more computing resources than their closed-source counterparts when executing the same tasks. This finding could challenge the perceived cost benefits of open-source AI and influence how enterprises assess their AI deployment strategies. Conducted by Nous Research, the study revealed that open-weight models utilize between 1.5 to 4 times more tokens— the fundamental units of AI computation—compared to closed models such as those offered by OpenAI and Anthropic. The disparity becomes even more pronounced for basic knowledge questions, with some open models using up to 10 times more tokens.
Understanding Token Efficiency
The researchers emphasized the importance of measuring “token efficiency”—the number of computational units used relative to the complexity of the solutions provided. This metric, which has not been systematically studied despite its significant cost implications, showed that open models can be more expensive per query, even though they generally have lower per-token costs. The findings challenge the common belief that open-source models inherently provide economic advantages over proprietary ones.
The Impact of Scaling Limits
As enterprises navigate the challenges of AI scaling, factors such as power caps, rising token costs, and inference delays are becoming increasingly critical. To gain insights into these issues, we invite you to join our exclusive salon, where leading teams will discuss strategies for:
– Transforming energy into a strategic advantage
– Architecting efficient inference for real throughput gains
– Unlocking competitive ROI with sustainable AI systems
Secure your spot today: [Join Now](https://bit.ly/4mwGngO)
Significant Findings Across AI Models
The research evaluated 19 different AI models across three categories: basic knowledge questions, mathematical problems, and logic puzzles. The results indicated that open-source AI models can consume up to 12 times more computational resources than the most efficient closed models, particularly for basic knowledge inquiries. The inefficiency is especially notable in Large Reasoning Models (LRMs), which utilize extended “chains of thought” to tackle complex problems. These models often expend thousands of tokens on relatively simple questions that should require minimal computation.
For instance, when asked a straightforward question such as “What is the capital of Australia?”, reasoning models were found to use “hundreds of tokens” when a one-word answer would suffice.
Performance Variability Among Providers
The research highlighted significant differences between various model providers. OpenAI’s models, particularly the o4-mini and the newly released open-source gpt-oss variants, showcased remarkable token efficiency, especially in mathematical problem-solving. OpenAI models were noted for their exceptional efficiency, using up to three times fewer tokens than other commercial models for math tasks. Among open-source options, Nvidia’s llama-3.3-nemotron-super-49b-v1 was identified as the most token-efficient model across all domains, while newer models from companies like Magistral exhibited “exceptionally high token usage” as outliers.
The efficiency gap varied notably by task type. While open models generally used about twice as many tokens for mathematical and logic problems, the difference surged for simple knowledge questions where efficient reasoning should be straightforward. OpenAI’s latest models achieved the lowest costs for these simple inquiries, while some open-source alternatives proved to be significantly more expensive despite their lower per-token pricing.
Implications for Enterprise AI Adoption
These findings carry immediate implications for enterprises considering AI adoption, where computing costs can escalate quickly with increased usage. Companies often prioritize accuracy benchmarks and per-token pricing when evaluating AI models, potentially overlooking the total computational requirements necessary for real-world applications.