OpenAI's o3 Model: A $1000 Query Challenge

The recently launched o3AI model by OpenAI has emerged as the organization's most powerful artificial intelligence offering to date, but its operational costs are raising eyebrows across the industry, with reports indicating that a single query could exceed $1000.

According to a report by TechCrunch, the o3 model utilizes a technique referred to as "computation during testing" to tackle complex problems. This approach allows the model to invest additional time in deliberation and exploration of various scenarios before arriving at a conclusion. OpenAI engineers are optimistic that this method will yield higher-quality responses when presented with intricate prompts.

In benchmark assessments conducted by François Chollet, the founder of the ARC-AGI benchmark test, o3 achieved a score of 87.5% in its "high computation mode," a significant leap compared to the 32% score of its predecessor, the o1 model. This notable performance enhancement underscores the advancements made in the o3 model; however, the cost associated with this increased capability is substantial. To reach this elevated score, o3 requires 170 times more computing power than its low-power counterpart, which operates at a cost of less than $4 per task.

The situation raises concerns within the industry regarding the dichotomy between the performance of the o3 model and its exorbitant operational expenses. On one hand, the dramatic increase in o3's score suggests that AI models can continue to progress through scaling, which involves augmenting processing power and training data. Conversely, there are growing criticisms about the diminishing returns from scaling, as the enhancements in o3's capabilities stem primarily from improvements in its reasoning abilities rather than mere scaling. Nonetheless, the high operational costs are a point of contention.

Even the low-computation version of o3 achieved a score of 76% in benchmark evaluations, with a per-task cost of approximately $20. While this remains relatively affordable compared to other AI solutions, it still represents a several-fold increase over its predecessor. Moreover, given that ChatGPT Plus charges users $25 monthly, OpenAI is under significant pressure to manage costs while enhancing user interaction capabilities.

In a recent blog post discussing the benchmark outcomes, Chollet remarked that although o3 operates at a level close to human capability, "the costs are still high and not yet economical." He highlighted that the labor cost to resolve ARC-AGI tasks is around $5 per task, while energy expenses are merely a few cents. However, he remains optimistic, suggesting that "cost-effectiveness may significantly improve in the coming months and years." Currently, the o3 model has not been made available to the public, but a mini version is anticipated to launch in January next year.

Key Points:

1. The cost of a single query with the o3AI model exceeds $1000, highlighting its high operating expenses.
2. In the ARC-AGI benchmark test, o3 scored 87.5%, nearly three times that of the previous generation o1 model.
3. Currently, o3 has not been released to the public, and its "mini version" is expected to launch in January next year.