When evaluating the return on investment in AI, companies are shifting from "how much was used" to "what was saved." Boris Cherny, head of the Anthropic programming tool Claude Code, said that looking at token consumption or dashboard usage alone only reflects activity levels and cannot directly indicate whether AI brings returns.

Not just looking at usage

In an article on the X platform discussing the path for enterprises to adopt AI, Cherny stated that once employees have integrated AI tools into their daily routines, the next step is to measure actual benefits. He believes that many companies will first look at usage data, but these metrics are closer to activity records than return on investment.

He proposed that a more meaningful question is: without AI, would this task have originally required an engineering team to dedicate time to completing? If the answer is yes, then companies can further estimate how many man-hours and corresponding labor costs would have been needed. This savings are closer to the direct returns brought by AI.

The benefits are reflected in the saving of working hours.

According to him, companies should not focus on the cost of model invocation itself, but rather view AI as a tool to replace some engineering labor. This approach is particularly direct for tasks such as code generation, fixing, and maintenance, because companies can usually estimate the development resources that would otherwise be required more clearly.

Cherny also mentioned that the greater benefit is not just "getting the same things done faster," but rather moving more fixes and maintenance to the background, freeing up teams to do new, constructive work. Only when teams are no longer tied up with a lot of repetitive tasks can companies begin to move forward with projects that were previously under-resourced and difficult to schedule.

Enterprises are starting to recalculate the cost of AI.

This statement also reflects a shift in companies' focus regarding AI spending. Previously, the industry popularized measuring AI progress by the scale of token usage, but as investment increases, companies are beginning to place greater emphasis on cost control and actual output.

Recently, executives from several technology and financial companies have publicly discussed this issue. JPMorgan Chase CEO Jamie Dimon recently stated that AI costs for enterprises are rising rapidly, therefore companies will evaluate their investments more rationally, just like managing other resources. OpenAI CEO Sam Altman also stated that reducing spending and increasing value have become frequently raised topics by enterprise clients.

This trend suggests that enterprises are shifting their focus in purchasing and deploying AI tools from pursuing higher usage rates to proving whether these tools truly reduce human input and expand the scope of work that teams can accomplish.