A report card for the era of artificial intelligence

If you've invested in AI tools but struggle to prove they're actually paying off, this one's for you. Sarah Friar, Chief Financial Officer at OpenAI, has laid out a straightforward scorecard that helps businesses judge AI's real value. Instead of vague promises about "transformation," her framework focuses on four concrete measures: useful work completed, cost per successful task, dependability, and return on compute. In this article, we'll break down each element of the scorecard and show you how to apply it to your own AI projects.

Why AI Needs a Scorecard in the First Place

Many companies jumped on AI over the past couple of years, but few can answer a simple question: is it worth it? Traditional ROI calculations don't always translate well to AI. A chatbot might handle thousands of conversations, but were those conversations actually resolved? An AI coding assistant might suggest plenty of code, but how much of it survived review?

Friar's point, as CFO of one of the world's most prominent AI companies, is that AI spending deserves the same scrutiny as any other business investment. That means moving beyond adoption statistics and headline demos, and measuring outcomes that matter to the bottom line. Her scorecard gives leaders a shared vocabulary for those conversations—something rare in a field where excitement often outruns evidence.

The Four Measures That Matter

1. Useful Work

The first question is deceptively simple: how much genuinely useful work is AI doing? Not tasks attempted, not outputs generated—work that actually moves the needle. This forces teams to define success before deploying AI. If an AI agent drafts customer emails, useful work means emails that were sent without major edits. If it summarizes documents, it means summaries people actually relied on to make decisions.

Framing ROI around useful work shifts the conversation from activity to impact. A system that runs constantly but produces little of value isn't a win, no matter how impressive the underlying technology.

2. Cost per Successful Task

The second measure tackles economics directly: what does it cost to complete one successful task? This is where AI starts to look like any other operational expense. If a human handles a support ticket for five dollars and the AI does it for fifty cents with the same quality, the math is compelling. If the AI costs more once you account for retries, human oversight, and failed attempts, that's worth knowing too.

Cost per successful task is honest in a way that overall spending figures are not. It captures hidden inefficiencies—like an agent that needs three tries to get something right—and makes comparisons across tools and vendors possible.

3. Dependability

Useful work and low cost mean little if performance is unpredictable. Dependability asks whether the AI delivers consistent results over time, across different situations, and without constant babysitting. A tool that works brilliantly on Mondays but fumbles edge cases on Fridays creates more friction than it removes.

For businesses, dependability is often the difference between a pilot project and a production system. Teams can design around occasional errors, but they can't build processes on top of results they can't trust. Friar's inclusion of this measure signals that reliability—not just raw capability—is what separates enterprise-grade AI from impressive demos.

4. Return on Compute

The final measure looks at the infrastructure side. AI runs on compute, and compute costs money—sometimes a lot of it. Return on compute asks whether the processing power you're paying for is generating proportional value. This matters both for companies buying AI services and for those running their own models.

As models grow more capable, the question isn't just "can this model do the task?" but "is this the right amount of compute for the task?" Using a heavyweight model for lightweight jobs wastes money; using an undersized model for complex work produces failures. Return on compute encourages matching resources to actual needs.

How to Put the Scorecard to Work

Applying Friar's framework doesn't require a data science team. Start small:

1. Pick one AI use case already running in your organization and define what a "successful task" looks like for it.
2. Track useful work for a set period—count only outputs that met your success definition.
3. Calculate cost per success by dividing total spending (licenses, usage fees, oversight time) by the number of successful tasks.
4. Monitor dependability by logging failure rates and how often humans had to intervene.
5. Review your compute spend and ask whether a different model or approach would deliver similar results at lower cost.

Run this check monthly, and you'll build a clear picture of where AI is earning its keep—and where it isn't.

The Bigger Picture

What makes Friar's scorecard notable is its source. When OpenAI's own CFO emphasizes disciplined measurement, it's a sign that the AI industry is entering a more mature phase—one where accountability matters as much as ambition. For finance leaders and operations teams, the message is clear: treat AI like any other investment. Demand evidence, count what works, and let the numbers guide where you go next.

It's also a useful corrective to hype on both sides. AI isn't magic, and it isn't a fad. Measured honestly—with useful work, task-level costs, dependability, and compute efficiency—it becomes what it should have been all along: a tool judged by results.