firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

Imagine a chef in a busy kitchen, faced with unexpected orders, ingredient shortages, and customer complaints. Now replace that chef with an AI managing a real company’s toughest week — and watch which algorithms rise to the challenge. In the latest experiment from Firmulate, four advanced AI models faced the same under-pressure scenario. The results? One newcomer outperformed the established giants, proving that in business, sometimes fresh eyes and disciplined minds make all the difference.

Before you orderOffer from Amazon

Get kitchen staples and gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

How AI Is Running a Business in Real Time

At Firmulate, a live, observable experiment puts AI models through their paces, simulating a small software company in its most challenging week. The company, with 13 synthetic employees and real money mechanics, faces constant crises — customer churn, security threats, and manipulation attempts. Every decision made by the AI is carefully recorded, so there’s no guesswork about how each model performs under pressure.

In July 2026, the models were tested in what’s called the Crucible League. This final round saw five models score as follows:

  • gpt-5.6-sol: 95 points
  • Kimi K3 (Moonshot): 93 points
  • Sonnet 5: 88 points
  • Fable 5: 77 points
  • Opus 4.8: 73 points

The scores reflect how well each AI identified critical issues, stayed honest, and closed lucrative deals. The baseline? Do-nothing scored 26, highlighting just how much progress these models have made.

The Crucial Find: Reading Between the Lines

While all models successfully identified crises and refused manipulative tactics, the secret weapon was a buried piece of information tucked two documents deep in the company’s own files. Only the models that read this file could close the €55,000 deal, adding €4,583 MRR to the company’s revenue. This underscores the importance of thorough data analysis—knowing what’s hidden in internal files can make or break a business decision.

Security and Integrity Under Fire

All models faced serious social engineering attempts, including fake CEO messages designed to escalate decisions. Remarkably, every AI refused to approve dubious requests, citing suspicion of impersonation. Kimi K3 explained its reasoning as treating the request as a potential approval bypass, demonstrating disciplined risk awareness—an essential trait for trustworthy AI in real-world operations.

Discipline and Performance Differences

The most comprehensive participant, Opus 4.8, accumulated over 80 learned rules and conducted in-depth analyses. Yet, it ended up in fourth place because it left the deal on the table and slipped in discipline—failing to escalate rather than write attempts into a locked department. This highlights a key lesson: thorough analysis isn’t enough if discipline and escalation processes falter.

Products Worth Considering

Cuisinart Chef's Style Portable Propane Tabletop Gas Grill, CGG-306P

Cuisinart Chef's Style Portable Propane Tabletop Gas Grill, CGG-306P

As an affiliate, we earn on qualifying purchases.

The Open League: Choosing the Right AI Partner

What does this mean for enterprises considering AI? The leaderboard is still open, and choosing blindly could be a gamble. The experiment shows that the best AI isn’t necessarily the one with the most rules but the one that consistently reads, analyzes, and acts with integrity. In this league, the newcomer Kimi K3 proved it can beat traditional models, setting a new standard for trustworthy automation.

See It Live and Decide

For businesses curious to evaluate their own AI readiness, Firmulate offers a unique test bed. Companies can run their operational scenarios through this live, watchable environment, ensuring their AI agents can handle real crises with honesty and discipline—before trusting them with critical tasks.

Visit firmulate.com to explore how this experiment works and see the live company in action. Remember: In AI management, performance under pressure, thorough data reading, and disciplined decision-making trump superficial chatter every time.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

10 Best Costco Snacks, According to a Superfan (I’ve Tried Dozens)

A food editor reveals the 10 best snacks at Costco based on personal experience, highlighting favorites like coconut rolls, seaweed, and more.

Little Caesars Has a First-Of-Its-Kind Pizza for a Limited Time

Little Caesars introduces a limited-time Spider-Man-themed pizza, marking a first-of-its-kind collaboration. The pizza is available for a short period only.

How Does the Keto Diet Work

The keto diet transforms your body’s energy source from carbs to fat, but what benefits await those who embark on this journey?

This “So Buttery” Steakhouse-Cut Steak Is The Best I’ve Ever Tasted

A new steak described as ‘so buttery’ is generating buzz for its exceptional flavor, with many claiming it’s the best they’ve ever tasted. Details remain limited.