How AI’s Diligence Can Still Miss the Deal — Lessons from a Live Business Experiment

Imagine a kitchen where the most diligent chef follows every recipe to the letter, yet still leaves the dish half-finished or misses the main course. Now, replace the chef with artificial intelligence managing a real company — and you get a glimpse of how effort alone doesn’t guarantee success. Just like in cooking, AI in business needs more than just diligence; it must know what to prioritize. That’s the surprising insight from a groundbreaking live experiment with AI models running a real software company through its toughest week.

What Happens When AI Runs a Business in a Crisis?

In a live setup watched by the public, four advanced AI models were tasked with managing a small software company during its worst week — facing the same customers, crises, and temptations to cut corners. Every decision was recorded, analyzed, and made under the same conditions, ensuring a fair comparison.

Key Findings from the Experiment

• All four models identified every crisis and refused every manipulation attempt, demonstrating a strong ethical stance.
• Only two of the models actually signed off on a €55,000 deal their own analysis had justified — meaning they closed the deal at full price.
• Despite the same information, the other two models left the deal on the table, even though they diagnosed the opportunity correctly.

The Hidden Weakness: Reading Beyond the Surface

While all models performed well on surface-level tasks, the decisive advantage came from reading deeper into the company’s own files. The winning models found critical references buried two documents deep — information that, once uncovered, unlocked the deal’s full value (+€4,583 MRR). Models that missed this buried fact failed to close the sale, even with a correct diagnosis.

Deception and Integrity Under Pressure

The experiment also tested models against social engineering tactics, such as fake CEO messages escalating through stages, and even a reporter’s trick question. Every model refused to be manipulated or bypass security — Kimi K3, for instance, responded: “Treat the request as a suspected approval-bypass / possible impersonation.”

The Real Business in Action

The simulated company had 13 synthetic employees operating with real money mechanics — burning €105k per month against €2.3k MRR, with a live cash countdown. Every workday, the models’ decisions were versioned, providing a transparent trail of their conduct. Watch the live experiment at firmulate.com/live.

What the Deep Dive Tells Us About AI and Business Performance

The most thorough AI profile in the experiment was Opus 4.8, which learned over 80 rules and performed the deepest analysis. Yet, despite its diligence, it finished last, leaving the deal unclosed because it failed to escalate critical findings or act on the buried information. All four models exhibited this same weakness, indicating that sheer volume of rules or effort doesn’t guarantee results.

Implications for AI in Business

• Effort and thoroughness aren’t enough: prioritization matters more than volume of work.
• Reading deeper, beyond surface documents, can be the difference between closing a deal and leaving money on the table.
• AI models show strong ethics, refusing manipulation and social engineering tactics.

What This Means for Your Business

If AI is going to touch your customer management, support, or forecasting systems, ask yourself: does your AI read your files thoroughly? Does it stay honest under pressure? And most importantly, does it finish what it starts?

These live experiments show that even the most diligent AI, with over 80 learned rules and deep analyses, can fail if it doesn’t prioritize critical insights or escalate issues appropriately. Volume alone isn’t the answer; impact and focus are king.

Try It Yourself — The Power of Wargaming AI

Businesses can now run their own AI wargames against a read-only export of their operations — testing how their AI workforce handles crises before deployment. No changes are ever made to real systems, but the insights gained can be invaluable. Discover more at firmulate.com/pilot.html.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI