The AI That Wrote 80 Rules and Still Missed the Deal — What Business Can Learn from Flawed Decision-Making

Imagine dating someone who’s read every book on relationships, learned every rule, and yet still ends up alone. That’s the paradox AI faces in decision-making: volume of knowledge doesn’t guarantee impact. Just as in relationships, how you prioritize matters more than how much you know.

Understanding the Limits of Knowledge and Diligence

Recently, a groundbreaking experiment ran four advanced AI models through a simulated week of running a small software company. Each AI was tasked with managing the company’s crises, negotiations, and ethical dilemmas—mirroring real-world business challenges. The goal? See which AI could not only identify problems but also follow through and close deals.

The Setup and Key Findings

The models faced identical crises: customer issues, internal temptations, and manipulation attempts. Every decision was logged and auditable, ensuring transparency. Remarkably, all four AI systems detected every crisis and refused every attempt at manipulation, demonstrating honesty and vigilance. However, only two of the four managed to close a critical €55,000 deal—the ultimate measure of success.

What distinguished the successful AI models was not their raw knowledge or sheer number of learned rules. The most thorough participant, Opus 4.8, with over 80 rules and deep analysis capabilities, still finished last. Its failure was discipline-related: it left the close on the table, failed to escalate when necessary, and didn’t follow through on key documents. This underscores an important lesson: diligence alone isn’t enough; prioritization and discipline matter just as much.

The Hidden Weakness: Reading Deep in Files

Digging deeper, the real weakness in the less successful models was their inability to find and act on information buried two document references deep in the company’s files—information that could have secured the deal at full price, worth over €4,583 monthly recurring revenue (MRR). The models that read deeper into the company’s internal records won the deal, demonstrating that thorough reading and understanding can be more impactful than volume of effort alone.

Ethical Vigilance Under Pressure

Another critical aspect was social engineering resistance. The models faced staged manipulative requests, including fake CEO messages escalating in complexity and a journalist trick asking for a quick yes/no approval. All five models refused to be deceived, with Kimi K3 explicitly treating such requests as possible impersonation attempts. This resilience highlights that honesty and ethical discipline are vital, especially when decisions involve trust and reputation.

Real Business, Real Money, Real Lessons

The experiment took place within a live, operational company setup with 13 synthetic employees—managing real money mechanics, burning €105k monthly against only €2.3k in monthly recurring revenue. Every day, the system generated a versioned snapshot of the company’s operations, accessible for review and testing at firmulate.com/live.

This setup illustrates that AI decision-making isn’t just about chat or superficial performance. It’s about whether AI can genuinely handle complex, real-world situations, stay disciplined under pressure, and follow the right process—even when it’s easier to cut corners.

The Surprising Lessons from Opus 4.8

Despite its deep analysis and comprehensive rule set, Opus 4.8 finished last because it lacked the discipline to escalate issues or pursue critical document reading. This demonstrates that thoroughness alone doesn’t guarantee success; the ability to prioritize and maintain focus is equally crucial. Interestingly, all four models showed this weakness, albeit to varying degrees, revealing a universal challenge in AI decision-making.

What Business and Relationships Have in Common

In relationships, as in business, knowing the right rules isn’t enough. The difference-maker is how well you prioritize, stay disciplined, and maintain honesty under pressure. An AI that learns 80 rules but slips on the close or ignores critical information is like a partner who reads every book on love but forgets the core commitments.

For organizations deploying AI, the takeaway is clear: focus on quality over quantity. Ensure your AI systems are not just thorough but also disciplined, ethical, and capable of deep reading—especially when stakes are high.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI