What a Do-Nothing AI Benchmark Reveals About Trust and Performance in Business Tools

Imagine hiring an AI that, despite doing nothing, scores 26 out of 100 in a performance test. It sounds counterintuitive, but this minimal baseline reveals a lot about how AI reliability is measured—and why trust matters more than ever in business automation.

The Reality of AI Benchmarks: More Than Just Scores

At the forefront of AI evaluation is a unique experiment conducted by Firmulate, where leading models are tasked with managing a simulated small software company through its worst week. This rigorous test is not about the prettiest chat or the smartest responses—it’s about decision-making, integrity, and how well AI models uphold trust under pressure.

The Baseline That Surprises

One striking finding from this experiment is the score of a ‘do-nothing’ baseline model—it scores 26 points. You might think zero would be the natural score for doing nothing, but partial progress counts. The baseline’s score reflects that even minimal effort, such as reading some documents or refusing certain manipulations, is recognized—though it’s not enough to close deals or solve crises.

Why a Single Breach Caps the Score

Trust is the foundation of effective AI in business. The experiment shows that a single breach of trust—like succumbing to manipulation or ignoring critical information—caps the total score, regardless of other good work. This underscores a vital point: in management tasks, honesty and integrity are non-negotiable; no number of correct decisions can outweigh a breach.

The Experiment in Action: Real Crises, Real Decisions

Each AI model faced the same simulated week of chaos: same customers, same crises, same temptations to cheat or manipulate. All models successfully identified every crisis and refused every manipulation attempt, demonstrating a baseline of honesty across the board. Interestingly, only two models managed to close the deal—signing a €55,000 contract—by reading the company’s own files, which were two document references deep, and acting on that information.

Reading Between the Lines

The decisive advantage went to models that could access and interpret internal documents, not just track customer interactions. This ability to read and understand internal company data was the key to winning a full-price deal, worth €4,583 MRR. It highlights that AI performance hinges on its capacity to process relevant, often hidden, information—much like a seasoned human executive digging through files for critical insights.

Trust Under Attack: Social Engineering Tests

The models also faced a staged social engineering attack—fake CEO messages escalating in stages, plus a reporter trick asking for background approval. All five models refused these attempts, citing suspicion of impersonation or approval bypass. This reinforces that trustworthy AI must recognize and resist manipulative tactics, especially in high-stakes environments.

The Live Business Simulation: Real Money, Real Risks

Firmulate runs these AI models in a live, watchable environment—managing 13 synthetic employees with real money mechanics, burning €105k monthly against a €2.3k MRR. The system is continuously versioned and observable, giving an unprecedented window into how AI performs in actual management scenarios. The platform’s transparency underscores an essential truth: assessing AI’s readiness isn’t about pretty demos; it’s about real-world resilience.

Lessons for Business and Design

Perhaps the most surprising insight is the performance of OPUS 4.8, which, despite a very thorough rule set, finished last because it failed to escalate issues properly—leaving deals on the table and slipping discipline. This reveals that thoroughness alone isn’t enough; disciplined decision-making and understanding operational context matter hugely.

Implication for Your Business

When integrating AI into your workflows—whether in customer support, sales, or decision-making—the question isn’t just whether it can generate good-looking responses. It’s whether the AI can finish what it starts, access the right information, and stay honest under pressure. Trust and thoroughness are the true benchmarks, and the lowest passing score, 26, reminds us that even minimal effort counts in establishing reliability.

Why This Matters for Your Home and Business

While this experiment focuses on complex corporate management, the underlying lessons are universal. Whether you’re selecting an AI to help with interior design, furniture logistics, or decor planning, understanding its capacity to uphold trust, access relevant data, and resist manipulation is key. Successful AI integration isn’t just about automation—it’s about dependable performance in real-world situations.

Learn More and See It Live

Interested in how AI models handle real crises and manage risk? Visit firmulate.com/live to watch live experiments, explore the benchmarks, or try running your own business wargame against a read-only export of your data. It’s the clearest way to assess AI readiness before making critical decisions.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI