Proof-of-Use Beats Proof-of-Vibe

Okay, I wanted to put my two cents on this post from Simon Willison.

The line to keep verbatim:

You want solutions that are proven to work before you take a risk on them.

Simon Willison, 2026-05-06^[1]

Where I Agree

Basically, I'm kind of super agreeing with it. What I wanted to add is some tokenized or ledgerized way of proving how software actually runs once it is in use.

When you're running software, there should be a record that it ran smoothly, or that it didn't. There should maybe be a record of the bugs, and a record of fixing those bugs.

So for me it is a lot about operations: how your software works within operations, within different people, and within different teams.

The Same Problem Existed Before AI

This also goes in directions that were already the same for pre-AI, pre-agentic engineering software.

I saw it many times: management decided to choose some software without proper discussion with engineering. Sometimes that discussion was not even possible, because they had to choose something and did not have resources for proper evaluation.

Many times it ended up not just suboptimal, I would say, but painful because of how it works.

Sales Upward, Execution Downward

The sales team of the software worked hard. They sold it to higher-level management. Then it went down to the execution team.

That often ended with painful migrations and painful operation.

Many times I considered it good if it was not worse than what we had before. And many times it was worse.

The Evaluation Gap

I don't know what the outcome is here, I guess.

But every time I saw new software being decided without proper evaluation, I pointed out that exact moment: we do not know what we are getting here.

We may want it to be perfect. We may want it to be good enough. But without proper evaluation, we do not have a way to know.

And many times you have these annual contracts where you have to sign up for something without actually using it beforehand.

Why I Prefer Pay-As-You-Go

That is why I always choose, for me, pay-as-you-go or pay-as-you-grow contracts as the best option.

On the other side, those are hated by sales teams.

No Big Moral Claim

I'm not sure if there is any morality here. It is just an observation.

I wanted to point out that probably it was the same way before, just on a bit different level.

The way software companies sell their software often has similar issues to what Simon pointed out: when you vibe-coded something and did not have proof of usage by anyone, or proof of operation for six months, as he mentions with other companies.