Build the data layer before you build the AI

The decision

Everyone wants to bolt a copilot onto their admin panel. I did too. But before I let it touch real money and real safety calls, I built the ground truth underneath it first: the instrumentation, the review loop, the data. The AI shipped. It just shipped second.

The stakes

The pressure to add AI everywhere is real, and most of it is noise. The dangerous version is AI that acts, adjusting fees, flagging practitioners, deciding what's safe, with no ground truth underneath it. A confident wrong decision on money or safety is the one you can't take back, which is exactly why I refused to automate anything until the data could stand behind it.

The argument

AI that acts is only as trustworthy as the data and the review loop beneath it. So I fixed the order and held to it: instrument first, build the human review loop, then automate. Run it in that sequence and the AI inherits ground truth; skip a step and it inherits your blind spots.

The proof

On wellness marketplace I shipped four-stage verification, a full analytics stack, and a tiered commission engine first. Only then did I scope Claude into the super-admin, split cleanly into "recommends, human approves" and "safe to automate." By the time the AI made its first call, it was reading from a system that already knew what good looked like.

The nuance

Don't read this as "never use AI early." You can, as decision-support a human signs off on, from day one, and I did. The line was never AI vs. no-AI; it's automated vs. assisted on anything touching money or safety. I let AI advise from the start, and only let it act on its own once the data had earned that trust.

The takeaway

So here is the rule I'd hand you: instrument before you automate, draw the human-in-the-loop line explicitly, and let AI run unattended only where a wrong call is cheap and reversible. Build the data layer first, and the AI you ship on top of it is the boring, trustworthy kind, which is the only kind worth shipping.