What I'm learning about building useful AI
Most AI product failures aren't model failures. They're product failures wearing a model-failure costume.
I've spent the last year shipping AI features, watching others ship AI features, and reviewing post-mortems. Almost every disappointment I've seen traces back to the same root cause — and it isn't the model.
What people think goes wrong
The common story: "The model wasn't smart enough. We need GPT-5." Or: "We need to fine-tune." Or: "We need a bigger context window."
Sometimes that's true. Usually it isn't.
What actually goes wrong
The product was unclear about what it was promising.
That's it. That's the whole essay, really.
When users are unsure what an AI feature is for — when the value proposition doesn't fit on a sticky note — they try a bunch of things, most of which the feature isn't designed for, and conclude that "the AI doesn't really work." The model performed fine on the queries it was built for; users just didn't know what those were.
A quick test
Take any AI feature you've built. Ask three random people: "What does this do? Give me one example of when you'd use it." If you get three different examples, you don't have a feature — you have a demo.
The best AI products I've used answer that question identically, every time. The model is in service of a clear, narrow promise the user already understood before they typed anything.
So what should you do?
- Cut the surface area. A focused feature that does one thing well beats a flexible one that does five things passably.
- Show, don't tell. Replace "Ask me anything" with three good example prompts. Then make those prompts work brilliantly.
- Constrain the user before constraining the model. A well-designed input box reduces ambiguity faster than any system-prompt engineering.
None of this is glamorous. It's product work, not model work. That's the secret.