One of the stranger things about building AI infrastructure is how quickly a good decision can become an old one.
We make these decisions before we have all the information. We have to. You evaluate a model against the use case you have today. The quality is good. The cost works. Security is comfortable with it. Procurement gets through the contract. Teams start building against it.
Then the model changes. Or the pricing does. Or another provider gets better at the thing you care about. Or usage looks nothing like the assumptions you made when twenty people were testing it.
None of that means the original decision was bad.
It means I don't want the architecture pretending the decision is permanent.
What are we attaching to it?
Where does provider-specific behavior live? What state or data lives with the provider? What workflows depend on it? What would we have to change or move if this stopped being the right answer?
Sometimes the answer is that we're going to commit pretty deeply. Fine. There are good reasons to do that.
I just want us to know we're doing it.
I want to know who's using it
When AI starts spreading through an engineering organization, aggregate usage doesn't tell me much.
A monthly bill tells me what we spent. It doesn't tell me which team spent it, what they were doing, whether the usage was expected, or whether anyone got enough value from it to justify the cost.
I want that information before usage gets large.
Requests need enough identity on them to say which team sent them and what it cost. Cost needs to be visible at a level where someone can do something about it. And I need to be able to look at usage and understand what is happening, not just get one enormous number at the end of the month.
I want to know whether it's working
Cost attribution tells me where the money went. It still doesn't tell me whether we got anything useful for it.
"AI usage went up" doesn't tell me whether anything got better. One engineering team might be using it heavily and removing hours of work they used to do by hand. Another might have an agent chewing through tokens every night without changing anything anyone cares about.
Those can look identical on a provider invoice.
They are not the same investment.
If we cannot see what is being used, where, and at what cost, we cannot make a particularly informed decision when somebody proposes changing it.
The provider should not own the architecture
I don't like downstream systems talking directly to model providers unless there is a reason for it.
The first integration is easy. A team gets credentials, calls an API, gets a response, and ships something.
Then another team does it.
Eventually authentication, routing, logging, cost attribution, rate controls, and provider-specific behavior are scattered through applications that were supposed to be solving completely different problems. Now changing the model means changing the applications.
I would rather put that boundary in the platform.
Where that boundary makes sense, I want the application asking for a capability and the platform deciding how to provide it.
That doesn't mean building a universal AI abstraction before anyone knows what they need. I've seen that version too. You end up designing for every provider, every model, and every theoretical use case before the second team has shipped anything.
The boundary only needs to contain the decisions we actually expect to change: identity, routing, cost attribution, common policy, or which provider handles a capability.
Models aren't interchangeable. Changing one can change quality, latency, cost, and failure behavior. Applications also start depending on those specifics over time, whether anyone meant them to or not.
The point of the boundary is to keep those decisions somewhere we can see them and test them instead of discovering the assumptions later across a pile of applications.
Maybe one provider is the right answer today. Six months from now, another one may be better for a particular workload. Pricing may change. A model may improve enough to change the economics. Availability or limits may change what you can reliably run through it.
Being able to route somewhere else is only useful if I can tell whether somewhere else is actually better.
That means keeping enough representative work to test a change against. Not a benchmark somebody else published. The things our systems actually do.
If we change a model, provider, or routing decision, I want to know what happened to quality, cost, latency, and failure behavior before the users are the ones who tell us.
A proof of concept is allowed to fail
This is harder than it should be.
A proof of concept does not have to become a platform because people spent three months on it.
A proof of concept should have an exit condition before it has an audience. Decide what the experiment needs to prove, what you're going to measure, and what result means you stop before everybody gets attached to proving that it worked.
Give it enough real usage to learn something.
Then look at what happened.
Maybe it worked. Maybe the model was good but the economics weren't. Maybe people used it differently than expected. Maybe the use case itself wasn't valuable enough. Maybe another approach solved the problem with less machinery.
If it isn't doing what you needed it to do, stop.
I've killed a proof of concept the week before it was supposed to go on a roadmap. The pilot numbers said the model was good and the economics weren't, and no amount of usage was going to change the second part. It was uncomfortable for about a day. Six months later nobody remembered it, which is the whole point.
That isn't wasted work. You paid to answer a question and got an answer.
Once enough people have seen the demo, put it on a roadmap, or started talking about when it launches, stopping starts to feel like failure. That is exactly when the technical evidence needs to matter more than the momentum around it.
What gets expensive is continuing because the organization has already committed to the story.
I've watched technology decisions become harder to reverse the longer people spend defending the original decision. AI makes that worse because the thing you evaluated may have changed while everyone was still arguing about it.
The cost of a decision isn't just what it costs to make
It's also what it costs to undo.
I think about that when I'm deciding where an integration belongs, what gets standardized, and how much provider-specific behavior we're willing to let into an application.
Sometimes the right answer is still to commit deeply to one thing. If a provider gives us something valuable enough to justify that dependency, use it.
I just want that tradeoff made on purpose.
We will know more six months after a decision than we knew when we made it. I want to be able to see what it's costing, who's using it, whether it's doing what we expected, and what it would take to change.
Then I need to be able to change it.
