The Criterion Is Not Mine

I built a system that matches names against a catalog. Two rows looked nearly identical to a string comparison. One bottle costs two hundred euros. The other, ten thousand.

The code did exactly what it was built to do. That's the point.

The error wasn't in the implementation. It was in the criterion. There was an entire dimension to the subject matter, a classification level, that didn't exist anywhere in the system. No field. No check. Not because anyone had forgotten it, but because none of us at the keyboard knew it was a dimension.

We didn't find it by reading our own code. We couldn't have. The code was entirely consistent with what we believed the task to be.

Two Kinds of Knowledge

There's a picture of AI work that says domain knowledge is the multiplier. It's true. It's just half the story.

My domain is how to build something. The data model. The matching. The confidence bands and the review queue. Making the threshold adjustable without anyone rewriting code. Thirty years count for something there, and no subject-matter expert in the world can do that part for me.

Their domain is what correct means. Whether two names are the same thing. Whether eighty percent is good enough. Whether the system should resolve to manufacturer, product, or individual SKU. There, thirty years of building systems is worth exactly nothing.

Neither of us can do the other's part. The interesting thing happens at the seam between them.

What I Actually Own

I own that the program is correct given the criterion. They own that the criterion is correct given reality.

That sounds like a division that removes responsibility from me. It doesn't. It makes my responsibility precise.

I can't determine whether two wines are the same wine. I can own that I asked. That the answer ended up in a file rather than a conversation. That the system doesn't pretend to have a certainty it doesn't have.

The Question Must Be Asked Before It Can Be Answered

Here's the catch. And it's worse than it looks.

Letting the subject-matter expert own the criterion only works if I know which criteria I need to go and ask about. In this case, the expert was available the entire time. We never asked. We didn't know there was anything to ask about.

So there's a third kind of knowledge sitting between the other two, and it's mine. Enough literacy in someone else's field to know where my own blind spots are. Not expertise. I don't need to be able to determine where an individual bottle belongs. I need to know that a hierarchy exists, so I can ask the question.

That's the part no one talks about. Everyone says you should talk to the client. Few say you have to understand their field well enough to know what you don't understand.

What Follows From That

A system where the criterion is hardcoded forces me to own a judgment I'm not equipped to make.

That's not a philosophical problem. It's a design requirement. If the threshold lives in the code, I'm the one who set it, regardless of who said the number once. If it lives somewhere the person who understands the subject matter can reach it, then the right person owns the right thing.

I can't determine whether two wines are the same wine. I can build so that the person who can gets to decide, and so that the answer can be changed when it turns out to be wrong. That's my part of the work. It doesn't become less by being bounded.