Spots

What AI Context Limitations Teach About Software Development

Ask an AI coding assistant to fix a bug in one file, and there's a decent chance it patches the symptom at the call site rather than the logic several layers away that's actually causing it. That originating logic never made it into the context window for this particular prompt. Ask it to add error handling to a new module, and it may quietly invent a new convention rather than use the one the codebase already settled on two months ago. The file where that convention lives wasn't part of what it was shown this time. Ask it to add a helper function, and there's a real chance it writes a third, slightly different version of a date formatter that already exists twice elsewhere in the codebase. Not out of carelessness. It never saw the other two.

None of this is primarily a training problem

None of this is primarily a training problem. It's a visibility problem. The model is only ever reasoning over what's in front of it. Even when a codebase is technically small enough to fit inside a large context window, there's now good evidence that long-context models don't attend to everything in that window with equal reliability. Information buried in the middle gets used less faithfully than information at the start or end. So even "just give it the whole file" doesn't reliably produce the thing a human means by "the assistant understands the codebase." What it produces is closer to a series of locally coherent, globally unaware decisions. Each one internally sound. None of them checked against the others.

That failure has a name, and it's worth

That failure has a name, and it's worth sitting with before reaching for the obvious fix. The honest reveal here isn't that AI has a context problem software development didn't already have. It's that software development has had exactly this problem for decades, and the industry never diagnosed it as a context problem. The actor doing the pattern-matching was human, and humans don't announce their context window the way a token limit does. The same failure, running on people

Large COBOL systems became notorious for a reason

Large COBOL systems became notorious for a reason that has nothing to do with the language. Nobody could hold the whole system in mind anymore. Nobody could say with confidence what depended on what, and eventually nobody dared touch the code. Object orientation was supposed to fix this. Java, and later C#, arrived with the promise that encapsulation and class design would finally make large systems reasoned-about rather than merely operated on. But there weren't enough developers who understood object orientation deeply enough to do that reasoning. A generation was trained quickly instead: bootcamps teaching the syntax of controllers, services, and dependency injection without ever teaching what a responsibility is or where it belongs. The result was code that used classes syntactically while remaining procedural in structure underneath.

This is the anemic domain model, named as

This is the anemic domain model, named as such over twenty years ago, and still the default shape of a great deal of enterprise Java and C# written today.

That developer wasn't limited by a context window

That developer wasn't limited by a context window. The code was sitting right there, fully readable, no retrieval problem at all. What was missing wasn't access to information. It was the skill of holding a coherent model of the domain and checking new work against it. Handed a new requirement, that developer reaches for the nearest template: Controller here, Service there, Repository underneath. The same template gets applied every time, regardless of whether this particular responsibility actually belongs in that shape. It is functionally the same act an AI agent performs when it pattern-matches a new prompt onto a scaffold it has seen many times in training. Select the nearest known shape, apply it, move on. The causes are different. One agent literally cannot see the rest of the codebase. The other was simply never taught to look.

Both arrive at the identical output: locally plausible

Both arrive at the identical output: locally plausible code that was never checked against a standing model of the whole, because no standing model was ever being held in the first place.

This is the distinction actually worth naming, because

This is the distinction actually worth naming, because it's the one that survives contact with every objection. There is a real difference between applying a known pattern and designing. Pattern application is recognizing that a situation resembles something already seen and reaching for the matching shape. Design is discovering, through friction with something that won't simply agree with you, where a responsibility actually belongs. That friction might come from an ambiguous domain expert, or from code that gets uglier the further a wrong assumption is pushed. Design means revising the model when the friction shows the current one is wrong. A framework-trained developer applying Controller-Service-Repository to everything is doing the first. An AI agent completing a prompt by reaching for the nearest scaffold in its training is doing the first. Neither is doing the second.

The reason is the same in both cases

The reason is the same in both cases: neither is holding, or being made to hold, a single, standing, revisable model of the actual problem, checked against something outside itself that's allowed to disagree. Brooks named this before either of them existed

This is where Fred Brooks is useful, not

This is where Fred Brooks is useful, not as decoration but as the vocabulary that already explains the failure. In No Silver Bullet, Brooks split software difficulty into two kinds. Essential complexity is the irreducible difficulty of the problem itself: the actual business rules, the actual domain logic, the things that would still be hard with a perfect language and infinite time. Accidental complexity is everything layered on top while trying to manage it: the frameworks, the deployment topology, the infrastructure. Essential complexity is inherent to the problem. Accidental complexity is a byproduct of the tools chosen to solve it.

News

What AI Context Limitations Teach About Software Development

Ask an AI coding assistant to fix a bug in one file, and there's a decent chance it patches the symptom at the call site rather than the logic several layers away that's actually causing it.

@spots #dev
Source: Dev.to
See more like this