Humans learned to program with red squiggles, go-to-definition and type errors that appear before a function is even finished. Giving the same tools to coding agents sounds obvious.
Ian Barber tried it. The result is less tidy than “LSP equals better agent.”12
In his Python experiments across local and API models, simply exposing an LSP definition tool did not make models reach for it. They often kept doing what training had made familiar: text search followed by partial file reads.1
A cheaper tool the agent ignores
In the tested setup, a definition call used about 1.3 times fewer tokens than a ranged file read. That looks like an easy optimisation.1
Then the models frequently did both. They fetched the definition and read the file anyway. The saving vanished.
Even injecting a span that contained the relevant fix usually did not prevent the extra read. Barber eventually fine-tuned Qwen 3.6 on trajectories where file reads were replaced with definition calls to change the behaviour reliably.1
That is a useful warning for agent systems: exposing a tool does not mean the model has learned when to trust it.
The gate beats the squiggle
The most interesting experiment is about timing.
Barber gives an agent a draft change that passes visible tests but fails a held-out case. Left alone, the model accepts the bad revision 11 times out of 12. When type checking gates submission, it accepts the bad change only once in 12 runs.1
By contrast, injecting diagnostics live during generation, like IDE squiggles, was neutral in this setup. Explicitly telling the model to act on tool feedback could even hurt its judgement. Batched feedback at the end of a turn or after an edit worked better, with end-of-turn feedback winning on total token spend.1
That changes how a harness might be designed. A lot of effort can go into making tools visible to the model when some checks may belong outside its decision loop: submit the work, run the gate, and only interrupt the agent when something actually fails.
Do not turn one experiment into a religion
Barber is explicit about the limits. Many tasks were synthetic and easy. The diagnostic-timing tests used a 7B model. The codebases fit in context, and larger private repositories may behave differently.1
He also points to an important counterexample: work where file exploration spends far more tokens than a structured graph tool but gets better answers. Optimising the cost of one operation can make the complete task worse.1
That may be the most transferable lesson.
An agent does not use software like a miniature human developer. Giving it our IDE, buttons and habits guarantees very little. Measure the whole trajectory: which tool it chooses, what it reads again, what the gate catches and whether the final work is actually better.
Sometimes the best tool for an agent is the one it never has to decide to call.