Agentic AI for development, from someone who built one

I built a coding assistant into Visual Studio. The interesting part was never the model — it was the loop around it, the context you feed it, and deciding what it is allowed to do without asking.

What "agentic" actually means

A chatbot answers. An agent acts, looks at the result, and decides what to do next. That is the whole distinction, and it is implemented by a loop that is almost disappointingly simple.

send conversation system, tools, history model replies text, or tool calls done — answer host runs them read, edit, test, shell end_turn tool_use append every result, then go round again
The model never touches your machine. It asks; the host decides whether to comply.

Two details in that loop matter more than they look.

One reply can contain several tool calls. When it does, run them and return all of the results in a single message back. Splitting them across several messages teaches the model, within that conversation, that parallel calls do not work — so it stops making them and everything gets slower.

A failed tool is still a result. Return the failure, marked as an error, rather than dropping it. An agent told "that path does not exist" corrects itself on the next turn. An agent told nothing repeats the same call, because from where it sits nothing happened.

Tools are the product

The model is a commodity; everyone has access to roughly the same ones. What differs between a useful assistant and a frustrating one is the tool surface: what it can see, what it can change, and how precisely it can express an intention.

The set that pulls its weight is small:

  • Search — find files by name and by content. This is how an agent orients itself in a codebase it has never seen.
  • Read — a file, or a range of lines. Ranges matter: reading a 4,000-line file to change one method is how you burn a context window.
  • Edit — a precise replacement of known text, not "write the whole file". Whole-file writes destroy anything the model did not think to reproduce.
  • Run — tests, build, linter. This is what turns a plausible answer into a verified one.
Fewer, sharper tools beat more tools Every tool definition sits in the context of every request, and a near-duplicate pair ("edit file" and "modify file") mostly produces wrong choices. One good edit tool with a clear description outperforms three overlapping ones.

Context is the real constraint

Context windows are large now, which has not made context management less important — it has made it less obvious when you are doing it badly. Everything in the window competes for attention, you pay for all of it on every turn, and a window full of noise produces worse answers than a smaller window full of signal.

In practice, a turn is assembled roughly like this:

tools rarely change system prompt rules, conventions history grows each turn this message always new cacheable prefix — identical bytes every turn volatile — changes, so it goes last order is fixed: tools, then system, then messages
Stable content first, volatile content last. That ordering is what makes caching possible at all.

The non-obvious part: the things you most want to inject — the current file, the time, a request id — are exactly the things that must go last, because anything placed before the history invalidates it.

Caching, and the silent invalidator

Prompt caching works on an exact prefix match. The provider hashes the request from the beginning; everything that matches a previous request can be served from cache, and the first byte that differs invalidates everything after it.

Which produces a failure mode worth knowing, because nothing errors:

// Looks harmless. Costs you the entire cache, every single turn.
system = $"You are a coding assistant. Current time: {DateTime.Now}";

The timestamp is in the prefix, so the prefix is different on every request, so nothing is ever a hit. The agent still works. It is just several times more expensive and slower to first token, and nothing in the response says so.

How to catch it The usage figures on every response report how many input tokens were read from cache. If that number is near zero from the second turn onwards, you have an invalidator — a timestamp, an unsorted dictionary serialised into the prompt, a tool list built in nondeterministic order, or a per-request id placed too early. Log it and alert on it.

The same reasoning explains a rule that surprises people: do not rewrite history. Editing or re-summarising earlier turns changes the prefix, so the cache dies and the cost of the conversation jumps. Append; do not revise.

When the conversation gets too long

Two different remedies, often confused:

  • Clearing removes old content outright — typically stale tool results, which are the bulkiest and least useful thing in a long session.
  • Compaction replaces old turns with a summary, keeping the thread of the conversation at a fraction of the tokens.

Clearing is cheaper and safer; reach for it first. Compaction keeps more meaning, at the cost of a summarisation pass and a rewritten prefix.

Permissions: the UX problem

An agent that can edit files and run commands is, in security terms, a program executing instructions derived from text. The hard question is not "can it do damage" but "how often do we interrupt the human to ask".

tool requested edit, run, delete policy what is pre-approved run it ask the human refuse the answer goes back as a tool result either way
A refusal is not an error. It is information the agent should receive and adapt to.

Modes that proved genuinely useful, rather than theoretically complete:

  • Plan first. The agent may read anything and write nothing until it has stated what it intends to do. This catches the "confidently solving the wrong problem" failure, which is the expensive one.
  • Approve edits. Reads and searches run freely; every write is shown as a diff and waits. This is the default I keep coming back to.
  • Approve commands. Edits flow, but anything touching a shell asks.
  • Autonomous. For a sandbox or a throwaway branch, where the cost of a bad action is a git reset.
The real risk is not the model going rogue It is that the agent reads a file, a web page or an issue containing text that reads like an instruction, and treats it as one. Content it fetches is data, not direction. The defence is the permission gate — not trusting the model to notice it is being manipulated.

Make it check its own work

The single biggest difference between an assistant that saves time and one that costs it is whether it verifies. A model producing a change that looks right is unreliable. A model producing a change and then running the test suite is doing engineering.

So: give it the ability to run tests, build, and type-check — and ask for the change to be verified rather than merely written. The loop above is what makes this possible: it sees the failure output and gets another turn.

The cheapest quality win A project file that states how to run the tests, how the code is laid out and what the conventions are. It is read once per session, it is cacheable, and it removes an entire category of confidently wrong output. Most of the "this tool is bad at our codebase" complaints I have seen were a missing page of context.

Where it helps, where it does not

Four questions decide whether an agent is the right shape for a task:

  • Is it multi-step and hard to fully specify up front? If you could write the exact steps, write the script instead — it is cheaper and deterministic.
  • Is the outcome worth the latency and cost? Agents are slower per task than a single call, because they take several turns.
  • Is the model actually good at this? Be honest about the task type.
  • Can a mistake be caught and undone? Version control and tests make errors cheap. Production data makes them expensive.

Where it genuinely earns its place, in my experience: exploring an unfamiliar codebase, mechanical refactors across many files, writing the tests nobody wants to write, turning a stack trace into a diagnosis, and first drafts of code that has a clear specification.

Where it does not: anything where the hard part is deciding what to build, anything load bearing that nobody will review, and anything whose correctness you cannot check. The agent is fast at producing plausible work. Deciding whether plausible is correct is still the job.

← All articles