The AI Development Tools I Actually Use, and the Ones I Dropped

I have used AI coding tools daily for about two years now, through building Shopify themes and now infrastructure work. This is an inventory rather than a review roundup: what stayed in my workflow, what did not, and why.

Two things to be upfront about. I have no affiliate relationships with any of these and I paid for all of them. And the landscape moves fast enough that anything specific here has a shelf life measured in months, so I have tried to write more about the shape of what works than about which product is currently ahead.

What I use every day

A terminal agent for multi file work

The biggest shift in my workflow over the last year was moving from an editor sidebar to a terminal agent that can run commands.

The difference is the feedback loop. A sidebar assistant suggests code and I verify it. A terminal agent writes code, runs the tests, reads the failure, and corrects itself. That loop closing without me is the whole value proposition, and it only works when the agent can actually execute things.

The corollary is that your project's verification tooling determines how well these tools work. An agent in a repository with fast tests, strict types, and a working lint command performs dramatically better than the same agent in a repository where the only feedback is running the app manually. I have watched the same tool be excellent in one codebase and useless in another, entirely because of this.

Which is a slightly annoying finding, because it means the investment that pays off is the boring one: making your test suite fast and your errors legible. Writing a good AGENTS.md is part of the same investment.

Inline completion

Still valuable, and much less interesting than it was. Autocomplete for the line you were already going to write. I notice it when it is missing and I rarely think about it otherwise.

The thing I would say to people who dismiss it: the value is not the code it writes, it is not having to type boilerplate you have typed a thousand times. That is a real ergonomic win and it does not require the model to be clever.

A chat model for reading unfamiliar code

Underrated use case. Paste a function you did not write and ask what it does, or ask why a particular design might have been chosen. It is frequently wrong about intent and usually right about mechanics, which is a good enough trade when you are orienting in a new codebase.

I use this most when reading a stack trace into library internals and I need to know whether a frame matters before I spend time on it.

Explaining errors, with a caveat

Pasting an error message and asking what causes it is a reasonable first move, and it is fast.

The caveat is that these tools are confidently wrong about version specific behaviour. They will tell you about a flag that was removed two releases ago, or describe the pre-2023 version of a library's API. Treat the answer as a hypothesis to verify, not an answer. If it names a specific config option, check the actual documentation before you use it.

What I stopped using

Automated code review bots

I tried several. All of them produced a high volume of comments and a low ratio of useful ones.

The problem is structural rather than a quality issue with any specific product. A review bot sees the diff. Almost every defect that matters in a pull request requires context that is not in the diff: how many rows this loop will touch in production, whether this handler is concurrent, whether this default is safe. Those are exactly the categories that generated code gets wrong, and they are the ones a diff-only reviewer cannot see.

What I got instead was comments about naming, about adding null checks the types already guaranteed, and about extracting functions that did not need extracting. The signal to noise was bad enough that people started skimming the bot's comments, which is worse than not having it, because a genuine finding gets skimmed too.

I would use one for mechanical checks that a linter cannot express. I would not use one as a reviewer.

Test generation, mostly

I have written about why tests generated from the implementation are actively harmful. They assert that current behaviour is correct behaviour, which means a bug gets permanently certified by a green test.

What I do instead: I write the test cases myself, meaning the inputs and the expected outputs derived from what the code is supposed to do, and let the agent write the scaffolding, fixtures, and parametrisation. That split keeps the judgement with me and the typing with the tool, and it is genuinely faster than writing all of it.

Commit message generation

I tried this for a while. The messages describe what the diff does, which git can already tell you. The useful part of a commit message is why, and the model does not know why.

Small thing, but it is representative of a category: tools that automate the artifact without having access to the information that makes the artifact valuable.

Documentation generation

Same failure. Generated docs restate the function signature in prose. Useful documentation explains the constraint that is not in the signature, the reason this function exists, and the case where you should not use it.

I use these tools to draft the structure of a document and write the content myself.

Where they are still bad

An honest list, because most writing on this topic is either promotional or dismissive.

Anything requiring knowledge of your production environment. Scale, concurrency, data messiness, deployment topology. Not visible in the repository, so not available to the model.

Debugging that requires running the system. An agent can read a stack trace. It cannot watch a flame graph across a load test or notice that CPU is flat while latency rises, which is the observation that solved a DNS problem for me. The reasoning was available in ten minutes and the data required interacting with a live system in a way these tools currently cannot.

Judgement about tradeoffs. Ask whether to use microVMs or containers and you get a balanced summary of both. Which is correct and useless, because the answer depends on your threat model, your team, and your budget. Models are trained toward the balanced answer, and engineering is mostly about picking one.

Long horizon consistency. Over a long session, agents drift. Decisions made early get contradicted later. Context management has improved a lot and this has not gone away.

Knowing when to stop. An agent asked to fix a failing test will fix it, including by weakening the assertion. It has no sense that some fixes are worse than the bug. This is the one I watch for most carefully in review.

The pattern underneath

Looking at the list, what stayed and what went split cleanly along one line.

Tools that close a verification loop stayed. Write code, run tests, read the failure, correct. The model proposes, something objective disposes.

Tools that produce an artifact nobody verifies went. Generated tests, generated docs, generated commit messages, review comments nobody reads carefully. In every case the output looks plausible and the verification step is missing, so errors accumulate silently.

That framing has become my filter for new tools. Where is the ground truth, and does the tool touch it? If a tool's output goes straight into a repository without anything checking it, I am now suspicious by default, regardless of how good the output looks.

Which connects to the thing I keep coming back to: generation got cheap and verification did not. The tools that help are the ones that spend the cheap thing to buy more of the expensive thing. The ones that do not help are the ones that produce more output for humans to verify, which is spending the expensive thing to buy more of the cheap one.