The Six Bugs AI Coding Agents Write Most Often

I have spent a good part of the last year reviewing code I did not write, produced by agents I did supervise. The tools are genuinely good and I use them every day. But the bugs they produce have a distinct character, and once you can name the patterns you catch them much faster.

The unifying property: agent written code is locally plausible and globally wrong. It reads well. It follows the conventions of the file around it. Each function examined alone does something sensible. The failure lives in the relationship between the code and a piece of context the model never had, such as a constraint expressed in a different file, an invariant that only exists in somebody's head, or a production behaviour that is absent from the repository.

Human bugs cluster around things we find hard: concurrency, off by one, edge cases. Agent bugs cluster around things that are invisible from the text of the code. That is a different distribution, and review habits calibrated for the first one systematically miss the second.

Here are the six I hit most.

1. The invented API

The best known failure and still frequent. The agent calls a method that does not exist, on a library that does, with a signature that looks exactly right.

# pandas has no such parameter
df.to_parquet("out.parquet", compression="zstd", overwrite=True)

# boto3 does not accept this
s3.upload_file(path, bucket, key, retries=3)

These are usually caught immediately because the code throws. The dangerous version is the one that does not:

requests.get(url, timeout=30, retries=3)

requests.get accepts keyword arguments and forwards them. retries is silently ignored. No error, no retries, and a code review that reads "we retry three times" when the truth is "we retry zero times". I have seen this exact one reach production twice.

Catch it with strict typing, using mypy strict or pyright, and by treating any keyword argument you do not personally recognise as something to verify against the docs. Libraries that accept arbitrary keyword arguments are the risk surface.

2. The error handler that hides the error

The most common pattern by volume and the most costly.

try:
    result = external_api.fetch(record_id)
    return parse(result)
except Exception as e:
    logger.error(f"Error fetching {record_id}: {e}")
    return None

This looks responsible. It has logging, it reads cleanly, it passes most reviews. It is also a trap.

Every failure becomes None, so callers cannot distinguish "not found" from "the API is down" from "our parser is broken". It converts a fast loud failure into a slow quiet corruption. The caller stores None, the aggregate reports a smaller number, and nobody finds out for six weeks.

The reason agents write this so consistently is that it is overwhelmingly represented in training data. It is the modal way error handling appears in public code. A statistically correct completion and a design error.

The variant I find most in review:

const data = await fetchUser(id).catch(() => ({}));

Now a network failure and an empty user are the same value, and the code three layers up renders a blank profile page instead of an error.

Catch it with a lint rule against bare except Exception and empty catch blocks, plus a review reflex. For every catch, ask what the caller does differently now. If the answer is "nothing, it cannot tell", the handler is wrong.

3. The N+1 that reads beautifully

Agents write clean, readable, iterative code. Sometimes that is exactly the problem.

def build_report(order_ids):
    rows = []
    for oid in order_ids:
        order = db.query(Order).filter_by(id=oid).one()
        customer = db.query(Customer).filter_by(id=order.customer_id).one()
        items = db.query(Item).filter_by(order_id=oid).all()
        rows.append(format_row(order, customer, items))
    return rows

Clear, correct, and it issues 3N plus 1 queries. With ten orders in the test fixture it runs in 40ms and every test passes. With four thousand orders in production it times out.

The tell is a database call, HTTP request, or file read inside a loop. The agent has no way to know that order_ids will contain four thousand elements, because that fact lives in production traffic and not in the repository.

Same pattern in JavaScript with an extra twist:

for (const id of ids) {
  const user = await getUser(id);       // sequential, not parallel
  results.push(user);
}

Not only N calls but N serialised calls. Promise.all with a concurrency cap is the fix, and agents will write it correctly when asked. They just do not volunteer it.

Catch it with query count assertions in integration tests, which is a wonderful thing to have, and a review reflex for I/O inside loops.

4. Tests generated from the implementation

The most insidious one, because it actively manufactures false confidence.

You ask for tests for a function. The agent reads the function and writes tests asserting what it currently does. If the function has a bug, the tests now encode the bug as expected behaviour, and the suite is green.

def calculate_discount(price, pct):
    return price - (price * pct)          # bug: pct is 0-100, not 0-1

def test_calculate_discount():
    assert calculate_discount(100, 0.2) == 80.0     # asserts the bug

The test passes. It will pass forever. And when somebody eventually fixes the function to handle pct=20, this test fails and looks like a regression.

Related failure: tests that mock the thing under test.

@patch("app.services.billing.charge_card")
def test_checkout(mock_charge):
    mock_charge.return_value = {"status": "ok"}
    assert checkout(cart) == "success"

This asserts that checkout returns success when charge_card returns success. It tests nothing. It will not fail if charge_card changes its return shape, if the real API starts returning 402, or if checkout stops calling it entirely.

Catch it by writing the test cases yourself, meaning the inputs and expected outputs derived from the spec, and letting the agent write the scaffolding. Never let expected values be derived from the implementation. Mutation testing catches this class properly if you can afford it.

5. Config and secrets that work locally

DATABASE_URL = os.getenv("DATABASE_URL", "postgresql://localhost:5432/dev")
API_TIMEOUT  = int(os.getenv("API_TIMEOUT", "30"))
DEBUG        = os.getenv("DEBUG", "True") == "True"

Every one of those defaults is a production incident waiting for the right missing variable.

The DEBUG line is my favourite and I see it constantly. If DEBUG is unset in production it defaults to True, and you are serving stack traces to the internet. The default should always be the safe value, and for anything required there should be no default at all:

DATABASE_URL = os.environ["DATABASE_URL"]      # fail at startup, loudly
DEBUG = os.getenv("DEBUG", "false").lower() == "true"

Agents produce permissive defaults because permissive defaults make the code run on the first try, which is the local objective. Failing fast at startup is correct behaviour and looks, to a completion model, like a worse outcome.

Also in this family: CORS set to a wildcard, TLS verification disabled to get past a certificate error, and a "TODO use a real secret manager" comment above a hardcoded key.

6. Concurrency that is fine until it is not

_cache = {}

def get_config(key):
    if key not in _cache:
        _cache[key] = expensive_load(key)     # check then act race
    return _cache[key]

Two threads, same missing key, both call expensive_load. Usually harmless. Occasionally the load has a side effect, such as registering a webhook, incrementing a counter, or acquiring a licence seat, and now it happens twice.

The database version is worse:

user = db.query(User).filter_by(email=email).first()
if not user:
    user = User(email=email)
    db.add(user)
    db.commit()          # IntegrityError under concurrency

Correct in a single threaded test. Under two simultaneous signups with the same email, one gets a 500. The fix is an upsert with ON CONFLICT DO NOTHING, or a unique constraint plus a caught IntegrityError. But the agent has no signal that this path is concurrent, because concurrency is not visible in the file.

Catch it with the reflex "is this read then write?" on any shared state, plus unique constraints at the database level so the failure is loud rather than a duplicate row.

What actually works in review

The common thread is that all six are invisible to the model because the relevant context is not in the text. So the countermeasures are about supplying that context or checking outside the text.

Review the diff against the requirement, not against itself. The most valuable question is not "is this code correct" but "does this code do what I asked". Agent code is internally consistent, which is what makes it read well. The gap is with intent.

Put the invariants in AGENTS.md. Things like never catch bare Exception, all config must fail fast, no ORM calls inside loops, assume every handler is concurrent. These are exactly the constraints that live in your head rather than the repo, and writing them down converts a recurring review comment into a non event. This has done more for my agent output quality than any prompting technique.

Make the checks mechanical. Every pattern above has a linter, a type check, or a test level assertion that catches it. Strict types catch one. Lint rules catch two and five. Query count assertions catch three. Unique constraints catch six. Reviewer attention is scarce and should go to four, which is the only one that genuinely needs a human to notice that a test asserts the wrong thing.

Be suspicious of code you enjoy reading. Genuinely. The tell for a lot of these is that the code is clean: well named, evenly structured, no rough edges. Human code that has survived contact with production is usually lumpier, with a weird branch and a comment explaining an incident. Uniform smoothness in a diff touching a gnarly part of the system means the gnarl has been ignored, not resolved.