How to review a pull request written by an AI agent

A pull request written by a coding agent usually looks finished. The style is consistent, there are tests, the description is tidy and confident. For years, that kind of polish was a decent sign that someone had been careful. With agents, it tells you almost nothing. The polish is free. The care is what you still have to check.

This is how we review agent pull requests: what to read first, the mistakes that come up again and again, and a short checklist at the end.

Start with the task, not the diff

Before you open a single file, put the original request next to the pull request description. What was asked? What does the description say was done? How does it say the change was verified?

Agent-written descriptions are summaries of intent, and they can drift from the code. “Added validation for empty input” is a claim, not a fact. Pick two or three claims from the description and check them in the diff. If one doesn't hold, read everything else more slowly.

1. Check the scope first

Agents are helpful in a way that makes diffs grow. A task to fix one endpoint comes back with a renamed helper, a reformatted file and an “improved” error message somewhere else. Each change may be fine. Together they make the pull request harder to review and harder to revert.

Look at the list of files before the content. Anything that has nothing to do with the task should go to its own pull request, or be dropped. Saying no to scope creep early is the cheapest review comment you will ever write.

2. Read the tests before the code

Tests are where agents cut corners most quietly, because a green check is what they are steering towards. Read the test changes first and look for:

• Tests that test the mock. The unit under test is mocked, so the test only proves the mock returns what it was told to return.
• Weakened assertions. An exact value replaced by toBeTruthy(), or an expected error replaced by “does not throw”.
• Skipped or deleted tests. A .skip, an xit, a @pytest.mark.skip, or a test file that simply got shorter.
• Snapshots updated wholesale. A snapshot diff of 400 lines usually means nobody read it.

If the tests are honest, the rest of the review gets much easier. If they aren't, nothing else in the pull request has really been checked.

3. Look for what the agent couldn't know

An agent knows your code. It doesn't know your team's history: the decision from last quarter, the incident from March, the convention nobody wrote down. This is where human review is irreplaceable, and where most real bugs in agent pull requests hide. Typical examples:

• A new /api route that skips the auth middleware every other route uses. It works, the tests pass, and anyone can call it.
• An npm install that adds a package-lock.json to a repo that uses pnpm.
• A change to a database migration that has already run in production, instead of a new migration.
• A direct call to a third-party API that your team agreed to always wrap in a client with retries.

None of these look wrong in isolation. You only catch them if you know the rule, so this part of the review is worth your full attention.

4. Find error handling that hides failures

Agents like code that doesn't crash. That often means code that fails silently. Watch for empty catch blocks, “log and continue” where the caller needed to know, retries with no limit, and defaults that mask missing configuration:

// Works on a laptop, quietly wrong in production
const apiUrl = process.env.BILLING_API_URL || "http://localhost:4000";

A crash at startup is better than a service that runs against the wrong backend for a week.

5. Check new dependencies and unfamiliar APIs

Models sometimes call functions that don't exist in your version of a library, or options that were removed two major versions ago. The type checker catches some of this, not all of it. When a call looks unfamiliar, check it against the version in your lockfile.

New dependencies deserve extra care. Check the package is the one you think it is, that it is maintained, and that its license is acceptable. Attackers have started registering package names that models tend to invent, so a plausible name is not proof of a real, trustworthy package.

6. Ask whether this already exists

Agents search less than you would expect. It is common to find a new date formatter, retry helper or validation function that duplicates one already in the codebase. Duplicates drift apart, and the next bug gets fixed in only one of them. “Is there already a helper for this?” is a good question to ask on every agent pull request.

7. Run it

“I tested it” from an agent means it ran some commands in its own environment. For anything user-facing, check out the branch and use the feature for two minutes. It is the fastest way to catch the bugs that unit tests were never going to see: the empty state, the slow network, the button that does nothing on mobile.

Keep agent pull requests small

Review quality drops fast as diffs grow, and agents can produce large diffs in minutes. Ask for one change per pull request, and split bigger tasks before the agent starts, not after. A reviewer who can hold the whole change in their head catches far more than one who skims.

Make every review comment count once

The frustrating part of reviewing agent code is repetition. You flag the missing auth check on Monday, and a different session makes the same mistake on Thursday, because nothing carried the lesson forward. When you write a review comment about a convention or a past incident, also write it down where the next agent will read it: a rule file, a nested AGENTS.md, or a shared memory. Otherwise you are reviewing the same mistake forever.

Carrying those lessons forward is what we work on at Memmo. Our reviewer cites the team memory behind each comment, and each approved lesson reaches the next agent before it writes code. We wrote about it in Code review that knows your codebase.

The checklist

1. Does the description match the task, and do its claims hold in the diff?
2. Is every changed file needed for this task?
3. Do the tests assert real behavior? Was any test skipped, deleted or weakened?
4. Does the change follow the team rules the agent couldn't know (auth, migrations, package manager, wrappers)?
5. Can any failure pass silently?
6. Are new dependencies real, maintained and needed? Do the API calls exist in our versions?
7. Does this duplicate something we already have?
8. Did someone actually run it?