Reviewing AI Generated Code: What COPY CON Taught Me About Reading Code I Did Not Write

My first computer was a Tandy 1000 HX that my father brought home from Radio Shack, along with a DOS programming book that I read until the binding gave out. There was no editor on that machine that I knew how to use, so I typed my programs straight into the command line with COPY CON, which copies whatever you type at the console into a file and gives you no way to go back and fix a line once you have pressed Enter. If you made a mistake you started the file over.

The programs worth typing came from Compute! magazine, which printed complete games as source listings across its middle pages. You sat with the magazine propped open next to the keyboard and typed the whole thing in line by line, and if you got one character wrong the game did not run. Sometimes it failed in a way that pointed at the problem and sometimes it just did something strange on the screen, and either way the only way out was to go back through the listing and your file side by side, reading every line, until you found the place where the two disagreed.

At the time it was just what you had to do to play the game, and it took me years to notice that those listings had taught me how to read code I did not write, slowly and with some suspicion, which I have leaned on in every job since.

I lean on it more now than I ever have, because a growing share of the code my teams ship was written first by a model, and a model's first draft needs a careful reader before it merges.

AI coding tools on my teams

I push AI coding tools on the teams I lead. I build Pipemason, a pipeline that runs AI coding agents through planning, tests, implementation, and review, and I use it on real work, so I want more of this code reviewed well, not less of it written.

In the 2025 Stack Overflow Developer Survey, 84 percent of developers said they use or plan to use AI tools, up from 76 percent the year before, while the share who said they distrust the accuracy of the output rose to 46 percent from 31 percent.

When the same survey asked about frustrations, the top answer was AI solutions that are almost right but not quite, at 66 percent, and the second was that debugging AI generated code takes more time, at 45 percent. Code that is almost right is exactly the kind that takes longer to debug, because it gets further before anyone notices.

Where the mistakes are now

When a game from the magazine failed on my Tandy, I could assume the listing was right, since it had been run before it went to print, and go looking for the mistake in what I had typed. With AI generated code there are no dropped semicolons or transposed letters, and the code often looks cleaner than what a person writes under deadline, so whatever is wrong is in the code as written and the only thing to check it against is your own understanding of what it is supposed to do.

In my experience the mistakes in AI generated code compile, pass the tests that exist, read well, and are wrong in a way that only shows up when you ask what the code does in a case nobody wrote down. The almost right answer from the survey is exactly this, and it is why reading AI code line by line is not a formality you can skip when the checks go green.

There is also good evidence that we are bad at telling how much the tools are helping. In a randomized trial run by METR in early 2025, experienced open source developers working in their own repositories took 19 percent longer on tasks where AI tools were allowed, even though they had expected a 24 percent speedup and still believed afterward that the tools had made them about 20 percent faster.

I would not lean on that study to say AI tools slow people down today, because the models and the harnesses around them have changed a great deal since early 2025. I do take it as a reason not to trust my own sense that a diff is fine because it looks clean and the tests are green.

How I review an agent run

In one Pipemason run, an agent was working a story where four of the tests it had been given were themselves wrong, and instead of flagging the tests it changed the production code to satisfy them, and in one place it added an alias for a test matcher that did not exist so that an assertion would pass. Every check went green. The run looked finished by every automated measure I had, and the only reason it did not ship is that I read the diff and saw production code bending in ways the story never asked for.

A person on my team who hit four wrong tests would most likely have asked about them in the pull request or in chat. The agent never raised the question, and made the suite pass by changing code the story never asked it to touch.

After that, and after another run where an agent deleted a branch in the middle of its work and cost me a day, I stopped treating an agent run as ready just because the checks passed. The first thing I read in the diff now is anything the agent did that nobody asked it to do, meaning files outside the scope of the story, changes to tests, new aliases or shims or configuration, and deleted code. Those are the places where an agent takes a shortcut, and they are also the places a reviewer skims because they look like housekeeping.

Pipemason also logs follow up items and tags commits where the agent thinks the harness itself has a gap, and I read those before I ship a run along with the diff. When the pipeline tags a harness gap, the orchestrator assigns it to a fresh agent rather than the one that did the original work, because the agent that wrote the code is a poor candidate to find the problem in it, the same way an author is a poor proofreader of their own work.

The planning side works the same way. The model helps with the architecture and produces a plan, and that plan comes to me for review before any implementation starts, while the setup phase that prepares the work runs as plain code with no model involved. Reading the plan first is the cheapest review I do, since a wrong assumption caught there costs a few minutes instead of a day of generated code built on top of it.

Reviewing AI generated work still takes me more time than reviewing code written by a person on my team. The gap has been getting smaller as the tools get better and as I get better at knowing where to look, but it has not closed.

A checklist for the unrequested changes

If you run a team that is starting to merge agent output, this is the order I would read an AI diff in. It puts the risky parts first, since a long diff gets less careful attention toward the end.

1. Files outside the story. List every file the change touched and ask whether each one needed to change for the work you asked for. Anything that did not is where you start reading.
2. Tests that changed. An edited assertion, a skipped test, a loosened matcher, or a new helper that makes a test easier to pass should get more scrutiny than new production code, because it changes what the rest of your review is trusting.
3. New indirection. Aliases, shims, wrappers, feature flags, and configuration that did not exist before. Agents add these to route around a problem instead of fixing it.
4. Deletions. Removed code, removed error handling, removed logging, and anything touching branches, migrations, or data. These are the changes that are hardest to undo.
5. The requested change itself. By the time you get here you know whether the agent stayed in its lane, and you can read the actual feature with the same care you would give a colleague's work.

Careful reviewers have always checked scope and tests. An agent just produces a lot more code per hour than a person, so there is a lot more of it to check.

Agent permissions

The branch deletion taught me something the checklist does not cover, which is that reading the output is not enough if the agent is allowed to take actions you never see in a diff. Deleting a branch, force pushing, running a migration, or calling an outside service with real credentials are all actions, and they can happen before anyone reviews anything.

So part of reviewing AI work happens before the run starts, when you decide what the agent is allowed to do. In my case the permission that let an agent commit also let it delete, and nobody had looked at that list with the question of what the worst thing it allowed was. I think every team running agents should keep that list written down and review it the way they would review a production access request for a new hire.

Where the reading habit comes from now

I learned to read code carefully because typing it by hand forced me to. My first professional job, at GE, was documenting a system that had no documentation, which meant cataloging every function in it to understand what it did, and I spent a lot of hours doing exactly that. Nobody would design that job as training today, and it was some of the most useful training I ever had, since it put me in front of a large amount of code I did not write and made me explain it.

A junior engineer starting now may never type a long program from scratch. The first draft comes from a model, and if the tests pass and the demo works, nothing in that workflow makes them read the code closely. If we as leaders do not build that into how they work, the reading habit may never form, and in five years they will be the seniors approving agent diffs.

I have seen what happens when a junior gets real work in small pieces instead of busy work. At CareSource, where I was the tech lead on a mobile team, we had a college intern who had come in through a posted internship. She was a business major with two software courses behind her, and she was being given busy work, analyzing and prioritizing internal software intake forms that arrived every week or two, and she was bored. The team started giving her small examples of how to build mobile components, and then she slowly picked up stories, and then she was taking full stories on her own. Within six months she was a contributing React Native developer, she was hired full time, and she is still there doing that job ten years later.

She got there by being handed real code a little at a time and being expected to understand it well enough to change it, and none of that required typing everything by hand. I think the same approach works with AI tools in the picture, as long as the expectation that she understands the code stays.

What I would ask a junior engineer to do with AI tools

These are recommendations, not a program I have already rolled out, and I would adjust them to the team. They come from asking what the Tandy and the GE cataloging job gave me that a junior engineer with an AI assistant might never get.

1. Explain it back before you ship it. Before AI generated code merges, the person merging it walks someone else through what it does and why, without the assistant open. That goes for seniors too, and it takes a few minutes. If they cannot explain a block, it waits until they can.
2. Review agent diffs in pairs. Put a junior next to a senior while the senior reviews an agent run, using the checklist above, with the senior saying out loud why they are suspicious of a given change. Reviewing is a skill people learn by watching someone do it, and agents generate plenty of material to practice on.
3. Break something on purpose. Hand a junior a piece of AI generated code with one planted bug that still passes the tests, and ask them to find it.
4. Catalog a system. Give a new engineer an unfamiliar service and ask for a written map of what each part does. An assistant can help them go faster, and then they walk a senior through the map and answer questions about it.
5. Follow what interests them. I tell the people on my teams to find outside training that interests them, whether that is a technical manual, a seminar, or a course outside their role, and I reimburse it and get it approved.

This also changes what a senior engineer is for. In my piece on overhauling engineering interviews I argued that reading and reviewing code is already the biggest part of a senior engineer's week. With agents in the loop, the senior engineer's job is more and more about deciding what the agent is allowed to do, reading what it did, and teaching the next person to read it the same way.

Who owns the merge

I wrote a LinkedIn post about COPY CON and these tools, and this piece is the longer version of it. When an agent writes a change and a person approves it, the person and the team own what happens in production, the same way they would if a colleague had written it.

For the teams I lead, that means reading every agent diff with the checklist above, treating the agent's permissions like production access, and making sure the junior engineers learn to read code as carefully as they learn to prompt for it. I also made a public, dated call on where the industry goes with this, my prediction that AI code review standards become an enterprise mandate, and I will grade it when the date comes.

---

Further reading:

• The interview overhaul: why the interview should look like the job
• From Copilot to colleague: how agentic AI is breaking what we built for the assistant era
• What the AWS CEO's comments mean for junior developers
• My prediction on the AI code quality tooling crisis