How to tell if code was written by AI

Detectors built for prose do not transfer to code

Hyunjae Suh and colleagues presented the measurement at ICSE 2025. They ran AI content detectors, GPTZero among them, over human-written and model-written source code in several languages, and reported that all of them performed poorly and lacked the generalisation needed for practical use. Their proposed fix was not a better prompt or a threshold: it was training new classifiers on code structure, using static metrics and abstract syntax tree embeddings.

The reason is mechanical. A prose detector measures the things that vary between writers: sentence length, vocabulary choice, punctuation rhythm. Source code strips most of that out on purpose. A formatter normalises the whitespace, a linter normalises the naming, and a language with one obvious way to write a loop leaves nothing personal in the loop.

The history is where the evidence lives

A file is a snapshot, and a snapshot cannot show how it got there. A repository can. This is the same argument as document version history in an essay case, and it is stronger here, because commits are timestamped, signed in many setups, and awkward to backfill convincingly.

• Commit size and shape. Work done over a week arrives in pieces that compile at each step. A single commit containing a finished feature, tests included, says the code arrived from somewhere else, whether that somewhere was a model or an old project.
• The order of things. Most people write a function, run it, fix it, then write the next one. Generated work often lands complete and then gets edited backwards to fit.
• Trailers on the commit. Coding agents write their own attribution. A Co-authored-by line naming an assistant, or a pull request opened by a bot account, is a declaration rather than an inference, and it is the closest thing to real provenance that code has.
• Timestamps against the rest of the record. Local commit times, CI runs and the clock on the machine all have to agree with the story.

Habits people read as tells, and what each is worth

One real measurement exists, and it is not about you

GitClear analysed 211 million changed lines of code and reported the shape of the change. Copy-pasted lines went from 8.3% of changes in 2020 to 12.3% in 2024. Lines that were moved, which is what refactoring looks like in a diff, fell from 24.1% to 9.5% across the same period. In 2024, for the first time in their data, repeated code was being added faster than existing code was being reorganised.

That is a population trend across millions of commits, and it says nothing about any individual commit. A single duplicated block is how a tired person writes a Tuesday. Anyone quoting GitClear at one developer is using a fleet statistic as a personal accusation, and the difference is the whole argument.

Stack Overflow ran this experiment already

In 2023 Stack Overflow's moderators were using AI detectors to remove generated answers. The company analysed the results and told them to stop. Its stated reason was that the detection tools had an alarmingly high false positive rate, and that their use had produced a sharp rise in suspensions of people with little or no prior posting history, which is to say newcomers writing their first honest answer.

Moderators went on strike over it, because the generated answers were a genuine problem and they had been left without a tool. Both halves of that are true at once, and the site is still working through it. It is the clearest case anyone has published of a large platform measuring detector false positives against real users and deciding the cost was too high.

Questions that come up in a code review or a viva

If you are the one being asked

• Do not rewrite the repository. Editing history now destroys the record that argues for you, exactly as editing a flagged essay does.
• Offer to walk through it: why this data structure, why this error is caught here, what breaks if a value arrives empty.
• Show the parts you got wrong first. Abandoned branches, a commit that reverts an approach, a bug you chased for two days. Generated code rarely comes with a graveyard.
• Ask what specifically prompted the question. If the answer is a detector percentage, ask which tool produced it, and ask what its published error rate on source code is.