Coding Agents Are Lazy Patchers

SCBench @Gorlanski

TL;DR: Leading coding agents become lazy patchers when faced with iterative changes. They fall into copy-paste habits instead of refactoring, leading to massive god functions that become unmaintainable. This is why they feel worse than their scores on leading benchmarks and why we need realistic benchmarks like SlopCodeBench.

Contents

Edits Concentrate into Hotspots
Obvious Abstractions Turn Into Quick Hacks
- GPT-5.2-Codex’s 140-line Monstrosity
- GPT-5.1-Codex-Max Shows Some Progress
- Slopus 4.5 is Slightly Better
Duplication Rises with Progress
Conclusion
Methodology

Instead of using the 2x holiday rate limits to vibe code some B2B SaaS, I decided to stress test the top coding agents with SlopCodeBench (SCBench). In SCBench, each “problem” is a sequence of checkpoints. The agent must extend its existing solution to satisfy the specification for the current checkpoint. Iterative change, not one-shot coding, is where maintainability matters.

We use a very similar setup to our announcement blog post, but have now expanded to 93 checkpoints across 20 problems (see the Methodology section for details).

This is the first part of our series discussing these results and the benchmark harness overhaul. We find that agents are exceptionally lazy in how they handle problem extensions. Rather than refactoring when appropriate, they fall into the common trap of copying and pasting code snippets. Thus, we are left with massive and unmaintainable “god” methods.

Edits Concentrate into Hotspots

Line chart showing Gini coefficient growth across checkpoints for Opus 4.5, GLM-4.7, GPT-5.2-Codex, and GPT-5.1-Codex-Max demonstrating increasing code concentration in fewer functions

Function Gini, Branches, and Complexity

Function Gini coefficient:
Measures how unevenly code is distributed among functions.

0: Code is spread out evenly across all functions.
1: All code resides in a single function. High values mean most code lives in a few “god” functions, making the codebase harder to maintain.

Branch count:
The total number of decision points (e.g., if, elif, else, case statements) present in the code. More branches can indicate tangled logic or a lack of proper abstraction.

Cyclomatic complexity:
A metric representing the number of independent paths through the code. Higher values suggest more complicated, harder-to-test functions and a greater potential for bugs or maintenance headaches.

Comparisons:
Counts of explicit conditional checks (e.g., ==, !=, <, >, in if/while/for conditions), which can reflect how tightly logic is entwined. More comparisons often mean increased complexity and a higher risk of logic errors.

Together, these metrics illustrate how code structure degrades: functions swell, branching logic proliferates, complexity rises, and maintainability plummets as agents focus only on passing the next test.

Across all setups, a clear pattern emerges: agents make large patches in a select few functions. The Gini coefficient climbs steadily as agents progress through checkpoints, highlighting the emergence of god functions that become nearly impossible to debug and maintain. These monstrosities also impose a significant cognitive burden during review. This would not be bad if these solutions were split into cohesive modules, but they are not. Each setup uses a single file spanning more than 2,000 lines across all problems.

Model	STMT Gini (Start → Final)	LOC Gini (Start → Final)
GLM-4.7	0.531 → 0.584	0.518 → 0.594
GPT-5.2-Codex	0.524 → 0.572	0.537 → 0.602
GPT-5.1-Codex-Max	0.525 → 0.559	0.494 → 0.567
Opus 4.5	0.513 → 0.585	0.532 → 0.602

Table legend

STMT = statements per function;
LOC = lines of code per function.
Higher Gini = more concentration in fewer functions.

Both statements (number of top-level expressions as counted by tree-sitter) and LOC further support this finding.

Obvious Abstractions Turn Into Quick Hacks

The worst offender is code search. At checkpoint 3, the agent must add tree-sitter pattern matching to an existing rule dispatcher. Across all agents, the LOC concentration increases 16% despite the clear motivation to refactor. Here is how I would have written the dispatcher:

1
def find_matches(text, rules, language):
2
    for rule in rules:
3
        if language not in rule.languages:
4
            continue
5
        yield from rule(text, language)

This is not great, but at least I won’t have to update it every time new rules are added. I could see the argument for using a dictionary or if statements as well—it’s very unlikely there will be more than five kinds of rules. Either approach would keep it simple as long as the logic for finding matches is encapsulated.

GPT-5.2-Codex’s 140-line Monstrosity

1
def find_matches(text: str, rules: List[Dict], language: str) -> List[Dict]:
2
def find_matches(text: str, rules: List[Dict], language: str, encoding: str) -> List[Dict]:
3
    line_starts = build_line_starts(text)
4
    ...
5
    for rule in rules:
6
        if rule["kind"] == "exact":
7
            ...
8
            while True:
9
                ...
10
                matches.append({
11
                    "rule_id": rule_id,
12
                    "start": {"line": start_line, "col": start_col},
13
                    "end": {"line": end_line, "col": end_col},
14
                    "match": pattern,
15
                })
16
        elif rule["kind"] == "regex":
17
            regex = rule["regex"]
18
            if regex is None:
19
                continue
20
            for match in regex.finditer(text):
21
                ...
22
                matches.append({
23
                    "rule_id": rule_id,
24
                    "start": {"line": start_line, "col": start_col},
25
                    "end": {"line": end_line, "col": end_col},
26
                    "match": match.group(0),
27
                })
28

29
    pattern_rules = [r for r in rules if r["kind"] == "pattern"]
30
    if pattern_rules:
31
        ...
32
        for rule in pattern_rules:
33
            ...
34
            if root_info["is_sequence"]:
35
                ...
36
                for node in nodes:
37
                    ...
38
                    matches.append({
39
                        "rule_id": rule_id,
40
                        "start": {"line": start_line, "col": start_col},
41
                        "end": {"line": end_line, "col": end_col},
42
                        "match": match_text,
43
                        "captures": build_captures_output(...),
44
                    })
45
            else:
46
                ...
47
                for node in nodes:
48
                    ...
49
                    matches.append({
50
                        "rule_id": rule_id,
51
                        "start": {"line": start_line, "col": start_col},
52
                        "end": {"line": end_line, "col": end_col},
53
                        "match": match_text,
54
                        "captures": build_captures_output(...),
55
                    })
56
    ...
57
    return matches

There are so many baffling decisions here. Why filter the rules list after iterating over it, only to iterate again? Why copy the match construction twice more when the structure already exists? It reeks of hacky coding. Worst of all, this function comes out to over 140 lines.

GPT-5.1-Codex-Max Shows Some Progress

At least GPT-5.1-Codex-Max does slightly better. The original dispatch function was renamed from scan_file to scan_text_rules. Then if rule.kind == "pattern": continue was added to skip pattern rules:

1
def scan_file(content: str, rules: Sequence[Rule], language: str):
2
def scan_text_rules(content: str, rules: Sequence[Rule], language: str):
3
    # ... initialization ...
4
    for rule in rules:
5
        if language not in rule.languages:
6
            continue
7
        if rule.kind == "pattern":
8
            continue
9
        # ... matching logic ...
10
        for start, end in spans:
11
            yield {
12
                "rule_id": rule.rule_id,
13
                "language": language,
14
                "start": {"line": start_line, "col": start_col},
15
                "end": {"line": end_line, "col": end_col},
16
                "slice": (start, end),
17
            }

Why not extend the dispatcher? Why skip pattern rules here only to handle them elsewhere? It’s infuriating and confusing, especially when looking at how these rules are handled in main:

1
for match in scan_file(content, applicable_rules, language):
2
for match in scan_text_rules(content, applicable_text_rules or [], language):
3
    matches.append({
4
        "rule_id": match["rule_id"],
5
        "file": file_rel,
6
        "language": match["language"],
7
        "start": match["start"],
8
        "end": match["end"],
9
        "match": content[match["slice"][0] : match["slice"][1]],
10
    })
11
if applicable_pattern_rules:
12
    content_bytes = content.encode("utf-8")
13
    line_start_bytes = build_line_start_bytes(content_bytes)
14
    tree = PARSERS[language].parse(content_bytes)
15
    for rule in applicable_pattern_rules:
16
        for match in collect_pattern_matches(
17
            tree, content_bytes, rule, language, line_start_bytes
18
        ):
19
            match["file"] = file_rel
20
            matches.append(match)

A new branch, loop, and source of match dictionaries. The construction used in collect_pattern_matches is nearly verbatim:

1
yield {
2
    "rule_id": rule.rule_id,
3
    "file": "",  # filled by caller
4
    "language": language,
5
    "start": {"line": start_line, "col": start_col},
6
    "end": {"line": end_line, "col": end_col},
7
    "match": content_bytes[start_byte:end_byte].decode("utf-8", errors="replace"),
8
    "captures": captures_out,
9
}

Four places to update when the format changes. Four places for one update to slip through. Everyone has dealt with this in the days before agents. The time and annoyance of fixing these bugs is immense, even for agents.

Slopus 4.5 is Slightly Better

Opus 4.5 extended the dispatcher correctly:

1
def find_matches(content, rule, language=None):
2
    if rule["kind"] == "exact":
3
        ...
4
        matches.append({"rule_id": rule["id"], "start": ..., "end": ..., "match": pattern})
5
    elif rule["kind"] == "regex":
6
        ...
7
        matches.append({"rule_id": rule["id"], "start": ..., "end": ..., "match": m.group(0)})
8
    elif rule["kind"] == "pattern":
9
        matcher = PatternMatcher(rule["pattern"], rule["languages"])
10
        for m in matcher.find_matches(content, language):
11
            matches.append({
12
                "rule_id": rule["id"],
13
                "start": m["start"], "end": m["end"],
14
                "match": m["match"], "captures": m["captures"]
15
            })
16
    return matches

One function with a proper elif extension. The 300-line PatternMatcher class encapsulates the tree-sitter logic. This is progress.

But look at the pattern branch. PatternMatcher.find_matches returns dicts with start, end, match, and captures. Then those dicts are immediately re-wrapped to add rule_id:

1
# PatternMatcher returns
2
{'start': ..., 'end': ..., 'match': match_text, 'captures': captures}
3

4
# Caller copies four fields to add one
5
{"rule_id": rule["id"], "start": m["start"], "end": m["end"], ...}

Why not pass rule_id to the class? A 300-line class with a single public method, and it cannot return a complete result. Additionally, a new instance is created for every rule on every file—no caching, pattern re-parsed each time.

The structure is correct. The integration was not thought through.

Duplication Rises with Progress

Chart showing code duplication ratio and churn rate across checkpoints for all tested AI coding agents

Figure metrics

Clone ratio = percentage of duplicated code blocks in the solution.
Churn = percentage of lines changed between checkpoints relative to total codebase size.

They all exhibit copy-pasting behavior while making significant changes to the codebase. GLM-4.7 is the worst offender—nearly 25% of its output is duplicated code, and it peaks at over 110% churn. GPT-5.1-Codex-Max is the best, with a ~9% clone ratio, but it still peaks at 80% churn. As agents progress through problems, their sole goal is to pass the next test, with no concern for downstream maintainability.

One of our newest problems, circuit_eval, spans 8 checkpoints. For Opus 4.5, this meant an 8x increase in lines of code for its single file, going from 627 to 5,204. Despite the churn, there are clear signs of hacky copy-pasting.

The entire literal parsing function is duplicated. Same regex, same validation, same errors—one just allows X in binary. First, parse_literal (65 lines):

1
def parse_literal(text: str, for_input: bool = False) -> tuple[int, int]:
2
    error_class = InputValueParseError if for_input else CircParseError
3
    text_clean = text.replace('_', '')
4

5
    if text_clean.upper() == 'X':
6
        raise error_class(f"Literal X value not allowed")
7

8
    sized_match = re.match(r"^(\d+)'([bhd])([0-9a-fA-F]+)$", text_clean)
9
    if sized_match:
10
        width = int(sized_match.group(1))
11
        base_char = sized_match.group(2).lower()
12
        digits = sized_match.group(3)
13
        if width == 0:
14
            raise error_class(f"Invalid literal width: 0")
15
        if base_char == 'b':
16
            if not all(c in '01' for c in digits):
17
                raise error_class(f"Invalid binary literal: {text}")
18
            value = int(digits, 2)
19
        elif base_char == 'h':
20
            value = int(digits, 16)
21
        else:  # 'd'
22
            value = int(digits, 10)
23
            if value >= (1 << width):
24
                raise error_class(f"Decimal value {value} exceeds width {width}")
25
        return value, width
26

27
    if text_clean.lower().startswith('0b'):
28
        digits = text_clean[2:]
29
        if not digits or not all(c in '01' for c in digits):
30
            raise error_class(f"Invalid binary literal: {text}")
31
        return int(digits, 2), len(digits)
32

33
    if text_clean.lower().startswith('0x'):
34
        digits = text_clean[2:]
35
        if not digits or not all(c in '0123456789abcdefABCDEF' for c in digits):
36
            raise error_class(f"Invalid hex literal: {text}")
37
        return int(digits, 16), len(digits) * 4
38

39
    if text_clean == '0': return 0, 1
40
    if text_clean == '1': return 1, 1
41
    raise error_class(f"Invalid literal: {text}")

Now look at parse_3val_literal (75 lines), introduced one checkpoint later:

1
def parse_3val_literal(text: str) -> tuple[int, int, int]:
2
    text_clean = text.replace('_', '')
3

4
    if text_clean.upper() == 'X':
5
        return (0, 1, 1)
6

7
    sized_match = re.match(r"^(\d+)'([bhd])([0-9a-fA-F]+)$", text_clean)
8
    if sized_match:
9
        width = int(sized_match.group(1))
10
        base_char = sized_match.group(2).lower()
11
        digits = sized_match.group(3)
12
        if width == 0:
13
            raise InputValueParseError(f"Invalid literal width: 0")
14
        if base_char == 'h':
15
            value = int(digits, 16)
16
        else:  # 'd'
17
            value = int(digits, 10)
18
            if value >= (1 << width):
19
                raise InputValueParseError(f"Decimal value {value} exceeds width {width}")
20
        return (value, 0, width)
21

22
    if text_clean.lower().startswith('0b'):
23
        digits = text_clean[2:]
24
        if not digits:
25
            raise InputValueParseError(f"Invalid binary literal: {text}")
26
        # ... X parsing logic ...
27
        return (value, x_mask, len(digits))
28

29
    if text_clean.lower().startswith('0x'):
30
        digits = text_clean[2:]
31
        if not digits or not all(c in '0123456789abcdefABCDEF' for c in digits):
32
            raise InputValueParseError(f"Invalid hex literal: {text}")
33
        return (int(digits, 16), 0, len(digits) * 4)
34

35
    if text_clean == '0': return (0, 0, 1)
36
    if text_clean == '1': return (1, 0, 1)
37
    raise InputValueParseError(f"Invalid literal: {text}")

Same structure. Same regex. Same error messages. One function with allow_x=True would eliminate 60+ lines.

Tracing through the checkpoints reveals exactly when this happened. Checkpoint 3 introduced parse_literal. Then Checkpoint 4 needed 3-valued logic support. So the machine spirit copy-pasted the entire function to parse_3val_literal with a minor tweak to allow X. From checkpoints 4 through 8, neither function changed. The agent added 4,500+ lines of new code across those checkpoints without these functions ever being consolidated.

The agent also duplicated the entire evaluator class. Evaluator (150 lines) and ThreeValuedEvaluator (193 lines):

1
class Evaluator:
2
    def __init__(self, circuit: Circuit):
3
        self.circuit = circuit
4
        self.values: dict[str, int] = {}
5
        self.assignment_map: dict[str, Assignment] = {}
6
        self.all_signals: dict[str, Signal] = {}
7
        for sig in circuit.inputs + circuit.outputs + circuit.wires:
8
            self.all_signals[sig.name] = sig
9
        for asgn in circuit.assignments:
10
            self.assignment_map[asgn.lhs] = asgn
11

12
    def evaluate(self, inputs: dict[str, int]) -> dict[str, int]:
13
        self.values = dict(inputs)
14
        outputs = {}
15
        for sig in sorted(self.circuit.outputs, key=lambda s: s.name):
16
            outputs[sig.name] = self._eval_signal(sig.name)
17
        return outputs
18

19
class ThreeValuedEvaluator:  # 150 lines later...
20
    def __init__(self, circuit: Circuit):
21
        self.circuit = circuit
22
        self.values: dict[str, tuple[int, int]] = {}  # only diff: tuple
23
        self.assignment_map: dict[str, Assignment] = {}
24
        self.all_signals: dict[str, Signal] = {}
25
        for sig in circuit.inputs + circuit.outputs + circuit.wires:
26
            self.all_signals[sig.name] = sig
27
        for asgn in circuit.assignments:
28
            self.assignment_map[asgn.lhs] = asgn
29

30
    def evaluate(self, inputs: dict[str, tuple[int, int]]) -> ...:
31
        self.values = dict(inputs)
32
        outputs = {}
33
        for sig in sorted(self.circuit.outputs, key=lambda s: s.name):
34
            outputs[sig.name] = self._eval_signal(sig.name)
35
        return outputs

The __init__ and evaluate methods are verbatim copies. The only difference is int vs tuple[int, int]. A generic Evaluator[T] or even just passing a value-wrapper would unify 300+ lines into one class.

This is the lazy patcher pattern in action: solve the immediate problem, move on, never look back.

Conclusion

One may be tempted to ask, Why does it matter if the code is correct? The SOTA checkpoint pass rate on SCBench is only 10.8%, so maybe these agents just aren’t good enough yet. But that misses the larger point: we should expect them to do better. It is precisely from this expectation that the gap between benchmark scores and real-world use emerges. As Dax from OpenCode put it, solutions from LLMs are always the 2nd or 3rd best way of doing something.

This can only be evaluated through an iterative specification benchmark like SlopCodeBench. That is exactly why we created it. If you care about the real quality of coding agents, join the Discord or look at our repo! We are always looking for new contributors.

Methodology

We now have 20 problems in total, with 93 checkpoints. GLM-4.7 and GPT-5.2-Codex results are now included—GLM-4.7 was run using Claude Code 2.0.75, and GPT-5.2-Codex was run using Codex 0.74.0. Following our original setup, we use only the “high” reasoning-effort setting for both harnesses and the just-solve prompt, which focuses purely on solving the problem.

Otherwise, it is nearly identical to our announcement post.