Skip to content

The AI Code Quality Maturity Model

After working with 22 engineering teams, I built a 5-level maturity model for AI code quality. Most teams overestimate their level. Here's how to assess and advance.

10 min read
Illustrative series of five small physical assembly studies arranged on gradually rising off-white plinths.
On this page

Every engineering team I've worked with in the last 18 months is at a different stage of AI adoption. Some are figuring out basic prompting. Others have fully automated quality pipelines. But almost none of them have a clear picture of where they are versus where they need to be.

That gap bothers me. So I built a maturity model.

After working with 22 engineering teams across startups, mid-size companies, and enterprises, I've identified 5 distinct maturity levels for AI code quality. Each level has observable behaviors, specific risks, and clear steps to advance to the next one.

Why Another Maturity Model?

I know. Maturity models can feel like consultant theater. But here's my contrarian take: most teams overestimate their AI maturity. They think they're at Level 3 because they use AI daily. In reality, they're at Level 1 because daily usage without quality controls is the least mature state.

Usage frequency has nothing to do with maturity. Quality awareness does.

The 5 Levels

Uncontrolled individual adoption Guidelines shared practices Enforcement automated checks Measured quality outcomes Adaptive update the checks
Figure 1. The proposed model groups practices into five levels.

Level 1: Uncontrolled Adoption

Observable behaviors:

  • Engineers use AI tools with personal accounts and preferences
  • No shared prompting practices or guidelines
  • AI-generated code goes through the same review process as human code
  • No metrics tracking AI code quality separately
  • "Everyone uses Copilot" is considered an AI strategy

Risks at this level:

  • Pattern inconsistency accelerating weekly
  • Unknown dependency additions
  • Security vulnerabilities introduced by AI going undetected
  • Growing duplicate code ratio (typically 8-15% after 6 months)

The assessment test: Ask 5 engineers on your team how they prompt AI for a database query. If you get 5 different answers with no shared conventions, you're at Level 1.

I surveyed teams at this level and found an average of 4.2 different error handling patterns per codebase. One team had 7. They didn't realize it because each pattern looked reasonable in isolation.

Level 2: Awareness and Guidelines

Observable behaviors:

  • Written guidelines for AI tool usage exist
  • Team has discussed AI code quality in at least one retro
  • Some engineers share effective prompts
  • Basic awareness that AI code needs different review attention
  • Shared .cursorrules or AI configuration files in the repo

What gets you here:

  • Creating an AI coding guidelines document
  • Adding AI code quality to your team's definition of done
  • Establishing a shared prompt library for common tasks

Typical metrics:

  • Duplicate code ratio: 5-8%
  • Pattern consistency: 2-3 patterns per concern
  • Code review catch rate for AI issues: 40-50%

In this model, Level 2 focuses on shared guidelines and discussion. Repository instructions can record the conventions an assistant should follow. Review generated changes against those conventions and check whether the team uses the guidance in practice.[1]

Level 3: Automated Enforcement

Observable behaviors:

  • Custom linting rules catch AI-specific antipatterns
  • CI pipeline includes AI code quality checks
  • Dependency additions require explicit approval
  • Duplicate code detection runs automatically
  • Architecture decisions are codified as automated rules

What gets you here:

  • Building an AI-specific quality gate in CI (see my CI/CD quality gate guide)
  • Implementing pattern conformance checks with tools like ts-morph
  • Adding dependency auditing to your pipeline
  • Creating custom ESLint rules for your architecture patterns

Typical metrics:

  • Duplicate code ratio: 3-5%
  • Pattern consistency: 1-2 patterns per concern (automated enforcement)
  • Code review catch rate for AI issues: 70-80% (human + machine)
  • PR cycle time: 15-25% faster (less back-and-forth on style issues)
typescript
// Example: Level 3 team's .eslintrc additions
module.exports = {
  rules: {
    // Enforce internal HTTP client usage
    "no-restricted-imports": ["error", {
      patterns: [
        { group: ["axios", "node-fetch", "got"],
          message: "Use @/lib/httpClient instead." }
      ]
    }],
    // Enforce Result pattern over try/catch
    "custom/no-try-catch-in-services": "error",
    // Flag AI-common antipatterns
    "custom/no-catch-all-error": "error",
    "custom/no-unused-generic-types": "warn",
  }
};

Automated enforcement can check configured rules, such as restricted imports, before a change reaches a reviewer. Choose rules that reflect the architecture and evaluate their findings and false positives.[2]

Level 4: Measured and Optimized

Observable behaviors: Measure outcomes against a baseline and interpret delivery measures together in the context of an application or service.[3]

  • AI code quality metrics tracked weekly with dashboards
  • A/B testing of prompting strategies for quality outcomes
  • Regular audits comparing AI-generated vs human-written code quality
  • Feedback loop from production issues back to quality gate rules
  • Team knows exactly which AI tasks produce high-quality output and which don't

What gets you here:

  • Building a quality metrics dashboard that separates AI and human code
  • Running monthly AI code audits
  • Tracking production incidents back to code origin
  • Measuring and optimizing review effectiveness

Typical metrics:

  • Duplicate code ratio: < 3%
  • Bug density (AI code): within 1.2x of human code
  • Pattern consistency: single enforced pattern per concern
  • Production incident attribution: tracked and categorized
  • Mean time to quality gate (new rules added within 1 week of discovery)

The jump from Level 3 to Level 4 requires investment in measurement. Most teams resist this because measuring feels like overhead. But I've seen measurement cut production incidents by 40% within 3 months because it creates a feedback loop: production issue discovered, root cause identified, quality gate rule added, issue prevented in future.

Level 5: Adaptive Intelligence

Observable behaviors: Automated review can use repository context, but missed issues and false positives still require careful human review.[4]

  • Quality gates evolve automatically based on codebase changes
  • AI-assisted review catches architecture drift before humans
  • Pre-generation context injection is automated (AI "knows" your codebase)
  • Quality metrics predict issues before they reach production
  • Team contributes to organizational AI coding standards

What gets you here:

  • Automated rule generation based on codebase analysis
  • Integration of AI review tools with full codebase context
  • Predictive quality metrics (flagging risky AI patterns before bugs appear)
  • Cross-team knowledge sharing about AI quality patterns

I'll be honest: I've only seen 2 teams reach Level 5, and both are at large tech companies with dedicated developer tooling teams. It's aspirational for most organizations right now. But it's the direction to aim for.

How to Assess Your Level

Here's a quick assessment. Answer each question honestly:

Question Yes = Points
Do you have written AI coding guidelines? +1
Does your CI pipeline have AI-specific checks? +2
Can you measure AI code quality separately from human code? +2
Do you track production incidents by code origin? +2
Do your quality gate rules update based on production feedback? +3
Does your team share a prompt library? +1
Is there a dependency approval process for AI-suggested packages? +1
Do you have custom linting rules for your architecture? +2
Can you detect pattern drift automatically? +2
Do you run regular AI code audits? +1

Scoring:

  • 0-2 points: Level 1
  • 3-5 points: Level 2
  • 6-9 points: Level 3
  • 10-14 points: Level 4
  • 15-17 points: Level 5

The 90-Day Plan to Jump Two Levels

Use this proposed 90-day plan to introduce shared guidance, automation and measurement. Choose the pace according to the team's constraints and verify each change:

Days 1-14: Awareness sprint

  • Run the team assessment above
  • Hold a 1-hour AI code quality workshop
  • Create your AI coding guidelines document
  • Set up a shared prompt library (even a Notion page works)

Days 15-45: Automation sprint

  • Add dependency auditing to CI (2 hours)
  • Add placeholder/TODO scanning (1 hour)
  • Write 3-5 custom ESLint rules for your architecture (4-6 hours)
  • Set up duplicate code detection with jscpd (1 hour)

Days 46-90: Measurement sprint

  • Build a basic quality dashboard (track weekly)
  • Run your first AI code audit (pick 10 random AI-generated files)
  • Add AI code quality metrics to your sprint retro
  • Tune false positives in your quality gate

After 90 days, run the assessment again. I've seen teams go from 1-2 points to 8-10 points following this plan. The key is consistency: don't try to build everything in week one. Build steadily and let each improvement compound.

Awareness days 1-14 Automation days 15-45 Measurement days 46-90
Figure 2. The proposed plan introduces awareness, automation and measurement in phases.

Where Teams Get Stuck

The most common stall point is between Level 2 and Level 3. Teams write guidelines but never automate enforcement. Guidelines without automation are suggestions. Suggestions get ignored under deadline pressure.

If you've been at Level 2 for more than a month, stop writing documents and start writing linting rules. One automated check is worth ten pages of guidelines.

The second stall point is between Level 3 and Level 4. Teams automate checks but never measure outcomes. Without measurement, you don't know if your checks are catching the right things. You need the feedback loop.

Wherever you are today, the path forward is clear. Pick the next level and start the work. Your codebase will thank you in 6 months.

Frequently asked questions

Is this maturity score an industry certification?

The score is the author's proposed assessment method. Its categories and thresholds help a team discuss its practices. They do not establish an industry benchmark, certify security or prove that one team will achieve another team's results.

What is the first step after assessing a team?

Identify the next control that would address a specific gap in your work. A team without shared conventions can start with written guidance. A team with guidance can choose an enforceable check and evaluate whether it catches the intended problem.

Does automated enforcement remove the need for review?

Automated checks can enforce rules they have been configured to recognise. Examine their findings and false positives, and keep human review for correctness, security and design. Review tools can miss issues even when they have repository context.

How should a team judge whether it has improved?

Record a baseline, apply a bounded change and track quality and delivery outcomes over time. Use incidents and review findings to revise the checks. A higher assessment score alone does not show that the codebase became easier or safer to change.

Sources

  1. [1]Adding repository custom instructions for GitHub Copilot - GitHub Docs · GitHub Docs
  2. [2]no-restricted-imports - ESLint - Pluggable JavaScript Linter · ESLint
  3. [3]DORA’s software delivery performance metrics · DORA
  4. [4]Application card: GitHub Copilot Agents - GitHub Docs · GitHub Docs

Checked against these sources on

Related Articles