Why AI-Generated Code Needs Guardrails: Part 2 - The Research

Want to implement solutions? Check out the Tutorial Series

---

In Part 1, we explored the problems with AI-generated code: code clones, forgotten await keywords, and maintenance debt. But here's the interesting part: not all AI collaboration looks the same.

The guardrails you need depend entirely on who is collaborating with whom.

The Three Collaboration Patterns

After researching how teams actually use AI coding assistants, three distinct patterns emerged:

1. AI-AI Collaboration

What it is: AI generates code that other AI tools will consume, modify, or interact with.

Examples:

• Code generation tools creating boilerplate
• AI-powered refactoring tools restructuring code
• Automated migration scripts
• Generated API clients
• Build tool output

Key characteristic: No human will read or maintain this code regularly.

What matters:

• Correctness (it must work)
• Consistency (predictable patterns)
• Completeness (comprehensive type safety)
• Performance (generated code can be verbose)

What doesn't matter:

• Readability (beyond basic comprehension)
• Clever optimizations
• Comments explaining "why"
• Elegant abstractions

Example:

// Generated API client (AI-AI)
// Maximum type safety, verbose, but machines don't care
export interface GetUserApiResponse {
  readonly data: {
    readonly user: {
      readonly id: string;
      readonly email: string;
      readonly profile: {
        readonly firstName: string;
        readonly lastName: string;
        readonly avatar: string | null;
      };
    };
  };
  readonly meta: {
    readonly timestamp: string;
    readonly version: string;
  };
}

export async function getUserById(
  userId: string,
  options?: RequestOptions
): Promise<GetUserApiResponse> {
  const response = await fetch(`/api/users/${userId}`, options);
  if (!response.ok) {
    throw new ApiError(response.status, await response.text());
  }
  return await response.json() as GetUserApiResponse;
}

This is verbose and repetitive. But it's type-safe, predictable, and machines can work with it easily.

2. AI-Human Collaboration

What it is: AI assists humans in writing code that humans will read, review, and maintain.

Examples:

• GitHub Copilot suggesting completions
• Claude Code helping implement features
• AI pair programming
• AI-assisted refactoring
• Most day-to-day development

Key characteristic: Humans are in the loop, making decisions and maintaining code.

What matters:

• Readability (humans need to understand it quickly)
• Maintainability (humans will change it later)
• Safety (catch obvious mistakes)
• Consistency (matches team patterns)
• Documentation (explain non-obvious decisions)

What doesn't matter as much:

• Perfect type coverage (humans can add types as needed)
• Exhaustive edge case handling (humans review and test)
• Maximum strictness (can be overly restrictive)

Example:

// Feature code (AI-Human)
// Readable, maintainable, good enough type safety
export async function createUser(userData: CreateUserData) {
  // Validate email format before saving
  if (!isValidEmail(userData.email)) {
    throw new ValidationError('Invalid email format');
  }

  // Check if user already exists
  const existing = await userRepository.findByEmail(userData.email);
  if (existing) {
    throw new ConflictError('User already exists');
  }

  // Create user with hashed password
  const hashedPassword = await hashPassword(userData.password);
  const user = await userRepository.create({
    ...userData,
    password: hashedPassword,
  });

  // Send welcome email asynchronously
  await emailService.sendWelcomeEmail(user.email);

  return user;
}

This is clear, well-commented, handles edge cases, and a human can understand the flow in 30 seconds.

3. Human-Human Collaboration

What it is: Traditional team development where humans write code for other humans, possibly with minimal AI assistance.

Examples:

• Core business logic
• Security-critical code
• Complex algorithms
• Legacy codebases
• Highly specialized domains

Key characteristic: Human expertise and judgment are primary, AI is supplementary.

What matters:

• Architecture (thoughtful design decisions)
• Readability (code is read 10x more than written)
• Maintainability (long-term sustainability)
• Team patterns (consistency with existing code)
• Context (understanding business domain)

Example:

// Core business logic (Human-Human)
// Deep domain knowledge, thoughtful design
export class OrderPricingEngine {
  /**
   * Calculates final order price including discounts, taxes, and shipping.
   *
   * Note: We apply discounts before tax per legal requirement in most jurisdictions.
   * Shipping is calculated separately and added after tax.
   *
   * @see https://docs.company.com/pricing-rules
   */
  async calculateFinalPrice(order: Order, context: PricingContext): Promise<Money> {
    const subtotal = this.calculateSubtotal(order.items);
    const discount = await this.calculateDiscount(order, context);
    const taxableAmount = subtotal.subtract(discount);
    const tax = await this.calculateTax(taxableAmount, context);
    const shipping = await this.calculateShipping(order, context);

    return taxableAmount.add(tax).add(shipping);
  }

  private calculateSubtotal(items: OrderItem[]): Money {
    // Use reduce to maintain precision throughout calculation
    return items.reduce(
      (total, item) => total.add(item.price.multiply(item.quantity)),
      Money.zero(items[0]?.price.currency ?? 'USD')
    );
  }

  // ... additional methods with deep domain logic
}

This code reflects business domain knowledge, legal requirements, and architectural decisions that AI can't infer.

The Evidence-Based Approach

The AI Programming Toolkit takes an evidence-based approach to these collaboration patterns.

Research Findings

1. AI-generated code has different characteristics

   • 4x more code clones
   • Higher code churn
   • Specific pattern mistakes (forgotten await, unused variables)
   • Less architectural coherence

2. Different patterns need different guardrails

   • Strictness appropriate for AI-AI may frustrate humans
   • Flexibility needed for Human-Human may let AI mistakes slip through
   • AI-Human needs balanced protection without excessive friction

3. Automated checks are more effective than manual review

   • ESLint catches 90%+ of common AI mistakes instantly
   • Type checking prevents unsafe operations before runtime
   • Tests validate behavior regardless of how code was written
   • Humans can focus on architecture and business logic

The Research That Informed the Toolkit

Code clone detection studies found that AI-generated code contains significantly more duplicated code blocks. This led to:

• ESLint rules targeting code duplication
• Emphasis on clear project structure
• Documentation patterns that help AI find existing abstractions

Code churn analysis revealed that AI-generated code gets modified more frequently. This led to:

• Stronger testing requirements
• More comprehensive type checking
• Review checklists specifically for AI-generated code

Pattern analysis identified specific recurring mistakes. This led to:

• ESLint rules targeting forgotten await
• Rules catching unused variables
• Rules preventing unsafe type operations

Choosing the Right Guardrails

So which collaboration pattern are you in?

Use AI-AI patterns when:

• Generating code that other tools consume
• Creating boilerplate or scaffolding
• Building codegen pipelines
• Maximum correctness is critical
• Readability is secondary

Use AI-Human patterns when:

• Day-to-day feature development
• AI assists but humans drive
• Code will be reviewed and maintained
• Balance between safety and velocity

Use Human-Human patterns when:

• Core business logic
• Security-critical code
• Legacy codebases with established patterns
• Deep domain expertise required
• Minimal AI assistance

Strictness Levels Within Each Pattern

Even within a collaboration pattern, teams have different needs. The toolkit offers three strictness levels:

Maximum Strictness

• All rules enabled
• No exceptions
• Maximum safety
• Best for: Critical systems, teams new to AI assistance, codegen

Balanced Strictness

• Most rules enabled
• Pragmatic exceptions
• Good safety with flexibility
• Best for: Most teams, production applications, balanced AI-Human collaboration

Relaxed Strictness

• Core rules only
• Maximum flexibility
• Minimum friction
• Best for: Prototypes, exploratory work, experienced teams

Real-World Examples

Startup Using AI-Human (Balanced)

Context: 5-person team building a SaaS product with GitHub Copilot

Needs:

• Move fast but maintain quality
• AI helps with boilerplate
• Humans make architectural decisions

Configuration:

• AI-Human ESLint config (Balanced)
• TypeScript strict mode
• Test coverage 80%+
• PR reviews required

Results:

• Caught 15+ forgotten await before code review
• Reduced code duplication by 40%
• Maintained velocity while improving quality

Enterprise Using Human-Human (Maximum)

Context: 50-person team maintaining a 5-year-old monolith

Needs:

• Consistency with existing patterns
• Minimal AI assistance (developers choose when to use it)
• Maximum code quality standards

Configuration:

• Human-Human ESLint config (Maximum)
• Strict TypeScript with no any
• Test coverage 90%+
• Architecture review for major changes

Results:

• New code matches existing patterns
• AI suggestions filtered through strict rules
• Maintained architectural coherence

Platform Team Using AI-AI (Maximum)

Context: Internal tools team generating API clients

Needs:

• Generate code from OpenAPI specs
• Other services consume generated clients
• Humans rarely modify generated code

Configuration:

• AI-AI ESLint config (Maximum)
• Generated code in separate directories
• Comprehensive type generation
• Automated tests for generated code

Results:

• 100% type-safe API clients
• Zero human involvement in client maintenance
• Other AI tools easily work with generated code

The Key Insight

The collaboration pattern matters more than the tools you use.

GitHub Copilot, Claude Code, and Cursor are all excellent tools. But they work best when you:

1. Identify your collaboration pattern
2. Choose appropriate guardrails
3. Automate enforcement
4. Iterate based on results

This is the evidence-based approach: observe patterns, apply research-backed solutions, measure results, adjust.

What's Next?

In Part 3, we'll explore the philosophy behind the toolkit's design. You'll learn:

• Why 24 ESLint configurations instead of one?
• How to choose the right configuration
• The gradual adoption philosophy
• Why context-specific tooling beats one-size-fits-all

Ready to implement? Jump to the Tutorial Series to start setting up your guardrails.

Continue reading: Part 3: The Philosophy →

---

What collaboration pattern best describes your team? Understanding this is the first step to choosing the right guardrails for your project.