VibeCoding Academy

Giao diện

Buổi 19: Self-Healing QA — Test Tự Tiến Hoá ​

Phần Lý Thuyết (30 phút) ​

The QA Immune System Metaphor ​

Tưởng tượng test suite như hệ thống miễn dịch của ứng dụng:

Immune System           →    QA System
─────────────────────────────────────────
White blood cells       →    Tests
Pathogens              →    Bugs
Antibodies             →    Assertions
Immune memory          →    learnings.json
Healing               →    Self-repair of tests

Healthy immune system:

  • Nhận diện mối đe dọa nhanh (quick bug detection)
  • Nhớ lâu (learning from past bugs)
  • Phát triển theo thời gian (evolve with codebase)
  • Tự chữa lành (fix broken tests automatically)

Weak immune system:

  • Tests pass → Đột nhiên fail (flaky)
  • Không nhớ lessons (repeat same bugs)
  • Test phá hủy khi code thay đổi (brittleness)
  • Manual fix tests (no evolution)

Three Self-Healing Skills ​

1. cm-skill-health: Audit Test Suite Health ​

Chẩn đoán: Test suite khỏe không?

Health Signals (từ shipped tests):

json
{
  "healthCheckId": "task-test-suite",
  "timestamp": "2026-04-24T14:00:00Z",
  "signals": {
    "docsDrift": {
      "status": "warning",
      "detail": "Test documentation says 'Task model handles timezone', but code doesn't. Docs outdated."
    },
    "brokenReferences": {
      "status": "critical",
      "detail": "Test imports TaskManager from old path: '../managers/old-task.js' (file deleted)"
    },
    "retroNotes": {
      "status": "ok",
      "detail": "learnings.json has 8 entries, last update 2 days ago"
    },
    "validationGaps": {
      "status": "warning",
      "detail": "Test for filterTasks() missing edge case: empty filter object"
    },
    "gates": {
      "status": "ok",
      "detail": "Last 10 quality gates passed without issue"
    }
  },
  "overallHealth": "FAIR",
  "recommendations": [
    "Fix broken import in test file",
    "Update docs to match code",
    "Add test case for empty filter",
    "Refresh learnings.json with new patterns"
  ]
}

Health Levels:

  • 🟢 HEALTHY: No drift, all references good, validation complete
  • 🟡 FAIR: Minor docs drift, 1-2 warning signals
  • 🔴 CRITICAL: Broken tests, missing validations, docs way off

2. cm-skill-evolution: Test Suite Evolves with Code ​

Khi code thay đổi, tests phải thích ứng. 3 modes:

Mode 1: FIX (Repair Broken Skill)

Khi codebase thay đổi, test break. AI tự fix:

javascript
// ❌ OLD CODE (test written for this)
class Task {
  getStatus() {
    return this.completed ? 'DONE' : 'PENDING';
  }
}

// ✅ NEW CODE (someone refactored)
class Task {
  getStatus() {
    return this.state; // state: 'active', 'paused', 'completed'
  }
}

// ❌ BROKEN TEST
test('getStatus returns DONE when completed', () => {
  const task = new Task({ completed: true });
  expect(task.getStatus()).toBe('DONE'); // ❌ FAIL, now 'undefined'
});

// ✅ AI FIXED TEST
test('getStatus returns completed when state is completed', () => {
  const task = new Task({ state: 'completed' });
  expect(task.getStatus()).toBe('completed'); // ✅ PASS
});

Mode 2: DERIVED (Clone + Modify)

Khi cần test variant của existing feature:

javascript
// EXISTING SKILL: filterTasksByStatus
function filterTasksByStatus(tasks, status) {
  return tasks.filter(t => t.status === status);
}

// NEW FEATURE: filterTasksByPriority
function filterTasksByPriority(tasks, priority) {
  return tasks.filter(t => t.priority === priority);
}

// AI DERIVES new test from old pattern:
// EXISTING TEST
test('filterTasksByStatus returns only active tasks', () => {
  const tasks = [
    { id: 1, status: 'active' },
    { id: 2, status: 'completed' }
  ];
  expect(filterTasksByStatus(tasks, 'active')).toEqual([tasks[0]]);
});

// DERIVED TEST (pattern: filter by field)
test('filterTasksByPriority returns only high priority tasks', () => {
  const tasks = [
    { id: 1, priority: 'high' },
    { id: 2, priority: 'low' }
  ];
  expect(filterTasksByPriority(tasks, 'high')).toEqual([tasks[0]]);
});

Mode 3: CAPTURED (Auto-Create from Patterns)

Khi pattern xuất hiện 3+ lần, tự động tạo reusable skill:

javascript
// PATTERN OBSERVED (appears 3x in codebase):
// Check 1: if (!array || array.length === 0) return []
// Check 2: if (!items || items.length === 0) return []
// Check 3: if (!results || results.length === 0) return []

// AI AUTO-CREATES SKILL:
/**
 * cm-learned-empty-array-guard
 * Pattern: Check array is not null/undefined AND not empty
 */
function guardEmptyArray(arr) {
  if (!arr || arr.length === 0) {
    return [];
  }
  return arr; // pass through if valid
}

// PATTERN NOW IN LEARNINGS:
{
  "id": "cm-learned-empty-array-guard",
  "pattern": "Guard clause for empty arrays",
  "signature": "guardEmptyArray(arr) → [] if empty, else arr",
  "usage": "Used in filterTasks, mapTasks, sortTasks",
  "frequency": 3,
  "autoCreated": true
}

3. cm-learning-promoter: Continuous Improvement Loop ​

Tự động promote learned patterns lên coded skills:

Task struggles (logged)
        ↓
    Analyze patterns
        ↓
    Pattern appears 3+ times?
        ↓
    Auto-create cm-learned-* skill
        ↓
    Test new skill
        ↓
    Add to codebase
        ↓
    Update learnings.json

Example Promotion Flow:

Week 1: Developers struggle with timezone handling in 3 different tests
Week 2: Pattern recognized: need UTC normalization
Week 3: Create cm-learned-timezone-normalize skill
Week 4: Integrate skill into test utils
Week 5: All timezone tests use skill, 0 timezone bugs

Savings: 3 manual fixes → 1 automated utility → prevention

Continuous QA Improvement Loop ​

┌─────────────────────────────────────────┐
│  1. MONITOR: Track test health signals  │
├─────────────────────────────────────────┤
│  2. DETECT: Find broken tests, drift    │
├─────────────────────────────────────────┤
│  3. DIAGNOSE: Root cause of health drop │
├─────────────────────────────────────────┤
│  4. EVOLVE: Fix, derive, or capture     │
├─────────────────────────────────────────┤
│  5. VALIDATE: Test new skill works      │
├─────────────────────────────────────────┤
│  6. INTEGRATE: Add to codebase          │
├─────────────────────────────────────────┤
│  7. RECORD: Update learnings.json       │
├─────────────────────────────────────────┤
│  8. REPEAT: Loop back to MONITOR        │
└─────────────────────────────────────────┘

Phần Practice (55 phút) ​

Lab 1: Audit TaskFlow Test Suite Health ​

Your Task: Chạy health check trên test suite

Step 1: Identify Signals

bash
cd /Users/todyle/Documents/qa-course

# Signal 1: Check test file status
ls -la tests/
# → Look for: old files, missing files, outdated names

# Signal 2: Check imports
grep -r "require.*models" tests/ | grep -E "old|deprecated|deleted"
# → Find broken references

# Signal 3: Check documentation
ls -la docs/ | grep test
cat docs/testing.md | head -20
# → See if docs match current codebase

# Signal 4: Check learnings
cat learnings.json | jq '.[] | select(.category == "testing")' | head
# → See last update, completeness

# Signal 5: Run tests and check coverage
npm test 2>&1 | tail -10
npm test -- --coverage 2>&1 | grep -E "Lines|Branches|Functions"

Step 2: Document Health Report

markdown
# TaskFlow Test Suite Health Check

## Health Signal Summary

### 1. Documentation Drift
- ✅ Docs updated 2026-04-20 (4 days ago)
- ✅ Test guide matches current structure
- ⚠️ Performance testing section incomplete

### 2. Broken References
- ✅ All imports point to valid files
- ✅ No deprecated modules used
- ⚠️ 1 test file has unused import: `./old-task-manager.js` (line 8)

### 3. Retro Notes (learnings.json)
- ✅ 15 entries total
- ✅ Last update: 2026-04-24 (today)
- ✅ Covers: bugs, patterns, timezones, filtering

### 4. Validation Gaps
- ✅ Happy path tests: 100%
- ⚠️ Edge cases: filterTasks() missing empty filter test
- ⚠️ Error handling: isOverdue() no timezone edge case test

### 5. Quality Gates
- ✅ Last 5 gates: all passed
- ✅ No flaky tests detected
- ✅ Coverage trend: stable 87-89%

## Overall Health: FAIR (🟡)

### Issues Found
1. **P2**: Remove unused import in task.test.js line 8
2. **P3**: Add test case for filterTasks({}) edge case
3. **P3**: Add timezone edge case for isOverdue()

### Recommendations
1. Remove dead imports (quick fix)
2. Add 2 missing test cases (2 hours)
3. Refresh learnings.json with new patterns (1 hour)

Status: READY to evolve (no blockers)

Lab 2: FIX Mode — Repair Broken Test After Code Change ​

Scenario: Feature change requires test update

Original Code:

javascript
// src/models/task.js (ORIGINAL)
class Task {
  constructor(data) {
    this.id = data.id;
    this.title = data.title;
    this.completed = data.completed || false;
  }

  getStatus() {
    return this.completed ? 'DONE' : 'PENDING';
  }
}

Original Test (passing):

javascript
// tests/models/task.test.js
test('getStatus returns DONE when task is completed', () => {
  const task = new Task({ 
    id: 1, 
    title: 'Buy milk', 
    completed: true 
  });
  expect(task.getStatus()).toBe('DONE');
});

Code Change (NEW FEATURE — "tags" field added):

javascript
// src/models/task.js (AFTER "tags" feature)
class Task {
  constructor(data) {
    this.id = data.id;
    this.title = data.title;
    this.completed = data.completed || false;
    this.tags = data.tags || []; // NEW FIELD
    this.state = this.completed ? 'completed' : 'active'; // NEW
  }

  getStatus() {
    // Refactored to use state instead of completed
    return this.state === 'completed' ? 'DONE' : 'PENDING';
  }

  addTag(tag) {
    if (!this.tags.includes(tag)) {
      this.tags.push(tag);
    }
  }
}

Test Now Breaks (because this.state not set before call):

javascript
// ❌ BROKEN TEST (doesn't account for new initialization)
test('getStatus returns DONE when task is completed', () => {
  const task = new Task({ 
    id: 1, 
    title: 'Buy milk', 
    completed: true 
  });
  expect(task.getStatus()).toBe('DONE'); // ❌ FAIL: state is undefined
});

Your Job: Apply FIX Mode

Step 1: Identify Problem

Before: Code sets completed=true → getStatus() checks completed
After: Code sets state=completed → getStatus() checks state
Test issue: Still passing completed=true, but constructor now expects state logic

Step 2: Fix Test

javascript
// ✅ FIXED TEST
test('getStatus returns DONE when task is completed', () => {
  const task = new Task({ 
    id: 1, 
    title: 'Buy milk', 
    completed: true  // Constructor sets state='completed'
  });
  // Now state is properly initialized in constructor
  expect(task.getStatus()).toBe('DONE'); // ✅ PASS
});

// ✅ ADD NEW TEST for tags feature
test('addTag adds tag to task', () => {
  const task = new Task({ 
    id: 1, 
    title: 'Buy milk'
  });
  task.addTag('shopping');
  expect(task.tags).toContain('shopping');
});

test('addTag prevents duplicate tags', () => {
  const task = new Task({ 
    id: 1, 
    title: 'Buy milk'
  });
  task.addTag('urgent');
  task.addTag('urgent');
  expect(task.tags).toEqual(['urgent']); // Only one 'urgent'
});

Step 3: Verify Fix

bash
npm test -- task.test.js
# ✅ All tests pass

Step 4: Record Learning

json
{
  "id": "fix-mode-example",
  "type": "evolution-fix",
  "whatChanged": "Task model added state field, refactored getStatus()",
  "whatBroke": "getStatus() test expected old completed field",
  "howFixed": "Constructor now initializes state based on completed param",
  "prevention": "When refactoring, update related tests same commit",
  "pattern": "Internal refactor + public interface change = test update needed"
}

Lab 3: DERIVED + CAPTURED Mode — New Test from Patterns ​

Scenario: Adding new filtering function; want to derive test from existing pattern

Existing Function + Test:

javascript
// src/models/task.js
function filterTasksByStatus(tasks, status) {
  if (!tasks || tasks.length === 0) return [];
  return tasks.filter(t => t.status === status);
}

// tests/models/task.test.js
describe('filterTasksByStatus', () => {
  it('should return tasks with matching status', () => {
    const tasks = [
      { id: 1, status: 'active' },
      { id: 2, status: 'completed' }
    ];
    const result = filterTasksByStatus(tasks, 'active');
    expect(result).toEqual([{ id: 1, status: 'active' }]);
  });

  it('should return empty array for empty input', () => {
    expect(filterTasksByStatus([], 'active')).toEqual([]);
  });

  it('should return empty array for null input', () => {
    expect(filterTasksByStatus(null, 'active')).toEqual([]);
  });
});

New Function (by product team):

javascript
// src/models/task.js (NEW)
function filterTasksByPriority(tasks, priority) {
  if (!tasks || tasks.length === 0) return [];
  return tasks.filter(t => t.priority === priority);
}

Your Job: DERIVE Test from Pattern

Step 1: Recognize Pattern

Pattern: Filter array by field === value
Guard: Check null/empty
Structure: Same as filterTasksByStatus
Action: Derive test suite by changing field names

Step 2: Create DERIVED Test

javascript
// ✅ DERIVED TEST (same pattern, new field)
describe('filterTasksByPriority', () => {
  it('should return tasks with matching priority', () => {
    const tasks = [
      { id: 1, priority: 'high' },
      { id: 2, priority: 'low' }
    ];
    const result = filterTasksByPriority(tasks, 'high');
    expect(result).toEqual([{ id: 1, priority: 'high' }]);
  });

  it('should return empty array for empty input', () => {
    expect(filterTasksByPriority([], 'high')).toEqual([]);
  });

  it('should return empty array for null input', () => {
    expect(filterTasksByPriority(null, 'high')).toEqual([]);
  });
});

Step 3: Pattern Recognition → CAPTURED Skill

javascript
// ✅ RECOGNIZE: "Filter array by field" pattern exists 3 times now
// 1. filterTasksByStatus
// 2. filterTasksByPriority
// 3. filterTasksByDueDate (might exist)

// AUTO-CREATE SKILL:
// src/utils/filterByField.js
/**
 * Reusable filter function for any field
 * Pattern: Filter array by matching a specific field value
 */
function filterByField(items, fieldName, fieldValue) {
  if (!items || items.length === 0) {
    return [];
  }
  return items.filter(item => item[fieldName] === fieldValue);
}

module.exports = filterByField;

// ✅ REPLACE old code with skill:
const filterByField = require('../utils/filterByField');

function filterTasksByStatus(tasks, status) {
  return filterByField(tasks, 'status', status);
}

function filterTasksByPriority(tasks, priority) {
  return filterByField(tasks, 'priority', priority);
}

// ✅ CAPTURED TEST (test the reusable skill)
describe('filterByField (CAPTURED SKILL)', () => {
  it('should filter by any field', () => {
    const tasks = [
      { id: 1, status: 'active', priority: 'high' },
      { id: 2, status: 'completed', priority: 'low' }
    ];
    
    // Same skill, different fields
    expect(filterByField(tasks, 'status', 'active')).toEqual([tasks[0]]);
    expect(filterByField(tasks, 'priority', 'low')).toEqual([tasks[1]]);
  });

  it('should handle edge cases', () => {
    expect(filterByField(null, 'status', 'active')).toEqual([]);
    expect(filterByField([], 'status', 'active')).toEqual([]);
    expect(filterByField(undefined, 'status', 'active')).toEqual([]);
  });
});

Step 4: Update Learnings

json
{
  "id": "cm-learned-filter-by-field",
  "type": "evolution-captured",
  "pattern": "Filter array by matching field value",
  "createdFrom": ["filterTasksByStatus", "filterTasksByPriority", "filterTasksByDueDate"],
  "frequency": 3,
  "implemented": "src/utils/filterByField.js",
  "testCoverage": "tests/utils/filterByField.test.js",
  "savings": "3 custom functions → 1 reusable utility + 3 thin wrappers",
  "autoCreated": true,
  "timestamp": "2026-04-24T15:00:00Z"
}

Lab 4: Monitoring Loop — Weekly Health Check ​

Automate continuous improvement:

bash
#!/bin/bash
# scripts/weekly-health-check.sh

echo "🏥 TaskFlow QA Health Check — $(date)"
echo "=========================================="

# Signal 1: Test results
echo -e "\n1️⃣ Test Results:"
npm test 2>&1 | tail -5

# Signal 2: Coverage
echo -e "\n2️⃣ Coverage:"
npm test -- --coverage 2>&1 | grep -E "Lines|Branches|Functions|Statements"

# Signal 3: Lint
echo -e "\n3️⃣ Code Quality:"
npm run lint 2>&1 | tail -3

# Signal 4: Type checking
echo -e "\n4️⃣ Type Safety:"
npm run type-check 2>&1 | tail -2

# Signal 5: Health verdict
echo -e "\n5️⃣ Verdict:"
if npm test &>/dev/null && npm run lint &>/dev/null; then
  echo "✅ HEALTHY — All systems green"
else
  echo "🔴 CRITICAL — Issues found, see details above"
fi

Run weekly, commit results to docs/health-reports/

Phần Quiz & Homework ​

Quiz (5 câu) ​

Q1: "Immune system metaphor" so sánh test suite với:

  • A) Bacteria
  • B) ✅ Immune system (detect bugs, remember patterns, evolve)
  • C) Hospital

Q2: 3 modes của cm-skill-evolution là:

  • A) Read, Write, Delete
  • B) ✅ FIX, DERIVED, CAPTURED
  • C) Old, New, Mixed

Q3 (Situation): Code thay đổi, test break. Anh/chị áp dụng mode nào?

  • A) CAPTURED
  • B) ✅ FIX (repair broken test)
  • C) DERIVED

Q4: cm-learning-promoter tự động tạo skill khi pattern xuất hiện bao nhiêu lần?

  • A) 1 lần
  • B) 2 lần
  • C) ✅ 3+ lần

Q5 (Situation): Thấy filterByStatus, filterByPriority, filterByDate có pattern tương tự. Làm gì?

  • A) Ignore, they're different
  • B) ✅ Recognize pattern, extract to reusable skill (CAPTURED)
  • C) Wait for more examples

Homework ​

  1. Conduct Health Check: Run health audit trên project cá nhân/course project. Ghi 5 health signals. Submit health-report.md.

  2. Apply FIX Mode: Tìm 1 test case đang fail hoặc outdated. Apply FIX mode (diagnosis → update → verify). Document process.

  3. Apply DERIVED Mode: Tìm 1 function có test. Tạo similar function. Derive test pattern từ original. Add new test.

  4. Identify CAPTURED Pattern: Scan codebase, tìm code pattern xuất hiện 3+. Propose extracted utility + test.

  5. Create Health Monitoring Script: Viết script (bash/node) để run weekly health check. Include: tests, coverage, lint, type-check.

Tài Liệu Tham Khảo ​

  • Test Suite: tests/
  • Learnings Database: learnings.json
  • Health Reports: docs/health-reports/
  • Skills Registry: src/utils/ (where CAPTURED skills live)
  • Weekly Monitoring: scripts/weekly-health-check.sh

Powered by CodyMaster × VitePress

© 2026 Todyle. All rights reserved.