Buổi 18: Evidence-Based QA — Không Claim Nếu Chưa Prove

Phần Lý Thuyết (35 phút)

Iron Law #2: Quality Assurance

"NO CLAIMS WITHOUT FRESH VERIFICATION OUTPUT"

Claim gì không có bằng chứng = tự đánh lừa. Đặc biệt với QA:

"Tests pass" ← cần thấy test output: ✅ 0 failures
"Bug fixed" ← cần thấy test case gốc pass
"No regression" ← cần full test suite green

Quality Gate Function: IDENTIFY → RUN → READ → VERIFY → CLAIM

Luôn tuân thủ 5 bước này. Không được bỏ qua bất kỳ bước nào.

Step 1: IDENTIFY

Xác định chính xác cái gì cần verify:

Cái claim nào? ("tests pass", "no bugs", "code is safe")
Cái gì sẽ chứng minh? (test output, coverage, static analysis)
Tool nào? (vitest, eslint, type checker)

Claim: "Task model is working correctly"
Verify with: npm test -- task.test.js
Expected output: ✅ All tests passed, 0 failures

Step 2: RUN

Thực thi lệnh, nhận kết quả:

bash

npm test -- src/models/task.test.js

Step 3: READ

Đọc TOÀN BỘ output, không skip:

PASS  src/models/task.test.js (234ms)
  Task
    ✓ isOverdue() returns true when 2 days past due (5ms)
    ✓ isOverdue() returns false when due in future (3ms)
    ✓ filterTasks() returns empty array on null input (2ms)
    ✓ filterTasks() filters by status correctly (4ms)

Test Suites: 1 passed, 1 total
Tests:       4 passed, 4 total

Ghi chú từng chi tiết:

Test Suite nào pass?
Bao nhiêu tests passed vs. total?
Có skipped tests không?
Performance: bao lâu?

Step 4: VERIFY

So sánh output với expectation:

Expected: ✅ All tests passed (4 passed, 0 failed)
Actual:   ✅ Test Suites: 1 passed, Tests: 4 passed, 4 total
Match? YES → Proceed to Step 5

Nếu mismatch:

NOT "should be fine"
Investigate: Why test failed? Different output?

Step 5: CLAIM

NGAY LẦN ĐÓ, đưa ra claim có bằng chứng:

✅ CLAIM: "Task model is working correctly"
Evidence: Test output shows 4 tests passed, 0 failures
Timestamp: 2026-04-24 14:30:00
Command: npm test -- src/models/task.test.js

Common Failures Table — Bẫy Thường Gặp

Scenario	❌ WRONG	✅ CORRECT	Why
"Tests pass"	Trust test description	Run tests, paste output	Description can be wrong
"Bug fixed"	Look at code	Run original failing test, see it pass	Code looks right ≠ actually works
"No regression"	Run 1 test	Run full suite (npm test)	One test pass ≠ no regression
"Code is safe"	Assume no bugs	Run linter + type check + tests	Multiple checks needed
"Deploy ready"	1 person approved	Verification dashboard + team review	Single approval insufficient
"Performance OK"	"Should be fast"	Benchmark: baseline vs. new, compare	Need numbers
"No breaking changes"	"I think it's compatible"	Test old + new together	Need proof

Red Flags: Phát Hiện Claim Ẩu

Nếu nghe được những câu này, dừng lại và yêu cầu evidence:

Red Flag	Nguy Hiểm	Cách Chối
"should work"	Speculative	"Let me verify with test output"
"probably fine"	Unverified	"Run the test suite first"
"seems to be fixed"	No evidence	"Show me the failing test passing"
"I believe it's ready"	Personal opinion	"Show me the verification dashboard"
"Tests usually pass"	Not current	"What's the latest run result?"
"It works on my machine"	Not reproducible	"Does it pass in CI/CD?"
"No issues found"	Incomplete scan	"What tools ran? Show output"
"Trust me, it's safe"	No verification	"Show vulnerability scan results"

Update Working Memory Protocol

Sau mỗi quality gate pass, cập nhật learnings:

json

{
  "gateId": "task-model-verification",
  "claim": "Task model tests pass",
  "verifiedAt": "2026-04-24T14:30:00Z",
  "evidence": {
    "tool": "vitest",
    "command": "npm test -- src/models/task.test.js",
    "result": "4 passed, 0 failed",
    "duration_ms": 234
  },
  "confidence": "high",
  "nextGate": "integration tests",
  "assumptions": [
    "Test data uses current date",
    "Timezone is UTC",
    "Database connection stable"
  ]
}

Phần Practice (50 phút)

Setup: Evidence Dashboard

Tạo file docs/evidence-report.md để tracking all verifications:

markdown

# QA Evidence Report — TaskFlow
Generated: 2026-04-24

## Unit Tests

| Module | Command | Result | Evidence |
|--------|---------|--------|----------|
| Task Model | `npm test -- src/models/task.test.js` | ✅ 4/4 passed | Output below |
| Task Manager | `npm test -- src/managers/taskManager.test.js` | ✅ 8/8 passed | Output below |
| Database | `npm test -- src/db/connection.test.js` | ✅ 2/2 passed | Output below |

## Integration Tests

| Scenario | Command | Result | Evidence |
|----------|---------|--------|----------|
| Create + Query Task | `npm test -- e2e/task-flow.test.js` | ✅ PASS | Output below |

## Quality Metrics

| Metric | Value | Target | Status |
|--------|-------|--------|--------|
| Line Coverage | 87% | >80% | ✅ PASS |
| Branch Coverage | 75% | >70% | ✅ PASS |
| Linting Issues | 0 | 0 | ✅ PASS |
| Type Errors | 0 | 0 | ✅ PASS |

## Deployment Readiness

- [ ] All unit tests pass
- [ ] Integration tests pass
- [ ] Coverage >80%
- [ ] No linting errors
- [ ] Security scan passed
- [ ] Performance baseline met

Practice 1: Claim "Tests Pass" — Nếu Có Bằng Chứng

Scenario: Developer claim "All tests pass, ready to deploy"

Your job: Verify using Gate Function

Step 1: IDENTIFY

Claim: "All tests pass"
Verify: Run full test suite
Tool: npm test
Expected: 0 failures

Step 2: RUN

bash

cd /Users/todyle/Documents/qa-course
npm test 2>&1 | tee test-output.log

Step 3: READ Output

PASS  src/models/task.test.js
  Task
    ✓ constructor initializes correctly
    ✓ isOverdue() works
    ✓ getStatus() returns correct status
    ...

PASS  src/managers/taskManager.test.js
  TaskManager
    ✓ filterTasks() handles null
    ✓ filterTasks() filters by status
    ...

Test Suites: 2 passed, 2 total
Tests:       12 passed, 12 total
Coverage: 87% lines, 75% branches
Duration: 450ms

Step 4: VERIFY

Expected: All tests pass ✅
Actual: Test Suites 2/2 ✅, Tests 12/12 ✅
Match: YES ✅

Step 5: CLAIM

✅ VERIFIED CLAIM: "All tests pass"
Evidence file: docs/evidence-report.md
Test output: 12/12 passed, 0 failed
Coverage: 87% (>80% threshold) ✅
Ready for next gate: integration tests

Practice 2: Claim "Bug Fixed" — Nếu Failing Test Giờ Pass

Scenario: Developer say "I fixed the isOverdue() bug"

Your job: Verify using Gate Function

Step 1: IDENTIFY

Claim: "isOverdue() bug is fixed"
Original failing test: task.test.js line 45
Verify: Run that specific test

Step 2: RUN — Before vs. After

BEFORE (before fix):

bash

npm test -- src/models/task.test.js -t "isOverdue"

FAIL  src/models/task.test.js
  ❌ isOverdue() returns true when 2 days past due
    Expected: true
    Received: false

AFTER (after developer's fix):

bash

npm test -- src/models/task.test.js -t "isOverdue"

PASS  src/models/task.test.js
  ✓ isOverdue() returns true when 2 days past due
  ✓ isOverdue() returns false when due in future

Step 3: READ

Before: test failed ❌
After: test passes ✅
Same test, different output

Step 4: VERIFY

Before: FAIL ❌
After: PASS ✅
Cause of fix: Extract ONE_DAY_MS constant (reviewed code)
Hypothesis confirmed: magic number was root cause

Step 5: CLAIM

✅ VERIFIED CLAIM: "isOverdue() bug is fixed"
Root cause: Magic number 86400000
Fix applied: const ONE_DAY_MS = 24*60*60*1000
Test evidence: Previously failing test now passes
Related tests: 3/3 isOverdue tests pass, 0 failures

Practice 3: Claim "No Regression" — Full Suite Green

Scenario: Merged code to main. "Should have no regression"

Your job: Verify full test suite

Step 1: IDENTIFY

Claim: "No regression after merge"
Verify: Run full test suite on new code
Baseline: Previous passing test count

Step 2: RUN

bash

npm test 2>&1 | tee regression-check.log

Step 3: READ FULL OUTPUT

Test Suites: 4 passed, 4 total
Tests:       24 passed, 0 failed, 0 skipped
Snapshots:   0 total
Duration:    1200ms

Coverage summary:
Lines:    89%
Branches: 78%
Functions: 88%

Step 4: VERIFY

Baseline (previous): 24 passed
Current: 24 passed
Change: 0 new failures, 0 skipped
Regression: NO ✅

Step 5: CLAIM

✅ VERIFIED CLAIM: "No regression detected"
Test suites: 4/4 passed ✅
Total tests: 24/24 passed ✅
Coverage maintained: 89% lines (was 89%)
Timestamp: 2026-04-24 14:35:00
Status: Safe to merge ✅

Create QA Summary Report for TaskFlow

Template để submit report:

markdown

# TaskFlow QA Summary Report
Date: 2026-04-24
Version: 1.2.0
Prepared by: [Your Name]

## Executive Summary
Status: ✅ READY FOR PRODUCTION
- All tests passing (24/24)
- Coverage 89% (exceeds 80% target)
- No security issues found
- No linting errors
- Performance within baseline

## Verification Details

### Unit Test Results

npm test → Task Model: 4 passed → Task Manager: 8 passed → Database: 12 passed Total: 24 passed, 0 failed


### Coverage Report

Lines: 89% (target: >80%) ✅ Branches: 78% (target: >70%) ✅ Functions: 88% (target: >80%) ✅


### Known Issues & Limitations
None critical. Minor items in backlog:
- Snapshot tests need update (non-critical)
- Performance test slow on CI (investigating)

### Blockers for Deployment
NONE ✅

### Recommendations
1. Monitor performance metrics in production
2. Set up automated alert for test coverage drops
3. Add integration test for multi-user scenarios

### Sign-off
✅ QA approved for deployment to staging
Confidence level: HIGH

Phần Quiz & Homework

Quiz (5 câu)

Q1: Iron Law #2 là gì?

A) Tests always pass
B) ✅ NO CLAIMS without fresh verification output
C) Code is always correct

Q2: Claim "tests pass" cần verify bằng gì?

A) Read test file
B) ✅ Run tests, read output, check 0 failures
C) Trust developer

Q3 (Red Flag): Anh/chị nghe "should work, probably fine". Làm gì?

A) Accept and move on
B) ✅ Stop and ask for test output verification
C) Trust the person

Q4: Quality Gate có mấy steps?

A) 3 steps
B) ✅ 5 steps (IDENTIFY → RUN → READ → VERIFY → CLAIM)
C) 7 steps

Q5 (Situation): Developer nói "Bug fixed", nhưng không run test. Anh/chị sẽ:

A) Trust the code looks right
B) ✅ Ask to run original failing test, verify it passes now
C) Approve immediately

Homework

Create Evidence Dashboard: Tạo docs/evidence-report.md theo template, fill với real test results từ dự án (nếu có).
Verify 3 Claims:
- Claim: "Function X works" → Run test → Paste output
- Claim: "No bugs in module Y" → Run linter → Paste output
- Claim: "Code is type-safe" → Run type checker → Paste output
Red Flag Spotting: Tìm 3 claim trong codebase comments hoặc commit messages không có evidence. Ghi lại.
Record Verification: Ghi 1 verification anh/chị làm tuần này vào learnings.json (claim, tool, evidence, result).

Tài Liệu Tham Khảo

Test Suite: tests/
Coverage Report: coverage/lcov-report/index.html
TaskFlow Codebase: src/
QA Checklist: docs/qa-checklist.md

Buổi 18: Evidence-Based QA — Không Claim Nếu Chưa Prove ​

Phần Lý Thuyết (35 phút) ​

Iron Law #2: Quality Assurance ​

Quality Gate Function: IDENTIFY → RUN → READ → VERIFY → CLAIM ​

Step 1: IDENTIFY ​

Step 2: RUN ​

Step 3: READ ​

Step 4: VERIFY ​

Step 5: CLAIM ​

Common Failures Table — Bẫy Thường Gặp ​

Red Flags: Phát Hiện Claim Ẩu ​

Update Working Memory Protocol ​

Phần Practice (50 phút) ​

Setup: Evidence Dashboard ​

Practice 1: Claim "Tests Pass" — Nếu Có Bằng Chứng ​

Practice 2: Claim "Bug Fixed" — Nếu Failing Test Giờ Pass ​

Practice 3: Claim "No Regression" — Full Suite Green ​

Create QA Summary Report for TaskFlow ​

Phần Quiz & Homework ​

Quiz (5 câu) ​

Homework ​

Tài Liệu Tham Khảo ​

Buổi 18: Evidence-Based QA — Không Claim Nếu Chưa Prove

Phần Lý Thuyết (35 phút)

Iron Law #2: Quality Assurance

Quality Gate Function: IDENTIFY → RUN → READ → VERIFY → CLAIM

Step 1: IDENTIFY

Step 2: RUN

Step 3: READ

Step 4: VERIFY

Step 5: CLAIM

Common Failures Table — Bẫy Thường Gặp

Red Flags: Phát Hiện Claim Ẩu

Update Working Memory Protocol

Phần Practice (50 phút)

Setup: Evidence Dashboard

Practice 1: Claim "Tests Pass" — Nếu Có Bằng Chứng

Practice 2: Claim "Bug Fixed" — Nếu Failing Test Giờ Pass

Practice 3: Claim "No Regression" — Full Suite Green

Create QA Summary Report for TaskFlow

Phần Quiz & Homework

Quiz (5 câu)

Homework

Tài Liệu Tham Khảo