Adopt AI_INFRA Harness-16 (3 broadcast 2026-06-29) qua 2-workflow mandate: WF1 implement wf_4c63e1bd-99e + WF2 review wf_13e3d35a-023 (PASS 0-blocking). MFE = coverage/retention eval (do nap-vs-nho-vs-dung bang SO DO). DISTINCT voi H6.7 memoryDelta-routing-fidelity (vocab-fork WF1 reviewer bat duoc). Built (em-main single-writer, 0 production code): - scripts/mfe-eval.ps1 deterministic NO-API (ASCII #30, exit 0): LEAD coverage-FIT (token-RANGE, cap live-read) + age-band flag-not-cut + Goodhart-anchor strikes/RCA; SUB per-role coverage do-that (prose->N/A khong 0%); sub-workflow N/A. Smoke 3-tier PASS. - memory-budget.json :mfe config + eval/mfe/ (seed sample-questions stable-id + README) + engine §H + artifact-row + C3-comment + wire session-start §2.1.6 / session-end §L.b(c) opt-in + agents/README S93. Judge layer = SCAFFOLD-only (honest nac). adap-report harness-16-mfe + harness-15-v3 (covered/folded) + email outbox/ai_infra. AS-10 dogfood: WF1 residual-write caught+reverted. State GIU NGUYEN (Mig 59 · 88 · 434 · gotcha 76). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
8.3 KiB
Adap-report — Harness-16 Memory-Fidelity-EVAL (MFE) → SOLUTION_ERP
TL;DR (nấc honest)
Adopt memory-fidelity-EVAL (MFE) = đo đã-nạp ≠ nhớ-ra ≠ dùng-đúng bằng SỐ ĐO, KHÔNG ước cảm-tính. Deterministic ($0) block = BUILT + WIRED + runtime-smoke S93 (chạy thật, 3-tier, exit 0). Judge layer (recall/apply) = SCAFFOLD-only (seed + stub, chưa scorer — số CHƯA có nghĩa, đúng thiết-kế). Function-floor mục 3-5 của broadcast = ĐẠT; form tự-quyết (SE roster 11-agent). 2-workflow mandate ĐẠT (WF1 implement + WF2 review, run-id bằng-chứng).
Function-floor coverage (broadcast mục 3 — SÀN bắt buộc)
| Floor | SE implementation | Nấc |
|---|---|---|
| (1) Mẫu-số "bắt-buộc-nhớ" deterministic, status-field KHÔNG grep, dedup | mfe-eval.ps1 LEAD: marks Active(-High) (ACTIVE-MARKS.md status-col) + AS-class + Active-Guards (error-ledger.md Verified-col) + recurring-gotchas (gotchas.md recurrence-token). Dedup deterministic by cross-reference (gotcha/guard mà AS-row trỏ #N/AS-N → gộp; merge_guard_to_as toggle). Hiện = 29 item (4 mark + 13 AS + 5 net-guard + 7 net-gotcha). |
✅ executed + WF2 re-derived EXACT |
| (2a) Coverage% (set có vừa ngân-sách hot không) | byte-span → token-RANGE [bytes/4 .. bytes/3.0] vs cap live-read token_governor.tier1_hotfeed_tokens.lead_tokens (220K). Hiện 29 item ~2.5-3.3K tok = 1.5% worst-case → FIT thừa. |
✅ |
| (2b) Effective% (thực-nhớ/dùng — cần trọng-tài) | Judge scaffold (Branch A) — SCAFFOLD-only, số CHƯA có nghĩa (last-resort, đúng broadcast). | 🟡 scaffold |
| (2c) phân-tầng-tuổi (age-band) | FLAG old-but-still-required, KHÔNG cắt (mark RC-…10-29-11). drop chỉ khi status đổi. |
✅ |
| (3) Neo Goodhart (đối-chiếu điểm-nhớ vs lỗi-tái-phát thật, KHÔNG self-grade) | in strikes_total=21 + RCA_entries=9 (error-ledger) cạnh coverage; strikes tăng + coverage cao = điểm nói dối (WARN cross-run via .mfe-state.json). |
✅ |
| (4) Per-tier + opt-in command param | -Tier lead/sub/all + -Judge. Wire opt-in session-start §2.1.6 + session-end §L.b(c) (param eval, no-param = như cũ). Baseline đầu + retention cuối → chênh = rot trong-phiên. |
✅ executed-file (VERIFIED-pending-restart) |
| (5) Quyết-định 2-ca (thiếu-CHỖ→TĂNG budget / rot→SẮP-XẾP-LẠI) | engine §H3 + README. 🔴 con-số budget = quyền anh (owner-authority); MFE chỉ báo. | ✅ |
Sub-branch (mfe-update + cd-judge-sub-update)
| Floor | SE | Nấc |
|---|---|---|
| trần đọc-LIVE single-source (không hardcode) | lead_tokens live-read từ budget.json (B1 derived-trỏ-canonical; số đã 60K→200K→220K nên hardcode = rot). |
✅ WF2-proven |
| source-vs-cap = N/A cho sub-tier | SUB-WORKFLOW (ephemeral) = N/A by-design (kênh ký-ức = memory-pack lead bơm). | ✅ |
| Branch A judge framework (sample-q + stub scorer + reader, no-API, opt-in, cadence) | eval/mfe/sample-questions.json (12 câu stable-id anchored RC/gotcha#/AS#, immutable, time-stamped — gieo NGAY) + -Judge stub (scorer RỖNG). |
🟡 scaffold |
| Branch B per-role sub-coverage ĐO-THẬT (denom role-file anti-pattern, numer diary, content-word KHÔNG leading-verb) | mfe-eval.ps1 SUB: denom = numbered anti-pattern/NEVER trong agents/<role>.md; numer = diary agent-memory/<role>/MEMORY.md (KHÔNG project/user MEMORY.md — bẫy H15-v3); match = exact-id OR ≥2 content-word sau stop-list. ĐO-THẬT: impl-backend 94% · impl-frontend 85% · reviewer 88%. |
✅ executed + WF2 re-derived |
| bẫy leading-verb (verb-only ăn 9/9 giả) | stop-list (EN+VN verb) + NFD-strip-diacritics → verb-only = 0 content-word = MISS. WF2 adversarial-probed: "NEVER commit push"→0 word→no false-max. | ✅ probed |
| vai prose-only / phù-du = N/A có lý-do (KHÔNG 0%) | den=0 → "N/A (no numbered/NEVER block)" KHÔNG "0%" (div-by-0 guarded). 3 role prose-only (database-agent/harvest/tooling). | ✅ (refinement em-main sau smoke) |
Tailored (form tự-quyết — SE-specific)
- Denominator = chính governance của SE (4 mark Active-High + 13 AS-class + Active-Guards + recurring-gotchas) — trùng
value_protect.patternslà CỐ Ý (cùng tập giá-trị-cao, markRC-…10-29-11), KHÔNG double-count (dedup by AS cross-reference). - Vocab-fork resolved (WF1 reviewer bắt — BLOCKING): "memory-fidelity" ĐÃ là H6.7
memoryDelta-routing-fidelity. → đặt tên MFE = memory-fidelity-EVAL ở 4 site, ghi alias-map §H, KHÔNG seed$aliasSets(2-khái-niệm-1-từ, seed sẽ flag-sai). C3-comment chống maintainer seed nhầm. - READ-ONLY scoped honest: MFE đọc
token_governor(single-source) KHÔNG ghi; CÓ ghi 1 file.mfe-state.json(strikes cross-run) —honest_caveats.read_only_scopenói rõ "KHÔNG phải writes-nothing".
2-workflow mandate (Harness-9) — ĐẠT
- WF1 IMPLEMENT
wf_4c63e1bd-99e(3 agent: lead-denominator-spec + sub-role-denominator + cross-harness-map) → ground-truth. Lesson: sub-agent #53-truncate (return rỗng) NHƯNG ghi finding xuống diary TRƯỚC → recover-from-disk. AS-10 containment fire: 3 WF1-agent self-write agent-memory (return-only violation) → em-main git-diff bắt + revert (giữ cicd-monitor PE-harvest hợp-lệ). Dogfood: memory-governance hoạt-động đúng giữa lúc adopt memory-eval. - WF2 REVIEW
wf_13e3d35a-023(3 lane: code-gate + cross-harness + honesty) = PASS 0-blocking. RETURN-ONLY honored (0 tracked write — instruction-fix sau WF1). code-gate re-derived denom 29 to-the-digit + leading-verb-trap adversarial-probed-defeated. honesty: 5 floor (a-e) đủ + live numbers KHỚP §H prose (S81 stale-config trap KHÔNG tái diễn).
Honest caveats (broadcast mục 5 — đưa vào doc, không giấu)
- Token = RANGE (char/4 ≠ token thật; VN ~3.0-3.5 byte/tok). FIT dùng worst-case bytes/3.0. CHƯA đo bằng tokenizer thật của model → hệ-số còn bất-định; 1.5% nằm xa trần nên kết-luận FIT an-toàn, nhưng đuôi-sổ phình thì theo-dõi.
- Judge = SCAFFOLD, số ĐẾN SAU. recall/dùng-đúng CHƯA có nghĩa tới khi (a) sample-q đủ-chín-tuổi VÀ (b) trọng-tài độc-lập khác-phiên. Tự-chấm-cùng-phiên = vô-nghĩa. KHÔNG nhận "đã đo độ-nhớ".
- Age = FLAG không cắt (mark
RC-…10-29-11). - Deterministic = executed + runtime-smoke, KHÔNG "đã đo retention toàn-hệ". LEAD = coverage/FIT (space), KHÔNG retention.
- +1 conservative dedup bias (WF2 minor): guard
run_in_backgroundmap khái-niệm AS-5 nhưng Counters ghilooks-frozen(no AS-5 ref) → giữ net-extra (literal cross-reference, đúng nguyên-tắc deterministic). Tighten = data-fix ledger (defer).
Evidence
- Artifact:
scripts/mfe-eval.ps1(302 dòng, NO-API, ASCII #30, exit 0) ·.claude/agent-memory/memory-budget.json:mfeconfig ·eval/mfe/{sample-questions.json,README.md}· engine§H+ artifact-row ·governance-detectors.ps1C3-comment · wire 2 cadence ·agents/READMES93 ·.gitignorestate-file. - Smoke: 3-tier exit 0; denom 29/220K=1.5%; SUB impl-backend 94%/impl-frontend 85%/reviewer 88%; 3 prose-role N/A; Goodhart strikes 21/RCA 9.
- 0 production code. State THẬT GIỮ NGUYÊN (Mig 59 · 88 bảng · 434 test · gotcha 76 · bundle admin
CDn40vpV/userCoPmPBg7). - ⚠️ restart CLI (command
.mdno hot-reload → MFE opt-in runtime).
Brutal-honest / phản-biện (§M)
KHÔNG có. Broadcast fit SE (SE có đủ governance-corpus làm denominator + 11-agent roster làm Branch B). 8 broadcast cũ chưa-match-tên (checklist-*/v2.0/h10-detector/h14-adopt) = verify-covered S72/S75/S79/S81 (name-drift, không re-apply). H15-v3-sub-load-channels = covered-S83 + fold-vào-H16 (report riêng).