958009858e
All checks were successful
Deploy SOLUTION_ERP / build-deploy (push) Successful in 5m2s
Adopt AI_INFRA Harness-16 (3 broadcast 2026-06-29) qua 2-workflow mandate: WF1 implement wf_4c63e1bd-99e + WF2 review wf_13e3d35a-023 (PASS 0-blocking). MFE = coverage/retention eval (do nap-vs-nho-vs-dung bang SO DO). DISTINCT voi H6.7 memoryDelta-routing-fidelity (vocab-fork WF1 reviewer bat duoc). Built (em-main single-writer, 0 production code): - scripts/mfe-eval.ps1 deterministic NO-API (ASCII #30, exit 0): LEAD coverage-FIT (token-RANGE, cap live-read) + age-band flag-not-cut + Goodhart-anchor strikes/RCA; SUB per-role coverage do-that (prose->N/A khong 0%); sub-workflow N/A. Smoke 3-tier PASS. - memory-budget.json :mfe config + eval/mfe/ (seed sample-questions stable-id + README) + engine §H + artifact-row + C3-comment + wire session-start §2.1.6 / session-end §L.b(c) opt-in + agents/README S93. Judge layer = SCAFFOLD-only (honest nac). adap-report harness-16-mfe + harness-15-v3 (covered/folded) + email outbox/ai_infra. AS-10 dogfood: WF1 residual-write caught+reverted. State GIU NGUYEN (Mig 59 · 88 · 434 · gotcha 76). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
44 lines
5.0 KiB
JSON
44 lines
5.0 KiB
JSON
{
|
|
"_note": "Harness-16 MFE Branch-A sample-questions SEED (S93, 2026-06-29). IMMUTABLE + time-stamped + anchored by STABLE-ID (RC-sig / gotcha# / AS# / budget-key) -- NEVER by line-number (files live, line-numbers drift). Seeded NOW so age-retention is measurable months later. Questions probe the LEAD must-remember set + a few SUB role-floors. Answers are the KEY FACT only (a judge checks recall/apply, paraphrase-tolerant). DO NOT rewrite existing questions (additive-only, like archive verbatim) -- append new ones with fresh ids.",
|
|
"seeded_date": "2026-06-29",
|
|
"scoring_status": "SCAFFOLD - no scorer wired. Same-session self-grade = MEANINGLESS. Needs (a) age-maturity AND (b) independent cross-session/different-model judge.",
|
|
"questions": [
|
|
{ "id": "Q-mark-09", "anchor": "RC-pqhuy1987-20-06-2026-10-29-09", "tier": "lead",
|
|
"q": "What does the architecture-decision mark assert about how to judge whether a feature is justified?",
|
|
"expect": "Objective criteria (pain / volume / quality), NOT team-size; 'overkill / too-much-for-solo-dev / gut-feeling' = rejected reasoning." },
|
|
{ "id": "Q-mark-11", "anchor": "RC-pqhuy1987-20-06-2026-10-29-11", "tier": "lead",
|
|
"q": "Why is time/age/recency a false proxy for memory-budget and drift decisions?",
|
|
"expect": "Age is same-family as team-size; cap = capacity/refresh-rate not an age-decay knob; drift = rolling baseline not an age window; age-decay cuts good memory = false economy (Goodhart). Applied to MFE: age = FLAG, never a cut." },
|
|
{ "id": "Q-mark-15", "anchor": "RC-pqhuy1987-20-06-2026-23-07-37", "tier": "lead",
|
|
"q": "What is the core principle of the H-15 memory-budget (token-governor)?",
|
|
"expect": "Budget = MINIMUM-to-USE floor (fill Tier-1 hot-feed with real work-state), NOT a ceiling to economize; token-saving = forgetting work. Numbers are the project-owner's authority; em-main executes + reports %." },
|
|
{ "id": "Q-cap-lead", "anchor": "memory-budget.json:token_governor.tier1_hotfeed_tokens.lead_tokens", "tier": "lead",
|
|
"q": "Where does the lead hot-feed token cap live, and may a tool hardcode it?",
|
|
"expect": "Lives ONLY in memory-budget.json (single-source); live-read it (B1 derived-tro-canonical); NEVER hardcode (it moved 60K->200K->220K). MFE READS it, never writes it." },
|
|
{ "id": "Q-as12", "anchor": "AS-12", "tier": "lead",
|
|
"q": "Before an identifier-based data op on prod (lock/seed/migrate-by-email), what must you do first?",
|
|
"expect": "DUMP the target-env table first; do not write the identifier list from CODE/Dev population. A 0-row / -1 assertion => suspect data-mismatch BEFORE code-bug. (gotcha #60 / E-008)" },
|
|
{ "id": "Q-as10", "anchor": "AS-10", "tier": "lead",
|
|
"q": "A sub-agent wrote a tracked file despite being return-only. What is the containment?",
|
|
"expect": "git-diff post-P2 catches it; em-main VERIFIES benign+accurate+placement then keep-if-correct or revert; NOT mechanized (G-015, sub keeps Bash). Defense-in-depth = git-diff + chunk-count." },
|
|
{ "id": "Q-gotcha-30", "anchor": "gotcha #30", "tier": "lead",
|
|
"q": "Why must a PowerShell .ps1 script body be pure ASCII?",
|
|
"expect": "Box-glyphs / Vietnamese literals in a PS 5.1 -File script body mojibake (even via Edit's render-normalize); use ASCII + code-points (e.g. [char]0x2705)." },
|
|
{ "id": "Q-gotcha-53", "anchor": "gotcha #53", "tier": "lead",
|
|
"q": "How is heavy-agent return-truncation mitigated?",
|
|
"expect": "em-main verify-on-disk + proxy-append (the agent often wrote the finding to disk before the empty return); lean memoryDelta return; 529 -> em-main solo fallback, no retry-loop." },
|
|
{ "id": "Q-gotcha-75", "anchor": "gotcha #75", "tier": "lead",
|
|
"q": "Why is a prod data-wipe not durable, and what is the fix?",
|
|
"expect": "Ungated per-code seeders RE-ADD the wiped data every restart; gate the seed behind an env-flag and verify by a REAL restart (not 'data clean right after wipe')." },
|
|
{ "id": "Q-h16-vocab", "anchor": "harness-16 vocab", "tier": "lead",
|
|
"q": "What is the difference between 'memory-fidelity-EVAL' and 'memoryDelta-routing-fidelity'?",
|
|
"expect": "MFE (H-16) = coverage/retention eval (does hot-feed retain the must-remember set). memoryDelta-routing-fidelity (H6.7) = the right delta lands in the right agent-memory under single-writer. Two distinct senses; do not conflate (C3 vocab-fork guard)." },
|
|
{ "id": "Q-sub-reviewer", "anchor": ".claude/agents/reviewer.md", "tier": "sub",
|
|
"q": "What is the reviewer sub-agent strictly forbidden from doing?",
|
|
"expect": "NEVER Edit/Write/commit/push; it produces a PASS/FAIL verdict with file:line, never writes code." },
|
|
{ "id": "Q-sub-implbackend", "anchor": ".claude/agents/implementer-backend.md", "tier": "sub",
|
|
"q": "What is implementer-backend forbidden to touch?",
|
|
"expect": "No FE 2-app (that is implementer-frontend); no test assertions (that is test-specialist); no schema/UX/cross-stack-bug reasoning (em-main solo). Auto-refuses out-of-scope." }
|
|
]
|
|
}
|