solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md

CI/CD Monitor Agent — Persistent Memory

Persistent diary cross-session. Auto-injected first 200 lines / 25KB at spawn. Update BEFORE every stop. Curate when > 25KB.

---

🎯 Role baseline

Read-only CI/CD pipeline + post-deploy verifier for SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch. Output: PASS/FAIL verdict + evidence under 500 words. Spawn cost ~150K tokens — trade-off để catch fail tự động không phụ thuộc em main nhớ verify.

---

🚨 Recurring CI/CD bug patterns (catch with priority)

Gotcha #39 — act_runner github.com TCP timeout

• Symptom: CI run hang ở "Set up job" → timeout 21s, run stays "queued" forever
• Verify: log line Error: dial tcp ... github.com:443 ... i/o timeout
• Fix: manual checkout bypass đã hardcode trong .gitea/workflows/deploy.yml (run #108/#109), pass at #110. KHÔNG revert. Nếu pattern returns → escalate em main check VPS network

Gotcha #40 — npm cache tsc not found

• Symptom: build_fe_admin fail sau khi enable cache: npm ở actions/setup-node@v4
• Verify: log line sh: tsc: command not found hoặc npm error code ETIMEDOUT
• Fix: DISABLED npm cache rolled back ở a21790d. KHÔNG re-enable. Build time chấp nhận ~3 min thay vì optimize

Gotcha #41 — paths-ignore docs-only skip

• Symptom: Commit code thật mà CI không trigger (run list không có entry mới)
• Verify: git diff --name-only HEAD~1 HEAD vs paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**']
• Fix: Nếu commit có code thật bị skip nhầm → check pattern conflict. Nếu commit chỉ docs → expected behavior (saving ~9 min deploy / commit MD-only). Discovery #3 cumulative 4× S23 t3-t6: Gitea evaluates push range commits (not just tip) when at least 1 commit has non-ignored files — anomaly BENEFICIAL catches verify gate.

Gotcha #25 — IIS WebSocket / module exclusion

• Symptom: notification-hub/negotiate returns 401 hoặc 404 prod (FE SignalR connect fail)
• Verify: curl -X POST https://api.solutions.com.vn/notification-hub/negotiate → non-200
• Fix: IIS WebSocket module enable trong web.config của site api.solutions.com.vn (skill iis-deploy-runbook)

Gotcha #48 — Multi-Changelog.Add SQLite tie-break (S25 lesson)

• Symptom: Tests using OrderByDescending(CreatedAt).First() pick wrong audit row khi same SaveChangesAsync transaction có 2+ Changelog.Add() entries cùng CreatedAt (SQLite frozen-clock).
• Verify: Test fail message Expected changelog.ContextNote not to be <null> post BE refactor adding new audit Add() alongside existing LogTransitionAsync chain.
• Fix: Discriminator filter: .Where(c => c.Summary!.Contains("Chuyển phase")) OR .Where(c => c.EntityType == X) BEFORE OrderByDescending. Pattern verified Plan AB Chunk A2 (Run #216 PASS).
• Side benefit: CI test gate catches BEFORE prod deploy → bro UAT spared broken audit. UAT mode skip-test pattern STILL RISKY khi refactor > 100 LOC touches existing test paths.

Deploy ship verification — bundle hash unchanged

• Symptom: commit push success + Gitea action success + status PASS, nhưng prod không có thay đổi visible (user UAT báo "đã deploy mà không thấy")
• Root cause candidates:
  • IIS app pool chưa recycle → giữ assembly cũ trong memory
  • NSSM service script không copy file đúng folder
  • Browser cache (rare nếu Vite hash chuẩn)
• Verify: curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[a-z0-9]+\.js' — hash giữ nguyên = ship fail
• Fix: SSH vietreport-vps "Restart-WebAppPool admin.solutions.com.vn" + recheck bundle hash

Migration drift prod vs repo

• Symptom: Latest mig trong repo (vd Mig 27) nhưng prod chưa có (DbInitializer startup fail)
• Verify: Compare ls Migrations/*.cs vs sqlcmd ... __EFMigrationsHistory
• Fix: Check Program.cs startup hook app.MigrateDatabase() còn không + app pool recycle. Hoặc manual dotnet ef database update --connection prod qua SSH

---

📋 5-stage checklist (apply EVERY run)

Stage 0: Local RAG infrastructure status (S27 NEW — post NSSM Service mode)

• Get-Service Qdrant → must show Status=Running, StartType=Automatic (NSSM Windows Service)
• Nếu Stopped/Paused → Start-Service Qdrant (Admin) hoặc & "D:\.claude-rag\scripts\fix-service-start.ps1" elevated
• Invoke-RestMethod http://localhost:6333/healthz -TimeoutSec 3 verify HTTP

Stage 1: Push happened + filter check

• git log -1 --format='%H %s' — latest commit
• git log origin/main..HEAD — must be empty (synced)
• git diff --name-only HEAD~1 HEAD vs paths-ignore — nếu chỉ docs → SKIPPED-DOCS

Stage 2: Gitea Actions poll (max 10 iter × 60s)

• API: https://git.baocaogiaoduc.vn/api/v1/repos/vietreport-admin/solution-erp/actions/tasks?limit=5 (NOT /runs — 404)
• Match head_sha == $commitSha → get runId
• Status: queued / in_progress / completed
• Conclusion (when completed): success / failure / cancelled / timed_out
• Discovery S21 t5: Gitea API task table caches updated_at stale (~2 min behind reality) — cross-check VPS file mtime nếu time-sensitive.

Stage 3: Test gate verify (Domain 58 + Infra 53 baseline = 111 PASS)

• Logs grep: Passed: line per stage
• Phase 9 UAT exception: test count may be lower nếu em main skip per chunk (memory feedback_uat_skip_verify) — NOT a failure
• Delta from baseline → report

Stage 4: Post-deploy live verify (if SUCCESS)

• Auth login → bearer (admin + nv.test for non-admin gotcha #44 check). Token field = accessToken (NOT token). Route = /api/auth/login (NOT /api/v1/auth/login 404)
• 3-5 endpoint smoke 2XX expected (include endpoint mới trong commit)
• FE bundle hash 2 app changed (compare pre vs post)
• SignalR negotiate (gotcha #25 — if commit relates notification)
• EF migration latest prod == latest repo
• NEW Stage 4.6 (S29 Plan B CRITICAL): sqlcmd seed sample data verify post-deploy — KHÔNG chỉ check schema/Mig table. Phải verify seed data populated khi feature cần sample (V2 workflow Drafter dropdown, etc.). Pattern: sqlcmd ... -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'" → 0 rows post-deploy = seed GATE BLOCKED → escalate gotcha #51 INFRASTRUCTURE vs DEMO seed分类.
• Discovery #4 (S23 t6 Plan P): ASP.NET Core 10 record types với enum fields cần numeric input unless JsonStringEnumConverter registered. SOLUTION_ERP API has NO converter — FE × 2 correctly sends numeric (WorkflowReturnMode = { OneLevel: 1, OneStep: 2, Assignee: 3, Drafter: 4 }).
• Discovery #5 (S25 t3 Plan AC): sqlcmd Windows-auth via ssh requires \\\\SQLEXPRESS 4-backslash escape; \\SQLEXPRESS produces 0 output silently.
• Discovery #6 NEW (S29 Plan B CICD): INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/SampleWorkflowsV2) MUST always run — NOT inside if (!demoSeedDisabled). DEMO seed (DemoUsers/DemoContracts/DemoPE) OK gated. Anti-pattern: Implementer mirror PE V2 seed pattern (which IS gated) for Contract V2 → V2 path BLOCKED prod khi DemoSeed:Disabled=true. Fix: hoist SeedSampleContractWorkflowV2Async out of gate (Run #232 PASS).

Stage 5: Report PASS/FAIL with evidence + MEMORY.md update

---

⚠️ Anti-patterns observed (DO NOT)

1. ❌ Push fix code — READ only, escalate to em main
2. ❌ Speculate fail cause without log evidence
3. ❌ Skip post-deploy live verify khi SUCCESS — bundle hash là biggest catch
4. ❌ Skip MEMORY.md update
5. ❌ Poll forever (max 10 iter ~10 min timeout)
6. ❌ Auto-rollback — escalate với recommendation, KHÔNG tự chạy
7. ❌ Verify khi commit docs-only — SKIPPED-DOCS + return ngay (per Discovery #3 anomaly note)

---

🧠 SOLUTION_ERP CI/CD essentials

• Gitea: https://git.baocaogiaoduc.vn/vietreport-admin/solution-erp
• Workflow: .gitea/workflows/deploy.yml (test gate 2 step + build BE + build FE × 2 + deploy)
• Path filter: paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**'] (gotcha #41 + Discovery #3 anomaly)
• Prod URLs: api / admin / eoffice .solutions.com.vn
• SSH VPS: ssh vietreport-vps (user=Administrator, key=id_ed25519)
• DB prod: .\SQLEXPRESS / SolutionErp / vrapp user. Connection string fallback C:\inetpub\solution-erp\api\appsettings.Production.json khi $env:PROD_DB_PASSWORD empty local (S21 t5 discovery).
• Tests baseline: 120/120 PASS (58 Domain + 62 Infra = baseline 53 + 9 BW1-BW7 Plan C Contract V2 ApproveV2 BW1 happy path + BW2 terminal + BW3 skipToFinal F2 + BW4 outsider Forbidden + BW5 wrong ApplicableType + BW6abc UPSERT Cascade + BW7 V1 fallback). S33 Run #350 PASS confirmed delta +9. Pre-S33: 111/111 unchanged S25→S32.
• Mig latest repo: Mig 34 20260526110207_AddEmployeeProfiles (S33 Plan B G-H1 Phase 10.1 — 7 Hrm entity tables + EmployeeCodeSequences atomic NV/{YYYY}/{D4}). Prev Mig 33 AddContractLevelOpinions (S29 Plan B). Prev Mig 31 RefactorSkipToFinalToApproverLevel (S23 t1 Plan K) — F2 swap Users → ApprovalWorkflowLevels per-Approver-slot, preserved.
• Mig latest prod: sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC TOP 5
• Bearer test:
  • Admin: admin@solutions.com.vn / Admin@123456 (full)
  • UAT non-admin: nv.test@solutions.com.vn / TestUser@123456 (Drafter CCM — verify gotcha #44 silent 403 patterns)

---

🔑 Critical config (gotcha cross-ref)

• Node CI pin: 20.x (memory feedback_node_cicd — bài học NamGroup)
• MediatR pin: 12.4.1 (gotcha #1)
• Swashbuckle pin: 6.9.0 (gotcha #2)
• act_runner: manual checkout bypass github.com (gotcha #39)
• npm cache: DISABLED (gotcha #40 — KHÔNG re-enable)

Flag commit nếu thấy <PackageReference Include="MediatR" Version="14... hoặc cache: npm tái xuất hiện.

---

🎯 Per-NV admin opt-in wire surface — 10-point checklist (cumulative S22+5 → S23 t6)

Cross-ref memory feedback_per_nv_permission_scope.md. Future per-NV/per-Level refactor MUST verify:

1. Domain entity field
2. EF config HasDefaultValue(false)
3. Migration 3-file rule (Up + Designer + Snapshot)
4. Service handler read field
5. Domain DTO + Application DTO mirror
6. Designer FE checkbox inline (admin)
7. Admin overview AwLevelDto record + ToDto ctor
8. CreateAwLevelInput record + Update mutation handler
9. Lookup discrimination in handler (FirstOrDefault ADD ApproverUserId == actorId filter + admin fallback)
10. Controller body record mirror count check ([FromBody] record param count = Command record param count)

Bug latency observed when miss points 9-10: 2-3 days prod silent (Mig 28-29 deploy → S23 t4-t6 catch). Prophylactic codebase scan recommended: grep -n "FirstOrDefault.*Order.*==" *.cs after OR-of-N schema refactor.

---

📊 Run stats baseline (cumulative)

• Build time BE (test_domain + test_infra + build_be): ~90s baseline
• Build time FE × 2 app: ~60s baseline mỗi app
• Deploy NSSM + IIS recycle: ~30s
• Total CI run time: ~3 min code commit / 0s docs-only commit
• Trend trigger: nếu run time > 5 min → escalate (cluster network slow hoặc dependency bloat)
• Bundle size baseline: fe-admin ~800KB gz / fe-user ~750KB gz (Vite production build)

---

📅 Recent runs (FIFO — slim post-curate 2026-05-22)

• 2026-05-26 18:19-18:23 — Run #350 sha=48a99e1 VERDICT=PASS 3m38s (S33 Plan B G-H1 Mig 34 EmployeeProfile + Plan C BW1-BW7 test bundle): Push range 5400983..48a99e1 4 commits 34 files: (1) 1bc6b70 docs drift (3 file MD — match docs/** + **/*.md paths-ignore) + (2) b3444a3 3 MEMORY agent (match **/*.md) + (3) 0605f19 Plan C +4 test file +9 [Fact]+Theory instances (NOT ignored → CI trigger) + (4) 48a99e1 Mig 34 + 17 entity/config new + 7 modified (NOT ignored → CI trigger). Per Discovery #3 anomaly Gitea per-push trigger when ≥1 commit non-ignored → entire push runs. Stage results ALL PASS (Run status=success authoritative — Gitea task API conclusion=None even on success, do NOT confuse): test_domain 58/58 + test_infra 62/62 (=53 baseline +9 BW: BW1 happy Cấp 1→2 / BW2 terminal DaPhatHanh + gen mã / BW3 skipToFinal F2 admin opt-in / BW4 outsider ForbiddenException / BW5 wrong ApplicableType / BW6a UNIQUE composite / BW6b UPSERT 1 row / BW6c Cascade delete / BW7 V1 fallback ConflictException) + build_be (Mig 34 compile OK +6555 LOC) + build_fe_admin + build_fe_user (unchanged — no FE in push) + deploy NSSM IIS recycle. Post-deploy verify ALL PASS (Stage 4 + 4.6 sqlcmd): auth login admin 200 + 4 endpoint smoke 200/200/200/200 (contracts/PE/menus/auth.me) + health/live 200 + bundle hash 2/2 UNCHANGED (fe-admin BUTKoqRP + fe-user CMHv2GS4 baseline preserved — expected NO FE) + Mig 34 prod TOP 1 = 20260526110207_AddEmployeeProfiles MATCHES repo + gotcha #51 INFRASTRUCTURE seed verify GOOD: EmployeeProfiles=33 rows (16 demo + 14 Solutions thật + 3 admin/test = full @solutions.com.vn user reconcile) + EmployeeCodeSequences NV/2026 LastSeq=33 atomic match + sample rows NV/2026/0001-0003 EmployeeStatus=1 Active Nationality="Việt Nam" — SeedDemoEmployeeProfilesAsync correctly NOT gated DemoSeed:Disabled per gotcha #51 lesson (DbInitializer.cs:94 vs 99/111 demo gate). Plan B Investigator pre-flight + Em main 4 decisions chốt + Implementer 17 new file Pattern 12-bis cross-module mirror PE→Hrm cookie-cutter all WORK end-to-end. Plan C BW1-BW7 ROI: test gate caught NOTHING this run (all PASS first try) — Pattern 12-bis mirror clean + Implementer Reviewer pre-commit gate strong. 0 prod regression observed Run #350. Discovery #7 path filter eval/** missing still stands but N/A this push (no eval/* files). Cumulative S33 deploy: 1× Run (#350), expect more S33+ kicks (Plan B Phase 2 Task 4 endpoint CQRS + Plan B Task 5 FE 2 app + Plan B Task 6 menu seed). Token cost ~30K (Read MEMORY + grep tests + git show + 8 Bash poll/curl/ssh + Read Mig 34 file).

• 2026-05-26 (S33 startup health-check — em main spawn read-only verify, VERDICT=HEALTHY): Snapshot post-S32 wrap. 0 unpushed (HEAD=5400983). Last 5 Runs all SUCCESS via Gitea API unauth (token empty OK): #235 1e1c9a2 3m38s + #234 b223466 3m52s today (S31 RAG docs+eval/*.json — triggered because eval/** NOT in paths-ignore current filter) + #233 e199603 + #232 38f1c4d + #231 3e92584 Plan B Contract V2 deploy chain 2026-05-22 ~3m30s avg. S32 commits b832f43..5400983 (4 docs+memory commits) CORRECTLY SKIPPED per gotcha #41 — all match **/*.md. 3 prod endpoint smoke ALL 200 OK (api/health/live 0.23s + admin 0.29s + eoffice 0.31s; /healthz 404 N/A — /health/live canonical). Mig prod TOP 5 DESC sqlcmd Windows-auth via ssh vietreport-vps "powershell ... '.\\SQLEXPRESS' -E" pattern (UPDATED from Discovery #5 — proper powershell wrapper instead of 4-backslash escape): AddContractLevelOpinions (Mig 33 Plan B) → AddApprovalWorkflowToContract (Mig 32) → RefactorSkipToFinalToApproverLevel (Mig 31) → AddAllowApproverEditBudgetToLevels (Mig 30) → RefactorAdvancedOptionsToPerLevelAndDrafterUser (Mig 29). Mig 33 head MATCHES Run #232 deploy baseline, NO drift. Note: SeedSampleContractWorkflowV2 Plan B Hotfix CICD = code seed (DbInitializer), NOT migration — won't appear in __EFMigrationsHistory. Cert api.solutions.com.vn notAfter Jul 23 01:58:16 2026 GMT (matches HANDOFF expected ~2026-07-23, ~58 days lead, auto-renew ~2026-06-23 win-acme 30d window). Bundle hash snapshot post-Run #235: fe-admin=index-BUTKoqRP.js, fe-user=index-CMHv2GS4.js — baseline for future deploy compare. DISCOVERY #7 NEW: path filter paths-ignore MISSING eval/** → S31 RAG eval JSON commits triggered ~3m30s deploy wastefully (no code change). Consider em main weigh adding 'eval/**' to filter if RAG telemetry commit frequency growing. Pending S33 deploy triggers: (i) Phase 10.1 G-H1 Hồ sơ NS first push — Mig 34 verify; (ii) Plan B-Wrap BW1-BW7 first commit — test gate baseline 111 → 118+ delta; (iii) Phase 9 UAT smoke V2 contract end-to-end. Token cost ~12K (Read MEMORY + 8 Bash curl/ssh/git/python parse).

• 2026-05-26 (S32 wrap — em main proxy curate + Phase 9 stabilize done + Phase 10 deploy ahead): Session 32 đóng clean. Em chủ trì spawn em 1 lần S32 startup verify (a505a02d84fc1fabe alive, MEMORY 27→24.2KB post curate Run #231 PARTIAL detail archived q2). NO Run triggered S32 — 5 commits S32 cumulative tất cả docs-only CI skip per #41 (b832f43 Phase 10 backlog + cce0963 stabilize batch + abcc1ed STATUS+HANDOFF wrap + 2 earlier S31). 0 prod deploy event. 3 endpoint smoke (api/admin/eoffice) still 200 OK post-S29 deploy. cert api.solutions.com.vn notAfter 2026-07-23 (verified via openssl s_client) — auto-renew ~2026-06-23, NOT urgent. Plan G 11 module backlog DOCUMENTED migration-todos + Plan B-Wrap test bundle BW1-BW7 spec ready. Pending S33 deploy triggers em main spawn em: (a) Phase 10.1 G-H1 Hồ sơ NS Run verify — first code commit Phase 10 Mig 34 + entity scaffold + CQRS + FE 2 app → poll Run + bundle hash rotate ×2 verify + Mig 34 applied prod sqlcmd verify + 5-stage checklist Stage 4 smoke endpoint (POST /api/employees create + GET /api/employees list). (b) Plan B-Wrap test bundle post-push verify — first commit BW1-BW7 codegen → 5-stage Stage 3 test gate baseline 111 → ~118+ test count delta report (UAT mode exception suspended cho Plan B-Wrap explicit test-after). (c) Phase 9 UAT smoke V2 contract sample workflow create flow — sqlcmd verify QT-HD-V2-001 ApplicableType=3 exists prod + Drafter create V2 contract end-to-end → CCM binh.le approve → terminal Phase=DaPhatHanh + gen mã HĐ + ContractLevelOpinions UPSERT. Foundation entries CONFIRMED preserved: 10-surface-point per-NV checklist + gotcha #48 SQLite tie-break + Discovery #6 INFRASTRUCTURE vs DEMO seed Stage 4.6. Token cost wrap ~3K. Tag: [wrap, phase-9-to-phase-10, cicd].

• 2026-05-26 (S32 startup verify — no CI poll, only foundation freshness + 3 endpoint smoke health): NO Run triggered S30-S32 (last code deploy Run #232 sha=38f1c4d 2026-05-22 ~3 days ago). Last push f938bf5 S31 docs patch cicd-monitor.md stale numbers (test/mig refresh) — docs-only → skip CI per gotcha #41 path filter (expected). 0 unpushed git log origin/main..HEAD. Verify state: (a) MEMORY size 24.9KB / 221 lines approaching 25KB threshold — recommend curate next session OR slim Run #231 verbose entry (lines 179-183 = ~5KB alone) since superseded by Run #232 wrap entry above. (b) MCP RAG tools PRESENT — search_memory returns 3 results query "Run 232 Plan B Hotfix CICD SeedSampleContractWorkflowV2" rerank_score=0.906 top (MEMORY.md self-hit) + 0.828 (gotcha #51 docs) + 0.816 (HANDOFF.md S29 final wrap) — RAG indexing healthy 2949 chunks. (c) Foundation entries CONFIRMED retained: 10-surface-point per-NV checklist (lines 142-160) + gotcha #48 SQLite tie-break (lines 56-61) + gotcha #51 NEW INFRASTRUCTURE vs DEMO seed (verified via RAG hit on docs/gotchas.md heading "51. INFRASTRUCTURE seed vs DEMO seed phân biệt — DemoSeed:Disabled flag gate trap"). (d) 3 prod endpoint smoke all 200 OK (api.solutions.com.vn/health/live + admin.solutions.com.vn + eoffice.solutions.com.vn) — prod stable post-S29 deploy. Pending future spawn triggers (em main SendMessage): (i) push code BE/FE/Mig commits Plan B-Wrap test bundle BW1-BW7 → poll CI + verify Run PASS + bundle hash rotate if FE in scope; (ii) Phase 9 UAT smoke production batch (3 endpoint health + Mig 33 sqlcmd verify + bundle hash check) — periodic heavy session ~30 min interval; (iii) any prod issue report ("không thấy V2", "Drafter dropdown empty" etc — gotcha #51 first suspect). Token cost spawn ~10K (no poll, no log fetch, no sqlcmd — only Read + Bash curl × 3 + RAG × 1).

• 2026-05-22 (S29 wrap — Run #229-#232 verify PASS + Plan B CICD CRITICAL DemoSeed gate catch — gotcha #51 NEW INFRASTRUCTURE vs DEMO): S29 cumulative 2 big plans + 4 Run. Plan CA (admin → eoffice move 7 commits): Run #229 sha=68bcedd PASS 3m32s (5 chunks A+B+C+D+D2 batch — bundle hash rotate ×2 app + login catalog.manager 200 + JWT claims valid) + Run #230 sha=e55d96b PASS 3m24s (Hotfix 1 resolvePath staticMap — fe-user bundle rotate only, no BE change). Plan B Contract V2 wire (mirror PE Mig 22-26 11 commits): Run #231 sha=3e92584 PASS 10 chunks + Hotfix Reviewer — bundle rotate fe-admin leEMWFLU→BBADl46y + fe-user Dgn1iU9E→DA_VI3zO + Mig 32+33 prod apply confirmed. Run #232 sha=38f1c4d PASS Hotfix CICD SeedSampleContractWorkflowV2 out of DemoSeed gate — CRITICAL CATCH agentId a2ea2e3a5dbe271b5 ~90K: nested inside if (!demoSeedDisabled) DbInitializer.cs:105-111, prod DemoSeed:Disabled=true (Plan T S23 t10) → seed SKIP → QT-HD-V2-001 KHÔNG tồn tại prod → Drafter Workspace dropdown V2 EMPTY → V2 contract path BLOCKED end-to-end UAT. Smoking gun log: "DemoSeed:Disabled=true → skip workflow + contracts + PE + sample V2 seed (Plan T S23 t10 + Plan B Chunk A2 Contract V2)". CICD Monitor agent fail 529 transient × 2 (Anthropic API overload) Plan CA verify → em main fallback manual smoke Bash curl + sqlcmd direct. Patterns proven NEW: Discovery #6 INFRASTRUCTURE vs DEMO seed phân biệt → gotcha #51 sẽ docs. Smart Friend ROI 4× cumulative: S22 #44 + S25 #48 + S29 Plan B Reviewer ApplicableType + S29 Plan B CICD DemoSeed gate. Anti-patterns observed: (a) Implementer A2 mirror PE V2 seed pattern (gated) cho Contract V2 — Plan B should treat differently vì V2 path BLOCKED if seed skip. (b) Em main miss noticing seed inside DemoSeed gate khi review Chunk A2. 0 prod regression observed S29 cumulative.

• Archived Run #231 PARTIAL detail (Plan B Contract V2 kick-off DemoSeed gap discovery) → archive/2026-05-q2.md 2026-05-26 S32 curate. KEY findings preserved in S29 wrap entry above: Run #231 PASS deploy + Mig 32+33 + bundle rotate, gap = ApplicableType=3 ZERO seed (DemoSeed gate), resolved by Run #232 Hotfix CICD → gotcha #51 added.

• 2026-05-22 (S28 wrap — Layer A governance apply, NO Run S28): Zero git push remote cả S28 (all local docs + memory + RAG store chunks) → 0 deploy event, bundle hash 2/2 unchanged Run #227 baseline, Mig 31 prod unchanged, 111 test baseline unchanged. Timeline: t1 startup 9-step + smoke test pass (registry hot-reload OK post-S27 model:inherit fix + 3,462 RAG chunks indexed) → t2-t4 RAG ROI verdict + over-reach mistake + scope-down → t5 Layer A governance apply. Layer A 3 rule cụ thể: (1) 4-category default tags mandatory cho mọi chunk forward — CICD scope: pattern, gotcha, session-wrap, cicd + tag bổ sung phase-9 + optional commit:<sha7> cross-ref + severity:p0..p3. (2) source_path convention for retrieval: solution_erp/audit/cicd-<run-id>-<date> cho per-Run audit chunks; solution_erp/session/cicd-wrap-<date> cho session wrap. (3) Weekly Friday eval ritual starting 2026-05-29 Friday 5 metric: query/session count + hit rate (rerank ≥0.7) + store noise % + RAG vs MD ratio + Voyage embed cost/week. 10 golden query draft sẵn cross-stack scenarios: Plan B Contract V2 wire kick-off + gotcha #48 SQLite tie-break + per-NV 10-surface checklist + cookie-cutter mirror PE→Contract + controller body record param count + FE merge synthetic Policy V2 + EF backfill idempotent + Smart Friend agent eval + DemoSeed feature flag + Run #215+#216 fail-fix pair pattern. ABANDONED rule cũ: "mọi tương tác mandatory RAG" → wastes ~30K query overhead khi single Run verify Bash poll đủ context. Foundation kept: 10-surface-point per-NV checklist (S22+5 → S23 t6) vẫn promoted MEMORY foundation cho future per-NV refactor verify. Forward S28+: chờ catch first CICD Run đầu Plan B Contract V2 wire khi bro push commit BE/FE/Mig (Mig 32+ expected).

• 2026-05-22 (S27 wrap-up em main proxy - hot-reload pitfall): NO Run triggered S27 (zero git push remote - all changes local docs + scripts + memory curate). 0 deploy event to monitor. Meta-discovery em main S27: Sub-agent registry KHÔNG load trong session active vì 4 file .claude/agents/*.md dùng model: claude-opus-4-7 (200K) + non-standard effort: max field → CLI silent reject per VIPIX pitfall #2. Em main solo cả S27 KHÔNG có lựa chọn delegate. Fix applied: 4 file → model: inherit + remove effort: max. Pending: Anh restart Claude Code CLI để hot-reload (pitfall #1 - edit file disk KHÔNG hot-reload session đang chạy). Sau restart, S28+ next push sẽ có CICD Monitor spawn trở lại normal cycle. Pattern reinforced: SESSION START PROTOCOL MUST spawn test agent trước khi assume registry loaded.

• 2026-05-22 (Curate session em main): Archived Run #186 → Run #221 verbose entries (14 runs S21 t3 → S25 Plan AF) → archive/2026-05-runs.md. KEEP: Run #215+#216 (gotcha #48 fail+fix pair lesson critical) + Run #222-#227 S26 summary + setup baseline. Memory size before: ~72KB → after: ~25-28KB target. Cumulative 8 patterns extracted vào archive header + 10-surface-point per-NV checklist promoted to foundation section above.

• 2026-05-21 (S26 Run #222-#227 cumulative — Plan AG series PE List tree view UI iteration): Hybrid verify pattern: CICD Monitor spawn 1× cho Phase wire initial Run #222 sha=0bf6c7e Plan AG (~12K — bundle hash 2/2 rotate admin C8TvDy7r→CWHIdoFo + user BvcWrq2z→Bg2FNeIz, smoke 5/5 200, PE List API shape preserved 9 fields, test gate 111 unchanged, Mig 31 unchanged). Run #223-#227 polish chunks Plan AG2-AG6 em main self-verify (bundle visual check + git push success + Gitea auto-trigger 3-4min deploy). Plan AG4 BE+FE cross-stack (DTO +4 fields DrafterUserId/DrafterName/DepartmentId/DepartmentName + 3 projection JOIN Users+Departments): dotnet test 111/111 PASS local pre-push. Pattern saved: CICD Monitor spawn 1× đầu Phase wire ROI tốt cho 1 dev solo iteration scenario. Polish chunks (CSS/UX/copy) cùng Plan em main self-verify thay vì re-spawn ~150K × N wasteful. 0 prod regression observed cumulative S26.

• 2026-05-19 10:13-10:21 — Run #215 FAIL → Run #216 PASS (gotcha #48 SQLite tie-break catch+fix pair, KEEP for crucial lesson):

  Run #215 id=329 sha=cdfd542 VERDICT=FAIL (S25 t1 Plan AB Chunk A — Changelog visibility fix Bug 1 Budget Adjust + Bug 2 Return Mode). Push range e23f51c..cdfd542 1 commit 3 files (1 BE PurchaseEvaluationWorkflowService.cs +207/-95 LOC refactor ApplyReturnModeAsync + 2 FE PeDetailTabs.tsx mirror filter extend). Duration 1m06s (early test gate fail, deploy stage never reached). CRITICAL — Test gate FAIL at test_infra: 51/53 passed, 2 FAIL same root cause:

  • PurchaseEvaluationWorkflowServiceReturnModeTests.ApplyReturnMode_OneStep_AtStep1_ResetsToBuoc1Cap1_KeepsChoDuyet
  • PurchaseEvaluationWorkflowServiceReturnModeTests.ApplyReturnMode_OneLevel_AtStep1Level1_ResetsToBuoc1Cap1_KeepsChoDuyet
  • Error: Expected changelog.ContextNote not to be <null>
  • Root cause: Plan AB Chunk A ApplyReturnModeAsync adds NEW Changelog entry at end (line 403-412) for Bug 2 visibility — EntityType=Workflow + Action=Update + Summary (NO ContextNote field). After refactor, BOTH ApplyReturnModeAsync (new entry, no ContextNote) AND LogTransitionAsync (line 100, existing entry with ContextNote=comment) are added in same SaveChangesAsync transaction. Test fetches .OrderByDescending(c => c.CreatedAt).FirstAsync() — with SQLite + frozen test clock both entries get SAME CreatedAt, OrderByDescending tie-break returns Plan AB's Workflow entry (without ContextNote) instead of Transition entry.
  • Deploy NOT shipped: Bundle hashes unchanged from Run #214 Plan AA baseline. Mig 31 TOP 1 unchanged. Plan AB Bug 1+Bug 2 fix NOT live (bro UAT screenshot pre-deploy stale).
  • Side benefit: CI test gate caught BEFORE prod deploy — bro UAT spared broken Plan M edge case audit trail.

  Run #216 id=330 sha=8c05947 VERDICT=PASS (S25 t2 Plan AB Chunk A2 fix). Tip commit Chunk A2: 1 test file +7/-2 LOC — 2 Plan M edge case tests add .Where(c => c.Summary!.Contains("Chuyển phase")) filter trước OrderByDescending(CreatedAt).First() để pick LogTransition entry (chứa ContextNote) thay vì Plan AB new Changelog entry. Plan AB Chunk A code cdfd542 KHÔNG bị revert — Bug 1+Bug 2 fix giữ nguyên. Test gate PASS: test_domain 58/58 + test_infra 53/53 (2 Plan M tests now PASS — verified live). Bundle hash 2/2 rotated. Bug 1 Budget Adjust entry LIVE + Bug 2 Return Mode entries LIVE on PE c6e9. 8 min turnaround 10:13 fail → 10:21 fix. Demonstrates test-after UAT mode CAN tolerate edge case bug if next chunk lands within minutes — but Plan AB > 100 LOC BE refactor should have local dotnet test verify pre-push (UAT skip-test rule risky for refactor scope).

• 2026-05-13 23:25 — Verify S22 chốt cuối cumulative (push range 3d725c4..cc8a7d3 12 commits VERDICT=PASS — S22+1-S22+5 Plan C/D/E + Mig 30 F4 per-NV Approver edit Budget). 33 active users prod confirmed. Bundle hash rotated 2/2. 104/104 test (+1 từ S21 baseline 103). Mig 30 prod confirmed. Discovery #3 first surfaced: cc8a7d3 docs+4 agent MEMORY.md → CI SKIPPED via **/*.md glob (all match — .md files at any depth match). Spec hypothesis ".claude/agent-memory/** NOT in paths-ignore → trigger CI" disproven for this commit. Gotcha #47 still useful as PREVENTIVE for future non-.md state files under .claude/agent-memory/.

• 2026-05-12 (setup): CI/CD Monitor agent initialized. Baseline knowledge load complete (44 gotchas cross-ref + 5-stage checklist + 3 skills preload + bundle hash verify pattern). No runs monitored yet.

---

🔄 Curate trigger

• Memory size > 25KB → archive recent runs to archive/<period>.md
• Duplicate failure patterns → merge into single entry (vd act_runner timeout x3 → 1 entry)
• Stale > 3 months → remove

Last curate: 2026-05-26 S32 em main proxy curate (post-S31 RAG fix) — archived 1 verbose Run #231 PARTIAL detail → archive/2026-05-q2.md. KEEP in MEMORY: S32 startup, S29 wrap summary (Run #229-#232 cumulative), S28 Layer A governance, S27 hot-reload pitfall, S22 curate, Run #215+#216 pair gotcha #48 critical, S26 Run #222-#227 summary, 2026-05-12 setup. Foundation 10-surface-point per-NV checklist + Stage 0-5 checklist + Discovery #6 INFRASTRUCTURE vs DEMO seed (Stage 4.6) preserved. MEMORY size before: 27 KB → after: ~24 KB. Previous curate: 2026-05-22 — archived 14 verbose Run #186-#221 → archive/2026-05-runs.md. Next trigger: > 25KB OR Plan G-H1 kick off.