solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md

CI/CD Monitor Agent — Persistent Memory

Persistent diary cross-session. Auto-injected first 200 lines / 25KB at spawn. Update BEFORE every stop. Curate when > 25KB.

---

🎯 Role baseline

Read-only CI/CD pipeline + post-deploy verifier for SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch. Output: PASS/FAIL verdict + evidence under 500 words. Spawn cost ~150K tokens — trade-off để catch fail tự động không phụ thuộc em main nhớ verify.

---

🚨 Recurring CI/CD bug patterns (catch with priority)

Gotcha #39 — act_runner github.com TCP timeout

• Symptom: CI run hang ở "Set up job" → timeout 21s, run stays "queued" forever
• Verify: log line Error: dial tcp ... github.com:443 ... i/o timeout
• Fix: manual checkout bypass đã hardcode trong .gitea/workflows/deploy.yml (run #108/#109), pass at #110. KHÔNG revert. Nếu pattern returns → escalate em main check VPS network

Gotcha #40 — npm cache tsc not found

• Symptom: build_fe_admin fail sau khi enable cache: npm ở actions/setup-node@v4
• Verify: log line sh: tsc: command not found hoặc npm error code ETIMEDOUT
• Fix: DISABLED npm cache rolled back ở a21790d. KHÔNG re-enable. Build time chấp nhận ~3 min thay vì optimize

Gotcha #41 — paths-ignore docs-only skip

• Symptom: Commit code thật mà CI không trigger (run list không có entry mới)
• Verify: git diff --name-only HEAD~1 HEAD vs paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**']
• Fix: Nếu commit có code thật bị skip nhầm → check pattern conflict. Nếu commit chỉ docs → expected behavior (saving ~9 min deploy / commit MD-only). Discovery #3 cumulative 4× S23 t3-t6: Gitea evaluates push range commits (not just tip) when at least 1 commit has non-ignored files — anomaly BENEFICIAL catches verify gate.

Gotcha #25 — IIS WebSocket / module exclusion

• Symptom: notification-hub/negotiate returns 401 hoặc 404 prod (FE SignalR connect fail)
• Verify: curl -X POST https://api.solutions.com.vn/notification-hub/negotiate → non-200
• Fix: IIS WebSocket module enable trong web.config của site api.solutions.com.vn (skill iis-deploy-runbook)

Gotcha #48 — Multi-Changelog.Add SQLite tie-break (S25 lesson)

• Symptom: Tests using OrderByDescending(CreatedAt).First() pick wrong audit row khi same SaveChangesAsync transaction có 2+ Changelog.Add() entries cùng CreatedAt (SQLite frozen-clock).
• Verify: Test fail message Expected changelog.ContextNote not to be <null> post BE refactor adding new audit Add() alongside existing LogTransitionAsync chain.
• Fix: Discriminator filter: .Where(c => c.Summary!.Contains("Chuyển phase")) OR .Where(c => c.EntityType == X) BEFORE OrderByDescending. Pattern verified Plan AB Chunk A2 (Run #216 PASS).
• Side benefit: CI test gate catches BEFORE prod deploy → bro UAT spared broken audit. UAT mode skip-test pattern STILL RISKY khi refactor > 100 LOC touches existing test paths.

Deploy ship verification — bundle hash unchanged

• Symptom: commit push success + Gitea action success + status PASS, nhưng prod không có thay đổi visible (user UAT báo "đã deploy mà không thấy")
• Root cause candidates:
  • IIS app pool chưa recycle → giữ assembly cũ trong memory
  • NSSM service script không copy file đúng folder
  • Browser cache (rare nếu Vite hash chuẩn)
• Verify: curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[a-z0-9]+\.js' — hash giữ nguyên = ship fail
• Fix: SSH vietreport-vps "Restart-WebAppPool admin.solutions.com.vn" + recheck bundle hash

Migration drift prod vs repo

• Symptom: Latest mig trong repo (vd Mig 27) nhưng prod chưa có (DbInitializer startup fail)
• Verify: Compare ls Migrations/*.cs vs sqlcmd ... __EFMigrationsHistory
• Fix: Check Program.cs startup hook app.MigrateDatabase() còn không + app pool recycle. Hoặc manual dotnet ef database update --connection prod qua SSH

---

📋 5-stage checklist (apply EVERY run)

Stage 0: Local RAG infrastructure status (S27 NEW — post NSSM Service mode)

• Get-Service Qdrant → must show Status=Running, StartType=Automatic (NSSM Windows Service)
• Nếu Stopped/Paused → Start-Service Qdrant (Admin) hoặc & "D:\.claude-rag\scripts\fix-service-start.ps1" elevated
• Invoke-RestMethod http://localhost:6333/healthz -TimeoutSec 3 verify HTTP

Stage 1: Push happened + filter check

• git log -1 --format='%H %s' — latest commit
• git log origin/main..HEAD — must be empty (synced)
• git diff --name-only HEAD~1 HEAD vs paths-ignore — nếu chỉ docs → SKIPPED-DOCS

Stage 2: Gitea Actions poll (max 10 iter × 60s)

• API: https://git.baocaogiaoduc.vn/api/v1/repos/vietreport-admin/solution-erp/actions/tasks?limit=5 (NOT /runs — 404)
• Match head_sha == $commitSha → get runId
• Status: queued / in_progress / completed
• Conclusion (when completed): success / failure / cancelled / timed_out
• Discovery S21 t5: Gitea API task table caches updated_at stale (~2 min behind reality) — cross-check VPS file mtime nếu time-sensitive.

Stage 3: Test gate verify (Domain 58 + Infra 53 baseline = 111 PASS)

• Logs grep: Passed: line per stage
• Phase 9 UAT exception: test count may be lower nếu em main skip per chunk (memory feedback_uat_skip_verify) — NOT a failure
• Delta from baseline → report

Stage 4: Post-deploy live verify (if SUCCESS)

• Auth login → bearer (admin + nv.test for non-admin gotcha #44 check). Token field = accessToken (NOT token). Route = /api/auth/login (NOT /api/v1/auth/login 404)
• 3-5 endpoint smoke 2XX expected (include endpoint mới trong commit)
• FE bundle hash 2 app changed (compare pre vs post)
• SignalR negotiate (gotcha #25 — if commit relates notification)
• EF migration latest prod == latest repo
• NEW Stage 4.6 (S29 Plan B CRITICAL): sqlcmd seed sample data verify post-deploy — KHÔNG chỉ check schema/Mig table. Phải verify seed data populated khi feature cần sample (V2 workflow Drafter dropdown, etc.). Pattern: sqlcmd ... -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'" → 0 rows post-deploy = seed GATE BLOCKED → escalate gotcha #51 INFRASTRUCTURE vs DEMO seed分类.
• Discovery #4 (S23 t6 Plan P): ASP.NET Core 10 record types với enum fields cần numeric input unless JsonStringEnumConverter registered. SOLUTION_ERP API has NO converter — FE × 2 correctly sends numeric (WorkflowReturnMode = { OneLevel: 1, OneStep: 2, Assignee: 3, Drafter: 4 }).
• Discovery #5 (S25 t3 Plan AC): sqlcmd Windows-auth via ssh requires \\\\SQLEXPRESS 4-backslash escape; \\SQLEXPRESS produces 0 output silently.
• Discovery #6 NEW (S29 Plan B CICD): INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/SampleWorkflowsV2) MUST always run — NOT inside if (!demoSeedDisabled). DEMO seed (DemoUsers/DemoContracts/DemoPE) OK gated. Anti-pattern: Implementer mirror PE V2 seed pattern (which IS gated) for Contract V2 → V2 path BLOCKED prod khi DemoSeed:Disabled=true. Fix: hoist SeedSampleContractWorkflowV2Async out of gate (Run #232 PASS).

Stage 5: Report PASS/FAIL with evidence + MEMORY.md update

---

⚠️ Anti-patterns observed (DO NOT)

1. ❌ Push fix code — READ only, escalate to em main
2. ❌ Speculate fail cause without log evidence
3. ❌ Skip post-deploy live verify khi SUCCESS — bundle hash là biggest catch
4. ❌ Skip MEMORY.md update
5. ❌ Poll forever (max 10 iter ~10 min timeout)
6. ❌ Auto-rollback — escalate với recommendation, KHÔNG tự chạy
7. ❌ Verify khi commit docs-only — SKIPPED-DOCS + return ngay (per Discovery #3 anomaly note)

---

🧠 SOLUTION_ERP CI/CD essentials

• Gitea: https://git.baocaogiaoduc.vn/vietreport-admin/solution-erp
• Workflow: .gitea/workflows/deploy.yml (test gate 2 step + build BE + build FE × 2 + deploy)
• Path filter: paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**'] (gotcha #41 + Discovery #3 anomaly)
• Prod URLs: api / admin / eoffice .solutions.com.vn
• SSH VPS: ssh vietreport-vps (user=Administrator, key=id_ed25519)
• DB prod: .\SQLEXPRESS / SolutionErp / vrapp user. Connection string fallback C:\inetpub\solution-erp\api\appsettings.Production.json khi $env:PROD_DB_PASSWORD empty local (S21 t5 discovery).
• Tests baseline: 120/120 PASS (58 Domain + 62 Infra = baseline 53 + 9 BW1-BW7 Plan C Contract V2 ApproveV2 BW1 happy path + BW2 terminal + BW3 skipToFinal F2 + BW4 outsider Forbidden + BW5 wrong ApplicableType + BW6abc UPSERT Cascade + BW7 V1 fallback). S33 Run #350 PASS confirmed delta +9. Pre-S33: 111/111 unchanged S25→S32.
• Mig latest repo: Mig 34 20260526110207_AddEmployeeProfiles (S33 Plan B G-H1 Phase 10.1 — 7 Hrm entity tables + EmployeeCodeSequences atomic NV/{YYYY}/{D4}). Prev Mig 33 AddContractLevelOpinions (S29 Plan B). Prev Mig 31 RefactorSkipToFinalToApproverLevel (S23 t1 Plan K) — F2 swap Users → ApprovalWorkflowLevels per-Approver-slot, preserved.
• Mig latest prod: sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC TOP 5
• Bearer test:
  • Admin: admin@solutions.com.vn / Admin@123456 (full)
  • UAT non-admin: nv.test@solutions.com.vn / TestUser@123456 (Drafter CCM — verify gotcha #44 silent 403 patterns)

---

🔑 Critical config (gotcha cross-ref)

• Node CI pin: 20.x (memory feedback_node_cicd — bài học NamGroup)
• MediatR pin: 12.4.1 (gotcha #1)
• Swashbuckle pin: 6.9.0 (gotcha #2)
• act_runner: manual checkout bypass github.com (gotcha #39)
• npm cache: DISABLED (gotcha #40 — KHÔNG re-enable)

Flag commit nếu thấy <PackageReference Include="MediatR" Version="14... hoặc cache: npm tái xuất hiện.

---

🎯 Per-NV admin opt-in wire surface — 10-point checklist (cumulative S22+5 → S23 t6)

Cross-ref memory feedback_per_nv_permission_scope.md. Future per-NV/per-Level refactor MUST verify:

1. Domain entity field
2. EF config HasDefaultValue(false)
3. Migration 3-file rule (Up + Designer + Snapshot)
4. Service handler read field
5. Domain DTO + Application DTO mirror
6. Designer FE checkbox inline (admin)
7. Admin overview AwLevelDto record + ToDto ctor
8. CreateAwLevelInput record + Update mutation handler
9. Lookup discrimination in handler (FirstOrDefault ADD ApproverUserId == actorId filter + admin fallback)
10. Controller body record mirror count check ([FromBody] record param count = Command record param count)

Bug latency observed when miss points 9-10: 2-3 days prod silent (Mig 28-29 deploy → S23 t4-t6 catch). Prophylactic codebase scan recommended: grep -n "FirstOrDefault.*Order.*==" *.cs after OR-of-N schema refactor.

---

📊 Run stats baseline (cumulative)

• Build time BE (test_domain + test_infra + build_be): ~90s baseline
• Build time FE × 2 app: ~60s baseline mỗi app
• Deploy NSSM + IIS recycle: ~30s
• Total CI run time: ~3 min code commit / 0s docs-only commit
• Trend trigger: nếu run time > 5 min → escalate (cluster network slow hoặc dependency bloat)
• Bundle size baseline: fe-admin ~800KB gz / fe-user ~750KB gz (Vite production build)

---

📅 Recent runs (FIFO — slim post-curate 2026-05-22)

• 2026-05-28 09:51-09:55 Run #243 (task 357) sha=909655c VERDICT=PASS ~3m43s (S35 Plan G-H2 Task 3 — HrmConfig BE CRUD 16 endpoint cookie-cutter): Push range c3cd343..909655c 1 commit 2 BE file NEW (+576 LOC, 0 mod): HrmConfigFeatures.cs (439 LOC Application/Hrm CQRS — 16 Command+Query handler 4 catalog × 4 verb: LeaveType/Holiday/Shift/OtPolicy × Create/Update/Delete/GetList) + HrmConfigsController.cs (137 LOC Api 16 endpoint route /api/hrm-configs/*). Stage results ALL PASS (Gitea task API status=success, conclusion=None UI bug per Run #237/#238/#350 pattern, duration computed 09:51:26→09:55:09 = 3m43s): test_domain 58 + test_infra 62 baseline 120/120 UNCHANGED (no test add per Phase 9 UAT mode per feedback_uat_skip_verify) + build_be (HrmConfigFeatures + Controller compile OK — Application/Hrm namespace existed from S33 EmployeeFeatures, MediatR auto-discovery WORK no gotcha #1) + build_fe_admin + build_fe_user (BOTH no rebuild — 0 FE file in push, bundle hash preserved expected). Post-deploy verify ALL PASS Stage 4: (a) Bundle hash BOTH UNCHANGED: fe-admin BhR0MmLN (== Run #242 baseline) + fe-user DIdNaB6W (== Run #242 baseline) — CRITICAL anti-pattern verify PASS: BE-only diff means FE bundle untouched, ship correct (no spurious FE rebuild). (b) 3 prod domain smoke ALL 200 OK: api/health/ready 200 0.16s + admin 200 0.11s + eoffice 200 0.11s. (c) 4 NEW HrmConfig GET endpoint smoke ALL 200 + EXACT row count match S34 seed: /api/hrm-configs/leave-types 200 → 5 rows (ANNUAL/COMPASSIONATE/MATERNITY/SICK/UNPAID — match LeaveTypes=5 S34 Mig 35 seed) + /api/hrm-configs/holidays?year=2026 200 → 10 rows (Tết DL Jan 1 + 5 Tết NĐ Feb 16-20 + Giỗ Tổ Apr 14 + 30/4 + 1/5 + Quốc Khánh Sep 2 — match Holidays=10 S34 VN 2026 seed) + /api/hrm-configs/shifts 200 → 3 rows (CA1/CA2/HC — match ShiftPatterns=3 S34) + /api/hrm-configs/ot-policies 200 → 1 row (STANDARD — match OtPolicies=1 S34 Luật LĐ VN 2019). Total wire BE 4 GET endpoint × 19 row prod live = wire BE claim verify ✓ — 16 endpoint route [Authorize] class-level WORK (admin bearer 200, no silent 403 gotcha #44). (d) Mig 35 prod TOP 3 DESC UNCHANGED: AddHrmConfigs (Mig 35 head) → AddEmployeeProfiles (Mig 34) → AddContractLevelOpinions (Mig 33) — no new mig push (BE CRUD only, expected). (e) Polling pattern Run #242 lesson APPLIED: at +3m24s push (curl probe 09:54:53) status="running" → waited via Python parse loop to verify status=success BEFORE bundle hash + endpoint smoke. Run #242 false-positive trap avoided. 0 prod regression observed Run #243. Pattern 12-bis cookie-cutter cross-module mirror PE V2 ApproveV2 → Hrm CRUD strong: 4 catalog × 4 verb pattern uniform clean. Cumulative S35 deploy: 2× Run (#242 FE inline forms + #243 BE CRUD 16 endpoint). Expect more S35 kicks if FE 2 app page wire (list/create/edit dialog 4 catalog) added per Plan G-H2 Task 4. Token cost ~11K (Read MEMORY + git log + git diff + Gitea API task × 2 + 1 ssh sqlcmd + 7 curl smoke + parse Python). Tag: [s35, run243, pass, be-only, hrm-configs-crud, 16-endpoint]. Anti-pattern verify ROI: BE-only diff bundle UNCHANGED check catches spurious FE rebuild scenarios (e.g. shared Vite config touch, dependency cache bust). Run #243 confirmed clean — no spurious rebuild.

• 2026-05-28 09:39-09:42 Run #242 sha=c3cd343 VERDICT=PASS ~3m30s (S35 Phase 1.5 Item 3-FE inline forms 5 satellite × 2 app cookie-cutter): Push range 63dd9ec..c3cd343 1 commit 4 FE files (+1560/-198 LOC): fe-admin EmployeesListPage.tsx 573→1200 +627 + fe-user mirror SHA256 IDENTICAL + types/employee.ts × 2 app +53 LOC each. Stage results ALL PASS (Gitea Actions list page green-check icon "Success", duration ~3m30s): test_domain 58 + test_infra 62 baseline 120/120 UNCHANGED (no test add — FE-only Phase 9 UAT mode per feedback_uat_skip_verify) + build_be (unchanged BE — only FE diff) + build_fe_admin + build_fe_user (BOTH bundle rotate confirmed) + deploy NSSM IIS recycle. Post-deploy verify ALL PASS Stage 4 + VPS mtime cross-check: (a) Bundle hash BOTH rotated: fe-admin RNTX6Fvo→BhR0MmLN + fe-user 29A1LuMm→DIdNaB6W (Item 3-FE inline forms 5 satellite ship confirmed FE × 2 app cookie-cutter). CSS hash also rotated (admin DQSzWczW→Dlk3cj6D + user D-PasqRK→25W039gj). (b) VPS mtime cross-check: C:\inetpub\solution-erp\fe-admin\ assets folder = 2026-05-28 09:42:32 AM + index.html = 09:41:37 AM + fe-user\ assets = 09:42:32 + index.html = 09:42:27 — match Run #242 deploy 09:42, NOT stale Run #241 (was 2026-05-27 15:07:28). NSSM service Restart-WebAppPool + xcopy ship verified end-to-end. (c) 3 prod domain smoke ALL 200 OK: api/health/ready 200 "Healthy" 0.32s + admin 200 0.32s + eoffice 200 0.31s. (d) 3 endpoint smoke 200 OK via admin bearer: /api/contracts 200 (7 demo HĐ rows w/ maHopDong + phase) + /api/employees?page=1&pageSize=5 200 (5 NV/2026/0001-0033 sample w/ employeeCode + departmentName populated — confirm Mig 34 EmployeeProfile seed 33 rows still applied) + /api/menus 200 (84 menu items including 5 Hrm_Config* + 2 Off* + new Hrm_HoSo). Token = accessToken (NOT token — gotcha preserved). (e) Mig 35 prod TOP 1 UNCHANGED = no BE Mig in push (FE-only diff). Plan 4 Phase 1.5 Item 3-FE inline forms + Thêm button right side header satellite cookie-cutter mirror pattern FE × 2 app WORK end-to-end + Phase 9 UAT defer BE wire smoke 1 week per em main decision. Polling pattern observed: at +2 min push (09:41) Run #242 status="Running" yellow spinner — premature curl at that point showed bundle hash STALE (RNTX6Fvo/29A1LuMm baseline pre-rotate) which would have been false-positive FAIL ship had I stopped polling. Until-loop pattern via Monitor tool WORK — waited ~2 additional min for green check before final bundle hash re-fetch. Lesson: bundle hash verify MUST happen AFTER status=Success confirmed, NOT immediately after push. 0 prod regression observed Run #242. Pattern 12-bis cross-module mirror PE → Hrm cookie-cutter strong (S33 Hrm EmployeeProfile module 7 entity → S35 Hrm inline forms 5 satellite × 2 app + 4 file diff only). Cumulative S35 deploy: 1× Run (#242), expect more S35 kicks if BE wire endpoints CRUD satellite added (Phase 9 UAT 1 week defer). Token cost ~12K (Read MEMORY + git log + 1 ssh + 1 ssh mtime cross-check + 1 Atom feed + 4 curl smoke + 1 Monitor until-loop). Tag: [s35, run242, pass, fe-only, hrm-inline-forms]. Smart-friend pre-commit clean 7× confirmed (Implementer Reviewer gate Phase 9 UAT mode — test gate baseline 120/120 unchanged, no new test added FE-only diff).

• 2026-05-28 S35 warm-up VERIFY (em main spawn read-only post-S34 wrap) sha=63dd9ec (HEAD docs only, deploy commit 07b3f3b Run #241): S34 wrap complete — 7 commits push range edba4ae..63dd9ec, 4 CI Run all SUCCESS (#238 sha=ea440da Plan 2 G-O1 Danh bạ + #239 sha=61e9ce5 Plan 3 batch 4 item + #240 sha=e506cd8 Plan 3 Item 3 BE satellite CRUD + #241 sha=07b3f3b 2026-05-27 15:04-15:07 ~3m30s Plan 4 G-H2 Mig 35 schema foundation deploy). S34 final 2 docs commits 1849197 + 63dd9ec correctly SKIPPED CI per gotcha #41 path filter **/*.md. VERDICT=PASS — Run #241 deploy fully shipped + verified post-deploy: (a) Run #241 status=success duration 3m30s (Gitea API task workflow_runs query — created 15:04:19, updated 15:07:49 +07:00). (b) 3 prod domain smoke ALL 200 OK: api.solutions.com.vn/health/ready returns "Healthy" 200/0.47s + admin.solutions.com.vn 200/0.33s + eoffice.solutions.com.vn 200/0.22s. (c) Mig 35 prod TOP 5 DESC sqlcmd via ssh vietreport-vps 'sqlcmd -S ".\SQLEXPRESS" -d SolutionErp -E -h -1 -Q "..."' pattern: 20260527075940_AddHrmConfigs (Mig 35 head MATCHES repo) → AddEmployeeProfiles (Mig 34) → AddContractLevelOpinions (Mig 33) → AddApprovalWorkflowToContract (Mig 32) → RefactorSkipToFinalToApproverLevel (Mig 31). (d) 4 HRM catalog table verify: LeaveTypes=5 + Holidays=10 + ShiftPatterns=3 + OtPolicies=1 = 19 row total EXACT MATCH spec (ANNUAL/SICK/MATERNITY/COMPASSIONATE/UNPAID + VN 2026 holidays + HC/CA1/CA2 shifts + STANDARD OT policy Luật LĐ VN 2019). (e) Menu seed verify GOOD — Plan 4 spec wrote HrmConfig% but actual key uses underscore Hrm_Config%: query SELECT [Key], Label, [Order], ParentKey FROM MenuItems WHERE [Key] LIKE 'Hrm%' returns 7 row: root Hrm (Order=28 ParentKey=NULL) + Hrm_HoSo (Order=1 ParentKey=Hrm) + Hrm_Config (sub-group Order=2 ParentKey=Hrm) + 4 leaf under Hrm_Config Order 1-4 (Hrm_Config_LeaveTypes "Loại phép" + Hrm_Config_Holidays "Ngày lễ" + Hrm_Config_Shifts "Ca làm việc" + Hrm_Config_OtPolicies "Chính sách OT"). 5 G-H2 menu rows present (sub-group + 4 leaf) — Pattern 16-bis MenuKeys.cs sync OK BE+FE. Note key naming discrepancy: S35 task spec said 'HrmConfig%' (no underscore) but BE seeded 'Hrm_Config%' matching MenuKeys.cs constants — verified by reading commit 07b3f3b message body. Recommend STATUS.md S35 task description fix key prefix. (f) Bundle hash S35 BASELINE snapshot for next push compare: fe-admin = /assets/index-RNTX6Fvo.js + fe-user = /assets/index-29A1LuMm.js. Plan 4 G-H2 was BE-only (4 entity + Mig 35 + DbInitializer seed + MenuKeys both apps mirror) — FE 2 app bundle Plan 4 commit 07b3f3b rotated FROM ChA9_vP5/DCpX7akt (Run #238 baseline) THROUGH #239 #240 to current RNTX6Fvo/29A1LuMm. (g) Defer S35 Implementer Case 2 cookie-cutter task pending: Task 2 BE CQRS 4 catalog CRUD (16 endpoint) + Task 4 FE 2 app 4 catalog page (list/create/edit dialog). Em main spawn cicd-monitor next when first S35 G-H2 wire push hits. 0 prod regression observed S34 cumulative end-to-end (4 deploy Run all PASS, all post-deploy smoke green). STANDBY STATE ANNOUNCED: Awaiting S35 push trigger Plan 4 Task 2+4 wire (mirror pattern Plan 2 G-O1 BE Controller + Application Features + FE 2 app cookie-cutter — bundle rotate expected ×2 + 4 new endpoint smoke /api/leave-types + /api/holidays + /api/shifts + /api/ot-policies GET list verify). Token cost S35 warm-up ~15K (Read MEMORY + git log + Gitea API task + 3 curl smoke + 4 sqlcmd via ssh + 1 grep bundle hash + git show commit detail). Tag: [s35, warm-up, standby, mig-35].

• 2026-05-27 13:40-13:43 — Run #238 (task 352) sha=ea440da VERDICT=PASS ~3m30s (S34 Plan 2 G-O1 Danh bạ nội bộ — BE+FE 2 app endpoint mới /api/directory): Push range edba4ae..ea440da 2 commits 23 files: (1) 7b0781b Plan 1 Curate 4 agent MEMORY (8 file MD — match **/*.md paths-ignore) + (2) ea440da Plan 2 G-O1 15 file BE+FE code (BE 4 new/modified: DirectoryFeatures.cs Application/Office namespace mới + DirectoryController.cs route /api/directory + MenuKeys.cs +Off/Off_DanhBa + DbInitializer.cs seed menu; FE × 2 app cookie-cutter: 5 new/modified each — types/directory.ts + pages/office/InternalDirectoryPage.tsx + App.tsx + Layout.tsx + menuKeys.ts). Per Discovery #3 anomaly: ≥1 commit non-ignored → entire push runs (commit 2 code triggers). Stage results ALL PASS (Run status=success, conclusion=None UI bug per Run #350/#237 pattern): test_domain 58 + test_infra 62 baseline 120/120 UNCHANGED (no test add G-O1 per UAT mode) + build_be (Application/Office namespace mới auto-discovered MediatR — KHÔNG gotcha #1) + build_fe_admin + build_fe_user (BOTH bundle rotate) + deploy NSSM IIS recycle. Post-deploy verify ALL PASS Stage 4 + 4.6 sqlcmd: auth login admin 200 + 5/5 endpoint smoke 200 (contracts 3.5KB + PE 6.2KB + employees 8.0KB + menus 10.8KB + NEW directory 13.6KB 34 rows — wire BE/FE confirmed: 34 @solutions.com.vn users w/ employeeCode+phone+departmentName populated, dept sample covers all 9 phòng ban: Ban GĐ + Cung ứng + Kiểm soát Chi phí + Kế toán + Nhân sự-HC + QS + Thiết bị + Tài chính + ...) + health/live 200 (0.14s) + admin/eoffice 200 (0.11s/0.10s) + bundle hash BOTH rotated (fe-admin CqGMUMOr→ChA9_vP5 + fe-user C_HKyxBe→DCpX7akt — Plan 2 FE × 2 app ship confirmed) + Mig 34 prod TOP 1 20260526110207_AddEmployeeProfiles UNCHANGED (no Mig in G-O1, only Application/Office layer + menu seed — expected) + menu seed verify GOOD: Off+Off_DanhBa keys present in /api/menus response + sqlcmd direct MenuItems WHERE Key IN ('Off', 'Off_DanhBa') returns 2 rows (Off root order 29, Off_DanhBa child Order 1 ParentKey=Off) + Permissions auto-grant 2 rows (MenuKey IN ('Off','Off_DanhBa') — SeedAdminPermissionsAsync All[] wire OK). Gotcha #44 silent 403 NOT observed: class-level [Authorize] on DirectoryController correctly allows any authenticated user (admin login 200 OK). 0 prod regression observed Run #238. Pattern: BE namespace mới SolutionErp.Application.Office (first Office-domain after Hrm) MediatR auto-discovery WORK — no manual services.AddMediatR registration needed (assembly scan từ Application root namespace, gotcha #1 pin v12.4.1 stable). Cumulative S33-S34 deploy: 3× Run PASS (#350 Phase 1 schema + #237 Phase 2 wire + #238 G-O1 Danh bạ). Token cost ~25K (Read MEMORY + grep git + 10 Bash poll/curl/ssh/sqlcmd + parse JSON).

• 2026-05-26 20:28-20:32 — Run #237 (task 351) sha=79a8343 VERDICT=PASS 3m50s (S33 Plan B G-H1 Phase 2 Task 4+5+6 — Hrm CQRS endpoint + FE 2 app + Menu seed): Push range 48a99e1..79a8343 3 commits 18 files: (1) 0e191de Task 4+6 (3 BE new — EmployeesController + EmployeeDtos + EmployeeFeatures + 2 modified MenuKeys + DbInitializer = CODE non-ignored CI trigger) + (2) 9616ae2 Task 5 FE × 2 app (12 file = 6 new × 2 + 6 modified App/Layout/menuKeys/types — FE CODE CI trigger + bundle rotate) + (3) 79a8343 MEMORY 3 agent (match **/*.md ignore). Per Discovery #3 anomaly: ≥1 commit non-ignored → entire push runs (commit 1+2 code triggers). Single Run #237 success authoritative (Gitea task API status=success, conclusion=None UI bug — same S33 #350 pattern). Stage results ALL PASS: test_domain 58 + test_infra 62 (baseline 120/120 unchanged — no test add Plan B Phase 2 per UAT mode) + build_be (Hrm CQRS compile OK) + build_fe_admin + build_fe_user (BOTH bundle rotate — 6 new file × 2 app each) + deploy NSSM IIS recycle. Post-deploy verify ALL PASS: auth login admin 200 + 6/6 endpoint smoke 200 (contracts/PE/menus/auth.me + 2 NEW employees endpoint 200/200 — wire BE Task 4 confirmed: route /api/employees GET list + GET paged page=1&pageSize=5) + health/live 200 (0.21s) + admin/eoffice 200 (0.23s/0.25s) + bundle hash BOTH rotated (fe-admin BUTKoqRP→CqGMUMOr + fe-user CMHv2GS4→C_HKyxBe per FE Task 5 ship confirmed) + Mig 34 prod TOP 1 = 20260526110207_AddEmployeeProfiles STILL applied (no rollback, Task 4-6 chỉ thêm CQRS + FE + menu seed, không touch Mig) + menu seed verify GOOD: Hrm + Hrm_HoSo keys present in /api/menus response (Task 6 SeedMenuTreeAsync + SeedAdminPermissionsAsync wire OK) + EmployeeProfiles total=33 (unchanged Run #350 baseline — idempotent guard skip). Sample row NV/2026/0007 (BOD 1 — Director, Ban Giám đốc, hireDate 2021-08-01) properly formed via API. Plan B Phase 2 Task 4+5+6 cookie-cutter mirror PE Workspace pattern WORK end-to-end FE+BE+DB+Menu. 0 prod regression observed Run #237. Pattern 12-bis cross-module mirror reaffirmed strong. Cumulative S33 deploy: 2× Run (#350 Phase 1 schema + #237 Phase 2 wire). Pending S33 next kick: Phase 1.5 backlog if any (e.g. Employee CRUD/Update/Detail Page) hoặc Phase 10.1 next G-H module port. Token cost ~30K (8 Bash curl/ssh/grep + Read MEMORY + parse JSON).

• 2026-05-26 18:19-18:23 — Run #350 sha=48a99e1 VERDICT=PASS 3m38s (S33 Plan B G-H1 Mig 34 EmployeeProfile + Plan C BW1-BW7 test bundle): Push range 5400983..48a99e1 4 commits 34 files: (1) 1bc6b70 docs drift (3 file MD — match docs/** + **/*.md paths-ignore) + (2) b3444a3 3 MEMORY agent (match **/*.md) + (3) 0605f19 Plan C +4 test file +9 [Fact]+Theory instances (NOT ignored → CI trigger) + (4) 48a99e1 Mig 34 + 17 entity/config new + 7 modified (NOT ignored → CI trigger). Per Discovery #3 anomaly Gitea per-push trigger when ≥1 commit non-ignored → entire push runs. Stage results ALL PASS (Run status=success authoritative — Gitea task API conclusion=None even on success, do NOT confuse): test_domain 58/58 + test_infra 62/62 (=53 baseline +9 BW: BW1 happy Cấp 1→2 / BW2 terminal DaPhatHanh + gen mã / BW3 skipToFinal F2 admin opt-in / BW4 outsider ForbiddenException / BW5 wrong ApplicableType / BW6a UNIQUE composite / BW6b UPSERT 1 row / BW6c Cascade delete / BW7 V1 fallback ConflictException) + build_be (Mig 34 compile OK +6555 LOC) + build_fe_admin + build_fe_user (unchanged — no FE in push) + deploy NSSM IIS recycle. Post-deploy verify ALL PASS (Stage 4 + 4.6 sqlcmd): auth login admin 200 + 4 endpoint smoke 200/200/200/200 (contracts/PE/menus/auth.me) + health/live 200 + bundle hash 2/2 UNCHANGED (fe-admin BUTKoqRP + fe-user CMHv2GS4 baseline preserved — expected NO FE) + Mig 34 prod TOP 1 = 20260526110207_AddEmployeeProfiles MATCHES repo + gotcha #51 INFRASTRUCTURE seed verify GOOD: EmployeeProfiles=33 rows (16 demo + 14 Solutions thật + 3 admin/test = full @solutions.com.vn user reconcile) + EmployeeCodeSequences NV/2026 LastSeq=33 atomic match + sample rows NV/2026/0001-0003 EmployeeStatus=1 Active Nationality="Việt Nam" — SeedDemoEmployeeProfilesAsync correctly NOT gated DemoSeed:Disabled per gotcha #51 lesson (DbInitializer.cs:94 vs 99/111 demo gate). Plan B Investigator pre-flight + Em main 4 decisions chốt + Implementer 17 new file Pattern 12-bis cross-module mirror PE→Hrm cookie-cutter all WORK end-to-end. Plan C BW1-BW7 ROI: test gate caught NOTHING this run (all PASS first try) — Pattern 12-bis mirror clean + Implementer Reviewer pre-commit gate strong. 0 prod regression observed Run #350. Discovery #7 path filter eval/** missing still stands but N/A this push (no eval/* files). Cumulative S33 deploy: 1× Run (#350), expect more S33+ kicks (Plan B Phase 2 Task 4 endpoint CQRS + Plan B Task 5 FE 2 app + Plan B Task 6 menu seed). Token cost ~30K (Read MEMORY + grep tests + git show + 8 Bash poll/curl/ssh + Read Mig 34 file).

• 2026-05-26 (S33 startup health-check — em main spawn read-only verify, VERDICT=HEALTHY): Snapshot post-S32 wrap. 0 unpushed (HEAD=5400983). Last 5 Runs all SUCCESS via Gitea API unauth (token empty OK): #235 1e1c9a2 3m38s + #234 b223466 3m52s today (S31 RAG docs+eval/*.json — triggered because eval/** NOT in paths-ignore current filter) + #233 e199603 + #232 38f1c4d + #231 3e92584 Plan B Contract V2 deploy chain 2026-05-22 ~3m30s avg. S32 commits b832f43..5400983 (4 docs+memory commits) CORRECTLY SKIPPED per gotcha #41 — all match **/*.md. 3 prod endpoint smoke ALL 200 OK (api/health/live 0.23s + admin 0.29s + eoffice 0.31s; /healthz 404 N/A — /health/live canonical). Mig prod TOP 5 DESC sqlcmd Windows-auth via ssh vietreport-vps "powershell ... '.\\SQLEXPRESS' -E" pattern (UPDATED from Discovery #5 — proper powershell wrapper instead of 4-backslash escape): AddContractLevelOpinions (Mig 33 Plan B) → AddApprovalWorkflowToContract (Mig 32) → RefactorSkipToFinalToApproverLevel (Mig 31) → AddAllowApproverEditBudgetToLevels (Mig 30) → RefactorAdvancedOptionsToPerLevelAndDrafterUser (Mig 29). Mig 33 head MATCHES Run #232 deploy baseline, NO drift. Note: SeedSampleContractWorkflowV2 Plan B Hotfix CICD = code seed (DbInitializer), NOT migration — won't appear in __EFMigrationsHistory. Cert api.solutions.com.vn notAfter Jul 23 01:58:16 2026 GMT (matches HANDOFF expected ~2026-07-23, ~58 days lead, auto-renew ~2026-06-23 win-acme 30d window). Bundle hash snapshot post-Run #235: fe-admin=index-BUTKoqRP.js, fe-user=index-CMHv2GS4.js — baseline for future deploy compare. DISCOVERY #7 NEW: path filter paths-ignore MISSING eval/** → S31 RAG eval JSON commits triggered ~3m30s deploy wastefully (no code change). Consider em main weigh adding 'eval/**' to filter if RAG telemetry commit frequency growing. Pending S33 deploy triggers: (i) Phase 10.1 G-H1 Hồ sơ NS first push — Mig 34 verify; (ii) Plan B-Wrap BW1-BW7 first commit — test gate baseline 111 → 118+ delta; (iii) Phase 9 UAT smoke V2 contract end-to-end. Token cost ~12K (Read MEMORY + 8 Bash curl/ssh/git/python parse).

• 2026-05-26 (S32 wrap — em main proxy curate + Phase 9 stabilize done + Phase 10 deploy ahead): Session 32 đóng clean. Em chủ trì spawn em 1 lần S32 startup verify (a505a02d84fc1fabe alive, MEMORY 27→24.2KB post curate Run #231 PARTIAL detail archived q2). NO Run triggered S32 — 5 commits S32 cumulative tất cả docs-only CI skip per #41 (b832f43 Phase 10 backlog + cce0963 stabilize batch + abcc1ed STATUS+HANDOFF wrap + 2 earlier S31). 0 prod deploy event. 3 endpoint smoke (api/admin/eoffice) still 200 OK post-S29 deploy. cert api.solutions.com.vn notAfter 2026-07-23 (verified via openssl s_client) — auto-renew ~2026-06-23, NOT urgent. Plan G 11 module backlog DOCUMENTED migration-todos + Plan B-Wrap test bundle BW1-BW7 spec ready. Pending S33 deploy triggers em main spawn em: (a) Phase 10.1 G-H1 Hồ sơ NS Run verify — first code commit Phase 10 Mig 34 + entity scaffold + CQRS + FE 2 app → poll Run + bundle hash rotate ×2 verify + Mig 34 applied prod sqlcmd verify + 5-stage checklist Stage 4 smoke endpoint (POST /api/employees create + GET /api/employees list). (b) Plan B-Wrap test bundle post-push verify — first commit BW1-BW7 codegen → 5-stage Stage 3 test gate baseline 111 → ~118+ test count delta report (UAT mode exception suspended cho Plan B-Wrap explicit test-after). (c) Phase 9 UAT smoke V2 contract sample workflow create flow — sqlcmd verify QT-HD-V2-001 ApplicableType=3 exists prod + Drafter create V2 contract end-to-end → CCM binh.le approve → terminal Phase=DaPhatHanh + gen mã HĐ + ContractLevelOpinions UPSERT. Foundation entries CONFIRMED preserved: 10-surface-point per-NV checklist + gotcha #48 SQLite tie-break + Discovery #6 INFRASTRUCTURE vs DEMO seed Stage 4.6. Token cost wrap ~3K. Tag: [wrap, phase-9-to-phase-10, cicd].

• 2026-05-26 (S32 startup verify — no CI poll, only foundation freshness + 3 endpoint smoke health): NO Run triggered S30-S32 (last code deploy Run #232 sha=38f1c4d 2026-05-22 ~3 days ago). Last push f938bf5 S31 docs patch cicd-monitor.md stale numbers (test/mig refresh) — docs-only → skip CI per gotcha #41 path filter (expected). 0 unpushed git log origin/main..HEAD. Verify state: (a) MEMORY size 24.9KB / 221 lines approaching 25KB threshold — recommend curate next session OR slim Run #231 verbose entry (lines 179-183 = ~5KB alone) since superseded by Run #232 wrap entry above. (b) MCP RAG tools PRESENT — search_memory returns 3 results query "Run 232 Plan B Hotfix CICD SeedSampleContractWorkflowV2" rerank_score=0.906 top (MEMORY.md self-hit) + 0.828 (gotcha #51 docs) + 0.816 (HANDOFF.md S29 final wrap) — RAG indexing healthy 2949 chunks. (c) Foundation entries CONFIRMED retained: 10-surface-point per-NV checklist (lines 142-160) + gotcha #48 SQLite tie-break (lines 56-61) + gotcha #51 NEW INFRASTRUCTURE vs DEMO seed (verified via RAG hit on docs/gotchas.md heading "51. INFRASTRUCTURE seed vs DEMO seed phân biệt — DemoSeed:Disabled flag gate trap"). (d) 3 prod endpoint smoke all 200 OK (api.solutions.com.vn/health/live + admin.solutions.com.vn + eoffice.solutions.com.vn) — prod stable post-S29 deploy. Pending future spawn triggers (em main SendMessage): (i) push code BE/FE/Mig commits Plan B-Wrap test bundle BW1-BW7 → poll CI + verify Run PASS + bundle hash rotate if FE in scope; (ii) Phase 9 UAT smoke production batch (3 endpoint health + Mig 33 sqlcmd verify + bundle hash check) — periodic heavy session ~30 min interval; (iii) any prod issue report ("không thấy V2", "Drafter dropdown empty" etc — gotcha #51 first suspect). Token cost spawn ~10K (no poll, no log fetch, no sqlcmd — only Read + Bash curl × 3 + RAG × 1).

• 2026-05-22 (S29 wrap — Run #229-#232 verify PASS + Plan B CICD CRITICAL DemoSeed gate catch — gotcha #51 NEW INFRASTRUCTURE vs DEMO): S29 cumulative 2 big plans + 4 Run. Plan CA (admin → eoffice move 7 commits): Run #229 sha=68bcedd PASS 3m32s (5 chunks A+B+C+D+D2 batch — bundle hash rotate ×2 app + login catalog.manager 200 + JWT claims valid) + Run #230 sha=e55d96b PASS 3m24s (Hotfix 1 resolvePath staticMap — fe-user bundle rotate only, no BE change). Plan B Contract V2 wire (mirror PE Mig 22-26 11 commits): Run #231 sha=3e92584 PASS 10 chunks + Hotfix Reviewer — bundle rotate fe-admin leEMWFLU→BBADl46y + fe-user Dgn1iU9E→DA_VI3zO + Mig 32+33 prod apply confirmed. Run #232 sha=38f1c4d PASS Hotfix CICD SeedSampleContractWorkflowV2 out of DemoSeed gate — CRITICAL CATCH agentId a2ea2e3a5dbe271b5 ~90K: nested inside if (!demoSeedDisabled) DbInitializer.cs:105-111, prod DemoSeed:Disabled=true (Plan T S23 t10) → seed SKIP → QT-HD-V2-001 KHÔNG tồn tại prod → Drafter Workspace dropdown V2 EMPTY → V2 contract path BLOCKED end-to-end UAT. Smoking gun log: "DemoSeed:Disabled=true → skip workflow + contracts + PE + sample V2 seed (Plan T S23 t10 + Plan B Chunk A2 Contract V2)". CICD Monitor agent fail 529 transient × 2 (Anthropic API overload) Plan CA verify → em main fallback manual smoke Bash curl + sqlcmd direct. Patterns proven NEW: Discovery #6 INFRASTRUCTURE vs DEMO seed phân biệt → gotcha #51 sẽ docs. Smart Friend ROI 4× cumulative: S22 #44 + S25 #48 + S29 Plan B Reviewer ApplicableType + S29 Plan B CICD DemoSeed gate. Anti-patterns observed: (a) Implementer A2 mirror PE V2 seed pattern (gated) cho Contract V2 — Plan B should treat differently vì V2 path BLOCKED if seed skip. (b) Em main miss noticing seed inside DemoSeed gate khi review Chunk A2. 0 prod regression observed S29 cumulative.

• Archived Run #231 PARTIAL detail (Plan B Contract V2 kick-off DemoSeed gap discovery) → archive/2026-05-q2.md 2026-05-26 S32 curate. KEY findings preserved in S29 wrap entry above: Run #231 PASS deploy + Mig 32+33 + bundle rotate, gap = ApplicableType=3 ZERO seed (DemoSeed gate), resolved by Run #232 Hotfix CICD → gotcha #51 added.

• Archived to archive/2026-05-q3.md 2026-05-27 S34 curate (em main proxy): S28 wrap Layer A governance apply (NO Run, governance only — tag schema absorbed forward) + S27 wrap-up hot-reload pitfall (model:inherit fix, registry hot-reload lesson preserved in foundation) + 2026-05-22 Curate session note (Run #186-#221 → archive/2026-05-runs.md). KEY pattern absorbed forward: SESSION START PROTOCOL MUST spawn test agent trước khi assume registry loaded.

• 2026-05-21 (S26 Run #222-#227 cumulative — Plan AG series PE List tree view UI iteration): Hybrid verify pattern: CICD Monitor spawn 1× cho Phase wire initial Run #222 sha=0bf6c7e Plan AG (~12K — bundle hash 2/2 rotate admin C8TvDy7r→CWHIdoFo + user BvcWrq2z→Bg2FNeIz, smoke 5/5 200, PE List API shape preserved 9 fields, test gate 111 unchanged, Mig 31 unchanged). Run #223-#227 polish chunks Plan AG2-AG6 em main self-verify (bundle visual check + git push success + Gitea auto-trigger 3-4min deploy). Plan AG4 BE+FE cross-stack (DTO +4 fields DrafterUserId/DrafterName/DepartmentId/DepartmentName + 3 projection JOIN Users+Departments): dotnet test 111/111 PASS local pre-push. Pattern saved: CICD Monitor spawn 1× đầu Phase wire ROI tốt cho 1 dev solo iteration scenario. Polish chunks (CSS/UX/copy) cùng Plan em main self-verify thay vì re-spawn ~150K × N wasteful. 0 prod regression observed cumulative S26.

• Archived to archive/2026-05-q3.md 2026-05-27 S34 curate (em main proxy): Run #215+#216 (gotcha #48 SQLite tie-break catch+fix pair, S25 t1-t2) — full detail preserved in archive. KEY lesson cumulative trong foundation gotcha #48 line 40-45 (test discriminator filter Summary contains "Chuyển phase"). 2026-05-13 S22 chốt cuối verify (Run #214 baseline + Discovery #3 docs-only skip). 2026-05-12 setup baseline.

---

🔄 Curate trigger

• Memory size > 25KB → archive recent runs to archive/<period>.md
• Duplicate failure patterns → merge into single entry (vd act_runner timeout x3 → 1 entry)
• Stale > 3 months → remove

Last curate: 2026-05-27 S34 em main proxy curate (post-S33 wrap, sequence 1/4) — archived Run #215+#216 verbose detail (gotcha #48 catch+fix pair, KEY lesson preserved in foundation line 40-45) + 2026-05-13 S22 verify + 2026-05-12 setup → archive/2026-05-q3.md. KEEP in MEMORY: Run #237 + Run #350 (S33 prod deploy), S33 startup health, S32 wrap + S32 startup, S29 wrap, S28 wrap, S26 Run #222-#227, S22 curate. Foundation 10-surface-point per-NV checklist + Stage 0-5 checklist + Stage 4.6 sqlcmd seed verify + Discovery #6 INFRASTRUCTURE vs DEMO seed + gotchas #39-#48 detail all preserved untouched. MEMORY size before: 32.9 KB → after: target ~22-24 KB. Previous curate: 2026-05-26 S32 — Run #231 PARTIAL detail → archive/2026-05-q2.md. Previous curate: 2026-05-22 — Run #186-#221 → archive/2026-05-runs.md. Next trigger: > 25KB OR Plan G-O1 Danh bạ kick off.