49134f4f41
🎊 PHASE 10 COMPLETE end-to-end cumulative 6 session S33-S38:
- G-H1 Hồ sơ NS (S33) + G-O1 Danh bạ (S34) + G-H2 Cấu hình HRM (S34+S35)
- G-O2 Phòng họp (S36) + G-O3 Đề xuất (S37)
- G-O4 + G-O5 + G-O6 + G-P1 + G-H3 SKELETON combo (S38)
S38 final wrap (code commit e54a22d + this docs wrap):
- Run #247 sha=e54a22d 3m25s PASS (fastest S38)
- Bundle rotate × 2 (admin CGueDk22→cWAXid0q + user CEt0QRgX→CX79e2kZ)
- Mig 39+40 prod head + 6 endpoint smoke + 6 new tables verified
- HR Dashboard prod live (TotalEmployees=33 / Male=17 / Female=16)
- 0 prod regression
Cumulative Phase 10 stats:
- 7 Mig (34→40) · 30+ new tables · ~75+ new endpoints · 17 FE pages × 2 app
- Pattern 16-bis 9× cumulative · Pattern 12-bis 12× cumulative · Smart Friend 9× clean
- 6 commit cumulative · 9+ CI Run all PASS
SKELETON Phase 1 trade-off rõ ràng — workflow ApproveV2 + LeaveBalance +
CodeGen + Vehicle catalog + ItTicketComments + Auto-assign + SLA timer DEFER Phase 11.
Next S39+: Phase 11 polish features OR Phase 9 Ops UAT 2-3 user 1 tuần.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
206 lines
35 KiB
Markdown
206 lines
35 KiB
Markdown
# CI/CD Monitor Agent — Persistent Memory
|
||
|
||
> **Persistent diary cross-session.** Auto-injected first 200 lines / 25KB at spawn.
|
||
> Update BEFORE every stop. Curate when > 25KB.
|
||
|
||
---
|
||
|
||
## 🎯 Role baseline
|
||
|
||
Read-only CI/CD pipeline + post-deploy verifier for SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch. Output: PASS/FAIL verdict + evidence under 500 words. **Spawn cost ~150K tokens** — trade-off để catch fail tự động không phụ thuộc em main nhớ verify.
|
||
|
||
---
|
||
|
||
## 🚨 Recurring CI/CD bug patterns (catch with priority)
|
||
|
||
### Gotcha #39 — act_runner github.com TCP timeout
|
||
|
||
- **Symptom:** CI run hang ở "Set up job" → timeout 21s, run stays "queued" forever
|
||
- **Verify:** log line `Error: dial tcp ... github.com:443 ... i/o timeout`
|
||
- **Fix:** manual checkout bypass đã hardcode trong `.gitea/workflows/deploy.yml` (run #108/#109), pass at #110. KHÔNG revert. Nếu pattern returns → escalate em main check VPS network
|
||
|
||
### Gotcha #40 — npm cache `tsc not found`
|
||
|
||
- **Symptom:** `build_fe_admin` fail sau khi enable `cache: npm` ở `actions/setup-node@v4`
|
||
- **Verify:** log line `sh: tsc: command not found` hoặc `npm error code ETIMEDOUT`
|
||
- **Fix:** DISABLED npm cache rolled back ở `a21790d`. KHÔNG re-enable. Build time chấp nhận ~3 min thay vì optimize
|
||
|
||
### Gotcha #41 — paths-ignore docs-only skip
|
||
|
||
- **Symptom:** Commit code thật mà CI không trigger (run list không có entry mới)
|
||
- **Verify:** `git diff --name-only HEAD~1 HEAD` vs `paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**']`
|
||
- **Fix:** Nếu commit có code thật bị skip nhầm → check pattern conflict. Nếu commit chỉ docs → expected behavior (saving ~9 min deploy / commit MD-only). **Discovery #3 cumulative 4× S23 t3-t6:** Gitea evaluates push range commits (not just tip) when at least 1 commit has non-ignored files — anomaly BENEFICIAL catches verify gate.
|
||
|
||
### Gotcha #25 — IIS WebSocket / module exclusion
|
||
|
||
- **Symptom:** `notification-hub/negotiate` returns 401 hoặc 404 prod (FE SignalR connect fail)
|
||
- **Verify:** `curl -X POST https://api.solutions.com.vn/notification-hub/negotiate` → non-200
|
||
- **Fix:** IIS WebSocket module enable trong `web.config` của site api.solutions.com.vn (skill `iis-deploy-runbook`)
|
||
|
||
### Gotcha #48 — Multi-Changelog.Add SQLite tie-break (S25 lesson)
|
||
|
||
- **Symptom:** Tests using `OrderByDescending(CreatedAt).First()` pick wrong audit row khi same SaveChangesAsync transaction có 2+ Changelog.Add() entries cùng CreatedAt (SQLite frozen-clock).
|
||
- **Verify:** Test fail message `Expected changelog.ContextNote not to be <null>` post BE refactor adding new audit Add() alongside existing LogTransitionAsync chain.
|
||
- **Fix:** Discriminator filter: `.Where(c => c.Summary!.Contains("Chuyển phase"))` OR `.Where(c => c.EntityType == X)` BEFORE OrderByDescending. Pattern verified Plan AB Chunk A2 (Run #216 PASS).
|
||
- **Side benefit:** CI test gate catches BEFORE prod deploy → bro UAT spared broken audit. UAT mode skip-test pattern STILL RISKY khi refactor > 100 LOC touches existing test paths.
|
||
|
||
### Deploy ship verification — bundle hash unchanged
|
||
|
||
- **Symptom:** commit push success + Gitea action success + status PASS, **nhưng prod không có thay đổi visible** (user UAT báo "đã deploy mà không thấy")
|
||
- **Root cause candidates:**
|
||
- IIS app pool chưa recycle → giữ assembly cũ trong memory
|
||
- NSSM service script không copy file đúng folder
|
||
- Browser cache (rare nếu Vite hash chuẩn)
|
||
- **Verify:** `curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[a-z0-9]+\.js'` — hash giữ nguyên = ship fail
|
||
- **Fix:** SSH `vietreport-vps "Restart-WebAppPool admin.solutions.com.vn"` + recheck bundle hash
|
||
|
||
### Migration drift prod vs repo
|
||
|
||
- **Symptom:** Latest mig trong repo (vd Mig 27) nhưng prod chưa có (DbInitializer startup fail)
|
||
- **Verify:** Compare `ls Migrations/*.cs` vs `sqlcmd ... __EFMigrationsHistory`
|
||
- **Fix:** Check `Program.cs` startup hook `app.MigrateDatabase()` còn không + app pool recycle. Hoặc manual `dotnet ef database update --connection prod` qua SSH
|
||
|
||
---
|
||
|
||
## 📋 5-stage checklist (apply EVERY run)
|
||
|
||
### Stage 0: Local RAG infrastructure status (S27 NEW — post NSSM Service mode)
|
||
- `Get-Service Qdrant` → must show Status=Running, StartType=Automatic (NSSM Windows Service)
|
||
- Nếu Stopped/Paused → `Start-Service Qdrant` (Admin) hoặc `& "D:\.claude-rag\scripts\fix-service-start.ps1"` elevated
|
||
- `Invoke-RestMethod http://localhost:6333/healthz -TimeoutSec 3` verify HTTP
|
||
|
||
### Stage 1: Push happened + filter check
|
||
- `git log -1 --format='%H %s'` — latest commit
|
||
- `git log origin/main..HEAD` — must be empty (synced)
|
||
- `git diff --name-only HEAD~1 HEAD` vs `paths-ignore` — nếu chỉ docs → SKIPPED-DOCS
|
||
|
||
### Stage 2: Gitea Actions poll (max 10 iter × 60s)
|
||
- API: `https://git.baocaogiaoduc.vn/api/v1/repos/vietreport-admin/solution-erp/actions/tasks?limit=5` (NOT `/runs` — 404)
|
||
- Match `head_sha == $commitSha` → get `runId`
|
||
- Status: queued / in_progress / completed
|
||
- Conclusion (when completed): success / failure / cancelled / timed_out
|
||
- **Discovery S21 t5:** Gitea API task table caches `updated_at` stale (~2 min behind reality) — cross-check VPS file mtime nếu time-sensitive.
|
||
|
||
### Stage 3: Test gate verify (Domain 58 + Infra 53 baseline = 111 PASS)
|
||
- Logs grep: `Passed:` line per stage
|
||
- Phase 9 UAT exception: test count may be lower nếu em main skip per chunk (memory `feedback_uat_skip_verify`) — NOT a failure
|
||
- Delta from baseline → report
|
||
|
||
### Stage 4: Post-deploy live verify (if SUCCESS)
|
||
- Auth login → bearer (admin + nv.test for non-admin gotcha #44 check). Token field = `accessToken` (NOT `token`). Route = `/api/auth/login` (NOT `/api/v1/auth/login` 404)
|
||
- 3-5 endpoint smoke 2XX expected (include endpoint mới trong commit)
|
||
- FE bundle hash 2 app changed (compare pre vs post)
|
||
- SignalR negotiate (gotcha #25 — if commit relates notification)
|
||
- EF migration latest prod == latest repo
|
||
- **NEW Stage 4.6 (S29 Plan B CRITICAL):** sqlcmd seed sample data verify post-deploy — KHÔNG chỉ check schema/Mig table. Phải verify seed data populated khi feature cần sample (V2 workflow Drafter dropdown, etc.). Pattern: `sqlcmd ... -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'"` → 0 rows post-deploy = seed GATE BLOCKED → escalate gotcha #51 INFRASTRUCTURE vs DEMO seed分类.
|
||
- **Discovery #4 (S23 t6 Plan P):** ASP.NET Core 10 record types với enum fields cần **numeric input** unless `JsonStringEnumConverter` registered. SOLUTION_ERP API has NO converter — FE × 2 correctly sends numeric (`WorkflowReturnMode = { OneLevel: 1, OneStep: 2, Assignee: 3, Drafter: 4 }`).
|
||
- **Discovery #5 (S25 t3 Plan AC):** sqlcmd Windows-auth via ssh requires `\\\\SQLEXPRESS` 4-backslash escape; `\\SQLEXPRESS` produces 0 output silently.
|
||
- **Discovery #6 NEW (S29 Plan B CICD):** INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/**SampleWorkflowsV2**) MUST always run — NOT inside `if (!demoSeedDisabled)`. DEMO seed (DemoUsers/DemoContracts/DemoPE) OK gated. Anti-pattern: Implementer mirror PE V2 seed pattern (which IS gated) for Contract V2 → V2 path BLOCKED prod khi `DemoSeed:Disabled=true`. Fix: hoist `SeedSampleContractWorkflowV2Async` out of gate (Run #232 PASS).
|
||
|
||
### Stage 5: Report PASS/FAIL with evidence + MEMORY.md update
|
||
|
||
---
|
||
|
||
## ⚠️ Anti-patterns observed (DO NOT)
|
||
|
||
1. ❌ Push fix code — READ only, escalate to em main
|
||
2. ❌ Speculate fail cause without log evidence
|
||
3. ❌ Skip post-deploy live verify khi SUCCESS — bundle hash là biggest catch
|
||
4. ❌ Skip MEMORY.md update
|
||
5. ❌ Poll forever (max 10 iter ~10 min timeout)
|
||
6. ❌ Auto-rollback — escalate với recommendation, KHÔNG tự chạy
|
||
7. ❌ Verify khi commit docs-only — SKIPPED-DOCS + return ngay (per Discovery #3 anomaly note)
|
||
|
||
---
|
||
|
||
## 🧠 SOLUTION_ERP CI/CD essentials
|
||
|
||
- **Gitea:** https://git.baocaogiaoduc.vn/vietreport-admin/solution-erp
|
||
- **Workflow:** `.gitea/workflows/deploy.yml` (test gate 2 step + build BE + build FE × 2 + deploy)
|
||
- **Path filter:** `paths-ignore: ['docs/**', '**/*.md', '.claude/skills/**']` (gotcha #41 + Discovery #3 anomaly)
|
||
- **Prod URLs:** api / admin / eoffice `.solutions.com.vn`
|
||
- **SSH VPS:** `ssh vietreport-vps` (user=Administrator, key=id_ed25519)
|
||
- **DB prod:** `.\SQLEXPRESS` / `SolutionErp` / vrapp user. Connection string fallback `C:\inetpub\solution-erp\api\appsettings.Production.json` khi `$env:PROD_DB_PASSWORD` empty local (S21 t5 discovery).
|
||
- **Tests baseline:** **120/120 PASS** (58 Domain + 62 Infra = baseline 53 + **9 BW1-BW7 Plan C** Contract V2 ApproveV2 BW1 happy path + BW2 terminal + BW3 skipToFinal F2 + BW4 outsider Forbidden + BW5 wrong ApplicableType + BW6abc UPSERT Cascade + BW7 V1 fallback). **S33 Run #350 PASS** confirmed delta +9. Pre-S33: 111/111 unchanged S25→S32.
|
||
- **Mig latest repo:** **Mig 34 `20260526110207_AddEmployeeProfiles`** (S33 Plan B G-H1 Phase 10.1 — 7 Hrm entity tables + EmployeeCodeSequences atomic NV/{YYYY}/{D4}). Prev Mig 33 `AddContractLevelOpinions` (S29 Plan B). Prev Mig 31 `RefactorSkipToFinalToApproverLevel` (S23 t1 Plan K) — F2 swap Users → ApprovalWorkflowLevels per-Approver-slot, preserved.
|
||
- **Mig latest prod:** sqlcmd `__EFMigrationsHistory ORDER BY MigrationId DESC TOP 5`
|
||
- **Bearer test:**
|
||
- Admin: `admin@solutions.com.vn / Admin@123456` (full)
|
||
- UAT non-admin: `nv.test@solutions.com.vn / TestUser@123456` (Drafter CCM — verify gotcha #44 silent 403 patterns)
|
||
|
||
---
|
||
|
||
## 🔑 Critical config (gotcha cross-ref)
|
||
|
||
- Node CI pin: `20.x` (memory `feedback_node_cicd` — bài học NamGroup)
|
||
- MediatR pin: `12.4.1` (gotcha #1)
|
||
- Swashbuckle pin: `6.9.0` (gotcha #2)
|
||
- act_runner: manual checkout bypass github.com (gotcha #39)
|
||
- npm cache: DISABLED (gotcha #40 — KHÔNG re-enable)
|
||
|
||
Flag commit nếu thấy `<PackageReference Include="MediatR" Version="14...` hoặc `cache: npm` tái xuất hiện.
|
||
|
||
---
|
||
|
||
## 🎯 Per-NV admin opt-in wire surface — 10-point checklist (cumulative S22+5 → S23 t6)
|
||
|
||
Cross-ref memory `feedback_per_nv_permission_scope.md`. Future per-NV/per-Level refactor MUST verify:
|
||
|
||
1. Domain entity field
|
||
2. EF config `HasDefaultValue(false)`
|
||
3. Migration 3-file rule (Up + Designer + Snapshot)
|
||
4. Service handler read field
|
||
5. Domain DTO + Application DTO mirror
|
||
6. Designer FE checkbox inline (admin)
|
||
7. Admin overview AwLevelDto record + ToDto ctor
|
||
8. CreateAwLevelInput record + Update mutation handler
|
||
9. **Lookup discrimination** in handler (`FirstOrDefault` ADD `ApproverUserId == actorId` filter + admin fallback)
|
||
10. **Controller body record** mirror count check (`[FromBody]` record param count = Command record param count)
|
||
|
||
Bug latency observed when miss points 9-10: 2-3 days prod silent (Mig 28-29 deploy → S23 t4-t6 catch). Prophylactic codebase scan recommended: `grep -n "FirstOrDefault.*Order.*==" *.cs` after OR-of-N schema refactor.
|
||
|
||
---
|
||
|
||
## 📊 Run stats baseline (cumulative)
|
||
|
||
- **Build time BE (test_domain + test_infra + build_be):** ~90s baseline
|
||
- **Build time FE × 2 app:** ~60s baseline mỗi app
|
||
- **Deploy NSSM + IIS recycle:** ~30s
|
||
- **Total CI run time:** ~3 min code commit / 0s docs-only commit
|
||
- **Trend trigger:** nếu run time > 5 min → escalate (cluster network slow hoặc dependency bloat)
|
||
- **Bundle size baseline:** fe-admin ~800KB gz / fe-user ~750KB gz (Vite production build)
|
||
|
||
---
|
||
|
||
## 📅 Recent runs (FIFO — slim post-curate 2026-05-22)
|
||
|
||
- **2026-05-28 16:20-16:23 Run #247 (task 361) sha=`e54a22d` VERDICT=PASS ~3m25s (S38 SKELETON 5 plan combo G-O4+G-O5+G-O6+G-P1+G-H3 full-stack Mig 39+40 dual):** Push range `de1c378..e54a22d` 1 commit massive scope `Domain+App+Infra+Api+FE-Admin+FE-User` — 4 Controllers NEW (AttendancesController + HrDashboardController + ItTicketsController + LeaveRequestsController) + FE × 2 cookie-cutter (HrmDashboardPage + ItTicketsPage + MyAttendancePage + WorkflowAppsListPage + types/workflowApps.ts + App.tsx + Layout.tsx + menuKeys.ts each app). **Stage results ALL PASS** (task #361 status=success, duration computed 16:20:23→16:23:48 = 3m25s, under baseline 3m30s — fastest S38 deploy despite massive scope). **Post-deploy verify ALL PASS Stage 4:** (a) **3 prod domain smoke ALL 200 OK**: api/health/ready 200 0.30s + admin 200 0.51s + eoffice 200 0.17s. (b) **Bundle hash BOTH rotated**: fe-admin `CGueDk22`→`cWAXid0q` + fe-user `CEt0QRgX`→`CX79e2kZ` (S38 SKELETON ship confirmed FE × 2 app). (c) **Mig 39+40 prod TOP 5 DESC head MATCHES repo**: `AddAttendances` (Mig 40 head 20260528090839) → `AddWorkflowApps` (Mig 39 20260528090830) → `AddProposals` (38) → `ExtendApplicableTypeForWorkflowApps` (37) → `AddMeetingRooms` (36). **Dual Mig deploy success** — DbInitializer auto-applied 2 mig in startup sequence (90830 → 90839 same push). (d) **6 NEW endpoint smoke ALL 200 OK**: `/api/leave-requests` 200 + `/api/ot-requests` 200 + `/api/travel-requests` 200 + `/api/vehicle-bookings` 200 + `/api/it-tickets` 200 + `/api/hr/dashboard` 200 with dto **`totalEmployees=33 activeEmployees=33 onLeaveEmployees=0 resignedEmployees=0 maleCount=17 femaleCount=16 birthdaysThisWeek=0 newHiresThisMonth=0`** — Plan G-H3 HR dashboard endpoint live + populated from Mig 34 EmployeeProfiles 33 NV seed. (e) **6 NEW tables sqlcmd EXACT MATCH spec**: Attendances + ItTickets + LeaveRequests + OtRequests + TravelRequests + VehicleBookings — all 6 rows returned from `sys.tables` filter. (f) **Menu seed verify 8 row MATCH ≥ 8 spec**: `Hrm_Dashboard` + `Off_ChamCong` + `Off_DatXe` + `Off_DonTu` + `Off_DonTu_Leave` + `Off_DonTu_Ot` + `Off_DonTu_Travel` + `Off_ItTicket` = **8 row** total (root Off_DonTu + 3 leaf Leave/Ot/Travel + 4 standalone Dashboard/ChamCong/DatXe/ItTicket). **0 prod regression observed Run #247.** Pattern 12-bis cross-module mirror PE V2 → 5 plan combo SKELETON full-stack mega scaffold STRONG: BE 4 Controller + FE × 2 page × 4 + Mig 39+40 dual + DbInitializer menu/seed 1 commit cohesive end-to-end. Polling pattern WORKED: 6 iter (1×0s + 5×45s sleep) ~3m45s wait running→success transition smooth, Run #247 false-positive trap avoided. Token cost ~12K (Read MEMORY + git log + git diff + Gitea API task poll × 7 + 3 smoke + 6 endpoint + 3 sqlcmd + 1 bundle hash + 1 auth login). Tag: `[s38, run247, pass, mig-39-40-dual, skeleton-5-plan-combo, full-stack]`. **Cumulative confirm:** Pattern 12-bis cross-module mirror end-to-end prod live; HR dashboard dto populated correctly from Mig 34 seed; 6 new modules SKELETON ready for next session BE wire CRUD + FE form scaffold.
|
||
|
||
- **2026-05-28 15:51-15:55 Run #246 (task 360) sha=`de1c378` VERDICT=PASS ~3m53s (S37 Plan G-O3 Đề xuất Mig 37+38 + BE Proposal CRUD + FE × 2):** Push range `f45090b..de1c378` 1 commit full-stack scope `Domain+App+Infra+Api+FE-Admin+FE-User`. **Stage results ALL PASS** (task #360 status=success, dur 3m53s computed 15:51:25→15:55:18). **Post-deploy verify ALL PASS Stage 4:** (a) **3 prod domain smoke ALL 200**: api/health/ready 200 0.27s + admin 200 0.13s + eoffice 200 0.11s. (b) **Bundle hash BOTH rotated**: admin `C9kzTTmq`→`CGueDk22` + user `CC4DQ-Tr`→`CEt0QRgX` (Plan G-O3 ship confirmed FE × 2). (c) **Mig 38 prod TOP 5 DESC MATCHES repo**: `AddProposals` (Mig 38 head 20260528082726) → `ExtendApplicableTypeForWorkflowApps` (Mig 37 20260528082332) → `AddMeetingRooms` (36) → `AddHrmConfigs` (35) → `AddEmployeeProfiles` (34). (d) **3 NEW endpoint smoke ALL 200**: `GET /api/proposals?page=1&pageSize=10` 200 0.38s → 0 items empty paged list expected + `GET /api/approval-workflows-v2?applicableType=4` 200 0.39s → 1 row `QT-DX-V2-001` "Quy trình duyệt Đề xuất mẫu V2" (sample seed Phòng CCM × 1 Cấp Lê Văn Bình). (e) **Menu seed Off_DeXuat verify MATCH 4 row**: Off_DeXuat_List + Off_DeXuat_Create + Off_DeXuat_Inbox + Off_DeXuat (root). (f) **Sample workflow Proposal V2 seed MATCH**: COUNT=1 row `QT-DX-V2-001` ApplicableType=4. **0 prod regression Run #246.** Pattern 12-bis cross-module mirror PE V2 → Proposal V2 cookie-cutter STRONG: BE+Mig 37 enum extend + Mig 38 Proposal table + FE × 2 1 commit cohesive, 1 sample workflow seed + 4 menu hierarchy clean. Stage 4.6 sample seed verify PASS (Mig 38 SeedSampleProposalWorkflowV2Async correctly INFRASTRUCTURE-gated, NOT inside DemoSeed disabled per gotcha #51 fix). Poll loop 6 iter × 30s ~3m wait running→success transition smooth. Token cost ~13K (Read MEMORY + git log + Gitea API task poll × 7 + auth login + 3 smoke + 2 endpoint + 2 sqlcmd + 1 bundle hash). Tag: `[s37, run246, pass, mig-37-38, proposal-v2, full-stack]`. **Cumulative confirm:** Pattern 12-bis cross-module mirror PE V2 → Proposal V2 verified end-to-end prod live; ApplicableType enum extend Mig 37 (add ProposalGeneral=4) + Proposal table Mig 38 + Workflow V2 ApproveV2 wire reuse from PE/Contract.
|
||
|
||
- **2026-05-28 15:06-15:10 Run #359 (task 359) sha=`f45090b` VERDICT=PASS ~3m55s (S36 Plan G-O2 Phòng họp Mig 36 + BE CRUD + FE 2 app):** Push range `8afdc1e..f45090b` 1 commit (scope `Domain+App+Infra+Api+FE-Admin+FE-User` full-stack). **Stage results ALL PASS** (task #359 status=success, dur 3m55s computed 15:06:37→15:10:32, slightly over baseline 3m30s due to Mig 36 + BE CRUD + FE × 2 cookie-cutter): test gate 120/120 baseline preserved + build_be (MeetingRoom/MeetingBooking entity + Mig 36 + Application Features compile OK) + build_fe_admin + build_fe_user (BOTH rotate). **Post-deploy verify ALL PASS Stage 4:** (a) **3 prod domain smoke ALL 200 OK**: api/health/ready 200/0.33s + admin 200/0.24s + eoffice 200/0.27s. (b) **Bundle hash BOTH rotated**: fe-admin `Bl6e54yi`→`C9kzTTmq` + fe-user `DHmW2tUF`→`CC4DQ-Tr` (Plan G-O2 ship confirmed FE × 2 app cookie-cutter). (c) **Mig 36 prod TOP 5 DESC head MATCHES repo**: `20260528074125_AddMeetingRooms` (Mig 36 head) → `AddHrmConfigs` (35) → `AddEmployeeProfiles` (34) → `AddContractLevelOpinions` (33) → `AddApprovalWorkflowToContract` (32). (d) **2 NEW endpoint smoke ALL 200 OK**: `GET /api/meeting-rooms` 200/0.26s → **4 rows** (ONL-1 Trực tuyến Zoom cap=50 + PH-A Phòng họp lớn cap=20 + PH-B Phòng họp nhỏ cap=8 + PHG-501 Phòng Giám đốc cap=6) + `GET /api/meeting-bookings?roomId=<first>&startDate=2026-05-28&endDate=2026-06-04` 200/0.29s → **0 rows** (no booking yet, expected). (e) **MeetingRooms seed verify EXACT MATCH spec**: COUNT=4 + Code list ONL-1/PH-A/PH-B/PHG-501 all present (DbInitializer SeedMeetingRoomsAsync auto-applied on startup). (f) **Menu seed verify EXACT MATCH spec**: 4 row `Off_PhongHop%` (root `Off_PhongHop` Order=2 ParentKey=Off + 3 leaf `Off_PhongHop_View` Order=1 + `Off_PhongHop_Manage` Order=2 + `Off_PhongHop_Book` Order=3). **0 prod regression observed Run #359.** **Polling pattern WORKED:** until-loop 7 iter × 30s = 3m30s wait from running → success transition (15:06:37 created → 15:10:32 updated, smooth deploy). Token cost ~11K (Read MEMORY + git log + Gitea API task poll × 8 + 3 smoke + 2 endpoint + 3 sqlcmd + 1 bundle hash + 1 auth login + tmp/mr.json parse). Tag: `[s36, run359, pass, mig-36, meeting-rooms-bookings, full-stack]`. **Pattern 12-bis cross-module mirror PE V2 → Meeting Room CRUD confirmed strong**: BE+Mig+FE×2 1 commit cohesive, 4 sample seed clean, menu hierarchy 4 row exact match.
|
||
|
||
- **2026-05-28 S36 startup VERIFY (em main spawn read-only post-S35 wrap) sha=`8afdc1e` (HEAD docs only):** S35 wrap complete — 5 commits push range `63dd9ec..8afdc1e`. **Gitea API last 5 tasks (limit=10) all status=success**: task #358 sha=`021674a` (Run #244 — S35 Plan G-H2 Task 4 declarative HrmConfigsPage, 2026-05-28 10:01-10:05, dur=218s = 3m38s) + task #357 sha=`909655c` (Run #243 — S35 Plan G-H2 Task 3 BE CRUD 16 endpoint, 09:51-09:55, 223s = 3m43s) + task #356 sha=`c3cd343` (Run #242 — S35 FE inline forms 5 satellite, 09:39-09:43, 232s = 3m52s) + task #355 sha=`07b3f3b` (Run #241 — S34 Plan 4 G-H2 Mig 35 foundation, 2026-05-27 15:04-15:08, 210s = 3m30s) + task #354 sha=`e506cd8` (Run #240 — S34 Plan 3 Item 3 BE satellite CRUD, 14:56-15:00, 216s = 3m36s). **Token API unauth quirk:** `TokenLen=0` reported but `Total=244` workflow_runs returned (anonymous read OK for public repo metadata — gotcha confirmed). Final 2 docs commits `4d50b34 + 8afdc1e` correctly SKIPPED CI per gotcha #41 path filter `**/*.md` (no task #359 in list — verified). **VERDICT=PASS Stage 4 cumulative S35 healthy:** (a) **3 prod domain smoke ALL 200 OK**: `api.solutions.com.vn/health/ready` 200/0.37s + `admin.solutions.com.vn` 200/0.50s + `eoffice.solutions.com.vn` 200/0.31s. Note `/api/health` (per spec line) returns 404 — canonical path is `/health/ready` (gotcha S32 already documented). (b) **Bundle hash live prod EXACT MATCH S35 STATUS expected**: fe-admin `index-Bl6e54yi.js` ✓ + fe-user `index-DHmW2tUF.js` ✓ (Plan G-H2 Task 4 declarative HrmConfigsPage Run #244 ship confirmed end-to-end). (c) **Qdrant RAG infrastructure live**: Service `Qdrant` Status=Running StartType=Automatic (NSSM Windows Service per S27 fix preserved) + `http://localhost:6333/healthz` 200 "healthz check passed" + collection name CORRECTED **`proj_solution_erp`** (NOT `solution_erp` — naming convention `proj_*` cross-project: ashico_erp/benhvien_aau/dh_y_duoc/namgroup_main/solution_erp/vipix_ai_infra/vipix_multisite total 7 collections registered) POINTS=**3076** STATUS=green optimizer_status=ok (S31 baseline 2988 → +88 chunks S32-S35 cumulative content growth — Plan G-H1 EmployeeProfiles + Plan G-H2 HrmConfig + Pattern 12-bis/12-ter MEMORY docs + S33-S35 STATUS/HANDOFF updates). (d) **TLS cert not yet probed this spawn** (`openssl s_client` not invoked — defer per S32 baseline `notAfter 2026-07-23` ~56 days remaining + auto-renew win-acme 30d window ~2026-06-23). **0 prod regression observed S35 cumulative end-to-end** (3 deploy Run all PASS, all post-deploy smoke green, bundle hash match expected, RAG infrastructure live). **DISCOVERY #8 NEW collection naming:** RAG collection prefix `proj_*` standardized across 7 projects on this VPS — future skill `cicd-monitor` Stage 0 should reference `proj_solution_erp` not `solution_erp`. Update Stage 0 spec in agent system prompt. **STANDBY STATE ANNOUNCED:** Awaiting S36 push trigger. Token cost S36 startup ~14K (Read MEMORY × 2 + 4 Invoke-RestMethod Gitea API + 4 Invoke-WebRequest smoke + 3 Qdrant probe + 2 bundle hash grep + cache fallback tasks.json). Tag: `[s36, startup, standby, discovery-8-rag-naming]`.
|
||
|
||
- **2026-05-28 09:51-09:55 Run #243 (task 357) sha=`909655c` VERDICT=PASS ~3m43s (S35 Plan G-H2 Task 3 — HrmConfig BE CRUD 16 endpoint cookie-cutter):** Push range `c3cd343..909655c` 1 commit 2 BE file NEW (+576 LOC, 0 mod): `HrmConfigFeatures.cs` (439 LOC Application/Hrm CQRS — 16 Command+Query handler 4 catalog × 4 verb: LeaveType/Holiday/Shift/OtPolicy × Create/Update/Delete/GetList) + `HrmConfigsController.cs` (137 LOC Api 16 endpoint route `/api/hrm-configs/*`). **Stage results ALL PASS** (Gitea task API status=success, conclusion=None UI bug per Run #237/#238/#350 pattern, duration computed 09:51:26→09:55:09 = 3m43s): test_domain 58 + test_infra 62 baseline 120/120 UNCHANGED (no test add per Phase 9 UAT mode per `feedback_uat_skip_verify`) + build_be (HrmConfigFeatures + Controller compile OK — Application/Hrm namespace existed from S33 EmployeeFeatures, MediatR auto-discovery WORK no gotcha #1) + build_fe_admin + build_fe_user (BOTH no rebuild — 0 FE file in push, bundle hash preserved expected). **Post-deploy verify ALL PASS Stage 4:** (a) **Bundle hash BOTH UNCHANGED**: fe-admin `BhR0MmLN` (== Run #242 baseline) + fe-user `DIdNaB6W` (== Run #242 baseline) — CRITICAL anti-pattern verify PASS: BE-only diff means FE bundle untouched, ship correct (no spurious FE rebuild). (b) **3 prod domain smoke ALL 200 OK**: api/health/ready 200 0.16s + admin 200 0.11s + eoffice 200 0.11s. (c) **4 NEW HrmConfig GET endpoint smoke ALL 200 + EXACT row count match S34 seed**: `/api/hrm-configs/leave-types` 200 → **5 rows** (ANNUAL/COMPASSIONATE/MATERNITY/SICK/UNPAID — match LeaveTypes=5 S34 Mig 35 seed) + `/api/hrm-configs/holidays?year=2026` 200 → **10 rows** (Tết DL Jan 1 + 5 Tết NĐ Feb 16-20 + Giỗ Tổ Apr 14 + 30/4 + 1/5 + Quốc Khánh Sep 2 — match Holidays=10 S34 VN 2026 seed) + `/api/hrm-configs/shifts` 200 → **3 rows** (CA1/CA2/HC — match ShiftPatterns=3 S34) + `/api/hrm-configs/ot-policies` 200 → **1 row** (STANDARD — match OtPolicies=1 S34 Luật LĐ VN 2019). **Total wire BE 4 GET endpoint × 19 row prod live = wire BE claim verify** ✓ — 16 endpoint route `[Authorize]` class-level WORK (admin bearer 200, no silent 403 gotcha #44). (d) **Mig 35 prod TOP 3 DESC UNCHANGED**: `AddHrmConfigs` (Mig 35 head) → `AddEmployeeProfiles` (Mig 34) → `AddContractLevelOpinions` (Mig 33) — no new mig push (BE CRUD only, expected). (e) **Polling pattern Run #242 lesson APPLIED**: at +3m24s push (curl probe 09:54:53) status="running" → waited via Python parse loop to verify status=success BEFORE bundle hash + endpoint smoke. Run #242 false-positive trap avoided. **0 prod regression observed Run #243.** Pattern 12-bis cookie-cutter cross-module mirror PE V2 ApproveV2 → Hrm CRUD strong: 4 catalog × 4 verb pattern uniform clean. **Cumulative S35 deploy:** 2× Run (#242 FE inline forms + #243 BE CRUD 16 endpoint). Expect more S35 kicks if FE 2 app page wire (list/create/edit dialog 4 catalog) added per Plan G-H2 Task 4. Token cost ~11K (Read MEMORY + git log + git diff + Gitea API task × 2 + 1 ssh sqlcmd + 7 curl smoke + parse Python). Tag: `[s35, run243, pass, be-only, hrm-configs-crud, 16-endpoint]`. **Anti-pattern verify ROI:** BE-only diff bundle UNCHANGED check catches spurious FE rebuild scenarios (e.g. shared Vite config touch, dependency cache bust). Run #243 confirmed clean — no spurious rebuild.
|
||
|
||
- **2026-05-28 09:39-09:42 Run #242 sha=`c3cd343` VERDICT=PASS ~3m30s (S35 Phase 1.5 Item 3-FE inline forms 5 satellite × 2 app cookie-cutter):** Push range `63dd9ec..c3cd343` 1 commit 4 FE files (+1560/-198 LOC): fe-admin EmployeesListPage.tsx 573→1200 +627 + fe-user mirror SHA256 IDENTICAL + types/employee.ts × 2 app +53 LOC each. **Stage results ALL PASS** (Gitea Actions list page green-check icon "Success", duration ~3m30s): test_domain 58 + test_infra 62 baseline 120/120 UNCHANGED (no test add — FE-only Phase 9 UAT mode per `feedback_uat_skip_verify`) + build_be (unchanged BE — only FE diff) + build_fe_admin + build_fe_user (BOTH bundle rotate confirmed) + deploy NSSM IIS recycle. **Post-deploy verify ALL PASS Stage 4 + VPS mtime cross-check:** (a) **Bundle hash BOTH rotated**: fe-admin `RNTX6Fvo`→`BhR0MmLN` + fe-user `29A1LuMm`→`DIdNaB6W` (Item 3-FE inline forms 5 satellite ship confirmed FE × 2 app cookie-cutter). CSS hash also rotated (admin `DQSzWczW`→`Dlk3cj6D` + user `D-PasqRK`→`25W039gj`). (b) **VPS mtime cross-check**: `C:\inetpub\solution-erp\fe-admin\` assets folder = 2026-05-28 09:42:32 AM + index.html = 09:41:37 AM + `fe-user\` assets = 09:42:32 + index.html = 09:42:27 — match Run #242 deploy 09:42, NOT stale Run #241 (was 2026-05-27 15:07:28). NSSM service `Restart-WebAppPool` + xcopy ship verified end-to-end. (c) **3 prod domain smoke ALL 200 OK**: api/health/ready 200 "Healthy" 0.32s + admin 200 0.32s + eoffice 200 0.31s. (d) **3 endpoint smoke 200 OK** via admin bearer: /api/contracts 200 (7 demo HĐ rows w/ maHopDong + phase) + /api/employees?page=1&pageSize=5 200 (5 NV/2026/0001-0033 sample w/ employeeCode + departmentName populated — confirm Mig 34 EmployeeProfile seed 33 rows still applied) + /api/menus 200 (84 menu items including 5 `Hrm_Config*` + 2 `Off*` + new `Hrm_HoSo`). Token = `accessToken` (NOT `token` — gotcha preserved). (e) **Mig 35 prod TOP 1 UNCHANGED** = no BE Mig in push (FE-only diff). Plan 4 Phase 1.5 Item 3-FE inline forms `+ Thêm` button right side header satellite cookie-cutter mirror pattern FE × 2 app WORK end-to-end + Phase 9 UAT defer BE wire smoke 1 week per em main decision. **Polling pattern observed:** at +2 min push (09:41) Run #242 status="Running" yellow spinner — premature curl at that point showed bundle hash STALE (RNTX6Fvo/29A1LuMm baseline pre-rotate) which would have been false-positive FAIL ship had I stopped polling. **Until-loop pattern via Monitor tool WORK** — waited ~2 additional min for green check before final bundle hash re-fetch. Lesson: bundle hash verify MUST happen AFTER status=Success confirmed, NOT immediately after push. **0 prod regression observed Run #242.** Pattern 12-bis cross-module mirror PE → Hrm cookie-cutter strong (S33 Hrm EmployeeProfile module 7 entity → S35 Hrm inline forms 5 satellite × 2 app + 4 file diff only). **Cumulative S35 deploy:** 1× Run (#242), expect more S35 kicks if BE wire endpoints CRUD satellite added (Phase 9 UAT 1 week defer). Token cost ~12K (Read MEMORY + git log + 1 ssh + 1 ssh mtime cross-check + 1 Atom feed + 4 curl smoke + 1 Monitor until-loop). Tag: `[s35, run242, pass, fe-only, hrm-inline-forms]`. **Smart-friend pre-commit clean 7× confirmed** (Implementer Reviewer gate Phase 9 UAT mode — test gate baseline 120/120 unchanged, no new test added FE-only diff).
|
||
|
||
- **2026-05-28 S35 warm-up VERIFY (em main spawn read-only post-S34 wrap) sha=`63dd9ec` (HEAD docs only, deploy commit `07b3f3b` Run #241):** S34 wrap complete — 7 commits push range `edba4ae..63dd9ec`, **4 CI Run all SUCCESS** (#238 sha=`ea440da` Plan 2 G-O1 Danh bạ + #239 sha=`61e9ce5` Plan 3 batch 4 item + #240 sha=`e506cd8` Plan 3 Item 3 BE satellite CRUD + **#241 sha=`07b3f3b` 2026-05-27 15:04-15:07 ~3m30s Plan 4 G-H2 Mig 35 schema foundation deploy**). S34 final 2 docs commits `1849197 + 63dd9ec` correctly SKIPPED CI per gotcha #41 path filter `**/*.md`. **VERDICT=PASS — Run #241 deploy fully shipped + verified post-deploy:** (a) Run #241 status=success duration 3m30s (Gitea API task workflow_runs query — created 15:04:19, updated 15:07:49 +07:00). (b) **3 prod domain smoke ALL 200 OK**: `api.solutions.com.vn/health/ready` returns "Healthy" 200/0.47s + `admin.solutions.com.vn` 200/0.33s + `eoffice.solutions.com.vn` 200/0.22s. (c) **Mig 35 prod TOP 5 DESC** sqlcmd via `ssh vietreport-vps 'sqlcmd -S ".\SQLEXPRESS" -d SolutionErp -E -h -1 -Q "..."'` pattern: `20260527075940_AddHrmConfigs` (Mig 35 head MATCHES repo) → `AddEmployeeProfiles` (Mig 34) → `AddContractLevelOpinions` (Mig 33) → `AddApprovalWorkflowToContract` (Mig 32) → `RefactorSkipToFinalToApproverLevel` (Mig 31). (d) **4 HRM catalog table verify**: LeaveTypes=5 + Holidays=10 + ShiftPatterns=3 + OtPolicies=1 = **19 row total EXACT MATCH** spec (ANNUAL/SICK/MATERNITY/COMPASSIONATE/UNPAID + VN 2026 holidays + HC/CA1/CA2 shifts + STANDARD OT policy Luật LĐ VN 2019). (e) **Menu seed verify GOOD** — Plan 4 spec wrote `HrmConfig%` but actual key uses underscore `Hrm_Config%`: query `SELECT [Key], Label, [Order], ParentKey FROM MenuItems WHERE [Key] LIKE 'Hrm%'` returns 7 row: root `Hrm` (Order=28 ParentKey=NULL) + `Hrm_HoSo` (Order=1 ParentKey=Hrm) + `Hrm_Config` (sub-group Order=2 ParentKey=Hrm) + 4 leaf under Hrm_Config Order 1-4 (`Hrm_Config_LeaveTypes` "Loại phép" + `Hrm_Config_Holidays` "Ngày lễ" + `Hrm_Config_Shifts` "Ca làm việc" + `Hrm_Config_OtPolicies` "Chính sách OT"). **5 G-H2 menu rows present** (sub-group + 4 leaf) — Pattern 16-bis MenuKeys.cs sync OK BE+FE. **Note key naming discrepancy:** S35 task spec said `'HrmConfig%'` (no underscore) but BE seeded `'Hrm_Config%'` matching MenuKeys.cs constants — verified by reading commit `07b3f3b` message body. Recommend STATUS.md S35 task description fix key prefix. (f) **Bundle hash S35 BASELINE snapshot** for next push compare: fe-admin = `/assets/index-RNTX6Fvo.js` + fe-user = `/assets/index-29A1LuMm.js`. Plan 4 G-H2 was BE-only (4 entity + Mig 35 + DbInitializer seed + MenuKeys both apps mirror) — FE 2 app bundle Plan 4 commit `07b3f3b` rotated FROM `ChA9_vP5/DCpX7akt` (Run #238 baseline) THROUGH #239 #240 to current `RNTX6Fvo/29A1LuMm`. (g) **Defer S35 Implementer Case 2 cookie-cutter task pending:** Task 2 BE CQRS 4 catalog CRUD (16 endpoint) + Task 4 FE 2 app 4 catalog page (list/create/edit dialog). Em main spawn cicd-monitor next when first S35 G-H2 wire push hits. **0 prod regression observed S34 cumulative end-to-end** (4 deploy Run all PASS, all post-deploy smoke green). **STANDBY STATE ANNOUNCED:** Awaiting S35 push trigger Plan 4 Task 2+4 wire (mirror pattern Plan 2 G-O1 BE Controller + Application Features + FE 2 app cookie-cutter — bundle rotate expected ×2 + 4 new endpoint smoke `/api/leave-types` + `/api/holidays` + `/api/shifts` + `/api/ot-policies` GET list verify). Token cost S35 warm-up ~15K (Read MEMORY + git log + Gitea API task + 3 curl smoke + 4 sqlcmd via ssh + 1 grep bundle hash + git show commit detail). Tag: `[s35, warm-up, standby, mig-35]`.
|
||
|
||
- **Archived 7 verbose Run entries → `archive/2026-05-q4.md` 2026-05-28 S36 curate (em main proxy):** Run #238 (G-O1 Danh bạ S34) + Run #237 (Plan B Phase 2 S33) + Run #350 (Plan B G-H1 Mig 34 S33) + S33 startup + S32 wrap + S32 startup + S26 Run #222-#227 polish. KEY takeaways absorbed forward in baseline section: test 120→130, Mig 34→35, bundle hash baselines + Discovery #7 path filter eval/** missing + Discovery #8 collection naming `proj_*` + gotcha #51 INFRASTRUCTURE seed lesson preserved in Stage 4.6 + 10-surface-point checklist intact.
|
||
|
||
- **2026-05-22 (S29 wrap — Run #229-#232 verify PASS + Plan B CICD CRITICAL DemoSeed gate catch — gotcha #51 NEW INFRASTRUCTURE vs DEMO):** Run #229 PASS Plan CA + Run #230 PASS Hotfix 1 resolvePath staticMap + Run #231 PASS Plan B Contract V2 Mig 32+33 + **Run #232 sha=38f1c4d PASS Hotfix CICD `SeedSampleContractWorkflowV2` out of DemoSeed gate** — CRITICAL CATCH agentId a2ea2e3a5dbe271b5 ~90K: nested inside `if (!demoSeedDisabled)` DbInitializer.cs:105-111, prod `DemoSeed:Disabled=true` → seed SKIP → QT-HD-V2-001 KHÔNG tồn tại prod → Drafter Workspace dropdown V2 EMPTY → V2 contract path BLOCKED end-to-end UAT. **Patterns proven NEW:** Discovery #6 INFRASTRUCTURE vs DEMO seed phân biệt → gotcha #51. **Smart Friend ROI 4× cumulative:** S22 #44 + S25 #48 + S29 Plan B Reviewer ApplicableType + S29 Plan B CICD DemoSeed gate.
|
||
|
||
- **Archived to `archive/2026-05-q3.md` 2026-05-27 S34 curate (em main proxy):** Run #215+#216 (gotcha #48 SQLite tie-break catch+fix pair, S25 t1-t2) — full detail preserved in archive. KEY lesson cumulative trong foundation gotcha #48 line 40-45 (test discriminator filter Summary contains "Chuyển phase"). 2026-05-13 S22 chốt cuối verify (Run #214 baseline + Discovery #3 docs-only skip). 2026-05-12 setup baseline.
|
||
|
||
---
|
||
|
||
## 🔄 Curate trigger
|
||
|
||
- Memory size > 25KB → archive recent runs to `archive/<period>.md`
|
||
- Duplicate failure patterns → merge into single entry (vd act_runner timeout x3 → 1 entry)
|
||
- Stale > 3 months → remove
|
||
|
||
**Last curate: 2026-05-27 S34 em main proxy curate** (post-S33 wrap, sequence 1/4) — archived Run #215+#216 verbose detail (gotcha #48 catch+fix pair, KEY lesson preserved in foundation line 40-45) + 2026-05-13 S22 verify + 2026-05-12 setup → `archive/2026-05-q3.md`. KEEP in MEMORY: Run #237 + Run #350 (S33 prod deploy), S33 startup health, S32 wrap + S32 startup, S29 wrap, S28 wrap, S26 Run #222-#227, S22 curate. Foundation 10-surface-point per-NV checklist + Stage 0-5 checklist + Stage 4.6 sqlcmd seed verify + Discovery #6 INFRASTRUCTURE vs DEMO seed + gotchas #39-#48 detail all preserved untouched. MEMORY size before: 32.9 KB → after: target ~22-24 KB. **Previous curate: 2026-05-26 S32** — Run #231 PARTIAL detail → `archive/2026-05-q2.md`. **Previous curate: 2026-05-22** — Run #186-#221 → `archive/2026-05-runs.md`. Next trigger: > 25KB OR Plan G-O1 Danh bạ kick off.
|