vietreport-admin/solution-erp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2a7269581c96bd0d3ce939b4dceaeb847df76a89
solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md
pqhuy1987 b5a7dd8a92 wal: flush 20260729T1207
2026-07-29 12:07:47 +07:00

18 KiB
Raw Blame History

CI/CD Monitor Agent — Persistent Memory

  • 07-29 verify b5799fc (FE-User: Tổng quan tất cả luồng duyệt — pipeline 4 GĐ) [S159 L8, FE-USER-ONLY 1 file-code, ghi-đĩa-suốt-lượt]: PASS 9/9 — Run #422 (id=535) success 5m59s (11:43:34→11:49:33+07); gate 562 EXACT (45 D + 517 I, Failed!×0 toàn log). NO-Mig đúng kỳ vọng (diff src/Backend/*+*Migrations*=0 ⇒ KHÔNG query DB; Mig 68/89-table). Range 8 commit/37 file (7 wal:) — chỉ fe-user/src/pages/UserDashboardPage.tsx thoát 7-glob ⇒ CI fire đúng. 🔑 KỸ THUẬT MỚI — marker PHÂN-BIỆT-PHIÊN-BẢN (mạnh hơn marker-hiện-diện): bản designer gốc ghi Pipeline 4 giai đoạn, bản sau review-fix ghi Toàn trình 4 giai đoạn ⇒ bundle prod có Toàn trình=1 & Pipeline=0 ⟹ chứng prod chạy bản ĐÃ-FIX, không chỉ 'có nội dung mới'. Dùng khi review sửa CHỮ: chọn cặp chuỗi cũ/mới làm phép phân-biệt. ⚠️ Bẫy comment-strip: grep -c source ra 2 mà bundle=1 — occurrence dư nằm ở COMMENT đầu file (minifier strip) ⇒ so bundle với occurrence-CODE, đừng so grep-source thô. ⚠️ Bẫy hình-dạng-truy-vấn (suýt hô cờ đỏ): control Pe ra 182/182 vs kỳ vọng 197/185 — do tao dùng LEFT(MenuKey,3)='Pe_' còn gốc là LIKE 'Pe%' (+Permissions1/1+Personal13/1+PeWorkflows1/1=+15/+3). LỆCH-số ⇒ truy PHẠM-VI-QUERY trước khi kết luận drift. Bundle: user js DptYR4wL→0GO8mgrG css BRCvEHvC→BARVqkP- (1.584.514B, LM 11:49:05 in-window); admin js 8-QYcMOg→BHodkPNV css DiNJjaoo FROZEN (#69 mode-A + REFINE). Control 2 tầng: fake-hash & hash CŨ đều ct=text/html 876B ⇒ bundle cũ biến mất = chứng ghi-đè. Smoke health live+ready 200 + 2 FE 200. M7 menu-data (data-change NGOÀI CI, verify sống sót recycle): 3 root IsVisible=0 · 38-key grant min=max=13 · Pe% 197/185. Evidence: runs/2026-07-29-S159-tong-quan-pipeline-menu/cicd-verify-b5799fc.md. Tag [run-422, pass-9-9, marker-phan-biet-phien-ban, comment-strip-trap, query-shape-not-drift, gate-562]

  • (S157 #419 a52a91e PE cảnh-báo amber ô Địa-điểm (FE 2-app) GO 9/9 — gate 562 [45+517, khớp canonical STATUS cả tách-phần]; NO-Mig; 🔑 byte-marker count=2/app = khớp SỐ FILE trong diff ["đúng-BỘI-SỐ" > "tìm thấy ≥1"]; js-rotate ×2 css-FROZEN ×2; control âm 2 tầng [fake-hash + hash CŨ đều ct=text/html ⇒ bundle cũ biến mất = chứng ghi-đè]; self-correct on-disk "6 file"→13 commit/26 file · verbatim → archive/2026-07.md @S159-curate)

  • (#416 S155 b1bae77 Workflow duyệt-TẠI-CHỖ +Mig 68 APPLIED-PROD 6/6 [gate 551; FLAG 'mig rớt khỏi commit' loại-trừ 3 lớp — mạnh nhất GET /approval-workflows-v2 trả cột ×54 materialize qua EF từ DB thật] · #407 S135 4510cea mã-dự-án normalize 5/5 [gate 532] · #399 S134 6664a61 PE lũy-kế 0-mig 5/5 [gate 528; 🔑 ct=text/html discriminator SẠCH HƠN size-919B vì shell-size drift] · #398 S133 d4fbced +Mig 67 budget-freeze 5/5 [gate 520; backfill 18/18; python text=True crash cp1252 → dùng bytes] · verbatim → archive/2026-07.md via _INDEX)

Persistent diary cross-session. Auto-injected first ~200 lines at spawn (L1 HOT). Update BEFORE every stop. Tiered Memory v1: L1 HOT soft-cap ~30KB · L2 archive/ on-demand · L3 RAG search_memory just-in-time. Keep entry ≤ 1.5K chars (gotcha #53). Full verbatim run history pre-S40 → git d2f52ba + archive/2026-05-{runs,q2,q3,q4}.md + archive/2026-06.md + archive/2026-07.md. 🗺️ Lookup map: archive/_INDEX.md — 1 dòng/bản-ghi + con-trỏ substring (sha-keyed, Ctrl-F fallback); đọc verbatim + 2026-0{5,6}.gist.md theo nhu cầu. See 🔄 Curate-trigger log for archive-move history.

🎯 Role baseline

Read-only CI/CD + post-deploy verifier SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch + 5 RAG MCP. Output: PASS/FAIL + evidence <500 words. Skills: iis-deploy-runbook + dependency-audit-erp + ef-core-migration. Spawn ~150K — trade-off catch fail tự động.

⚠️ cicd-lane return-garble tracker (#53)

cicd-lane đôi khi trả nhầm slice transcript (memory-curation) thay verdict — tao TÁI PHẠM dai nhất trong roster (×5/6 qua S114; S117/S118 ×3 nữa; cumulative all-agents ×66-cận-dưới → feedback_agent_return_garble_recover). ⚠️ Từng tự dán nhãn 'CLEAN-return' SAI (harvest-curator bắt S118) ⇒ KHÔNG tự phong return sạch — mình không quan sát được return của chính mình. 🔒 Đối-sách đã chứng minh (S158, xác nhận lần 5): ghi-đĩa TRONG LÚC LÀM. Prompt đúng KHÔNG chặn được garble; artifact mới chặn — vai có-ép-ghi-đĩa mất 0 B, vai không-ép mất TOÀN BỘ. ⇒ Mỗi lượt: file evidence trong run-folder là SẢN PHẨM CHÍNH, ghi từng mục ngay khi có bằng chứng, kết bằng dòng END … VERDICT=; return chỉ là tóm tắt. Recover khi garble: em-main re-curl + sqlcmd 2-nguồn-khớp, hoặc đọc thẳng file evidence.

🚨 Recurring CI/CD bug patterns (catch priority)

  • #39 act_runner github.com TCP timeout — run hang "Set up job" 21s. Log dial tcp github.com:443 i/o timeout. Fix: manual checkout bypass hardcoded .gitea/workflows/deploy.yml. KHÔNG revert.
  • #40 npm cache tsc not found — build_fe_admin fail post cache: npm. DISABLED rolled back a21790d. KHÔNG re-enable.
  • #41 paths-ignore docs-only skip — code commit không trigger CI? Check git diff --name-only HEAD~1 HEAD vs paths-ignore 7-entry từ D10 S111 8aa3869 (#495): ['docs/**','**/*.md','.claude/skills/**','.claude/agent-memory/**','.claude/workflows/runs/**','.gitignore','scripts/**.md']. Gitea evaluates push range — ≥1 commit có non-ignored file → toàn range build. ⚠️ AS-6 BASELINE SHIFT: từ #495, memory-push chứa memory-budget.json/agent-memory-file/run-trace HẾT trigger CI (pattern #490/#492 DEAD).
  • #25 IIS WebSocket — notification-hub/negotiate 401/404 prod. Fix: WebSocket module enable web.config site api (skill iis-deploy-runbook).
  • #48 SQLite tie-break — OrderByDescending(CreatedAt).First() pick wrong khi 2+ .Add() cùng frozen-clock. Fix: discriminator filter BEFORE OrderBy.
  • Bundle hash unchanged = ship FAIL — push+action success nhưng prod không đổi. Verify via INDEX.HTML ref (curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[A-Za-z0-9_-]+\.(js|css)'), NOT by GETting a hash-named asset directly. ⚠️ verify MUST sau status=success (Run #242 false-positive: check khi "running" → stale hash).
    • 🔴 #69 TWO-MODE (S72 #312 + S83 #333) — FE hash vừa có thể ROTATE vừa có thể FROZEN. deploy.yml copy dist\* UNCONDITIONALLY mọi run (path-filter gate cả workflow, không per-step) ⇒ 0-FE-change vẫn rotate được (mode-A); ngược lại src y hệt ⇒ Vite ra byte-identical ⇒ hash FROZEN (mode-B). Cả hai lành tính ⟹ hash-delta KHÔNG phải ship-proof. Ship-proof THẬT = Last-Modified in deploy-window + real-size + byte-marker. Muốn biết FE có đổi thật → diff fe-*/src trong range.
    • ⚠️ SPA-fallback 200 trap (S72): server rewrite /*→index.html ⇒ GET /assets/index-<BỊA>.js vẫn 200. Discriminator sạch = Content-Type (text/html ~876-919B = trap · application/javascript ~1,6MB = thật); size drift per-build nên đừng neo số byte. Hash CŨ rơi về text/html = bundle cũ biến mất khỏi đĩa = chứng ghi-đè (control âm tầng-2).
  • Migration drift prod vs repo — compare ls .../Persistence/Migrations/*.cs vs sqlcmd __EFMigrationsHistory. Fix: check Program.cs/DbInitializer app.MigrateDatabase() + app pool recycle. Mig-applied proof = __EFMigrationsHistory top==repo-HEAD; stuck-old ⟹ pool didn't recycle ⟹ FAIL even if status=success+bundle-rotated.

📋 5-stage checklist (EVERY run)

  • Stage 0 RAG infra: Get-Service Qdrant Running + http://localhost:6333/healthz. Collection proj_solution_erp.
  • Stage 1 Push+filter: git log -1 --format='%H %s' + git log origin/main..HEAD empty + diff vs paths-ignore (docs-only → SKIPPED-DOCS return).
  • Stage 2 Gitea poll (max 10 iter × 60s): API .../actions/tasks?limit=5 (NOT /runs 404). Match head_sha (NOT run_number — cancelled runs shift numbering). ⚠️ updated_at stale ~2min (gotcha #46) → cross-check VPS mtime. Foreground-sleep BLOCKED Windows → Monitor-style busy-wait-curl-spin ~30s/iter.
  • Stage 3 Test gate: baseline 562 PASS (45 Domain + 517 Infra; run #422 b5799fc 07-29; climb 562/551/532/528/520/509/505/501/495/486/458/440/421). CI chạy cả 2 proj TRƯỚC build/deploy ⇒ status:success = gate passed (tasks KHÔNG populate conclusion — tin status). Local grep undercount (Theory/InlineData). Phase 9 UAT skip OK.
  • Stage 4 Post-deploy (if SUCCESS): auth login bearer (admin + nv.test gotcha #44; token=accessToken route /api/auth/login) → 3-5 endpoint smoke 2XX (incl new — NEW-endpoint bodyless-POST returns 411 IIS Length-Required pre-auth NOT 404; re-probe WITH body → 401 confirms wired) → FE bundle hash 2 app (per #69 = real-size-vs-fake not hash-delta) → SignalR negotiate (gotcha #25) → EF mig prod==repo.
    • Stage 4.6 (S29 CRITICAL): sqlcmd seed sample verify post-deploy (NOT chỉ schema). Seed=runtime-row-insert (no history/tables advance) → verify via DB-query (e.g. Permissions count), NOT bundle-frozen-alone.
    • Discovery: ASP.NET 10 record enum needs numeric input unless JsonStringEnumConverter (SOL has NO converter → FE sends numeric). sqlcmd ssh Windows-auth \\\\SQLEXPRESS 4-backslash. INFRASTRUCTURE seed MUST run (NOT inside if(!demoSeedDisabled)); DEMO seed gated → gotcha #51.
  • Stage 5 Report PASS/FAIL + evidence + MEMORY update.

⚠️ Anti-patterns (DO NOT)

  1. ❌ Push fix code — READ only, escalate em main · 2. ❌ Speculate fail without log · 3. ❌ Skip post-deploy bundle hash (biggest catch) · 4. ❌ Skip MEMORY · 5. ❌ Poll forever (max 10 iter) · 6. ❌ Auto-rollback (escalate + recommend) · 7. ❌ Verify docs-only (SKIPPED-DOCS return ngay)

🧠 SOLUTION_ERP CI/CD essentials (S80 verified — re-ground từ docs/STATUS.md canonical)

  • Gitea: git.baocaogiaoduc.vn/vietreport-admin/solution-erp · workflow .gitea/workflows/deploy.yml · paths-ignore 7-entry (D10 S111 #495): ['docs/**','**/*.md','.claude/skills/**','.claude/agent-memory/**','.claude/workflows/runs/**','.gitignore','scripts/**.md']. Anon API works (public repo) when GITEA_TOKEN absent.
  • Prod: api/admin/eoffice .solutions.com.vn · SSH ssh vietreport-vps (Administrator, id_ed25519) · IIS phys paths: API C:\inetpub\solution-erp\api · admin \fe-admin · user \fe-user. DB .\SQLEXPRESS/SolutionErp/vrapp SQL-auth. Conn key = ConnectionStrings.Default (NOT DefaultConnection). DB pw = buKL3TGBkD0wDDbYVw65QeX9 (User Id=vrapp; ⚠️ Password field has NO vrapp/ prefix — S89 login-failed when prefix included; read verbatim from prod appsettings.Production.json ConnectionStrings.Default when $env:PROD_DB_PASSWORD empty). ⚠️ skill-doc path C:\inetpub\apps\SolutionErp\Api STALE.
    • SSH→PS quoting (S42): nested bash→ssh→powershell mangles $var/\" → use iconv UTF-16LE | base64 → powershell -EncodedCommand $B64; OR write ps1-file + powershell -File $(cygpath -w). sqlcmd over SSH direct -W -h -1 pw-inline. sys-catalog string-concat → COLLATE DATABASE_DEFAULT (collation conflict).
  • Tests baseline: 562 PASS (45 Domain + 517 Infra · 0 fail/skip; run #422 b5799fc 07-29; khớp canonical docs/STATUS.md). 🔑 Số cụ thể lấy ở web-UI log https://git.baocaogiaoduc.vn/vietreport-admin/solution-erp/actions/runs/{run_number}/jobs/0/logs (~18KB, KHÔNG có /api/v1/ — api/v1 runs+tasks logs đều 404) → mỗi project 1 dòng Passed! - Failed: 0, Passed: N, Skipped: 0; đối chiếu TÁCH-PHẦN (45/517) chứ đừng chỉ khớp tổng. Delta test mới = so run trước.
  • Mig latest repo: Mig 68 AddPeAllowApproverDelete (20260727033522, S155 b1bae77 — AddColumn AllowApproverDelete bit NOT NULL def-0 trên ApprovalWorkflowLevels, no-new-table; VERIFIED-APPLIED-PROD history-top + col-present + sys.tables 89). Prev Mig 67 AddPeApprovedBudgetSnapshot / 66 AddPeApprovedPriceRange / 65 backfill-invariant / 62 ExpandSupplierFields (full table → docs/CLAUDE.md). Path src/Backend/SolutionErp.Infrastructure/Persistence/Migrations/ (68 .cs excl Designer/Snapshot). Prod check sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC. ⚠️ sys.tables(is_ms_shipped=0)=89 — AddColumn/data-only/enum-additive don't change count; no-new-table commit ⟹ 89 correct, don't FAIL. NO-Mig proof = history NOT-advance + empty git diff <base>..<head> -- '*Migrations*'.
  • Bundle hash live: ship-baseline = run #422 (07-29) admin js BHodkPNV/css DiNJjaoo · eoffice js 0GO8mgrG/css BARVqkP- (user real-size 1.584.514B, LM 11:49:05+07). ⚠️ Đây là SNAPSHOT-per-run, KHÔNG phải mốc ổn định (#69 TWO-MODE) → LUÔN re-snapshot LIVE sau status=success; ship-proof = LM-in-window + real-size + marker, KHÔNG phải hash-delta. ⚠️ id ≠ run_number (key theo head_sha).
  • Bearer: admin admin@solutions.com.vn/Admin@123456 (full) · UAT nv.test@solutions.com.vn/TestUser@123456 (Drafter CCM, gotcha #44 check).
  • API Serilog log (S118 correction): real sink = C:\inetpub\solution-erp\api\logs\solution-erp-YYYYMMDD.log (lowercase logs, rolling daily). ⚠️ skill-runbook path …\Logs\log-YYYYMMDD.txt STALE → empty grep. MinimumLevel Default=Information (DbInitializer LogInformation seed/revoke lines DO persist) — Microsoft/System overridden Warning (no "Now listening" INFO). Startup seed-race audit pattern (durability deploys): grep this log for revoke-line THEN seeder-line same-second → "N added + M upgraded" proves seeder actively flipped rows revoke knocked down (0-upgraded = seeder no-op = FAIL flag).

🔑 Critical config (flag commit nếu tái xuất)

Node CI 20.x (feedback_node_cicd) · MediatR 12.4.1 (gotcha #1, flag Version="14) · Swashbuckle 6.9.0 (gotcha #2) · act_runner manual checkout (#39) · npm cache DISABLED (#40, flag cache: npm)

🎯 Per-NV admin opt-in wire — 10-point checklist (cumulative S22→S23)

Cross-ref feedback_per_nv_permission_scope. Refactor per-NV/per-Level phải verify 10 điểm: Domain field · EF HasDefaultValue(false) · Mig 3-file · Service read · Domain+App DTO mirror · Designer FE checkbox · AwLevelDto+ToDto · Create/Update input · (9) Lookup discrimination (FirstOrDefault thêm ApproverUserId==actorId + admin fallback) · (10) Controller body record count == Command record count. Miss 9-10 ⇒ bug prod im-lặng 2-3 ngày. Sau refactor OR-of-N: grep -n "FirstOrDefault.*Order.*==" *.cs.

📊 Run stats baseline

BE (test+build) ~90s · FE × 2 ~60s/app · deploy ~30s · total ~3min code / 0s docs-only. >5min → escalate. Recent runs all ~4m40-5m.

📅 Recent runs (compressed — verbatim → archive via _INDEX.md)

  • (S98-S132 #383-#397/#485-#496 — GOVERNANCE/FE/SECURITY streak, all GO (gate leo 440→509): 🔑 bài-học còn dùng — AS-6 #41 soi TỪNG file vs 7-glob (.ps1/.json thoát scripts/**.md ⇒ governance non-.md VẪN trigger CI; memory-push thì HẾT trigger từ #495) · conclusion:None dù success ⇒ tin status · 411-refine: endpoint MỚI bodyless-POST trả 411 (IIS Length-Required, pre-auth) KHÔNG phải 404 → re-probe CÓ body, 401/415 ⇒ đã wire · bare /health=404 UNMAPPED, thật là /health/live+/ready (đừng FAIL nhầm) · LM mới hơn cửa-sổ đã verify = detector 'có run mình bỏ sót' · Mig 63/64/65/66 + Mig 67 đều APPLIED-PROD, 89-table giữ · verbatim → archive/2026-07.md via _INDEX)
  • (PASS-pattern library S69b→S104 #307-#494 all GO — 🔑 reusable verify-recipes: SCHEMA-ADD+DATA-UN-CRAM [#488 Supplier 9→27 Mig62: AddColumn×18-exact + 89-invariant + un-cram 4-NCC + DTO-live] · DATA-FIX-BACKFILL [#484 Mig61: Mig-top==HEAD + target-rows-flip-EXACT + control-untouched + derived-API-FLIP-live] · DATA-SEED-PASS [#477: pre/post-DB-row-delta + DLL-mtime + w3wp-recycle]; #482 Mig60 EF-ternary-SQL 200≠500; #472 PROD Identity NON-prefix Roles; #470 no-resurrect DB-count-diff; Mig54-62 applied-prod; #348 411→401 · verbatim → git + 2026-06/07.md)
  • Older (S83 #325 ← S29 #232) → git d2f52ba + archive/2026-06.md (incl #291 06-16 FAIL [gotcha #65]); pre-S38 → archive/2026-05-*.

🔄 Curate trigger

  • ~30KB → archive recent runs → L2 archive/<period>.md (byte-exact append) + _INDEX.md substring pointer. Dup failure patterns → merge. Stale >3mo → remove.

  • Last curate: 2026-07-14 S118 (self #389-run: S117 #387 verbatim→archive/2026-07.md moved-not-cut byte-id, baselines 505→509; prev S115 self 21.6→<17KB): #384/#383/#496/#495 (S114/S113/S112/S111) verbatim → archive/2026-07.md + _INDEX +4 pointers → S114 compact-stub + S111-113 merged-stub; top S114 garble-block → 1-line tracker.
  • Prev: S111 07-12 self 20.5→17 (#490-#494→archive) · S107 07-10 self 20.4→16.9 (#489/#488→archive) · S104 07-06 self 20.5→17.9 · S98 07-02 em-main 19.9→17.06 (2026-07.md NEW) · S80 06-20 em-main 38.8→16 · S70 06-17 Harness-9 65.2→23.2 · earlier S66 · S40.