Files
solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md
pqhuy1987 88bd8e666d [CLAUDE] Docs: S115 presence-not-age adopt closeout (STATUS/HANDOFF/session-log/synthesis/WAL)
Session-end closeout for S115 (AI_INFRA presence-not-age selector guardrail +
rec-3 hmw.js STOP-HARD, shipped 81b59f0, CI #385 PASS). STATUS + HANDOFF
mega-line blurb, session-log, ensemble-synthesis (closes H10/HCV gate), WAL
reset to clean template. No canonical-count change (Mig 65 / 89 tables / 486
tests / gotcha 81). Owner backfill target 210K + reviewer/cicd harvest already
in 81b59f0. §5.0 squashed the trailing wal: flush.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:01:14 +07:00

97 lines
18 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# CI/CD Monitor Agent — Persistent Memory
> **Persistent diary cross-session.** Auto-injected first ~200 lines at spawn (L1 HOT).
> Update BEFORE every stop. Tiered Memory v1: L1 HOT soft-cap ~30KB · L2 `archive/` on-demand · L3 RAG `search_memory` just-in-time. Keep entry ≤ 1.5K chars (gotcha #53).
> Full verbatim run history pre-S40 → git `d2f52ba` + `archive/2026-05-{runs,q2,q3,q4}.md` + `archive/2026-06.md` + `archive/2026-07.md`. 🗺️ **Lookup map: `archive/_INDEX.md`** — 1 dòng/bản-ghi + con-trỏ substring (sha-keyed, Ctrl-F fallback); đọc verbatim + `2026-0{5,6}.gist.md` theo nhu cầu. See 🔄 Curate-trigger log for archive-move history.
---
## 🎯 Role baseline
Read-only CI/CD + post-deploy verifier SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch + 5 RAG MCP. Output: PASS/FAIL + evidence <500 words. Skills: `iis-deploy-runbook` + `dependency-audit-erp` + `ef-core-migration`. Spawn ~150K trade-off catch fail tự động.
---
## ⚠️ cicd-lane return-garble tracker (#53)
cicd-spawn đôi khi trả memory-curation slice ("Compressing S112/S111") thay verdict (×5/6 qua S114; cumulative ×18 all-agents [[feedback_agent_return_garble_recover]], root-cause = curate-at-return). Recover: em-main-solo curl-verify. **S115 (#385) cicd-lane CLEAN-return no-garble — verdict delivered OK.**
---
## 🚨 Recurring CI/CD bug patterns (catch priority)
- **#39 act_runner github.com TCP timeout** run hang "Set up job" 21s. Log `dial tcp github.com:443 i/o timeout`. Fix: manual checkout bypass hardcoded `.gitea/workflows/deploy.yml`. KHÔNG revert.
- **#40 npm cache `tsc not found`** `build_fe_admin` fail post `cache: npm`. DISABLED rolled back `a21790d`. KHÔNG re-enable.
- **#41 paths-ignore docs-only skip** code commit không trigger CI? Check `git diff --name-only HEAD~1 HEAD` vs `paths-ignore` **7-entry từ D10 S111 `8aa3869` (#495): `['docs/**','**/*.md','.claude/skills/**','.claude/agent-memory/**','.claude/workflows/runs/**','.gitignore','scripts/**.md']`**. Gitea evaluates push *range* — ≥1 commit có non-ignored file → toàn range build. ⚠️ **AS-6 BASELINE SHIFT: từ #495, memory-push chứa `memory-budget.json`/agent-memory-file/run-trace HẾT trigger CI (pattern #490/#492 DEAD).**
- **#25 IIS WebSocket** `notification-hub/negotiate` 401/404 prod. Fix: WebSocket module enable `web.config` site api (skill `iis-deploy-runbook`).
- **#48 SQLite tie-break** `OrderByDescending(CreatedAt).First()` pick wrong khi 2+ `.Add()` cùng frozen-clock. Fix: discriminator filter BEFORE OrderBy.
- **Bundle hash unchanged = ship FAIL** push+action success nhưng prod không đổi. Verify via INDEX.HTML ref (`curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[A-Za-z0-9_-]+\.(js|css)'`), NOT by GETting a hash-named asset directly. verify MUST sau status=success (Run #242 false-positive: check khi "running" stale hash).
- **🔴 #69 (S72 #312) FE bundle hash NON-DETERMINISTIC per rebuild.** `deploy.yml` `Remove-Item fe-*\* -Exclude web.config`+`Copy-Item dist\*` runs UNCONDITIONALLY every run (path-filter gates whole-workflow, NOT per-step). "BE/governance frozen" can be FALSE; rotation possible every deploy. **To detect REAL FE change, diff `fe-*/src` in commit/range, NOT hash-delta.**
- **⚠ SPA-fallback 200 trap (S72):** server rewrites `/*``/index.html` so GET `/assets/index-<ANYTHING>.js` returns **200** even for fake hash (control `ZZdoesnotexist0.js`200, size~919b text/html). Old-hash-still-200 MEANINGLESS. RELIABLE signal = parse index.html `<script src>`, GET that exact path + check `size_download` LARGE (real ~1.6MB vs fake ~919b) + `Last-Modified` in deploy window + 2nd-fetch byte-stable (no mid-deploy transient).
- **🟢 #69-REFINE (S83 #333) hash can stay IDENTICAL across deploy when fe-*/src 0-change.** #69 "rotates EVERY deploy" NOT absolute: identical Vite source byte-identical output same content-hash. **MOST RELIABLE ship-proof = `Last-Modified` on real bundle IN deploy-window + real-size, NOT hash-delta** (hash-delta false-NEG when src unchanged, false-POS risk per #69). 0-FE-change hash EITHER same (#333/#485-487/#495/#385-S115) OR rotated (#312/#489) both benign; ship-proof = fresh LM-in-window + real-size + marker.
- **Migration drift prod vs repo** compare `ls .../Persistence/Migrations/*.cs` vs `sqlcmd __EFMigrationsHistory`. Fix: check `Program.cs`/DbInitializer `app.MigrateDatabase()` + app pool recycle. Mig-applied proof = `__EFMigrationsHistory` top==repo-HEAD; stuck-old pool didn't recycle FAIL even if status=success+bundle-rotated.
---
## 📋 5-stage checklist (EVERY run)
- **Stage 0 RAG infra:** `Get-Service Qdrant` Running + `http://localhost:6333/healthz`. Collection `proj_solution_erp`.
- **Stage 1 Push+filter:** `git log -1 --format='%H %s'` + `git log origin/main..HEAD` empty + diff vs paths-ignore (docs-only SKIPPED-DOCS return).
- **Stage 2 Gitea poll** (max 10 iter × 60s): API `.../actions/tasks?limit=5` (NOT `/runs` 404). Match `head_sha` (NOT run_number cancelled runs shift numbering). `updated_at` stale ~2min (gotcha #46) cross-check VPS mtime. Foreground-sleep BLOCKED Windows Monitor-style busy-wait-curl-spin ~30s/iter.
- **Stage 3 Test gate:** baseline **486 PASS** (45 Domain + 441 Infra; S114 #384 `d436af2` 477+9 PE multi-NCC tests [PeMultiWinner/PeSelectWinnerClear/PeListWinnerNamesProjection/CreateContractFromEvaluationMultiWinner +3 modified]; prior 477 [S113 #383], 458, 440, 436, 434, 421). CI runs both proj BEFORE build/deploy status=success test gate passed (`tasks` terminal=`status:success`, `conclusion` NOT populated trust success NOT log-numeric). Local grep undercounts (Theory/InlineData). Phase 9 UAT skip OK.
- **Stage 4 Post-deploy** (if SUCCESS): auth login bearer (admin + nv.test gotcha #44; token=`accessToken` route `/api/auth/login`) 3-5 endpoint smoke 2XX (incl new NEW-endpoint bodyless-POST returns **411 IIS Length-Required pre-auth NOT 404**; re-probe WITH body 401 confirms wired) FE bundle hash 2 app (per #69 = real-size-vs-fake not hash-delta) SignalR negotiate (gotcha #25) EF mig prod==repo.
- **Stage 4.6 (S29 CRITICAL):** sqlcmd seed sample verify post-deploy (NOT chỉ schema). Seed=runtime-row-insert (no history/tables advance) verify via DB-query (e.g. Permissions count), NOT bundle-frozen-alone.
- Discovery: ASP.NET 10 record enum needs numeric input unless `JsonStringEnumConverter` (SOL has NO converter FE sends numeric). sqlcmd ssh Windows-auth `\\\\SQLEXPRESS` 4-backslash. INFRASTRUCTURE seed MUST run (NOT inside `if(!demoSeedDisabled)`); DEMO seed gated gotcha #51.
- **Stage 5 Report** PASS/FAIL + evidence + MEMORY update.
---
## ⚠️ Anti-patterns (DO NOT)
1. Push fix code READ only, escalate em main · 2. Speculate fail without log · 3. Skip post-deploy bundle hash (biggest catch) · 4. Skip MEMORY · 5. Poll forever (max 10 iter) · 6. Auto-rollback (escalate + recommend) · 7. Verify docs-only (SKIPPED-DOCS return ngay)
---
## 🧠 SOLUTION_ERP CI/CD essentials (S80 verified — re-ground từ docs/STATUS.md canonical)
- **Gitea:** `git.baocaogiaoduc.vn/vietreport-admin/solution-erp` · workflow `.gitea/workflows/deploy.yml` · paths-ignore 7-entry (D10 S111 #495): `['docs/**','**/*.md','.claude/skills/**','.claude/agent-memory/**','.claude/workflows/runs/**','.gitignore','scripts/**.md']`. Anon API works (public repo) when GITEA_TOKEN absent.
- **Prod:** api/admin/eoffice `.solutions.com.vn` · SSH `ssh vietreport-vps` (Administrator, id_ed25519) · IIS phys paths: API `C:\inetpub\solution-erp\api` · admin `\fe-admin` · user `\fe-user`. DB `.\SQLEXPRESS`/`SolutionErp`/`vrapp` SQL-auth. **Conn key = `ConnectionStrings.Default` (NOT `DefaultConnection`)**. DB pw = `buKL3TGBkD0wDDbYVw65QeX9` (User Id=`vrapp`; Password field has NO `vrapp/` prefix S89 login-failed when prefix included; read verbatim from prod `appsettings.Production.json` ConnectionStrings.Default when `$env:PROD_DB_PASSWORD` empty). skill-doc path `C:\inetpub\apps\SolutionErp\Api` STALE.
- **SSHPS quoting (S42):** nested bashsshpowershell mangles `$var`/`\"` use `iconv UTF-16LE | base64` `powershell -EncodedCommand $B64`; OR write ps1-file + `powershell -File $(cygpath -w)`. sqlcmd over SSH direct `-W -h -1` pw-inline. sys-catalog string-concat `COLLATE DATABASE_DEFAULT` (collation conflict).
- **Tests baseline:** **486 PASS** (45 Domain + 441 Infra · 0 fail/skip; S114 #384 `d436af2`; prior 477/458/440/436/434/421). Phase 9 UAT skip OK. 🔑 **Concrete count via anon log `.../actions/runs/{run_number}/jobs/0/logs` (~18KB) → per-project `Passed! - Failed: 0, Passed: N, Skipped: 0` (NOT just status=success; `/api/v1/.../tasks/{id}/logs` variant 404s — use web-UI runs path).**
- **Mig latest repo: Mig 65 `BackfillPeQuoteIsSelectedAndWinnerInvariant`** (S114 `d436af2`; DATA-ONLY no-schema 2-dir backfill, INVARIANT `Supplier.IsWinner ⟺ Any(Quote.IsSelected)`; Down=no-op, NO ModelSnapshot change [2 mig-file = correct data-only], SQL-Server-only [SQLite EnsureCreated skips]; VERIFIED-APPLIED-PROD history-top + violations=0 + sys.tables=89). Prev Mig 64 `AddSupplierPublishState` / 63 / 62 `ExpandSupplierFields` (see docs/CLAUDE.md migration table). Path `src/Backend/SolutionErp.Infrastructure/Persistence/Migrations/`. Prod check `sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC`. **`sys.tables`(is_ms_shipped=0)=89** AddColumn/data-only don't change count; commit touching no new table 89 correct, don't FAIL. enum-additive = int NO-mig (history NOT-advance + empty `git diff -- '*Migrations*'`).
- **Bundle hash live:** current ship-baseline = S114/S115 (admin js `D51OYyGV`/css `CfunQHbd` · user js `BVdssm5S`/css `DGqqHQXS`; size 1,655,105/1,561,453b) see S114/S115 recent-run entries for LM+markers. idrun_number (head_sha-keyed). PER #69 TWO-MODE hash is SNAPSHOT-per-run NOT stable-baseline (rotate on real-src OR benign-0-src; freeze on 0-src) ALWAYS re-snapshot LIVE post-success; ship-proof = LM-in-window + real-size (vs 919b SPA-trap) + marker-grep, NOT hash-delta. js-rotate/css-frozen = FE-logic-no-style fingerprint.
- **Bearer:** admin `admin@solutions.com.vn/Admin@123456` (full) · UAT `nv.test@solutions.com.vn/TestUser@123456` (Drafter CCM, gotcha #44 check).
## 🔑 Critical config (flag commit nếu tái xuất)
Node CI `20.x` (`feedback_node_cicd`) · MediatR `12.4.1` (gotcha #1, flag `Version="14`) · Swashbuckle `6.9.0` (gotcha #2) · act_runner manual checkout (#39) · npm cache DISABLED (#40, flag `cache: npm`)
---
## 🎯 Per-NV admin opt-in wire — 10-point checklist (cumulative S22→S23)
Cross-ref `feedback_per_nv_permission_scope`. Per-NV/per-Level refactor MUST verify: 1 Domain field · 2 EF `HasDefaultValue(false)` · 3 Mig 3-file · 4 Service read · 5 Domain+App DTO mirror · 6 Designer FE checkbox · 7 AwLevelDto+ToDto · 8 CreateAwLevelInput+Update mutation · 9 **Lookup discrimination** (`FirstOrDefault` ADD `ApproverUserId==actorId` + admin fallback) · 10 **Controller body record count == Command record count**. Bug latency 2-3 days prod silent khi miss 9-10. Scan `grep -n "FirstOrDefault.*Order.*==" *.cs` after OR-of-N refactor.
## 📊 Run stats baseline
BE (test+build) ~90s · FE × 2 ~60s/app · deploy ~30s · **total ~3min code / 0s docs-only**. >5min → escalate. Recent runs all ~4m40-5m.
---
## 📅 Recent runs (compressed — verbatim → archive via `_INDEX.md`)
- **🟢 S115 #385 (rn 385, id 499) `81b59f0` GOVERNANCE/TOOLING-ONLY GO 5m23s (12:25:31→12:30:54+07 2026-07-13, harness adopt presence-not-age + hmw.js STOP-HARD; 20-file: 0 src/fe/mig/test — non-ignored trigger = `.claude/workflows/hmw.js` + `scripts/memory-archive-gate.ps1` [`.ps1``scripts/**.md` ⟹ CI-run ĐÚNG #41; rest .md/.json under docs/agent-memory/workflows-runs/broadcasts IGNORED]):** status=success · **gate CONCRETE `runs/385/jobs/0/logs`: Domain 45 + Infra 441 = 486/0-fail/0-skip = S114 baseline EXACT** (0 src/tests diff → count frozen ✓) · **FE bundle FROZEN ×4** admin js `D51OYyGV`/css `CfunQHbd` · user js `BVdssm5S`/css `DGqqHQXS` = S114 verbatim (NO rotate) — deploy physically re-copied (LM 05:29:32/05:30:26 GMT = 12:29:32/12:30:26+07 IN-window) BUT byte-identical (size 1,655,105/1,561,453b = S114 EXACT) ⟹ #69-REFINE 0-fe-src=identical-Vite=same-hash+size+fresh-LM · SPA-trap ctrl fake-hash 919b text/html · Mig FROZEN 65 (0 migration-file diff; API health/ready 200=startup OK) · smoke 4/4 200 (api ready+live, admin, eoffice). 🔑 GOVERNANCE-ONLY PASS: non-.md-in-.claude (hmw.js/.ps1) STILL trigger CI (#41 LIVE post-#495 D10); FROZEN-hash+fresh-LM+identical-size = no-accidental-rotation proof. cicd-lane clean-return no-garble.
- **🟢 S114 #384 (rn 384, id 498) `d436af2` GO 5m17s — PE multi-NCC per hạng mục + Mig 65 `BackfillPeQuoteIsSelectedAndWinnerInvariant` APPLIED-PROD (data-only 2-dir backfill, invariant-violations=0 [IsWinner⟺Any(IsSelected)], sys.tables 89) · gate 486 (45D+441I) · FE js-ROTATE×2/css-frozen admin `acprMqfB``D51OYyGV`/user `Ck3BTDdA``BVdssm5S` (= CURRENT ship-baseline, size 1,655,105/1,561,453b) · byte-verify `trúng thầu theo từng hạng mục` · select-winner WIRED 401 · smoke ALL 200 · 🔑 js-rotate+css-frozen = FE-logic-no-style fingerprint — verbatim → `archive/2026-07.md` via `_INDEX`.**
- _(**S111 #495 `8aa3869` H22-WAL-adopt** [deploy.yml SELF-change +2 paths-ignore, run parsed+ran-full = workflow-self-change PASS; bundle FROZEN ×6-streak; 🔑 **AS-6 baseline-shift**: post-#495 memory-push (budget.json/agent-memory/run-trace) HẾT trigger CI, phân-loại giờ soi file ngoài 7-glob] **+ S112 #496 `e100ef0`** PE-workflow+Supplier-import-Phase-B+Mig63 `AddSupplierImportSourceFields` [gate 458, 🔑 411-refine: bodyless-POST→411 ANY-route → re-probe WITH-body → 401/415-matched vs 404-missing] **+ S113 #383 `5fa11b5`** Supplier-import-v2+Mig64 `AddSupplierPublishState`-APPLIED-PROD [IsPublic 37/37, gate 477] — all GO ~5m, FE real-ship, Mig-applied-prod, smoke ALL 200 · full verbatim → `archive/2026-07.md` via `_INDEX`)_
- _(**S107-closeout→S110-spot #490-#494** GOVERNANCE/MEMORY-push GO ×5 [`d74f67f`·`69957fd`·`f5a5c3e`·`6ee60cd`·`ac3f122` — trigger #41 non-.md-in-push: budget.json ×2 + hmw.js ×3, pattern DEAD từ D10 #495] — bundle FROZEN-streak ×5 sau ROTATE #489 [#69 TWO-MODE; LM-in-window + real-size 1,644,931/1,551,280b = signal duy nhất] · Mig 62 + sys.tables 89 frozen · gate 440 · smoke 3×200 ALL · 🔑 #494 spot-detection: LM-prod-newer-than-last-verified-window = missed-run detector · full verbatim → `archive/2026-07.md` via `_INDEX`)_
- _(**S107 #489 `21387cd` GOVERNANCE-ONLY (+office-document 12th sub) GO 5m33s** — JS ROTATED-benign `ChSNAJBK``BVyH-FY7` / `Bn2w1znH``EDsZebvn` CSS-frozen, 🔑 #69 TWO-MODE CONFIRMED (0-fe-src ROTATE #312/#489 hoặc FROZEN #333/#485-487 — ship-proof = LM-in-window + real-size, hash-delta phụ) · **S104 #488 `778fc98` FULL-STACK Supplier 9→27 + Mig 62 `ExpandSupplierFields` APPLIED-PROD** — AddColumn×18-exact + sys.tables-89-invariant + un-cram 4-NCC Note=NULL/Pkg/Bank/LegalRep-populated-EXACT + DTO-18-new-keys-live + JS-rotate-real-src + 🔑 FULL-STACK-SCHEMA-ADD+DATA-UN-CRAM PASS-pattern — full verbatim → `archive/2026-07.md` via `_INDEX`)_
- _(**S103 #487 + S100 #486 + S98 #485** GOVERNANCE/SCRIPTS-ONLY GO ×3 [.ps1/.json/.md-mix non-ignored → #41] — bundle FROZEN #69-REFINE ×4, GOVERNANCE-PASS stable [#476/#485-487] · 🔧 python `time.sleep` foreground-OK (chỉ bash-sleep-block) · full verbatim → `archive/2026-07.md` (+#485 git `c112be9`) via `_INDEX`)_
- _(**S96+S97 FINAL-REVIEW #478-#484** 11-commit ALL-HEALTHY whole-session-GO [backfill A/005=1/A/008=1/A/010=0-exact + 88-excl/89-incl reconcile; #69 rotation≠ship-signal] · **S97 #484 `fb2dfa9` DATA-FIX Mig61** 🔑 DATA-FIX-BACKFILL PASS = CI + Mig-top==HEAD + target-rows-flip-EXACT + control-untouched + derived-API-field-FLIP-live — verbatim → `archive/2026-07.md` via `_INDEX`)_
- _(S96-S97 endsBeforeCeo-saga + PE-fixes #479-#484 all PASS 440 — key: #482 Mig60 CROSS-STACK [🔴 EF-TERNARY-SQL-translate-LIVE list-200-NOT-500 · #77 field-PRESENT] · #481/#483 FE-only [#77 new-str-PRESENT/old-ABSENT] · #480 baseline-440 +PeSuggestedPriceSetterAuthzTests · JS-rotate/CSS-frozen · verbatim → git + `2026-06.md` via `_INDEX`)_
- _(S90→S96 #465·#470·#472·#473·#476·#477 all PASS — key: #477 🔑 **DATA-SEED-PASS = pre/post-DB-row-EXACT-delta + DLL-mtime + w3wp-recycle + Mig-frozen + API-payload-contains-row** [⚠️ sqlcmd `-W` diacritic-LIKE false-NEG → `_`-wildcard; base64-EncodedCmd-FAIL → `.ps1`-File] · #472 ⚠️ **PROD Identity tables NON-prefixed `Roles` NOT `AspNetRoles`** · #470 GO-LIVE 🔑 no-resurrect = pre+post-DB-count-diff + dll-in-window + w3wp≥run-window [integrated-auth `-E`] · #476 GOVERNANCE-PASS baseline · verbatim → git + `2026-06.md` via `_INDEX`)_
- _(S69b→S89 runs #307·#313·#315·#318·#330·#333·#342-#348 all PASS — key: Mig54-59 applied-prod · #345 PATCH-401+7-authz · #344 backfill-winners=13 · #343 BE-only ship-proof=Infra.dll-mtime · #342 INVERTED-PASS-FROZEN [#73 TZ] · #346 FE-only js+css-BOTH-rotate · #348 411→401 endpoint-wired · full verbatim → git + `2026-06.md` via `_INDEX`)_
- **Older runs (S83 #325 FE-budget-indent ← S77 #329 banner ← S75 #301 ← S62 #286 ← … → S29 #232) full verbatim → git `d2f52ba` + `archive/2026-06.md`** (incl #291 06-16 FAIL forensic [gotcha #65]); pre-S38 → `archive/2026-05-{runs,q2,q3,q4}.md`.
---
## 🔄 Curate trigger
- >~30KB → archive recent runs → L2 `archive/<period>.md` (byte-exact append) + `_INDEX.md` substring pointer. Dup failure patterns → merge. Stale >3mo → remove.
- **Last curate: 2026-07-13 S115 (self, PostToolUse 21.6→<17KB)**: #384/#383/#496/#495 (S114/S113/S112/S111) verbatim `archive/2026-07.md` + `_INDEX` +4 pointers S114 compact-stub + S111-113 merged-stub; top S114 garble-block 1-line tracker.
- **Prev: S111 07-12 self 20.517 (#490-#494archive) · S107 07-10 self 20.416.9 (#489/#488archive) · S104 07-06 self 20.517.9 · S98 07-02 em-main 19.917.06 (`2026-07.md` NEW) · S80 06-20 em-main 38.816 · S70 06-17 Harness-9 65.223.2 · earlier S66 · S40.**