39d55d4402
cicd-monitor verified Run #312 (commit 18fced6) = true no-op (306 / Mig 53 /
sys.tables 88 / health 4x200, DB untouched). BUT bundle ROTATED
BgNCjwsG/CBvh0vtf -> fc_xkNpJ/DP-tBcg0 despite 0 FE source change.
- gotcha #69: FE bundle hash non-deterministic + deploy.yml rebuilds FE
unconditionally every run -> bundle rotates even on BE-only/governance commits.
Corollary: SPA-fallback 200 trap (verify bundle via index.html refs+size, not
direct GET of hash-named asset). "BE-only => frozen" invariant overturned.
- Correct STATUS/HANDOFF live bundle to fc_xkNpJ/DP-tBcg0; my prior "bundle
frozen" claim was wrong (cosmetic rotation, prod state/DB unchanged).
- gotcha count 68 -> 69. cicd-monitor MEMORY updated (Run #312 + bundle pattern).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
89 lines
29 KiB
Markdown
89 lines
29 KiB
Markdown
# CI/CD Monitor Agent — Persistent Memory
|
||
|
||
> **Persistent diary cross-session.** Auto-injected first ~200 lines at spawn (L1 HOT).
|
||
> Update BEFORE every stop. Tiered Memory v1: L1 HOT soft-cap ~30KB · L2 `archive/` on-demand · L3 RAG `search_memory` just-in-time. Keep entry ≤ 1.5K chars (gotcha #53).
|
||
> Full verbatim run history pre-S40 → git `d2f52ba` + `archive/2026-05-{runs,q2,q3,q4}.md` + `archive/2026-06.md`. 🗺️ **Lookup map (Harness-9 S70): `archive/_INDEX.md`** — 1 dòng/bản-ghi + con-trỏ substring (sha-keyed, Ctrl-F fallback); đọc verbatim + `2026-0{5,6}.gist.md` (nén 4-field) theo nhu cầu.
|
||
|
||
---
|
||
|
||
## 🎯 Role baseline
|
||
|
||
Read-only CI/CD + post-deploy verifier SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch + 5 RAG MCP. Output: PASS/FAIL + evidence <500 words. Skills: `iis-deploy-runbook` + `dependency-audit-erp` + `ef-core-migration`. Spawn ~150K — trade-off catch fail tự động.
|
||
|
||
---
|
||
|
||
## 🚨 Recurring CI/CD bug patterns (catch priority)
|
||
|
||
- **#39 act_runner github.com TCP timeout** — run hang "Set up job" 21s. Log `dial tcp github.com:443 i/o timeout`. Fix: manual checkout bypass hardcoded `.gitea/workflows/deploy.yml` (pass #110). KHÔNG revert.
|
||
- **#40 npm cache `tsc not found`** — `build_fe_admin` fail post `cache: npm`. DISABLED rolled back `a21790d`. KHÔNG re-enable.
|
||
- **#41 paths-ignore docs-only skip** — code commit không trigger CI? Check `git diff --name-only HEAD~1 HEAD` vs `paths-ignore: ['docs/**','**/*.md','.claude/skills/**']`. Discovery #3: Gitea evaluates push *range* commits — nếu ≥1 commit có non-ignored file → toàn range build (BENEFICIAL).
|
||
- **#25 IIS WebSocket** — `notification-hub/negotiate` 401/404 prod. Fix: WebSocket module enable `web.config` site api (skill `iis-deploy-runbook`).
|
||
- **#48 SQLite tie-break** — `OrderByDescending(CreatedAt).First()` pick wrong khi 2+ `.Add()` cùng frozen-clock. Fix: discriminator filter `.Where(Summary.Contains("Chuyển phase"))` BEFORE OrderBy.
|
||
- **Bundle hash unchanged = ship FAIL** — push+action success nhưng prod không đổi. Verify via INDEX.HTML ref (`curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[A-Za-z0-9_-]+\.(js|css)'`), NOT by GETting a hash-named asset directly. Fix: SSH `Restart-WebAppPool`. ⚠️ Bundle hash verify MUST sau status=success (Run #242 false-positive lesson: check khi "running" → stale hash).
|
||
- **🔴 #69 (S72 Run #312 OVERTURNS prior invariant) — FE bundle hash is NON-DETERMINISTIC per rebuild. `deploy.yml` `Remove-Item fe-{admin,user}\* -Exclude web.config` + `Copy-Item dist\*` runs UNCONDITIONALLY every run (path-filter gates whole-workflow trigger, NOT per-step). Identical FE source ⟹ DIFFERENT hash each deploy (Vite/rolldown non-reproducible chunk-id). PROVEN: governance-only `18fced6` (0 files in fe-*/src) rotated BOTH bundles admin `BgNCjwsG→fc_xkNpJ` (+717b) + user `CBvh0vtf→DP-tBcg0`, index.html `Last-Modified` matched deploy window. ⟹ "BE-only/governance ⟹ bundle MUST stay frozen" is FALSE; rotation is EXPECTED on every deploy. Bundle-rotate alone is NOT proof of FE content change — to detect real FE change, diff `fe-*/src` in commit/range, NOT hash delta.**
|
||
- **⚠️ SPA-fallback 200 trap (S72):** server rewrites `/*`→`/index.html` so GET `/assets/index-<ANYTHING>.js` returns **200** even for non-existent/fake hash (control `ZZdoesnotexist0.js`→200 confirmed). Old-hash-still-200 is MEANINGLESS as persistence signal. RELIABLE bundle signal = parse index.html `<script src>`/`<link href>`, then GET that exact path + check `size_download` large (not white-screen) + `Last-Modified` in deploy window.
|
||
- **Migration drift prod vs repo** — compare `ls .../Persistence/Migrations/*.cs` vs `sqlcmd __EFMigrationsHistory`. Fix: check `Program.cs` `app.MigrateDatabase()` + app pool recycle.
|
||
|
||
---
|
||
|
||
## 📋 5-stage checklist (EVERY run)
|
||
|
||
- **Stage 0 RAG infra:** `Get-Service Qdrant` Running + `http://localhost:6333/healthz`. Collection `proj_solution_erp` (prefix `proj_*` 7 project — Discovery #8).
|
||
- **Stage 1 Push+filter:** `git log -1 --format='%H %s'` + `git log origin/main..HEAD` empty + diff vs paths-ignore (docs-only → SKIPPED-DOCS return).
|
||
- **Stage 2 Gitea poll** (max 10 iter × 60s): API `.../actions/tasks?limit=5` (NOT `/runs` 404). Match `head_sha`. ⚠️ task table `updated_at` stale ~2min (gotcha #46) → cross-check VPS mtime.
|
||
- **Stage 3 Test gate:** baseline **130 PASS** (58 Domain + 72 Infra). Phase 9 UAT exception lower OK (`feedback_uat_skip_verify`).
|
||
- **Stage 4 Post-deploy** (if SUCCESS): auth login bearer (admin + nv.test gotcha #44; token=`accessToken` route `/api/auth/login`) → 3-5 endpoint smoke 2XX (incl new) → FE bundle hash 2 app changed → SignalR negotiate (gotcha #25 if relevant) → EF mig prod==repo.
|
||
- **Stage 4.6 (S29 CRITICAL):** sqlcmd seed sample verify post-deploy (NOT chỉ schema). `sqlcmd -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'"` → 0 rows = seed GATE BLOCKED → gotcha #51.
|
||
- Discovery #4: ASP.NET 10 record enum cần numeric input unless `JsonStringEnumConverter` (SOL has NO converter → FE sends numeric). #5: sqlcmd ssh Windows-auth cần `\\\\SQLEXPRESS` 4-backslash. #6: INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/SampleWorkflowsV2) MUST run, NOT inside `if(!demoSeedDisabled)`; DEMO seed (DemoUsers/Contracts/PE) OK gated → gotcha #51.
|
||
- **Stage 5 Report** PASS/FAIL + evidence + MEMORY update.
|
||
|
||
---
|
||
|
||
## ⚠️ Anti-patterns (DO NOT)
|
||
1. ❌ Push fix code — READ only, escalate em main · 2. ❌ Speculate fail without log · 3. ❌ Skip post-deploy bundle hash (biggest catch) · 4. ❌ Skip MEMORY · 5. ❌ Poll forever (max 10 iter) · 6. ❌ Auto-rollback (escalate + recommend) · 7. ❌ Verify docs-only (SKIPPED-DOCS return ngay)
|
||
|
||
---
|
||
|
||
## 🧠 SOLUTION_ERP CI/CD essentials (S40 verified)
|
||
|
||
- **Gitea:** `git.baocaogiaoduc.vn/vietreport-admin/solution-erp` · workflow `.gitea/workflows/deploy.yml` · paths-ignore `['docs/**','**/*.md','.claude/skills/**']`
|
||
- **Prod:** api/admin/eoffice `.solutions.com.vn` · SSH `ssh vietreport-vps` (Administrator, id_ed25519) · IIS site phys paths (S42 verified): API `C:\inetpub\solution-erp\api` · admin `\fe-admin` · user `\fe-user` (3 sites Started). DB `.\SQLEXPRESS`/`SolutionErp`/`vrapp` SQL-auth. **Conn string key = `ConnectionStrings.Default` (NOT `DefaultConnection`!)** — read pw from prod appsettings.Production.json when `$env:PROD_DB_PASSWORD` empty.
|
||
- **SSH→PS quoting (S42 lesson):** nested bash→ssh→powershell mangles `$var`/`\"`. Use `iconv UTF-16LE | base64` → `powershell -EncodedCommand $B64`. Single-quote literal paths.
|
||
- **Tests baseline:** **263 PASS** (S62 Run #286 sha 7926c21 spec; 45 Domain + 218 Infra — em-main supplied; supersedes prev 228/240/256). CI gate runs both test projects BEFORE build/deploy → status=success ⟹ test gate passed (`tasks` endpoint reports terminal as `status:success`, `conclusion` field NOT populated). Local grep undercounts (Theory/InlineData) — trust CI conclusion. Phase 9 UAT mode skip per chunk OK.
|
||
- **Mig latest repo:** **Mig 53 `20260617060207_AddPeUrgentAndCeoApprovalThreshold`** (S71 Run#308; PE +IsUrgentByPro/+IsUrgentByCcm bit-default-0 + ApprovalWorkflows +CeoApprovalThreshold decimal(18,2)-nullable, 3 AddColumn no new table — VERIFIED APPLIED PROD). Prev Mig 52 `AddHoSoLinkToPurchaseEvaluation` (S65 PE HoSoLink hyperlink-NAS) + Mig 51 `AddDepartmentParentId` (S65 org-tree loose-Guid) + Mig 50 `ReplaceBudgetModuleWithPeWorkItemBudgets` (S61 net-reduce). Path `src/Backend/SolutionErp.Infrastructure/Persistence/Migrations/` (53 mig .cs non-designer total). Prod check `sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC TOP 5`. ⚠️ Table-count: `sys.tables` (is_ms_shipped=0, excl mighist) = **88** (S62 Run #286 verified — S61 Budget-replace DROPPED tables 93→88). Narrative-93 is STALE pre-S61 — when commit touches no schema, 88 is correct, don't FAIL on 88↔93. Always cross-ref COMMIT scope vs ambient count.
|
||
- **Bearer:** admin `admin@solutions.com.vn/Admin@123456` (full) · UAT `nv.test@solutions.com.vn/TestUser@123456` (Drafter CCM, gotcha #44 check)
|
||
- **Bundle hash live S72:** admin `fc_xkNpJ` + css `C2Zvnd2f` · user `DP-tBcg0` + css `1bS4xeUF` (Run #312 sha 18fced6; index.html `Last-Modified` 2026-06-18T07:07:49Z = deploy window). Prev S71 #308: admin `BgNCjwsG`/user `CBvh0vtf`. ⚠️ **PER #69 (S72): bundle hash ROTATES on EVERY deploy regardless of FE change (non-deterministic rebuild). So "live hash" value is only a SNAPSHOT for THIS run — do NOT treat it as a frozen-baseline to compare next run against.** To detect a real FE ship, diff `fe-*/src` in commit/range — NOT hash delta. ASYMMETRIC-deploy "anomaly" framing (S66) is RETIRED by #69: both apps rebuilt+rotated every deploy, asymmetry impossible to read from hashes alone. S50 mid-deploy transient lesson still holds: re-confirm hash AFTER status=success ALWAYS (anti-pattern #3).
|
||
- **DB pw (S42, when `$PROD_DB_PASSWORD` empty):** `vrapp/buKL3TGBkD0wDDbYVw65QeX9` read from `C:\inetpub\solution-erp\api\appsettings.Production.json`→`ConnectionStrings.Default`. ⚠️ Skill-doc path `C:\inetpub\apps\SolutionErp\Api` is STALE → real path `C:\inetpub\solution-erp\api`. sqlcmd over SSH works direct (no UTF-16 encode needed). ⚠️ sys-catalog string-concat queries hit collation conflict (`Latin1_General_CI_AS_KS_WS` vs `SQL_Latin1_General_CP1_CI_AS`) → add `COLLATE DATABASE_DEFAULT` per concatenated column.
|
||
|
||
## 🔑 Critical config (flag commit nếu tái xuất)
|
||
Node CI `20.x` (`feedback_node_cicd`) · MediatR `12.4.1` (gotcha #1, flag `Version="14`) · Swashbuckle `6.9.0` (gotcha #2) · act_runner manual checkout (#39) · npm cache DISABLED (#40, flag `cache: npm`)
|
||
|
||
---
|
||
|
||
## 🎯 Per-NV admin opt-in wire — 10-point checklist (cumulative S22→S23)
|
||
Cross-ref `feedback_per_nv_permission_scope`. Per-NV/per-Level refactor MUST verify: 1 Domain field · 2 EF `HasDefaultValue(false)` · 3 Mig 3-file · 4 Service read · 5 Domain+App DTO mirror · 6 Designer FE checkbox · 7 AwLevelDto+ToDto · 8 CreateAwLevelInput+Update mutation · 9 **Lookup discrimination** (`FirstOrDefault` ADD `ApproverUserId==actorId` + admin fallback) · 10 **Controller body record count == Command record count**. Bug latency 2-3 days prod silent khi miss 9-10. Scan `grep -n "FirstOrDefault.*Order.*==" *.cs` after OR-of-N refactor.
|
||
|
||
## 📊 Run stats baseline
|
||
BE (test+build) ~90s · FE × 2 ~60s/app · deploy ~30s · **total ~3min code / 0s docs-only**. >5min → escalate.
|
||
|
||
---
|
||
|
||
## 📅 Recent runs (FIFO — older → archive/git)
|
||
|
||
- **2026-06-18 S72 Run #312 (run_number 312) sha=`18fced6` PASS ~5m16s (GOVERNANCE-ONLY discipline-check: 26 `.md` + `.claude/workflows/hmw.js` + `.claude/agent-memory/memory-budget.json`. Harness-10 flat-refine adoption. ZERO production BE/FE/Mig code. Expected = true no-op deploy. deploy 13/13 session after #297–#308 all PASS):** Push origin/main = HEAD `18fced6` (nothing unpushed). `git diff --name-only 18fced6~1 18fced6` = 27 files; non-path-ignored = ONLY `hmw.js` + `memory-budget.json` (both tooling/governance, NOT prod code) → CI TRIGGERED (correct — `.js`/`.json` not in paths-ignore `['docs/**','**/*.md','.claude/skills/**']`). Migration files in diff = ZERO. GITEA_TOKEN absent in PS-scope → anon Gitea API (worked, public repo). PROD_DB_PW empty → DB pw from prod `appsettings.Production.json`→`ConnectionStrings.Default` (`vrapp`/`buKL3TGBkD0wDDbYVw65QeX9` len24, path `C:\inetpub\solution-erp\api`). Run IN-PROGRESS at spawn (running 14:04:11→14:06:24) — exact `head_sha` match RUN#312 (deterministic, not gotcha-#46 stale). Poll-loop iter4 status=success (created 14:04:11 → success 14:09:27 ≈5m16s). CI gate (both proj pre-build ⟹ status=success ⟹ test **306** baseline (45D+261I) passed; `conclusion` empty — `tasks` terminal=`status:success` doesn't populate conclusion, trust success; 306 INFERRED gate-passes-pre-build invariant NOT log-confirmed numeric). Smoke **4×200**: api `/health/ready`+`/health/live` + admin root + eoffice root (both pre- AND post-success). **🔑★ MIG VERIFIED NO-CHANGE (sqlcmd over SSH ground-truth): prod `__EFMigrationsHistory` top5 = `AddPeUrgentAndCeoApprovalThreshold`(Mig53)→`AddHoSoLinkToPurchaseEvaluation`(52)→`AddDepartmentParentId`(51)→`ReplaceBudgetModuleWithPeWorkItemBudgets`(50)→`AddWorkItemToPurchaseEvaluation`(49) == repo HEAD, NO new mig ✓. sys.tables=88 (excl mighist) UNCHANGED ✓.** ⟹ DB side = perfect no-op as expected for governance commit. **⚠️★ BUNDLE SURPRISE → RESOLVED BENIGN (NEW gotcha #69): expected FROZEN admin `BgNCjwsG`/user `CBvh0vtf`, but AFTER status=success BOTH ROTATED admin `BgNCjwsG→fc_xkNpJ` (+717b, 1,593,048b) + user `CBvh0vtf→DP-tBcg0` (1,497,701b), stable on 2nd-fetch (not mid-deploy transient). FORENSIC: commits ebd7e1c..18fced6 (9 commits, all `[CLAUDE] Docs:`/`Workflow:`) touched 0 files in `fe-{admin,user}/src/` + 0 in package.json/lock/vite/tsconfig (verified per-file). ROOT CAUSE: `deploy.yml` lines 161-167 `Remove-Item fe-*\* -Exclude web.config` + `Copy-Item dist\*` runs UNCONDITIONALLY every run (path-filter gates WHOLE-workflow trigger, not per-step) → FE rebuilt every deploy → Vite/rolldown emits NON-DETERMINISTIC hash from identical source. PROOF index.html `Last-Modified` 07:07:49Z(14:07VN) = inside deploy window → physically rewritten this run. CONTROL TEST: fake `/assets/index-ZZdoesnotexist0.js` → 200 (SPA `/*`→index.html fallback) ⟹ "old-hash-still-200" is MEANINGLESS, reliable signal = parse index.html refs (admin `fc_xkNpJ.js`+`C2Zvnd2f.css` · user `DP-tBcg0.js`+`1bS4xeUF.css`).** ⟹ **rotation is NOT an accidental FE change — it's the pipeline's normal non-deterministic-rebuild behavior; OVERTURNS prior "BE-only/governance ⟹ bundle frozen" invariant used in every prior verdict (held by coincidence, not mechanism).** Assets reachable 200 + large (not white-screen) ⟹ FE healthy. 0 regression. NO prod-data mutation (read-only curls + sqlcmd SELECT-only; FE re-copy = expected unconditional deploy side-effect, governance content unchanged). **VERDICT PASS (true no-op for the COMMIT's intent): green CI + test 306 + Mig 53 unchanged + tables 88 + health 4×200; bundle rotation flagged-then-cleared as benign pipeline mechanism (#69), NOT accidental ship — confirmed by 0 fe-src diff. LESSON: governance/BE-only verify must NOT FAIL on bundle-rotate; reframe check = (a) Mig+tables unchanged ground-truth, (b) health 200, (c) FE-content-change detection via `git diff fe-*/src` NOT hash, (d) bundle hash useful only to confirm a DELIBERATE FE deploy DID ship (rotate present), never to assert FE DIDN'T change (rotate is unconditional). TOOLING: bash→`$LOCALAPPDATA/Temp/x.ps1` + `powershell.exe -File $(cygpath -w ...)`; Gitea `tasks` via Invoke-RestMethod with `-Body @{limit=10}` hashtable (string-interp `?limit=N` into encoded path → 'hostname could not be parsed' URI error — use -Body not query-string); poll-loop backgrounded + Monitor grep-until FINAL; SSH→PS base64 `-EncodedCommand` UTF-16LE for appsettings-read AND sqlcmd; sqlcmd pw via PS here-string `$P=` + `[char]39` for literal object-name quotes; `-W -h -1 -s '|'`; read DB pw from prod appsettings when `$PROD_DB_PASSWORD` empty. NEVER fixed code (READ-only).** Tag `[s72, run312, pass, governance-only-no-prod-code, hmw-js+memory-budget-json-triggers-ci, mig53-unchanged-VERIFIED-prod, tables88-unchanged, health-4x200, test306-inferred, BUNDLE-BOTH-ROTATE-fc_xkNpJ-DP-tBcg0-but-BENIGN, NEW-gotcha-69-fe-hash-nondeterministic-per-rebuild-deploy-unconditional, spa-fallback-200-trap-control-test, frozen-invariant-OVERTURNED, 0-fe-src-diff-confirms-no-accidental-ship, deploy13of13, no-regression]`. (id422) sha=`ebd7e1c` PASS ~4m41s (FULL-STACK: BE Mig 53 + ApproveV2 CCM-finalize-by-threshold + endpoint PUT /urgent + FE 2-app PE cờ-gấp UI + designer ngưỡng. PE "cờ gấp PRO/CCM + CCM duyệt-final theo ngưỡng giá trị" anh Kiệt FDC. 27 files: BE 9 {Controller +SSetUrgent, App PurchaseEvaluationUrgentFeatures.cs NEW, PurchaseEvaluationFeatures, CreateContractFromEvaluation, AwV2AdminFeatures, Dtos, Domain PE+AW, Config} + 3 Mig-file + Service PurchaseEvaluationWorkflowService + FE 6 {PeDetailTabs+ListPage+types ×2-app + ApprovalWorkflowsV2Page admin-only} + 2 test {PeUrgentToggleAuthzTests, PeCcmThresholdFinalizeTests} + 3 agent-memory .md. deploy 12/12 session after #297–#307 all PASS):** Push parent `1f8947e..ebd7e1c` (1 commit). `git diff --name-only 1f8947e ebd7e1c -- '*Persistence/Migrations*'` = 3 files (Mig 53 .cs/.Designer.cs/ModelSnapshot.cs) — REAL EF migration this time (contrast S69b/S70 which had ZERO). `.cs/.tsx` non-ignored → full pipeline RAN. GITEA_TOKEN present (PS-scope) → authed Gitea API; PROD_DB_PW empty → DB pw từ prod `appsettings.Production.json`→`ConnectionStrings.Default` (`vrapp`/`buKL3TGBkD0wDDbYVw65QeX9`, path `C:\inetpub\solution-erp\api`). Run IN-PROGRESS at spawn (running 13:27:58, 1st-seen updated 13:28:54) — pre-deploy NOT snapshotted as baseline (anti#3: prev-live = #306/#307 spec admin `Wt54PHYl`/user `B99fMU6X`). Poll-loop iter6 status=success (created 13:27:58 → success-update 13:32:39 ≈4m41s). CI gate (both proj pre-deploy ⟹ status=success ⟹ test **306** baseline (45D+261I; +14 PeUrgentToggleAuthz+PeCcmThresholdFinalize) passed; `conclusion` empty — `tasks` terminal=`status:success` không populate conclusion, trust success; 306 INFERRED gate-passes-pre-build invariant NOT log-confirmed numeric). **★ BUNDLE BOTH ROTATE (FE-both-app PE-UI ⟹ both MUST rotate; verified AFTER status=success + re-confirm STABLE 2nd-fetch identical no-transient — anti#3): admin ROTATE `Wt54PHYl→BgNCjwsG`** ✓ **+ user ROTATE `B99fMU6X→CBvh0vtf`** ✓. Asset reachable 200: admin js 1,592,331b + user js 1,496,984b (not white). Smoke **4×200** health: api `/health/ready`+`/health/live` + admin root + eoffice root. **★ ENDPOINT /urgent PROBE: `PUT /api/purchase-evaluations/{guid}/urgent` unauth → 401 (NOT 404)** ✓ — route EXISTS + class-level `[Authorize]` any-auth enforced (controller `[HttpPut("{id:guid}/urgent")] SetUrgent → SetPurchaseEvaluationUrgentCommand`; behavioral PRO/CCM-only authz = unit-test PeUrgentToggleAuthzTests, not probed). PE list unauth → 401 ✓ (reachable, gated). **🔑★ MIG 53 VERIFIED APPLIED PROD (sqlcmd over SSH, ground-truth not inference): prod `__EFMigrationsHistory` top5 = `20260617060207_AddPeUrgentAndCeoApprovalThreshold`(53)→`AddHoSoLinkToPurchaseEvaluation`(52)→`AddDepartmentParentId`(51)→`ReplaceBudgetModuleWithPeWorkItemBudgets`(50)→`AddWorkItemToPurchaseEvaluation`(49) == repo HEAD ✓ (DbInitializer auto-migrate-on-boot ran; app pool recycled w/ new binary). 3 NEW COLUMNS confirmed via sys.columns: `PurchaseEvaluations.IsUrgentByCcm`=bit is_nullable=0 ✓ + `PurchaseEvaluations.IsUrgentByPro`=bit is_nullable=0 ✓ + `ApprovalWorkflows.CeoApprovalThreshold`=decimal is_nullable=1 precision=18 scale=2 ✓ — ALL match Mig 53 Up() spec exactly. sys.tables=88 (excl mighist) UNCHANGED — 3 AddColumn no new table ✓.** 0 regression. NO prod-data mutation (read-only curls + sqlcmd SELECT-only; migration-apply = expected boot-side-effect of deploy). Behavioral CCM-finalize-by-threshold + cờ-gấp UI render = anh Kiệt UAT (em confirm Mig applied + columns + endpoint route + bundles shipped per spec). **LESSON: full-stack PE commit w/ REAL Mig = verify ALL 4 axes: (1) BE+FE both-bundle ROTATE (FE 2-app), (2) Mig history-top advance + 3 columns sys.columns ground-truth (NOT just "schema looks right" — query each col type/nullable/precision vs Up() spec), (3) sys.tables stays 88 (AddColumn-only ⟹ no table delta; rotate-to-89 would = unexpected CREATE TABLE leak), (4) new endpoint 401-not-404 unauth probe (route-exists proof; authz-logic = unit-test domain not curl). Mig-applied-on-prod proof = `__EFMigrationsHistory` top == repo-HEAD (DbInitializer MigrateAsync boot-hook) — if top stuck at Mig 52 ⟹ app pool didn't recycle / migrate-on-boot failed ⟹ FAIL even if status=success+bundle-rotated (deploy shipped binary but DB-side incomplete). Distinguishes this run from S69b/S70 (zero-migration BE/FE — those had NO history-advance EXPECTED). TOOLING: Bash=POSIX → write PS to `$LOCALAPPDATA/Temp/x.ps1` + `powershell.exe -NoProfile -File "C:\Users\pqhuy\AppData\Local\Temp\x.ps1"` (⚠️ bash→Windows path auto-convert mangles `/tmp/` → use `$LOCALAPPDATA/Temp` explicit Windows path); Invoke-RestMethod for Gitea `tasks` (match `head_sha -eq sha`, poll-loop in PS not bash — bash jq absent/python3 broken); SSH→PS base64 `-EncodedCommand` (UTF-16LE Unicode) for BOTH appsettings-read AND sqlcmd; sqlcmd pw-literal via PS here-string `$P=` var inside encoded payload + query-string built w/ `[char]39` for embedded single-quotes (object-name literals); `-W -h -1` no-header + `-s '|'` pipe-delim; read DB pw from prod appsettings when `$PROD_DB_PASSWORD` empty. NEVER fixed code (READ-only).** Tag `[s71, run308, pass, full-stack-pe-urgent-ccm-threshold, mig53-AddPeUrgentAndCeoApprovalThreshold-VERIFIED-APPLIED-PROD, 3-cols-sys-columns-confirmed-IsUrgentByPro-IsUrgentByCcm-bit0-CeoApprovalThreshold-decimal18-2-null, history-top-advance-53, tables88-unchanged-addcolumn-only, bundle-BOTH-rotate-BgNCjwsG-CBvh0vtf, asset-200-reachable, endpoint-urgent-401-not-404-route-exists, pe-list-401-gated, health-4x200, test306-inferred, deploy12of12, no-regression, behavioral-ccm-finalize-anh-kiet-uat]`.
|
||
|
||
- **2026-06-17 S69b Run #307 (run_number 307, id421) sha=`1f8947e` PASS ~4m33s (BE-ONLY GOLIVE security hướng-ra-prod "Văn phòng số" public Read+Create mọi role — NEW `SeedAllRolesOfficeModulePermissionsAsync` DbInitializer.cs chạy lúc API boot SAU `RevokeTemporarilyHiddenModulesAsync` để THẮNG revoke (mirror S65 HRM/Hồ-sơ-NS pattern), allow-list 16 Office key grant CanRead+CanCreate=true upgrade-only (row tồn tại→nâng, chưa có→tạo R+C=true U/D=false, KHÔNG hạ KHÔNG đụng Update/Delete); +6 test `OfficeModulePermissionSeedTests.cs` (286→292). KHÔNG FE KHÔNG migration. deploy 11/11 session after #297–#306 all PASS):** Push `c556f6c..1f8947e` (1 commit, 5 files: DbInitializer.cs + 1 test + 3 agent-memory `.md` cicd/reviewer/test-spec). Diff `git diff --name-only c556f6c 1f8947e -- '*Migrations*' '*Persistence/Migrations*'` = EMPTY ✓ (DbInitializer.cs ở `Persistence/` NOT `Persistence/Migrations/` — seed runtime-idempotent NOT EF-migration). `.cs` non-ignored → full pipeline RAN. GITEA_TOKEN+PROD_DB_PW env empty trong bash-scope (env:GITEA_TOKEN có ở PS-scope; PROD_DB_PW thật rỗng) → anon Gitea API + DB pw từ prod `appsettings.Production.json`→`ConnectionStrings.Default` (path `C:\inetpub\solution-erp\api`, uid `vrapp` len24). Run IN-PROGRESS at spawn (running 10:33:40→10:38) — pre-deploy bundle baseline captured BEFORE poll-loop (anti#3): admin `Wt54PHYl` + user `B99fMU6X` — both == S70 #306 spec baseline (still live, deploy not yet shipped). Polled iter5 status=success (created 10:33:40 → success-update 10:38:13 ≈4m33s). CI gate (both proj pre-deploy ⟹ status=success ⟹ test **292** baseline (45D+247I; +6 OfficeModulePermissionSeedTests) passed; `conclusion` empty — `tasks` endpoint terminal=`status:success` không populate `conclusion`, trust success; 292 INFERRED gate-passes-pre-build invariant NOT log-confirmed numeric). **★ BUNDLE BOTH FROZEN (BE-only ⟹ both MUST stay = pre-deploy live; verified AFTER status=success): admin `Wt54PHYl` UNCHANGED** ✓ **+ user `B99fMU6X` UNCHANGED** ✓ (no FE touch → no content-hash rotation; rotate-w/o-FE-change = anomaly, did NOT happen). Smoke **4×200**: api `/health/ready`+`/health/live` + admin root + eoffice root. **NO migration** — prod `__EFMigrationsHistory` top5 = `AddHoSoLinkToPurchaseEvaluation`(Mig52)→`AddDepartmentParentId`(51)→`ReplaceBudgetModuleWithPeWorkItemBudgets`(50)→`AddWorkItemToPurchaseEvaluation`(49)→`AddProjectMasterFields`(48) == repo HEAD GIỮ NGUYÊN ✓. **sys.tables=88** (sqlcmd COUNT excl mighist — unchanged, seed inserts/updates ROWS not tables). **🔑★ GOLIVE DB VERIFIED THỰC SỰ ÁP PROD (seed chạy lúc boot ⟹ proof = prod Permissions, NOT chỉ binary-shipped): sqlcmd `Roles` (13 total: Accounting/Admin/AuthorizedSigner/CatalogManager/CostControl/DeptManager/Director/Drafter/Equipment/Finance/HrAdmin/Procurement/ProjectManager). (1) ALLOW-LIST 16/16 R=1∧C=1 cho Drafter (non-admin demo-role) — Off,Off_Dashboard,Off_DanhBa,Off_PhongHop(+View+Book),Off_DeXuat(+List+Create+Inbox),Off_DonTu(+Leave+Ot+Travel),Off_DatXe,Off_ItTicket ✓. (2) CROSS-ROLE: MỌI 13 role count=16 R∧C (golive ÁP TOÀN BỘ không Drafter-fluke) ✓. (3) EXCLUDED-3 (Off_PhongHop_Manage/Off_AttendanceReport/Off_ChamCong) Drafter R=0∧C=0 ✓ + LEAK-check: CHỈ Admin có R=1 trên 3 key này (0 non-admin leak) ✓. (4) HRM/Personal Drafter KHÔNG mở: Hrm_Dashboard=0, all 7 Hrm_Config*=0, Personal=0 ✓; Hrm=1 + Hrm_HoSo=1 (S65 public-read GIỮ NGUYÊN không đổi) ✓. (5) Admin KHÔNG hạ: full Office incl 3 excluded all R=1∧C=1 ✓.** ⟹ boot-time seed CHẠY THẬT trên prod (app pool recycled w/ new binary post-deploy; nếu seed CHƯA chạy thì non-admin sẽ vẫn 0 — nhưng 16/16 across 13 roles = đã chạy). Menu-tree non-admin via API NOT tested (DB-query mục 4 đủ proof per spec). 0 regression. NO prod-data mutation ngoài chính golive seed (read-only curls + sqlcmd SELECT-only; seed-write là chủ đích của deploy). Visual menu-render = anh UAT. **LESSON: BE-only GOLIVE security-seed verify = both bundles MUST stay FROZEN (no FE → no rotate; rotate=anomaly) + Mig-top + sys.tables MUST stay prev (DbInitializer seed = runtime row insert/update, NOT EF-migration → KHÔNG advance __EFMigrationsHistory NOR sys.tables) + **CORE proof = prod Permissions DB query**, NOT bundle-frozen alone (frozen chỉ chứng minh FE không đổi, KHÔNG chứng minh API binary mới deploy + seed chạy — phải query Permissions cho non-admin role thấy allow-list grant + cross-role để loại fluke + leak-check excluded chỉ-Admin). upgrade-only seed THẮNG revoke vì chạy SAU trong SeedAsync. golive-state AMBIENT-after-this-deploy (parent commit c556f6c chưa có method này → pre-1f8947e non-admin Office = 0; bằng chứng "đã chạy" = 16/16). TOOLING (re-confirmed S70): Bash=POSIX (`$var`/`$env:` mangle through bash→ssh→PS layers) → write PS to `/tmp/x.ps1` + `powershell.exe -NoProfile -File "C:/.../Temp/x.ps1"` (⚠️ heredoc em-dash/non-ASCII corrupts → ASCII-only in PS source); jq ABSENT + system python3 broken → Invoke-RestMethod cho Gitea `tasks` (match `head_sha -eq sha`, `?limit=N` URI-direct); SSH→PS base64 `-EncodedCommand` (UTF-16LE `[Text.Encoding]::Unicode`) cho BOTH appsettings-read AND sqlcmd; sqlcmd pw-literal qua single-quote in B64-payload (no `$` survives clean), `-U/-P` direct + `-W -s '|'` pipe-delim + `-h -1` no-header; read DB pw from prod appsettings when `$PROD_DB_PASSWORD` empty. NEVER fixed code (READ-only).** Tag `[s69b, run307, pass, be-only-golive-vanphongso-public-read-create-allowlist16, SeedAllRolesOfficeModulePermissionsAsync-boot-after-revoke, bundle-BOTH-frozen-Wt54PHYl-B99fMU6X, no-mig-top-stays-mig52, tables88, GOLIVE-DB-VERIFIED-16of16-across-13-roles, excluded3-canread0-only-admin-leak0, hrm-personal-still-0, hrm-hoso-public-unchanged, admin-not-downgraded, test292-inferred, deploy11of11, seed-not-migration-no-history-advance, no-regression]`.
|
||
- _(S77 #303 sha `6983609`, S76 #302, S75/S74 … pre-S78 verbatim → git `764fe70` + archive — FIFO trimmed to keep L1 under soft-cap)_
|
||
- _(S75 Run #301 sha=`6df1b2d` PASS ~2.5m [FE-both-app PE Link-hồ-sơ auto-detect render, bundle-BOTH-rotate I1fpLeYw/DrQYkzh0, no-mig-mig52, tables88, test286, fastest-streak] → FIFO-trimmed, full verbatim git + `archive/2026-06.md`)_
|
||
- _(S74 Run #300 sha=`91aaf05` PASS [FE-both-app list-redesign flex-row, bundle-BOTH-rotate PxiZQkaw/B36hGoKd, user-unfroze-from-s71-streak, no-mig-mig52, tables88, test286, TOOLING: ps1-file-not-inline-dollar + ssh-encodedcommand-base64] → FIFO-trimmed, full verbatim git + `archive/2026-06.md`)_
|
||
- _(S67 Run #292 [dept-parentid Mig51-applied-after-#291-fail, retry-chain CS7036-fix, bundle-asymmetric, tree-endpoint-200] + S65 #289 [hrm-hoso public-readonly seed-only, asymmetric read200/write403] → FIFO-trimmed, full verbatim git `d2f52ba` + `archive/2026-06.md`)_
|
||
- **Older runs (S66 #290 ← S62 #286 06-13 ← … → S29 #232) full verbatim archived → git `d2f52ba` + `archive/2026-06.md`** (incl #291 06-16 FAIL forensic [lesson=gotcha #65 + #292-inline] + #383 ex-VITRILAC); pre-S38 → `archive/2026-05-{runs,q2,q3,q4}.md`.
|
||
|
||
---
|
||
|
||
## 🔄 Curate trigger
|
||
- >~30KB → archive recent runs → L2 `archive/<period>.md`. Dup failure patterns → merge. Stale >3mo → remove.
|
||
- **Last curate: 2026-06-17 S70 Harness-9 (em-main + Stage-B `wf_a58e0d15-beb`)** (65.2→23.2KB): L2 dark-matter recovery — 10 oldest run-records → `archive/2026-06.md` (ADDITIVE, original 58378B prefix byte-exact) + `archive/_INDEX.md` (mục-lục substring sha-keyed, Ctrl-F fallback, no line-hint) + `2026-0{5,6}.gist.md` (4-field, distill-gen:1). 0-byte-loss git `+13 -0` + sha (Stage C `wf_9520d8cd-4fe` + em-main self-gate, 2 reviewer no-return). Kept foundation + 2 newest full #308/#307 + stubs.
|
||
- **Prev curate: 2026-06-16 S66 em main** (86.8→29.2KB sed Run #286→#232 incl #291 forensic) · S40 q4.
|
||
- **Prev curate: 2026-05-29 S40 em main proxy** (35.3→~21KB): archived Run #359/#243/#242/#241/#240 + S35/S36 startup → q4 + git d2f52ba; refreshed stale 120→130 test + Mig 34→40. Prev: S34 q3 · S32 q2 · S22 runs.
|