solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md

CI/CD Monitor Agent — Persistent Memory

Persistent diary cross-session. Auto-injected first ~200 lines at spawn (L1 HOT). Update BEFORE every stop. Tiered Memory v1: L1 HOT soft-cap ~30KB · L2 archive/ on-demand · L3 RAG search_memory just-in-time. Keep entry ≤ 1.5K chars (gotcha #53). Full verbatim run history pre-S40 → git d2f52ba + archive/2026-05-{runs,q2,q3,q4}.md + archive/2026-06.md. 🗺️ Lookup map (Harness-9 S70): archive/_INDEX.md — 1 dòng/bản-ghi + con-trỏ substring (sha-keyed, Ctrl-F fallback); đọc verbatim + 2026-0{5,6}.gist.md (nén 4-field) theo nhu cầu.

---

🎯 Role baseline

Read-only CI/CD + post-deploy verifier SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch + 5 RAG MCP. Output: PASS/FAIL + evidence <500 words. Skills: iis-deploy-runbook + dependency-audit-erp + ef-core-migration. Spawn ~150K — trade-off catch fail tự động.

---

🚨 Recurring CI/CD bug patterns (catch priority)

• #39 act_runner github.com TCP timeout — run hang "Set up job" 21s. Log dial tcp github.com:443 i/o timeout. Fix: manual checkout bypass hardcoded .gitea/workflows/deploy.yml (pass #110). KHÔNG revert.
• #40 npm cache tsc not found — build_fe_admin fail post cache: npm. DISABLED rolled back a21790d. KHÔNG re-enable.
• #41 paths-ignore docs-only skip — code commit không trigger CI? Check git diff --name-only HEAD~1 HEAD vs paths-ignore: ['docs/**','**/*.md','.claude/skills/**']. Discovery #3: Gitea evaluates push range commits — nếu ≥1 commit có non-ignored file → toàn range build (BENEFICIAL).
• #25 IIS WebSocket — notification-hub/negotiate 401/404 prod. Fix: WebSocket module enable web.config site api (skill iis-deploy-runbook).
• #48 SQLite tie-break — OrderByDescending(CreatedAt).First() pick wrong khi 2+ .Add() cùng frozen-clock. Fix: discriminator filter .Where(Summary.Contains("Chuyển phase")) BEFORE OrderBy.
• Bundle hash unchanged = ship FAIL — push+action success nhưng prod không đổi. Verify via INDEX.HTML ref (curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[A-Za-z0-9_-]+\.(js|css)'), NOT by GETting a hash-named asset directly. Fix: SSH Restart-WebAppPool. ⚠️ Bundle hash verify MUST sau status=success (Run #242 false-positive lesson: check khi "running" → stale hash).
  • 🔴 #69 (S72 Run #312 OVERTURNS prior invariant) — FE bundle hash is NON-DETERMINISTIC per rebuild. deploy.yml Remove-Item fe-{admin,user}\* -Exclude web.config + Copy-Item dist\* runs UNCONDITIONALLY every run (path-filter gates whole-workflow trigger, NOT per-step). Identical FE source ⟹ DIFFERENT hash each deploy (Vite/rolldown non-reproducible chunk-id). PROVEN: governance-only 18fced6 (0 files in fe-*/src) rotated BOTH bundles admin BgNCjwsG→fc_xkNpJ (+717b) + user CBvh0vtf→DP-tBcg0, index.html Last-Modified matched deploy window. ⟹ "BE-only/governance ⟹ bundle MUST stay frozen" is FALSE; rotation is EXPECTED on every deploy. Bundle-rotate alone is NOT proof of FE content change — to detect real FE change, diff fe-*/src in commit/range, NOT hash delta.
  • ⚠️ SPA-fallback 200 trap (S72): server rewrites /*→/index.html so GET /assets/index-<ANYTHING>.js returns 200 even for non-existent/fake hash (control ZZdoesnotexist0.js→200 confirmed). Old-hash-still-200 is MEANINGLESS as persistence signal. RELIABLE bundle signal = parse index.html <script src>/<link href>, then GET that exact path + check size_download large (not white-screen) + Last-Modified in deploy window.
• Migration drift prod vs repo — compare ls .../Persistence/Migrations/*.cs vs sqlcmd __EFMigrationsHistory. Fix: check Program.cs app.MigrateDatabase() + app pool recycle.

---

📋 5-stage checklist (EVERY run)

• Stage 0 RAG infra: Get-Service Qdrant Running + http://localhost:6333/healthz. Collection proj_solution_erp (prefix proj_* 7 project — Discovery #8).
• Stage 1 Push+filter: git log -1 --format='%H %s' + git log origin/main..HEAD empty + diff vs paths-ignore (docs-only → SKIPPED-DOCS return).
• Stage 2 Gitea poll (max 10 iter × 60s): API .../actions/tasks?limit=5 (NOT /runs 404). Match head_sha. ⚠️ task table updated_at stale ~2min (gotcha #46) → cross-check VPS mtime.
• Stage 3 Test gate: baseline 130 PASS (58 Domain + 72 Infra). Phase 9 UAT exception lower OK (feedback_uat_skip_verify).
• Stage 4 Post-deploy (if SUCCESS): auth login bearer (admin + nv.test gotcha #44; token=accessToken route /api/auth/login) → 3-5 endpoint smoke 2XX (incl new) → FE bundle hash 2 app changed → SignalR negotiate (gotcha #25 if relevant) → EF mig prod==repo.
  • Stage 4.6 (S29 CRITICAL): sqlcmd seed sample verify post-deploy (NOT chỉ schema). sqlcmd -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'" → 0 rows = seed GATE BLOCKED → gotcha #51.
  • Discovery #4: ASP.NET 10 record enum cần numeric input unless JsonStringEnumConverter (SOL has NO converter → FE sends numeric). #5: sqlcmd ssh Windows-auth cần \\\\SQLEXPRESS 4-backslash. #6: INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/SampleWorkflowsV2) MUST run, NOT inside if(!demoSeedDisabled); DEMO seed (DemoUsers/Contracts/PE) OK gated → gotcha #51.
• Stage 5 Report PASS/FAIL + evidence + MEMORY update.

---

⚠️ Anti-patterns (DO NOT)

1. ❌ Push fix code — READ only, escalate em main · 2. ❌ Speculate fail without log · 3. ❌ Skip post-deploy bundle hash (biggest catch) · 4. ❌ Skip MEMORY · 5. ❌ Poll forever (max 10 iter) · 6. ❌ Auto-rollback (escalate + recommend) · 7. ❌ Verify docs-only (SKIPPED-DOCS return ngay)

---

🧠 SOLUTION_ERP CI/CD essentials (S40 verified)

• Gitea: git.baocaogiaoduc.vn/vietreport-admin/solution-erp · workflow .gitea/workflows/deploy.yml · paths-ignore ['docs/**','**/*.md','.claude/skills/**']
• Prod: api/admin/eoffice .solutions.com.vn · SSH ssh vietreport-vps (Administrator, id_ed25519) · IIS site phys paths (S42 verified): API C:\inetpub\solution-erp\api · admin \fe-admin · user \fe-user (3 sites Started). DB .\SQLEXPRESS/SolutionErp/vrapp SQL-auth. Conn string key = ConnectionStrings.Default (NOT DefaultConnection!) — read pw from prod appsettings.Production.json when $env:PROD_DB_PASSWORD empty.
  • SSH→PS quoting (S42 lesson): nested bash→ssh→powershell mangles $var/\". Use iconv UTF-16LE | base64 → powershell -EncodedCommand $B64. Single-quote literal paths.
• Tests baseline: 263 PASS (S62 Run #286 sha 7926c21 spec; 45 Domain + 218 Infra — em-main supplied; supersedes prev 228/240/256). CI gate runs both test projects BEFORE build/deploy → status=success ⟹ test gate passed (tasks endpoint reports terminal as status:success, conclusion field NOT populated). Local grep undercounts (Theory/InlineData) — trust CI conclusion. Phase 9 UAT mode skip per chunk OK.
• Mig latest repo: Mig 55 20260618105426_AddCcmNoteToPeWorkItemBudget (S74-bis Run#315 sha 8655ebf; PeWorkItemBudgets +CcmNote nvarchar(1000) nullable — 1 AddColumn no new table — VERIFIED APPLIED PROD: history-top==repo, sys.columns nvarchar maxlen=2000 null=1 [2000=byte-len 1000char×2], tables88). Prev Mig 54 AddPeSuggestedAndApprovedPrice (S73 #313, 5 PE price cols) + Mig 53 20260617060207_AddPeUrgentAndCeoApprovalThreshold (S71 Run#308; PE +IsUrgentByPro/+IsUrgentByCcm bit-default-0 + ApprovalWorkflows +CeoApprovalThreshold decimal(18,2)-nullable, 3 AddColumn no new table — VERIFIED APPLIED PROD). Prev Mig 52 AddHoSoLinkToPurchaseEvaluation (S65 PE HoSoLink hyperlink-NAS) + Mig 51 AddDepartmentParentId (S65 org-tree loose-Guid) + Mig 50 ReplaceBudgetModuleWithPeWorkItemBudgets (S61 net-reduce). Path src/Backend/SolutionErp.Infrastructure/Persistence/Migrations/ (53 mig .cs non-designer total). Prod check sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC TOP 5. ⚠️ Table-count: sys.tables (is_ms_shipped=0, excl mighist) = 88 (S62 Run #286 verified — S61 Budget-replace DROPPED tables 93→88). Narrative-93 is STALE pre-S61 — when commit touches no schema, 88 is correct, don't FAIL on 88↔93. Always cross-ref COMMIT scope vs ambient count.
• Bearer: admin admin@solutions.com.vn/Admin@123456 (full) · UAT nv.test@solutions.com.vn/TestUser@123456 (Drafter CCM, gotcha #44 check)
• Bundle hash live S74-bis: admin BYF5vIMJ + css X_45M1jX · user CB-tiRxd + css Bbbo0M5H (Run #315 sha 8655ebf; index.html Last-Modified admin 2026-06-18T12:08:53Z + user 12:09:45Z = deploy window; CSS unchanged from #313/#314 = PE-tab logic touched not style). Prev #314: admin Bv3jUCNo/user BWlMBQz6. Prev #313: admin OlNyG9OD/user DSzSLVtL. ⚠️ PER #69 (S72): bundle hash ROTATES on EVERY deploy regardless of FE change (non-deterministic rebuild). So "live hash" value is only a SNAPSHOT for THIS run — do NOT treat it as a frozen-baseline to compare next run against. To detect a real FE ship, diff fe-*/src in commit/range — NOT hash delta. ASYMMETRIC-deploy "anomaly" framing (S66) is RETIRED by #69: both apps rebuilt+rotated every deploy, asymmetry impossible to read from hashes alone. S50 mid-deploy transient lesson still holds: re-confirm hash AFTER status=success ALWAYS (anti-pattern #3).
• DB pw (S42, when $PROD_DB_PASSWORD empty): vrapp/buKL3TGBkD0wDDbYVw65QeX9 read from C:\inetpub\solution-erp\api\appsettings.Production.json→ConnectionStrings.Default. ⚠️ Skill-doc path C:\inetpub\apps\SolutionErp\Api is STALE → real path C:\inetpub\solution-erp\api. sqlcmd over SSH works direct (no UTF-16 encode needed). ⚠️ sys-catalog string-concat queries hit collation conflict (Latin1_General_CI_AS_KS_WS vs SQL_Latin1_General_CP1_CI_AS) → add COLLATE DATABASE_DEFAULT per concatenated column.

🔑 Critical config (flag commit nếu tái xuất)

Node CI 20.x (feedback_node_cicd) · MediatR 12.4.1 (gotcha #1, flag Version="14) · Swashbuckle 6.9.0 (gotcha #2) · act_runner manual checkout (#39) · npm cache DISABLED (#40, flag cache: npm)

---

🎯 Per-NV admin opt-in wire — 10-point checklist (cumulative S22→S23)

Cross-ref feedback_per_nv_permission_scope. Per-NV/per-Level refactor MUST verify: 1 Domain field · 2 EF HasDefaultValue(false) · 3 Mig 3-file · 4 Service read · 5 Domain+App DTO mirror · 6 Designer FE checkbox · 7 AwLevelDto+ToDto · 8 CreateAwLevelInput+Update mutation · 9 Lookup discrimination (FirstOrDefault ADD ApproverUserId==actorId + admin fallback) · 10 Controller body record count == Command record count. Bug latency 2-3 days prod silent khi miss 9-10. Scan grep -n "FirstOrDefault.*Order.*==" *.cs after OR-of-N refactor.

📊 Run stats baseline

BE (test+build) ~90s · FE × 2 ~60s/app · deploy ~30s · total ~3min code / 0s docs-only. >5min → escalate.

---

📅 Recent runs (FIFO — older → archive/git)

• (S72 Run #312 sha=18fced6 PASS ~5m16s [governance-only 26-md+hmw.js+memory-budget.json triggers-CI, ZERO prod code, Mig53-unchanged-VERIFIED-prod, tables88, health-4x200, test306-inferred, BUNDLE-BOTH-ROTATE-but-BENIGN fc_xkNpJ/DP-tBcg0 → NEW gotcha #69 (FE-hash NON-deterministic per-rebuild, deploy.yml Remove+Copy fe-* UNCONDITIONAL every run, "frozen-on-BE-only" invariant OVERTURNED, SPA-200-trap control-test, real-FE-change = git diff fe-*/src NOT hash-delta), 0-fe-src-diff-confirms-no-accidental-ship] → gotcha #69 promoted to recurring-patterns §top; FIFO-trimmed, full verbatim git 18fced6+39d55d4)

• (S71 Run #308 sha=ebd7e1c PASS ~4m41s [FULL-STACK PE-urgent+CCM-threshold, Mig53 AddPeUrgentAndCeoApprovalThreshold VERIFIED-APPLIED-PROD 3-cols-sys-columns (IsUrgentByPro/Ccm bit0 + CeoApprovalThreshold decimal18-2-null), history-top-advance-53, tables88-addcolumn-only, bundle-BOTH-rotate BgNCjwsG/CBvh0vtf, endpoint-urgent-401-not-404, test306] → 4-axis full-stack pattern (BE+FE-both-rotate · history-top-advance + N-cols sys.columns type-verify · tables-stay-88 · endpoint-401-not-404); FIFO-trimmed, full verbatim git ebd7e1c)

• 2026-06-18 S74-bis Run #315 (run_number 315, id=429) sha=8655ebf PASS ~4m54s (FULL-STACK: BE Mig 55 AddCcmNoteToPeWorkItemBudget ô "Ghi chú từ CCM" ngân-sách-gói-thầu — CCM nhập số + ghi lý do giống PRO, anh Kiệt FDC continuation of Mig 50/53/54. 14 files: BE 9 {Controller, App Dtos+PeWorkItemBudgetFeatures+PurchaseEvaluationFeatures, Domain PeWorkItemBudget.cs, Config PeWorkItemBudgetConfiguration} + 3 Mig-file + FE 4 {PeDetailTabs+types ×2-app} + 1 test PeWorkItemBudgetTests. deploy 16/16 session after #297–#314 all PASS): Push HEAD=8655ebf (nothing unpushed; range e7e99d1..8655ebf single commit). git diff --name-only e7e99d1..8655ebf -- '*Persistence/Migrations*' = 3 files (Mig 55 .cs/.Designer.cs/Snapshot) — REAL EF migration. Mig55 Up() read = single AddColumn<string> CcmNote table=PeWorkItemBudgets nvarchar(1000) maxLength:1000 nullable:true; Down() DropColumn — matches spec exactly, AddColumn-only no new table. .cs/.tsx non-ignored → full pipeline RAN. GITEA_TOKEN+PROD_DB_PW both absent env → anon Gitea API (worked, public repo) + DB pw read prod appsettings.Production.json→ConnectionStrings.Default (vrapp/buKL3TGBkD0wDDbYVw65QeX9, .\SQLEXPRESS/SolutionErp, path C:\inetpub\solution-erp\api). Run IN-PROGRESS at spawn (running 19:05:26, exact head_sha match #315 deterministic). ★ PRE-DEPLOY DB SNAPSHOT captured (proves deploy hadn't shipped at spawn): prod history-top = Mig 54 AddPeSuggestedAndApprovedPrice, CcmNote col ABSENT, tables88. Pre-poll bundle baseline (anti#3): admin Bv3jUCNo/css X_45M1jX + user BWlMBQz6/css Bbbo0M5H = still #314 live. Poll-loop iter3 status=success (created 19:05:26→success 19:10:20 ≈4m54s; iter1-2 running). CI gate (both proj pre-build ⟹ status=success ⟹ test 339 expected (45D+294I; +5 new PeWorkItemBudgetTests vs #313's 334) passed; conclusion empty — tasks terminal=status:success doesn't populate conclusion, trust success; 339 INFERRED gate-passes-pre-build NOT log-confirmed numeric). 🔑★ MIG 55 VERIFIED APPLIED PROD (sqlcmd-over-SSH ground-truth, POST-deploy re-query): history-top advanced Mig54→Mig55 20260618105426_AddCcmNoteToPeWorkItemBudget == repo HEAD ✓ (DbInitializer auto-migrate-on-boot ran on app-pool recycle). Col CcmNote now EXISTS sys.columns: nvarchar | maxlen=2000 | null=1 ✓ (2000 = SQL byte-len 1000char×2 ⟹ confirms nvarchar(1000) spec; nullable ✓). sys.tables=88 UNCHANGED — AddColumn no new table ✓. ★ BUNDLE BOTH ROTATE (FE 2-app PeDetailTabs change ⟹ both EXPECTED per gotcha #69; verified AFTER status=success + STABLE 2nd-fetch no-transient per anti#3): admin JS ROTATE Bv3jUCNo→BYF5vIMJ ✓ (css X_45M1jX UNCHANGED — tab logic not style) + user JS ROTATE BWlMBQz6→CB-tiRxd ✓ (css Bbbo0M5H UNCHANGED). Asset reachable 200 + LARGE: admin js 1,600,221b + user js 1,504,880b (control fake /assets/index-ZZfakehash0.js→200 size 919b/895b SPA-fallback ⟹ real JS shipped, gotcha #69 SPA-200-trap distinguished by size). index.html Last-Modified admin 12:08:53Z + user 12:09:45Z (=19:08-19:09VN deploy window) ✓. Smoke 4×200 health: api /health/ready+/health/live + admin root + eoffice root. 0 regression. NO prod-data mutation (read-only curls + sqlcmd SELECT-only; mig-apply+FE-copy = expected boot/deploy side-effects). Behavioral CCM-note input on PeWorkItemBudgets = anh Kiệt UAT (em confirm Mig55+col+bundles shipped per spec). VERDICT PASS: green CI + test 339 + Mig 55 applied (CcmNote nvarchar(1000) sys.columns ground-truth, history advanced 54→55) + tables 88 + bundle BOTH rotate + health 4×200. LESSON: textbook full-stack-single-AddColumn = same 4-axis as S73 #313/S71 #308 (BE+FE-both-rotate · Mig history-top advance + col sys.columns type-verify[byte-len 2×char] · sys.tables stays 88 AddColumn-only · health 4×200). PRE-deploy DB snapshot at spawn (Mig54+col-absent) made the POST-deploy advance (Mig55+col-present) an unambiguous proof — captured baseline BEFORE poll-loop = best practice when run still building. Mig-applied proof = __EFMigrationsHistory top==repo-HEAD; if stuck Mig 54 ⟹ app pool didn't recycle ⟹ FAIL even if status=success+bundle-rotated. NOTE: prompt called this "Run #314" but #314 was prior FE-only 6aa4dcb — actual run for 8655ebf = #315 (matched by head_sha, NOT run_number). Rate-limit env this session (a sibling sub died on 529) — this verify completed without hitting it. TOOLING: bash POSIX → write PS to $LOCALAPPDATA/Temp/x.ps1 + powershell.exe -File $(cygpath -w ...); Gitea tasks via Invoke-RestMethod -Body @{limit=N} hashtable; poll-loop backgrounded + Monitor grep-until FINAL; SSH→PS base64 -EncodedCommand UTF-16LE for appsettings-read AND sqlcmd; sqlcmd pw via [char]39 literal quotes + -W -h -1. NEVER fixed code (READ-only). Tag [s74bis, run315, pass, full-stack-pe-ccmnote, mig55-AddCcmNoteToPeWorkItemBudget-VERIFIED-APPLIED-PROD, ccmnote-nvarchar1000-sys-columns-maxlen2000-bytelen-null, history-top-advance-54-to-55, tables88-unchanged-addcolumn-only, bundle-BOTH-rotate-BYF5vIMJ-CB-tiRxd, css-unchanged-X_45M1jX-Bbbo0M5H, asset-200-large-1.6MB-vs-fake-919b-spa-trap, last-modified-deploy-window-12-08-12-09z, health-4x200, test339-inferred-plus5-PeWorkItemBudgetTests, deploy16of16, pre-deploy-db-snapshot-mig54-then-post-mig55-unambiguous, prompt-said-314-but-actual-315-matched-by-head-sha, rate-limit-env-sibling-died-529-this-verify-ok, anon-gitea-api-both-token-absent, behavioral-anh-kiet-uat].

• (S74 Run #314 sha=6aa4dcb PASS ~5m12s [FE-ONLY empty-candidates guard PeWorkflowPanel ×2-app, no-mig-stays-54, bundle-BOTH-js-rotate OlNyG9OD→Bv3jUCNo/DSzSLVtL→BWlMBQz6, css-frozen, asset-200-large-vs-fake-919b, health-4x200, test334, no-DB-query-needed-FE-only] → LESSON: FE-only verify lightweight, skip Mig/sqlcmd when 0 migration in diff; FIFO-trimmed, full verbatim git 6aa4dcb)

• (S69b Run #307 sha=1f8947e PASS ~4m33s [BE-ONLY GOLIVE security "Văn phòng số" public Read+Create — NEW SeedAllRolesOfficeModulePermissionsAsync boot-after-revoke, allow-list 16 Office key upgrade-only; GOLIVE-DB-VERIFIED-16of16-across-13-roles sqlcmd prod Permissions (ALLOW-16 R∧C all roles · EXCLUDED-3 only-Admin leak0 · HRM/Personal Drafter still-0 · admin-not-downgraded), bundle-BOTH-frozen Wt54PHYl/B99fMU6X (BE-only no-rotate), no-mig-top-mig52, tables88, test292] → LESSON: BE-only golive-seed verify CORE-proof = prod Permissions DB query NOT bundle-frozen alone (seed=runtime-row-insert NOT EF-mig → no history/tables advance); FIFO-trimmed, full verbatim git 1f8947e)

• (S77 #303 sha 6983609, S76 #302, S75/S74 … pre-S78 verbatim → git 764fe70 + archive — FIFO trimmed to keep L1 under soft-cap)

• (S75 Run #301 sha=6df1b2d PASS ~2.5m [FE-both-app PE Link-hồ-sơ auto-detect render, bundle-BOTH-rotate I1fpLeYw/DrQYkzh0, no-mig-mig52, tables88, test286, fastest-streak] → FIFO-trimmed, full verbatim git + archive/2026-06.md)

• 2026-06-18 S73 Run #313 (run_number 313, id=427) sha=1d86abc PASS ~5m22s (FULL-STACK: BE Mig 54 PE giá-đề-xuất PRO/CCM + CEO chọn giá-chốt + CCM duyệt-done ô-tích, anh Kiệt FDC. 19 files: BE 9 {Controller +2 endpoint suggested-price/{pro,ccm} + ApprovedPrice in finalize, App PeSuggestedPriceFeatures.cs NEW, PurchaseEvaluationFeatures, Dtos, IPurchaseEvaluationWorkflowService, Domain PE, Config} + 3 Mig-file + Service PurchaseEvaluationWorkflowService + FE 6 {PeDetailTabs+PeWorkflowPanel+types ×2-app} + 3 test {PeSuggestedPriceSetterAuthzTests, PeApprovedPriceFinalizeTests, PeCcmThresholdFinalizeTests}. deploy 14/14 session after #297–#312 all PASS): Push HEAD=1d86abc (nothing unpushed). git diff --name-only 1d86abc~1 1d86abc -- '*Persistence/Migrations*' = 3 files (Mig 54 .cs/.Designer.cs/Snapshot) — REAL EF migration. .cs/.tsx non-ignored → full pipeline RAN. GITEA_TOKEN + PROD_DB_PW BOTH absent env → anon Gitea API (worked, public repo) + DB pw from prod appsettings.Production.json→ConnectionStrings.Default (vrapp/buKL3TGBkD0wDDbYVw65QeX9 len24, path C:\inetpub\solution-erp\api). Run IN-PROGRESS at spawn (running 15:51:47, exact head_sha match #313 deterministic). Poll-loop iter5 status=success (created 15:51:47→success 15:57:09 ≈5m22s). CI gate (both proj pre-deploy ⟹ status=success ⟹ test 334 baseline (45D+289I; +3 PeSuggestedPriceSetterAuthz+PeApprovedPriceFinalize+PeCcmThresholdFinalize) passed; conclusion empty — tasks terminal=status:success doesn't populate conclusion, trust success; 334 INFERRED gate-passes-pre-build NOT log-confirmed numeric). 🔑★ MIG 54 VERIFIED APPLIED PROD (sqlcmd over SSH ground-truth): prod __EFMigrationsHistory top5 = 20260618081758_AddPeSuggestedAndApprovedPrice(54)→AddPeUrgentAndCeoApprovalThreshold(53)→AddHoSoLinkToPurchaseEvaluation(52)→AddDepartmentParentId(51)→ReplaceBudgetModuleWithPeWorkItemBudgets(50) == repo HEAD ✓ (DbInitializer auto-migrate-on-boot ran). ALL 5 NEW COLUMNS confirmed via sys.columns w/ types: ProSuggestedMinPrice decimal(18,2) null ✓ + ProSuggestedMaxPrice decimal(18,2) null ✓ + CcmSuggestedPrice decimal(18,2) null ✓ + ApprovedPriceAmount decimal(18,2) null ✓ + ApprovedPriceSource nvarchar null ✓ — match Mig 54 Up() spec. sys.tables=88 (excl mighist) UNCHANGED — 5 AddColumn no new table ✓. ★ BUNDLE BOTH ROTATE (FE 2-app PE-UI change ⟹ both MUST rotate; verified AFTER status=success + STABLE 2nd-fetch no-transient per anti#3): admin ROTATE fc_xkNpJ→OlNyG9OD ✓ (+css C2Zvnd2f→X_45M1jX) + user ROTATE DP-tBcg0→DSzSLVtL ✓ (+css 1bS4xeUF→Bbbo0M5H). Asset reachable 200 + LARGE: admin js 1,599,106b + user js 1,503,765b (control fake /assets/index-ZZfakehash0.js→200 size=919b SPA-fallback ⟹ real JS shipped, gotcha #69 SPA-200-trap distinguished by size). index.html Last-Modified admin 08:55:32Z + user 08:56:33Z (=15:55VN deploy window) ✓. Smoke 4×200 health: api /health/ready+/health/live + admin root + eoffice root. ★ NEW ENDPOINT PROBE: PUT /api/purchase-evaluations/{guid}/suggested-price/pro unauth → 401 (NOT 404) ✓ (route EXISTS controller line 102 SetSuggestedPricePro→UpdatePeSuggestedPriceProCommand, class-level [Authorize] enforced; PRO-only authz = unit-test PeSuggestedPriceSetterAuthzTests) + /suggested-price/ccm unauth → 401 ✓ + PE list unauth → 401 ✓ (all reachable+gated). 0 regression. NO prod-data mutation (read-only curls + sqlcmd SELECT-only; mig-apply+FE-copy = expected boot/deploy side-effects). Behavioral PRO/CCM giá-đề-xuất + CEO chọn-giá + CCM duyệt-done = anh Kiệt UAT (em confirm Mig+5cols+endpoint+bundles shipped per spec). VERDICT PASS: green CI + test 334 + Mig 54 applied (5 cols sys.columns ground-truth) + tables 88 + bundle BOTH rotate + new endpoint 401-not-404 + health 4×200. LESSON: full-stack PE w/ REAL Mig 54 = same 4-axis pattern as S71 #308 (BE+FE-both-rotate · Mig history-top advance + N cols sys.columns type-verify · sys.tables stays 88 AddColumn-only · new-endpoint 401-not-404 probe). Mig-applied proof = __EFMigrationsHistory top==repo-HEAD; if stuck Mig 53 ⟹ app pool didn't recycle ⟹ FAIL even if status=success+bundle-rotated. TOOLING: bash POSIX → write PS to $LOCALAPPDATA/Temp/x.ps1 + powershell.exe -File $(cygpath -w ...); Gitea tasks via Invoke-RestMethod -Body @{limit=N} hashtable (NOT query-string — URI parse error); poll-loop backgrounded + Monitor grep-until FINAL; SSH→PS base64 -EncodedCommand UTF-16LE for appsettings-read AND sqlcmd; sqlcmd pw via here-string $P= + [char]39 literal object-name quotes + -W -h -1; read DB pw from prod appsettings when both env empty. NEVER fixed code (READ-only). Tag [s73, run313, pass, full-stack-pe-suggested-price, mig54-AddPeSuggestedAndApprovedPrice-VERIFIED-APPLIED-PROD, 5-cols-sys-columns-confirmed-ProSuggestedMin/Max-CcmSuggested-decimal18-2-ApprovedPriceAmount-decimal-ApprovedPriceSource-nvarchar-all-null, history-top-advance-54, tables88-unchanged-addcolumn-only, bundle-BOTH-rotate-OlNyG9OD-DSzSLVtL, asset-200-large-1.5MB-vs-fake-919b-spa-trap, endpoint-suggested-price-pro-401-not-404-route-exists-ccm-401, pe-list-401-gated, health-4x200, test334-inferred, deploy14of14, no-regression, anon-gitea-api-both-token-absent, behavioral-anh-kiet-uat].

• (S74 Run #300 sha=91aaf05 PASS [FE-both-app list-redesign flex-row, bundle-BOTH-rotate PxiZQkaw/B36hGoKd, user-unfroze-from-s71-streak, no-mig-mig52, tables88, test286, TOOLING: ps1-file-not-inline-dollar + ssh-encodedcommand-base64] → FIFO-trimmed, full verbatim git + archive/2026-06.md)

• (S67 Run #292 [dept-parentid Mig51-applied-after-#291-fail, retry-chain CS7036-fix, bundle-asymmetric, tree-endpoint-200] + S65 #289 [hrm-hoso public-readonly seed-only, asymmetric read200/write403] → FIFO-trimmed, full verbatim git d2f52ba + archive/2026-06.md)

• Older runs (S66 #290 ← S62 #286 06-13 ← … → S29 #232) full verbatim archived → git d2f52ba + archive/2026-06.md (incl #291 06-16 FAIL forensic [lesson=gotcha #65 + #292-inline] + #383 ex-VITRILAC); pre-S38 → archive/2026-05-{runs,q2,q3,q4}.md.

---

🔄 Curate trigger

• ~30KB → archive recent runs → L2 archive/<period>.md. Dup failure patterns → merge. Stale >3mo → remove.

• Last curate: 2026-06-17 S70 Harness-9 (em-main + Stage-B wf_a58e0d15-beb) (65.2→23.2KB): L2 dark-matter recovery — 10 oldest run-records → archive/2026-06.md (ADDITIVE, original 58378B prefix byte-exact) + archive/_INDEX.md (mục-lục substring sha-keyed, Ctrl-F fallback, no line-hint) + 2026-0{5,6}.gist.md (4-field, distill-gen:1). 0-byte-loss git +13 -0 + sha (Stage C wf_9520d8cd-4fe + em-main self-gate, 2 reviewer no-return). Kept foundation + 2 newest full #308/#307 + stubs.
• Prev curate: 2026-06-16 S66 em main (86.8→29.2KB sed Run #286→#232 incl #291 forensic) · S40 q4.
• Prev curate: 2026-05-29 S40 em main proxy (35.3→~21KB): archived Run #359/#243/#242/#241/#240 + S35/S36 startup → q4 + git d2f52ba; refreshed stale 120→130 test + Mig 34→40. Prev: S34 q3 · S32 q2 · S22 runs.