vietreport-admin/solution-erp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f0e616fd5a2f290aa293d6c4f14625db779f6ca6
solution-erp/.claude/agent-memory/cicd-monitor/MEMORY.md
pqhuy1987 f0e616fd5a [CLAUDE] Docs: S78 closeout — PE attach-file khi duyet (Run #330) + gotcha #71 + cicd flush 
STATUS/HANDOFF tier +S78 (bundle CsJetgZH/BVS0ApIm, no migration, test 354 unchanged) + trim S75 row. gotcha #71 (them enum value vao entity dung-chung -> pollute UI/guard proxy-predicate supplierId===null). session log S78. cicd-monitor MEMORY self-flush Run #330. curate-debt carry: reviewer 45.2KB + cicd 38.8KB + inv 35.7KB keep-floor-hit (archive-gate A7 PASS 186/186). Docs-only -> CI skip.
2026-06-19 19:32:41 +07:00

38 KiB
Raw Blame History

CI/CD Monitor Agent — Persistent Memory

Persistent diary cross-session. Auto-injected first ~200 lines at spawn (L1 HOT). Update BEFORE every stop. Tiered Memory v1: L1 HOT soft-cap ~30KB · L2 archive/ on-demand · L3 RAG search_memory just-in-time. Keep entry ≤ 1.5K chars (gotcha #53). Full verbatim run history pre-S40 → git d2f52ba + archive/2026-05-{runs,q2,q3,q4}.md + archive/2026-06.md. 🗺️ Lookup map (Harness-9 S70): archive/_INDEX.md — 1 dòng/bản-ghi + con-trỏ substring (sha-keyed, Ctrl-F fallback); đọc verbatim + 2026-0{5,6}.gist.md (nén 4-field) theo nhu cầu.

🎯 Role baseline

Read-only CI/CD + post-deploy verifier SOLUTION_ERP. Polls Gitea Actions API, verifies test gate + deploy ship + prod health. Tools: Read, Grep, Glob, Bash, WebFetch + 5 RAG MCP. Output: PASS/FAIL + evidence <500 words. Skills: iis-deploy-runbook + dependency-audit-erp + ef-core-migration. Spawn ~150K — trade-off catch fail tự động.

🚨 Recurring CI/CD bug patterns (catch priority)

  • #39 act_runner github.com TCP timeout — run hang "Set up job" 21s. Log dial tcp github.com:443 i/o timeout. Fix: manual checkout bypass hardcoded .gitea/workflows/deploy.yml (pass #110). KHÔNG revert.
  • #40 npm cache tsc not found — build_fe_admin fail post cache: npm. DISABLED rolled back a21790d. KHÔNG re-enable.
  • #41 paths-ignore docs-only skip — code commit không trigger CI? Check git diff --name-only HEAD~1 HEAD vs paths-ignore: ['docs/**','**/*.md','.claude/skills/**']. Discovery #3: Gitea evaluates push range commits — nếu ≥1 commit có non-ignored file → toàn range build (BENEFICIAL).
  • #25 IIS WebSocket — notification-hub/negotiate 401/404 prod. Fix: WebSocket module enable web.config site api (skill iis-deploy-runbook).
  • #48 SQLite tie-break — OrderByDescending(CreatedAt).First() pick wrong khi 2+ .Add() cùng frozen-clock. Fix: discriminator filter .Where(Summary.Contains("Chuyển phase")) BEFORE OrderBy.
  • Bundle hash unchanged = ship FAIL — push+action success nhưng prod không đổi. Verify via INDEX.HTML ref (curl -s https://admin.solutions.com.vn/ | grep -oE '/assets/index-[A-Za-z0-9_-]+\.(js|css)'), NOT by GETting a hash-named asset directly. Fix: SSH Restart-WebAppPool. ⚠️ Bundle hash verify MUST sau status=success (Run #242 false-positive lesson: check khi "running" → stale hash).
    • 🔴 #69 (S72 Run #312 OVERTURNS prior invariant) — FE bundle hash is NON-DETERMINISTIC per rebuild. deploy.yml Remove-Item fe-{admin,user}\* -Exclude web.config + Copy-Item dist\* runs UNCONDITIONALLY every run (path-filter gates whole-workflow trigger, NOT per-step). Identical FE source ⟹ DIFFERENT hash each deploy (Vite/rolldown non-reproducible chunk-id). PROVEN: governance-only 18fced6 (0 files in fe-*/src) rotated BOTH bundles admin BgNCjwsG→fc_xkNpJ (+717b) + user CBvh0vtf→DP-tBcg0, index.html Last-Modified matched deploy window. ⟹ "BE-only/governance ⟹ bundle MUST stay frozen" is FALSE; rotation is EXPECTED on every deploy. Bundle-rotate alone is NOT proof of FE content change — to detect real FE change, diff fe-*/src in commit/range, NOT hash delta.
    • ⚠️ SPA-fallback 200 trap (S72): server rewrites /*→/index.html so GET /assets/index-<ANYTHING>.js returns 200 even for non-existent/fake hash (control ZZdoesnotexist0.js→200 confirmed). Old-hash-still-200 is MEANINGLESS as persistence signal. RELIABLE bundle signal = parse index.html <script src>/<link href>, then GET that exact path + check size_download large (not white-screen) + Last-Modified in deploy window.
  • Migration drift prod vs repo — compare ls .../Persistence/Migrations/*.cs vs sqlcmd __EFMigrationsHistory. Fix: check Program.cs app.MigrateDatabase() + app pool recycle.

📋 5-stage checklist (EVERY run)

  • Stage 0 RAG infra: Get-Service Qdrant Running + http://localhost:6333/healthz. Collection proj_solution_erp (prefix proj_* 7 project — Discovery #8).
  • Stage 1 Push+filter: git log -1 --format='%H %s' + git log origin/main..HEAD empty + diff vs paths-ignore (docs-only → SKIPPED-DOCS return).
  • Stage 2 Gitea poll (max 10 iter × 60s): API .../actions/tasks?limit=5 (NOT /runs 404). Match head_sha. ⚠️ task table updated_at stale ~2min (gotcha #46) → cross-check VPS mtime.
  • Stage 3 Test gate: baseline 130 PASS (58 Domain + 72 Infra). Phase 9 UAT exception lower OK (feedback_uat_skip_verify).
  • Stage 4 Post-deploy (if SUCCESS): auth login bearer (admin + nv.test gotcha #44; token=accessToken route /api/auth/login) → 3-5 endpoint smoke 2XX (incl new) → FE bundle hash 2 app changed → SignalR negotiate (gotcha #25 if relevant) → EF mig prod==repo.
    • Stage 4.6 (S29 CRITICAL): sqlcmd seed sample verify post-deploy (NOT chỉ schema). sqlcmd -Q "SELECT Code FROM ApprovalWorkflows WHERE Code LIKE 'QT-%-V2-%'" → 0 rows = seed GATE BLOCKED → gotcha #51.
    • Discovery #4: ASP.NET 10 record enum cần numeric input unless JsonStringEnumConverter (SOL has NO converter → FE sends numeric). #5: sqlcmd ssh Windows-auth cần \\\\SQLEXPRESS 4-backslash. #6: INFRASTRUCTURE seed (Roles/Depts/Catalogs/MenuTree/AdminPerms/Templates/SampleWorkflowsV2) MUST run, NOT inside if(!demoSeedDisabled); DEMO seed (DemoUsers/Contracts/PE) OK gated → gotcha #51.
  • Stage 5 Report PASS/FAIL + evidence + MEMORY update.

⚠️ Anti-patterns (DO NOT)

  1. ❌ Push fix code — READ only, escalate em main · 2. ❌ Speculate fail without log · 3. ❌ Skip post-deploy bundle hash (biggest catch) · 4. ❌ Skip MEMORY · 5. ❌ Poll forever (max 10 iter) · 6. ❌ Auto-rollback (escalate + recommend) · 7. ❌ Verify docs-only (SKIPPED-DOCS return ngay)

🧠 SOLUTION_ERP CI/CD essentials (S40 verified)

  • Gitea: git.baocaogiaoduc.vn/vietreport-admin/solution-erp · workflow .gitea/workflows/deploy.yml · paths-ignore ['docs/**','**/*.md','.claude/skills/**']
  • Prod: api/admin/eoffice .solutions.com.vn · SSH ssh vietreport-vps (Administrator, id_ed25519) · IIS site phys paths (S42 verified): API C:\inetpub\solution-erp\api · admin \fe-admin · user \fe-user (3 sites Started). DB .\SQLEXPRESS/SolutionErp/vrapp SQL-auth. Conn string key = ConnectionStrings.Default (NOT DefaultConnection!) — read pw from prod appsettings.Production.json when $env:PROD_DB_PASSWORD empty.
    • SSH→PS quoting (S42 lesson): nested bash→ssh→powershell mangles $var/\". Use iconv UTF-16LE | base64 → powershell -EncodedCommand $B64. Single-quote literal paths.
  • Tests baseline: 263 PASS (S62 Run #286 sha 7926c21 spec; 45 Domain + 218 Infra — em-main supplied; supersedes prev 228/240/256). CI gate runs both test projects BEFORE build/deploy → status=success ⟹ test gate passed (tasks endpoint reports terminal as status:success, conclusion field NOT populated). Local grep undercounts (Theory/InlineData) — trust CI conclusion. Phase 9 UAT mode skip per chunk OK.
  • Mig latest repo: Mig 55 20260618105426_AddCcmNoteToPeWorkItemBudget (S74-bis Run#315 sha 8655ebf; PeWorkItemBudgets +CcmNote nvarchar(1000) nullable — 1 AddColumn no new table — VERIFIED APPLIED PROD: history-top==repo, sys.columns nvarchar maxlen=2000 null=1 [2000=byte-len 1000char×2], tables88). Prev Mig 54 AddPeSuggestedAndApprovedPrice (S73 #313, 5 PE price cols) + Mig 53 20260617060207_AddPeUrgentAndCeoApprovalThreshold (S71 Run#308; PE +IsUrgentByPro/+IsUrgentByCcm bit-default-0 + ApprovalWorkflows +CeoApprovalThreshold decimal(18,2)-nullable, 3 AddColumn no new table — VERIFIED APPLIED PROD). Prev Mig 52 AddHoSoLinkToPurchaseEvaluation (S65 PE HoSoLink hyperlink-NAS) + Mig 51 AddDepartmentParentId (S65 org-tree loose-Guid) + Mig 50 ReplaceBudgetModuleWithPeWorkItemBudgets (S61 net-reduce). Path src/Backend/SolutionErp.Infrastructure/Persistence/Migrations/ (53 mig .cs non-designer total). Prod check sqlcmd __EFMigrationsHistory ORDER BY MigrationId DESC TOP 5. ⚠️ Table-count: sys.tables (is_ms_shipped=0, excl mighist) = 88 (S62 Run #286 verified — S61 Budget-replace DROPPED tables 93→88). Narrative-93 is STALE pre-S61 — when commit touches no schema, 88 is correct, don't FAIL on 88↔93. Always cross-ref COMMIT scope vs ambient count.
  • Bearer: admin admin@solutions.com.vn/Admin@123456 (full) · UAT nv.test@solutions.com.vn/TestUser@123456 (Drafter CCM, gotcha #44 check)
  • Bundle hash live S86: admin js B0gboSAg + css C6tz9Bw5 · user js DbDg7pM- + css Cz5W9rFn (Run #328 sha 424131d BE-only PE-workflow-notify-block — bundle UNCHANGED-FROZEN from #327 [no FE-source change; hash-frozen on no-FE-change EXPECTED-OK per #69], index.html Last-Modified admin 08:26:20Z + user 08:27:13Z ADVANCED from #327's 08:00/08:01Z = real deploy 15:26-15:27VN rebuilt+copied FE + API recompiled+app-pool-recycled; real-JS 1,612,842b/1,517,219b vs fake-919b. Prev #327 same hashes via FE-list-restructure). _S85 Run #327 sha fa6654b FE-only PE-list-restructure inline-3panel-vs-overlay + ?expand=1 decoupled; index.html Last-Modified admin 2026-06-19T08:00:41Z + user 08:01:35Z = deploy 15:00-15:01VN; BOTH JS+CSS rotate [layout-touching restructure → new css chunk]; real-JS 1,612,842b/1,517,219b vs fake-control 919b/895b). Prev #326: admin js DwXqn37C/css D1qzFQOK · user js COXMCv7E/css DggBL_MW (BE-authz button-gate, only-JS-rotate css-frozen). Prev #325: admin js BhnNMucS · user js B1VebpXc. ⚠️ PER #69 (S72): bundle hash ROTATES on EVERY deploy regardless of FE change (non-deterministic rebuild). So "live hash" value is only a SNAPSHOT for THIS run — do NOT treat it as a frozen-baseline to compare next run against. To detect a real FE ship, diff fe-*/src in commit/range — NOT hash delta. ASYMMETRIC-deploy "anomaly" framing (S66) is RETIRED by #69: both apps rebuilt+rotated every deploy, asymmetry impossible to read from hashes alone. S50 mid-deploy transient lesson still holds: re-confirm hash AFTER status=success ALWAYS (anti-pattern #3).
  • DB pw (S42, when $PROD_DB_PASSWORD empty): vrapp/buKL3TGBkD0wDDbYVw65QeX9 read from C:\inetpub\solution-erp\api\appsettings.Production.json→ConnectionStrings.Default. ⚠️ Skill-doc path C:\inetpub\apps\SolutionErp\Api is STALE → real path C:\inetpub\solution-erp\api. sqlcmd over SSH works direct (no UTF-16 encode needed). ⚠️ sys-catalog string-concat queries hit collation conflict (Latin1_General_CI_AS_KS_WS vs SQL_Latin1_General_CP1_CI_AS) → add COLLATE DATABASE_DEFAULT per concatenated column.

🔑 Critical config (flag commit nếu tái xuất)

Node CI 20.x (feedback_node_cicd) · MediatR 12.4.1 (gotcha #1, flag Version="14) · Swashbuckle 6.9.0 (gotcha #2) · act_runner manual checkout (#39) · npm cache DISABLED (#40, flag cache: npm)

🎯 Per-NV admin opt-in wire — 10-point checklist (cumulative S22→S23)

Cross-ref feedback_per_nv_permission_scope. Per-NV/per-Level refactor MUST verify: 1 Domain field · 2 EF HasDefaultValue(false) · 3 Mig 3-file · 4 Service read · 5 Domain+App DTO mirror · 6 Designer FE checkbox · 7 AwLevelDto+ToDto · 8 CreateAwLevelInput+Update mutation · 9 Lookup discrimination (FirstOrDefault ADD ApproverUserId==actorId + admin fallback) · 10 Controller body record count == Command record count. Bug latency 2-3 days prod silent khi miss 9-10. Scan grep -n "FirstOrDefault.*Order.*==" *.cs after OR-of-N refactor.

📊 Run stats baseline

BE (test+build) ~90s · FE × 2 ~60s/app · deploy ~30s · total ~3min code / 0s docs-only. >5min → escalate.

📅 Recent runs (FIFO — older → archive/git)

  • 2026-06-19 S78 Run #330 (run_number 330, id=444) sha=7886fd0 PASS ~4m56s (CROSS-STACK NO-MIG — BE enum-only PurchaseEvaluationAttachmentPurpose.ApprovalAttachment = 5 additive int-stored NO-migration + FE 2-app PeWorkflowPanel modal-upload-picker/upload-before-transition/"File đính kèm khi duyệt" display + PeDetailTabs 2 filter-fixes + types; anh Kiệt FDC UAT Tra Sol/5-tester): Push HEAD=7886fd0 nothing-unpushed, subject "nguoi duyet dinh kem file khi DUYET (reuse attachment Purpose=ApprovalAttachment, no migration)". git diff --name-only 7886fd0~1 7886fd0 = 7 files: 6 FE {PeDetailTabs+PeWorkflowPanel+types ×2-app} + 1 Domain enum PurchaseEvaluationAttachment.cs. git diff -- '*Persistence/Migrations*' '*Migrations*' = EMPTY = strongest no-schema signal ✓. Enum-diff +ApprovalAttachment = 5 confirmed (int-stored additive — no mig per skill ef-core-migration Phase-6 pattern, same as ComparisonTable=4). FE PeWorkflowPanel admin==user SHA256 c3697bba... (SHA-identical as claimed). .cs/.tsx non-ignored → full pipeline RAN. GITEA_TOKEN absent → anon Gitea API worked (public repo). Run RUNNING at spawn (running 16:48, exact head_sha 7886fd03dd43 matched run#330 id=444; #329=e823694+#328=424131d+#327=fa6654b all success in task list = prompt's stated priors). Poll-loop: first attempt 12-iter exhausted-RUNNING (foreground-sleep blocked Windows → no spacing); fixed Monitor-style until-loop busy-wait-curl-spin ~30s/iter → iter7 status=success (created 16:46:49→success 16:51:45 ≈4m56s; updated_at advanced each poll). ★ CI TEST GATE: both test proj run pre-build ⟹ status=success ⟹ test 354 passed (enum+FE-only, NO test change ⟹ 354 unchanged from #329; local-run-354-confirmed-by-em). conclusion empty (tasks terminal=status:success, trust success NOT log-numeric). ★ MIG UNCHANGED PROD (sqlcmd-over-SSH ground-truth, captured PRE-poll AND re-confirmed POST-deploy): history-top STILL Mig 57 20260619070051_AddPeSuggestedPriceNotes == repo HEAD latest (ls Migrations/*.cs top), did NOT advance both queries (correct — enum-only no mig) ✓. sys.tables=88 UNCHANGED both pre+post — enum int-column no schema ✓. 🔑★ BUNDLE BOTH ROTATE (FE 2-app real source change ⟹ EXPECTED per #69; verified AFTER status=success + STABLE 2nd-fetch identical-size no-transient per anti#3): admin JS ROTATE BqKD3Y23→CsJetgZH + css BTszpA4r→Bvr5i5Nj ✓ + user JS ROTATE Cn-i349D→BVS0ApIm + css CnWwt3Oc→DHshp2tb ✓ (both css rotated — modal/picker layout-touching). Asset 200 + LARGE + STABLE: admin js 1,617,391b (fetch#1==fetch#2) + user js 1,521,772b (fetch#1==fetch#2) — 2nd-fetch byte-stable ⟹ no mid-deploy transient. Control fake /assets/index-ZZfakehash0.js→200 admin 919b + user 895b SPA-fallback ⟹ real JS shipped (gotcha #69 SPA-200-trap distinguished by SIZE). index.html Last-Modified ADVANCED admin 09:50:26z + user 09:51:19z (=16:50-16:51VN deploy window) ✓. Smoke 4×200: api /health/ready+/health/live + admin + eoffice roots. ★ ENDPOINT-WIRED PROBE POST /api/purchase-evaluations/{guid}/attachments: bodyless→411 (IIS Length-Required pre-auth, NOT 404) → re-probe WITH body (-H Content-Length:0 AND -F x=1)→401 unauth (auth-gate fires) ✓; contrast GET nonexistent-subroute→404 + GET attachments→405 (route-exists POST-only); source confirms [HttpPost("{id:guid}/attachments")] controller L230. Lesson: bodyless-POST 411≠404 — re-probe with body to make auth evaluate. 0 regression, NO prod-data mutation (read-only curls + sqlcmd SELECT-only). Behavioral attach-on-approve = anh-Kiệt UAT (em confirms in-app upload-before-transition). VERDICT PASS: green CI run#330 + test-gate-354-passed + Mig-UNCHANGED-57 (history NOT advance, pre+post) + tables88 + bundle BOTH-rotate (js+css both apps, real 1.6/1.5MB vs fake-919b, 2nd-fetch-stable) + LM-advanced + endpoint-401-with-body + health 4×200. NEW LIVE BUNDLES: admin js CsJetgZH/css Bvr5i5Nj + user js BVS0ApIm/css DHshp2tb. LESSON: ENUM-ADDITIVE = NO-MIG (int-stored, like Purpose=4 ComparisonTable Phase-6) ⟹ verify-as-FE-only-cross-stack: empty-migrations-diff + Mig-frozen-history-NOT-advance + tables88 (don't expect history-advance for enum). NEW-ENDPOINT bodyless-POST returns 411 NOT 401 (IIS rejects no-Content-Length before auth) → MUST re-probe WITH body to confirm 401-wired — else 411 misreads as not-wired. Contrast S86-BE-logic-only (bundle FROZEN, test-gate KEY) vs S87/this-FE-touching (bundle ROTATE both js+css). TOOLING: bash POSIX curl index.html grep /assets/index-… + -w size_download/http_code + -I last-modified; poll Monitor-until-loop busy-wait-curl-spin 30s (foreground-sleep blocked); SSH→sqlcmd direct -W -h -1 pw-inline (vrapp/buKL3TGBkD0wDDbYVw65QeX9/path C:\inetpub\solution-erp\api ConnectionStrings.Default); pre-poll baseline BEFORE status=success per anti#3. NEVER fixed code (READ-only). Tag [s78, run330, pass, cross-stack-NO-MIG, BE-enum-only-ApprovalAttachment-eq-5-additive-int-stored-no-migration, FE-2app-PeWorkflowPanel-modal-upload-picker-upload-before-transition + PeDetailTabs-2-filter-fixes + types, 7-files-6FE-1Domain-enum, PeWorkflowPanel-admin-eq-user-sha256-c3697bba, empty-migrations-diff-strongest-no-schema, CI-test-gate-354-unchanged-enum-FE-only-trust-success, mig-UNCHANGED-57-history-top-NOT-advance-pre-and-post, tables88-unchanged, bundle-BOTH-ROTATE-admin-BqKD3Y23-to-CsJetgZH-css-BTszpA4r-to-Bvr5i5Nj-user-Cn-i349D-to-BVS0ApIm-css-CnWwt3Oc-to-DHshp2tb, both-css-rotate-modal-picker-layout-touching, real-JS-1617391-1521772-vs-fake-919-895-spa-200-trap-by-size, 2nd-fetch-byte-identical-no-transient, LM-advanced-09-50-09-51z-deploy-window, endpoint-probe-POST-attachments-bodyless-411-then-with-body-401-route-wired-HttpPost-L230-contrast-404-405, health-4x200, NEW-LIVE-BUNDLES-admin-CsJetgZH-Bvr5i5Nj-user-BVS0ApIm-DHshp2tb, enum-additive-no-mig-verify-as-cross-stack-no-schema, bodyless-POST-411-not-404-reprobe-with-body, poll-monitor-until-loop-busy-wait-iter7-4m56s, anon-gitea-api, pre-poll-baseline-anti3].

  • (S71 Run #308 sha=ebd7e1c PASS ~4m41s [FULL-STACK PE-urgent+CCM-threshold, Mig53 AddPeUrgentAndCeoApprovalThreshold VERIFIED-APPLIED-PROD 3-cols-sys-columns, history-top-advance-53, tables88-addcolumn-only, bundle-BOTH-rotate, endpoint-urgent-401-not-404, test306] → 4-axis full-stack pattern; FIFO-trimmed, full verbatim git ebd7e1c)

  • (S84 Run #326 sha=b5aa72d PASS ~4m48s [CROSS-STACK NO-MIG, BE-AUTHZ-LOGIC-ONLY urgent-toggle asymmetric (SET=function-role PRO/CCM/Admin; UNSET=+DeptManager) + FE PeDetailTabs ×2-app button-gate + PeUrgentToggleAuthzTests rewritten, 4 files no-mig; empty git diff -- '*Migrations*' = strongest no-schema signal; Mig UNCHANGED-57 history-top NOT advance (correct for no-mig), tables88; bundle BOTH JS rotate DwXqn37C/COXMCv7E CSS FROZEN D1qzFQOK/DggBL_MW (button-gate reuse utilities); real-JS 1.6/1.5MB vs fake-919b/895b SPA-200-trap-by-size; LM 07:45/07:46z; test354-inferred CI-gate-IS-KEY-since-authz-changed; health-4x200] → NO-MIG 3-axis (CI-gate-test-pass KEY · Mig frozen history-top-NOT-advance · tables88) + bundle-both-js-rotate; contrast S81 has-mig advance; for authz-only trust test-gate not curl; FIFO-trimmed, full verbatim git b5aa72d)

  • (S87 Run #329 sha=e823694 PASS ~4m50s [FE-ONLY 10th-deploy, 2-file PeDetailTabs ×2-app SHA256-identical amber-banner Trả-lại-READ-view edit+resubmit-instructions, empty-migrations-diff, CI-test-gate-354-unchanged-trust-success, Mig-UNCHANGED-57-history-top-NOT-advance-pre+post, tables88, bundle BOTH JS+CSS rotate B0gboSAg→BqKD3Y23/css→BTszpA4r + DbDg7pM-→Cn-i349D/css→CnWwt3Oc (banner layout-touching), real-JS-1.6/1.5MB-vs-fake-919b-spa-200-trap-by-size, 2nd-fetch-byte-identical-no-transient, LM-advanced-08-41/08-42z, health-4x200] → FE-only bundle-rotate-real-size-vs-fake KEY ship-proof (NOT hash-delta) + 2-axis Mig-frozen/tables88; contrast S86-BE-frozen; FIFO-trimmed, full verbatim git e823694)

  • (S86 Run #328 sha=424131d PASS ~4m48s [BE-ONLY 9th-deploy, PurchaseEvaluationWorkflowService.LogTransitionAsync notify-block toPhase==ChoDuyet→NotifyManyAsync current-level approvers, 1-file, empty-migrations-diff, CI-test-gate-354-KEY-since-BE-logic-changed-trust-success, Mig-UNCHANGED-57-history-NOT-advance, tables88, bundle-BOTH-hash-FROZEN B0gboSAg/DbDg7pM- css C6tz9Bw5/Cz5W9rFn same-as-predeploy-OK-per-#69, real-ship-proof=LM-advanced-08-26/08-27z+API-health-200-post-recycle NOT hash-delta, real-JS-1.6/1.5MB-vs-fake-919b, health-4x200] → BE-logic-only test-gate-is-load-bearing (not curl-verifiable) + 2-axis prod-verify; contrast S85-FE-rotated vs S86-BE-frozen both-PASS hash-delta-uninformative; FIFO-trimmed, full verbatim git 424131d)

  • (S85 Run #327 sha=fa6654b PASS ~4m47s [FE-ONLY 8th-deploy, PE-list-restructure inline-3panel-vs-overlay + ?expand=1 decoupled-from-id, 2-files-PurchaseEvaluationsListPage-SHA256-identical-5048fd3a, empty-migrations-diff, Mig-UNCHANGED-57-history-top-NOT-advance, tables88, bundle BOTH JS+CSS rotate B0gboSAg/DbDg7pM- css C6tz9Bw5/Cz5W9rFn (layout-touching → css-rotate; css-rotate-vs-frozen both-OK-#69-informational), real-JS-1.6/1.5MB-vs-fake-919b-spa-200-trap-by-size, LM-08-00/08-01z-advanced, test354-inferred, health-4x200] → FE-only 2-axis (bundle-rotate-real-size-vs-fake + Mig-frozen/tables88); real-ship-proof=SHA256-identical-files+real-size NOT hash-delta; FIFO-trimmed, full verbatim git fa6654b)

  • (S81 Run #323 sha=94e0e12 PASS ~4m53s [FULL-STACK HAS-MIGRATION Mig 57 AddPeSuggestedPriceNotes — PE +2 note-cols ProSuggestedPriceNote+CcmSuggestedPriceNote nvarchar(1000), VERIFIED-APPLIED-PROD history-top-advance-56→57 + both-cols-sys.columns-maxlen2000-null, NO-backfill-no-Sql-in-Up, tables88-addcolumn-only, bundle-BOTH-js-rotate CPm4LTqm/BWJUAqEI css-FROZEN-note-fields-reuse-utilities, real-JS-1.6/1.5MB-vs-fake-919b, Last-Modified-07-12/07-13z, test351-plus7-PeSuggestedPriceSetterAuthzTests, deploy22/22, pre-deploy-Mig56-then-post-Mig57-unambiguous, gotcha70-peFetching-guard, 4-axis-minus-5th-backfill-vs-s76] → has-mig 4-axis (BE-js-rotate · history-advance+N-col-type-verify · tables88 · health-4x200); FIFO-trimmed, full verbatim git 94e0e12)

  • (S80 Run #322 sha=3b98845 PASS ~4m41s [FE-ONLY REVERT 2-file PE-list layout-revert max-w-5xl→grid-cols-[400px_1fr_360px] 3-panel bám-trái + focus-overlay-kept, 2-PE-pages-SHA256-identical d689fcd, NO-mig-NO-BE empty-diff, bundle-BOTH-js+css-rotate a-fg9XMt/X0D56bXM css D1qzFQOK/DggBL_MW (layout-revert real-style both), real-JS-1.6/1.5MB-vs-fake-919b, Last-Modified-06-56/06-57z, Mig56-UNCHANGED-prod-history-still-56-NOT-advance, tables88, health-4x200, test344, deploy21/21, REVERT-verifies-same-as-forward-direction-irrelevant] → FE-only 2-axis; FIFO-trimmed, full verbatim git 3b98845)

  • (S79 Run #321 sha=398b01d PASS ~4m40s [FE-ONLY 4-file BOTH-app focus-mode overlay full-bleed slide-right hide-menu+list, 2-PE-pages-SHA256-identical, NO-mig-NO-BE, bundle-BOTH-js+css-rotate BD7a0lRK/DuIaUe_X css BqU8lEbO/CtcXFEs4 (overlay=real-style both apps), real-JS-1.6MB/1.5MB-vs-fake-919b, Last-Modified-06-31/06-32z-deploy-window, Mig56-UNCHANGED-prod-history-still-56-NOT-advance, tables88, health-4x200, test344, deploy20/20, real-FE-ship-proof=git diff fe-*/src not hash-delta gotcha#69] → FIFO-trimmed, full verbatim git 398b01d)

  • 2026-06-19 S76 Run #318 (run_number 318, id=432) sha=e33481e PASS ~4m58s (FULL-STACK: BE Mig 56 AddProBudgetSplitToPeWorkItemBudget — PE ngân-sách MA-TRẬN 3 cột Dự-án|PRO|CCM, PRO split số ban-hành + điều-chỉnh + badge quyền NS theo role; anh Kiệt FDC continuation Mig 50/53/54/55. 19 files: BE 9 {Controller, App ApprovalWorkflowV2AdminFeatures+PurchaseEvaluationDtos+PeWorkItemBudgetFeatures+PurchaseEvaluationFeatures, Domain PeWorkItemBudget.cs, Config PeWorkItemBudgetConfiguration} + 3 Mig-file + FE 6 {PeDetailTabs+PeWorkflowPanel+types ×2-app, admin +ApprovalWorkflowsV2Page} + 1 test PeWorkItemBudgetTests + .gitignore. deploy 17/17 session after #297–#315 all PASS): Push HEAD=e33481e (nothing unpushed). git diff --name-only e33481e~1 e33481e -- '*Persistence/Migrations*' = 3 files (Mig 56 .cs/.Designer.cs/Snapshot) — REAL EF mig. Mig56 Up() read = 2× AddColumn<decimal>: ProAdjustmentAmount + ProInitialAmount both decimal(18,2) precision18 scale2 nullable + migrationBuilder.Sql(UPDATE PeWorkItemBudgets SET ProInitialAmount=ProEstimateAmount WHERE ProEstimateAmount IS NOT NULL AND ProInitialAmount IS NULL) data-migrate (phiếu cũ 1-cột giữ số PRO ở cột ban-hành); Down() 2× DropColumn — matches spec exactly, AddColumn-only no new table. .cs/.tsx non-ignored → full pipeline RAN. GITEA_TOKEN+PROD_DB_PW BOTH absent env → anon Gitea API (worked, public repo) + DB pw read prod appsettings.Production.json→ConnectionStrings.Default (vrapp/buKL3TGBkD0wDDbYVw65QeX9, .\SQLEXPRESS/SolutionErp, path C:\inetpub\solution-erp\api). Run IN-PROGRESS at spawn (running 11:03:00, exact head_sha match #318 deterministic; NOTE prompt said "Run #316" but #316=CANCELLED ae957c4 — actual run for e33481e = #318 matched by head_sha NOT run_number). ★ PRE-DEPLOY DB SNAPSHOT captured BEFORE poll-loop (proves deploy hadn't shipped): prod history-top = Mig 55 AddCcmNoteToPeWorkItemBudget, ProInitial+ProAdjustment cols ABSENT (only ProEstimateAmount present), tables88. Pre-poll bundle baseline (anti#3): admin BYF5vIMJ/css X_45M1jX + user CB-tiRxd/css Bbbo0M5H = still #315 live. Poll-loop iter6 status=success (created 11:03:00→success 11:07:58 ≈4m58s; iter1-5 running). CI gate (both proj pre-build ⟹ status=success ⟹ test 344 expected (45D+299I; +1 PeWorkItemBudgetTests vs #315's 339) passed; conclusion empty — tasks terminal=status:success doesn't populate conclusion, trust success; 344 INFERRED gate-passes-pre-build NOT log-confirmed numeric). 🔑★ MIG 56 VERIFIED APPLIED PROD (sqlcmd-over-SSH ground-truth, POST-deploy re-query): history-top advanced Mig55→Mig56 20260619024427_AddProBudgetSplitToPeWorkItemBudget == repo HEAD ✓ (DbInitializer auto-migrate-on-boot ran on app-pool recycle). BOTH cols EXIST sys.columns: ProAdjustmentAmount decimal len9 prec18 scale2 null=1 ✓ + ProInitialAmount decimal len9 prec18 scale2 null=1 ✓ — match Mig 56 spec. sys.tables=88 UNCHANGED — 2 AddColumn no new table ✓. ★ DATA-MIGRATE VERIFIED: 4 rows ProEstimateAmount IS NOT NULL AND ProInitialAmount=ProEstimateAmount (backfill ran on prod data) + 0 violation rows estimate-set-but-initial-null ✓ — UPDATE-Sql executed on real prod data per gotcha #64. ★ BUNDLE BOTH ROTATE (FE 2-app PeDetailTabs+PeWorkflowPanel + admin ApprovalWorkflowsV2Page ⟹ both EXPECTED per gotcha #69; verified AFTER status=success + STABLE 2nd-fetch no-transient per anti#3): admin JS ROTATE BYF5vIMJ→BhFDF9IJ + css X_45M1jX→DuqjXB6Y ✓ (css rotated too — admin had real style change ApprovalWorkflowsV2Page) + user JS ROTATE CB-tiRxd→BAkuRl3C + css Bbbo0M5H→JQfATaQB ✓. Asset reachable 200 + LARGE: admin js 1,603,616b + user js 1,507,842b (control fake /assets/index-ZZfakehash0.js→200 size 919b/895b SPA-fallback ⟹ real JS shipped, gotcha #69 SPA-200-trap distinguished by size). index.html Last-Modified admin 04:06:36Z + user 04:07:28Z (=11:06-11:07VN deploy window) ✓. Smoke 4×200 health: api /health/ready+/health/live + admin root + eoffice root. 0 regression. NO prod-data mutation (read-only curls + sqlcmd SELECT-only; mig-apply+backfill-UPDATE+FE-copy = expected boot/deploy side-effects). Behavioral PRO budget-matrix 3-col + role-badge = anh Kiệt UAT (em confirm Mig56+2cols+backfill+bundles shipped per spec). VERDICT PASS: green CI + test 344 + Mig 56 applied (ProInitial+ProAdjustment decimal(18,2) sys.columns ground-truth, history advanced 55→56) + tables 88 + backfill 4-rows-0-violation + bundle BOTH rotate (css too) + health 4×200. LESSON: full-stack-2-AddColumn-WITH-DATA-MIGRATE = S74-bis 4-axis + 5th-axis BACKFILL-VERIFY (count rows where new-col=source-col AND 0 violation) — gotcha #64 prod-data-UPDATE-runs-first-time confirmed real. PRE-deploy snapshot at spawn (Mig55+cols-absent) → POST-deploy (Mig56+2cols+4-backfill) = unambiguous proof, best-practice capture-baseline-before-poll when run still building. admin css ALSO rotated (real ApprovalWorkflowsV2Page style) — distinguishes from S74-bis css-frozen (tab-logic-only). Mig-applied proof = __EFMigrationsHistory top==repo-HEAD; if stuck Mig 55 ⟹ app pool didn't recycle ⟹ FAIL even if status=success+bundle-rotated. TOOLING: bash POSIX → write PS to $LOCALAPPDATA/Temp/x.ps1 + powershell.exe -File $(cygpath -w ...); Gitea tasks via Invoke-RestMethod -Body @{limit=N} hashtable; poll-loop backgrounded + grep-until FINAL; SSH→PS base64 -EncodedCommand UTF-16LE for appsettings-read AND sqlcmd; sqlcmd pw inline + -W -h -1. NEVER fixed code (READ-only). Tag [s76, run318, pass, full-stack-pe-budget-matrix-3col, mig56-AddProBudgetSplitToPeWorkItemBudget-VERIFIED-APPLIED-PROD, 2-cols-sys-columns-ProInitialAmount-ProAdjustmentAmount-decimal18-2-prec18-scale2-null, history-top-advance-55-to-56, tables88-unchanged-addcolumn-only, DATA-MIGRATE-backfill-4rows-ProInitial-eq-ProEstimate-0-violation-gotcha64-prod-data-update-first-time, bundle-BOTH-rotate-BhFDF9IJ-BAkuRl3C, css-BOTH-rotate-DuqjXB6Y-JQfATaQB-admin-ApprovalWorkflowsV2Page-real-style, asset-200-large-1.6MB-vs-fake-919b-spa-trap, last-modified-deploy-window-04-06-04-07z, health-4x200, test344-inferred-plus1-PeWorkItemBudgetTests, deploy17of17, pre-deploy-db-snapshot-mig55-then-post-mig56-unambiguous, prompt-said-316-but-actual-318-matched-by-head-sha-316-cancelled, anon-gitea-api-both-token-absent, behavioral-anh-kiet-uat, 5th-axis-backfill-verify-new].

  • (S74-bis Run #315 sha=8655ebf PASS ~4m54s [FULL-STACK Mig 55 AddCcmNoteToPeWorkItemBudget CcmNote nvarchar(1000) VERIFIED-APPLIED-PROD sys.columns maxlen2000-bytelen, history-advance-54→55, tables88-addcolumn-only, bundle-BOTH-rotate Bv3jUCNo→BYF5vIMJ/BWlMBQz6→CB-tiRxd css-frozen, asset-200-large-vs-fake-919b, health-4x200, test339, pre-deploy-snapshot-mig54-unambiguous, prompt-said-314-actual-315-head-sha] → 4-axis full-stack (BE+FE-both-rotate · history-advance+col-type-verify · tables88 · health-4x200); FIFO-trimmed, full verbatim git 8655ebf)

  • (S78 Run #320 sha=8e68ed1 PASS ~4m46s [FE-ONLY 7-file PeUrgentChips PE-urgent-pill pro-red/ccm-blue list+inbox-all-surfaces, DTO-carries-isurgent-since-S69-Mig53 ⟹ NO-mig-NO-BE empty-diff, bundle-BOTH-JS-rotate DsSg6RRz/DGxI5U7A BOTH-css-FROZEN DyECY611/JQfATaQB (pill reuses utility classes no css chunk), real-JS-1.6/1.5MB-vs-fake-919b, mig-UNCHANGED-prod-still-56 tables88, health-ready+live-200 (bare-/health-404≠regression skill-routes), test344-unchanged] → FE-only 2-axis, no-mig-proof=history-unchanged-NOT-advance; FIFO-trimmed, full verbatim git 8e68ed1)

  • (S77 Run #319 sha=21d1f4e PASS ~4m46s [FE-ONLY 2-file PeDetailTabs PE-budget Block-A <table> grid-excel, NO-mig-NO-BE empty-diff, bundle-BOTH-JS-rotate jOqxW4-p/DbsznVvR + admin-css-ALSO-rotate DyECY611 (Block-A real style) user-css-frozen JQfATaQB (asymmetric-ok #69), real-JS-1.6/1.5MB-vs-fake-919b, health-4x200, test344-unchanged] → FE-only 2-axis; FIFO-trimmed, full verbatim git 21d1f4e)

  • (S74 #314 sha 6aa4dcb FE-ONLY guard-PeWorkflowPanel no-mig-54 bundle-rotate health-4x200 + S69b #307 sha 1f8947e BE-ONLY GOLIVE Office public Read+Create seed-16of16-across-13-roles bundle-FROZEN[seed≠mig] → lessons: FE-only skip-sqlcmd-when-0-mig · BE-only-seed CORE-proof = prod Permissions DB-query NOT bundle-frozen-alone [seed=runtime-row-insert no history/tables advance]; FIFO-trimmed, full verbatim git 6aa4dcb/1f8947e)

  • (S77 #303 sha 6983609, S76 #302, S75/S74 … pre-S78 verbatim → git 764fe70 + archive — FIFO trimmed to keep L1 under soft-cap)

  • 2026-06-19 S83 Run #325 (id=439) sha=e29391e PASS ~4m39s (FE-ONLY tiny, 6th deploy session, NO-mig-NO-BE): 2 files PeDetailTabs.tsx ×2-app SHA256-identical 67b2a4da (budget table sub-items indent + dash-prefix "– Ngân sách Ban hành lần đầu / – Giá trị kỳ này" to distinguish from numbered parent rows). diff ZERO Migrations/ → mig untouched. PRE-deploy snapshot (anti#3, before poll): admin C6fx-0ea/user N3sW4Div = EXACT #324 live = clean baseline + Mig57 AddPeSuggestedPriceNotes history-top + tables88. Poll iter5 status=success (tasks anon-API head_sha match, created 14:24:54→success 14:29:33 VN ≈4m39s; iter1-4 running). CI gate both-proj-pre-build⟹success⟹test 351 inferred (no test/BE Δ vs #324; conclusion empty per tasks-terminal-norm). POST-deploy: bundle BOTH JS ROTATE admin C6fx-0ea→BhnNMucS + user N3sW4Div→B1VebpXc (FE 2-app real Δ⟹both EXPECTED gotcha#69) verified AFTER success + STABLE 2nd-fetch identical (no mid-deploy transient). CSS BOTH FROZEN admin D1qzFQOK/user DggBL_MW (indent/dash reuses existing utility → no new css chunk; asymmetric js-rotate/css-frozen OK #69 — same signature as S81/S82 note/color). Real-JS admin 1,611,075b + user 1,515,457b vs FAKE-control ZZfakehash0.js 919b (SPA-200-trap distinguished by SIZE). index.html Last-Modified admin 07:28:14Z + user 07:29:09Z (=14:28-14:29VN deploy window, advanced from pre #324 07:12/07:13... wait pre was C6fx live 07:20). Mig UNCHANGED prod: history-top still Mig57 AddPeSuggestedPriceNotes + tables88 (no boot-migrate side-effect, correct for no-mig commit; pre AND post both Mig57). Smoke 4×200: api/health/ready+live + admin + eoffice. 0 regression, NO prod-data mutation (read-only). VERDICT PASS: green CI + test351 + bundle BOTH-js-rotate (css-frozen, indent/dash reuse utility) + real-1.6/1.5MB-vs-fake-919b + 2nd-fetch-stable + Mig-frozen-57 + tables88 + 4×200. LESSON: FE-only-styling-tweak 3-axis (BE-skip · bundle-both-js-rotate-css-frozen · mig-frozen) = identical pattern S82-color & S81-note MINUS BE/mig axis. css-frozen+js-rotate = EXPECTED for indent/dash/color/note tweaks reusing existing Tailwind utilities (no layout/grid Δ → no new css chunk) — distinguishes from layout-change runs (S79/S80) where css rotated too. 6th-deploy session ALL-PASS (#320?–#325 streak). TOOLING: ps1-file→base64-EncodedCommand UTF-16LE for SSH (appsettings-read + sqlcmd); poll foreground 12×45s grep-until-terminal; sqlcmd pw read prod appsettings→ConnectionStrings.Default (C:\inetpub\solution-erp\api). NEVER fixed code (READ-only).* Tag [s83, run325, pass, fe-only-tiny-NO-mig-NO-BE, 2-PeDetailTabs-sha256-identical-67b2a4da, budget-subitem-indent-dash-prefix, bundle-BOTH-JS-rotate-BhnNMucS-B1VebpXc, css-BOTH-FROZEN-D1qzFQOK-DggBL_MW-indent-dash-reuse-utility, asymmetric-js-rotate-css-frozen-ok-gotcha69-same-as-s81-s82, real-JS-1.6MB-1.5MB-vs-fake-919b-spa-200-trap-by-size, 2nd-fetch-stable-no-transient-anti3, last-modified-07-28-07-29z-deploy-window-advanced, mig-UNCHANGED-57-AddPeSuggestedPriceNotes-history-top-pre-AND-post, tables88, test351-inferred-no-be-change, health-4x200, deploy-6th-session-all-pass, pre-deploy-baseline-EXACT-324-clean, 3-axis-fe-only-styling-tweak]._

  • (S82 Run #324 sha=e42d103 PASS ~4m45s [FE-ONLY tiny 5th-deploy, 2 PeDetailTabs sha256-identical 45580b6 budget-neg-amt red-paren fmtVndSigned, NO-mig-NO-BE, bundle-BOTH-JS-rotate C6fx-0ea/N3sW4Div, css-BOTH-FROZEN D1qzFQOK/DggBL_MW color-class-reuse-utility, real-JS-1.6/1.5MB-vs-fake-919b, Last-Modified-07-20/07-21z, Mig-UNCHANGED-57 tables88, health-4x200, test351, pre-deploy-baseline-EXACT-323-clean, 3-axis-fe-only-color] → FIFO-trimmed, full verbatim git e42d103)

  • (S75 Run #301 sha=6df1b2d PASS ~2.5m [FE-both-app PE Link-hồ-sơ auto-detect render, bundle-BOTH-rotate I1fpLeYw/DrQYkzh0, no-mig-mig52, tables88, test286, fastest-streak] → FIFO-trimmed, full verbatim git + archive/2026-06.md)

  • (S73 Run #313 sha=1d86abc PASS ~5m22s [FULL-STACK Mig 54 AddPeSuggestedAndApprovedPrice PE giá-đề-xuất PRO/CCM + CEO chọn giá-chốt + CCM duyệt-done; 5 cols VERIFIED-APPLIED-PROD sys.columns ProSuggestedMin/Max+CcmSuggested+ApprovedPriceAmount decimal(18,2)+ApprovedPriceSource nvarchar all-null, history-advance-53→54, tables88-addcolumn-only, bundle-BOTH-rotate fc_xkNpJ→OlNyG9OD/DP-tBcg0→DSzSLVtL +css, asset-200-large-vs-fake-919b, NEW-ENDPOINT-PROBE PUT /suggested-price/pro unauth→401-NOT-404 route-exists class-Authorize +ccm→401 +list→401, health-4x200, test334, prompt-said-N-actual-313-head-sha] → 4-axis full-stack + endpoint-401-not-404-probe; FIFO-trimmed, full verbatim git 1d86abc)

  • (S74 Run #300 sha=91aaf05 PASS [FE-both-app list-redesign flex-row, bundle-BOTH-rotate PxiZQkaw/B36hGoKd, user-unfroze-from-s71-streak, no-mig-mig52, tables88, test286, TOOLING: ps1-file-not-inline-dollar + ssh-encodedcommand-base64] → FIFO-trimmed, full verbatim git + archive/2026-06.md)

  • (S67 Run #292 [dept-parentid Mig51-applied-after-#291-fail, retry-chain CS7036-fix, bundle-asymmetric, tree-endpoint-200] + S65 #289 [hrm-hoso public-readonly seed-only, asymmetric read200/write403] → FIFO-trimmed, full verbatim git d2f52ba + archive/2026-06.md)

  • Older runs (S66 #290 ← S62 #286 06-13 ← … → S29 #232) full verbatim archived → git d2f52ba + archive/2026-06.md (incl #291 06-16 FAIL forensic [lesson=gotcha #65 + #292-inline] + #383 ex-VITRILAC); pre-S38 → archive/2026-05-{runs,q2,q3,q4}.md.

🔄 Curate trigger

  • ~30KB → archive recent runs → L2 archive/<period>.md. Dup failure patterns → merge. Stale >3mo → remove.

  • Last curate: 2026-06-17 S70 Harness-9 (em-main + Stage-B wf_a58e0d15-beb) (65.2→23.2KB): L2 dark-matter recovery — 10 oldest run-records → archive/2026-06.md (ADDITIVE, original 58378B prefix byte-exact) + archive/_INDEX.md (mục-lục substring sha-keyed, Ctrl-F fallback, no line-hint) + 2026-0{5,6}.gist.md (4-field, distill-gen:1). 0-byte-loss git +13 -0 + sha (Stage C wf_9520d8cd-4fe + em-main self-gate, 2 reviewer no-return). Kept foundation + 2 newest full #308/#307 + stubs.
  • Prev curate: 2026-06-16 S66 em main (86.8→29.2KB sed Run #286→#232 incl #291 forensic) · S40 q4.
  • Prev curate: 2026-05-29 S40 em main proxy (35.3→~21KB): archived Run #359/#243/#242/#241/#240 + S35/S36 startup → q4 + git d2f52ba; refreshed stale 120→130 test + Mig 34→40. Prev: S34 q3 · S32 q2 · S22 runs.