f0874dd2c9
All checks were successful
Deploy SOLUTION_ERP / build-deploy (push) Successful in 5m26s
Adap 6 broadcast (hash MATCH cả 6 indep-recompute) qua 2-workflow Harness-9: - Harness-20 fix-all-structure -> engine §L + dogfood model-note 4-tầng-sweep (incl PERSONA) + 220K→380K neo-supersede + 2-đầu-catch (review bắt 2 DẪN-XUẤT stale-trailer sweep-sót → fixed) - F4 crystallized-backfill = NEW scripts/crystallized-backfill.ps1 (measure-then-fill DRY NO-API; default target=0=OFF owner-authority; calibrated-range; precondition; file-floor=UPPER-BOUND honest) + budget.json config + budget-table wire - F1/F3 already-met (0 hardcoded-violation persona-clean; BƯỚC 0.6 bắt S103=Opus real-metadata) - session-cmd budget-table (Up1) + run-loop 2-tầng (Up2 largely-already-met §L.b(c)) - H16 no-key Applied% = NEW scripts/applied-eval-nokey.ps1 (adversarial-gate fail-closed 3-ca + fault-inject FLIP; mechanism-built numbers-DEFERRED) - memory-hygiene A/B already-met (archive-gate DRY-RUN + governance-detectors 4-layer 1:1); C n/a JS-sandbox no-CLI-launcher - consolidation reference-ack (crystallization≠backfill) engine §L+§M · 6 adap-report + 1 adap-request (#53-garble reasoning-lane mitigation) · send-email fc1e22ac selftest-MATCH · check-email 3 pulled+verified+STAGE-2. 2-workflow: WF1 implement wf_55ef6778-a0d (3 lane schema-CLEAN, selftest exit-0 ×2 + fault-inject teeth) + WF2 review wf_3c5c37cb-aa2 fable-clone (Lane-C PASS returned + Lane-B recover-from-memory PASS; 0-BLOCKING 2-CONCERN-addressed). #53 garble ×3 → em-main recover; investigator AS-10 garble-curation REVERTED (git-backstop). Runtime S103 = Opus (Fable outage lại, per-session flip; H5-fallback revert-free). State GIỮ NGUYÊN: Mig 61 · 89 bảng · 440 test · gotcha 79 · menu 54. 0 prod-code. Co-Authored-By: Claude Opus 4.8 (1M) <noreply@anthropic.com>
255 lines
13 KiB
PowerShell
255 lines
13 KiB
PowerShell
# crystallized-backfill.ps1 - Harness-... crystallized-backfill = "measure-then-fill"
|
|
# Adopt: AI_INFRA broadcast F4 + session-cmd-budget-display Upgrade-1 (SE adapt 07-06).
|
|
#
|
|
# WHAT THIS IS
|
|
# A PLANNER (DRY, NO-API) that answers ONE question for the LEAD hot-feed:
|
|
# "After the always-loaded hot-load sources are in Tier-1, how much REAL-TOKEN
|
|
# headroom is left under the live cap, and how many tokens of crystallized
|
|
# (value-gated) backfill may we pull in WITHOUT over-stuffing?"
|
|
# It MEASURES (byte-count the live hot-load files) then computes an EXPECTED
|
|
# backfill under a hard precondition. It does NOT pour anything (no memory write,
|
|
# no context change) - exactly like memory-archive-gate.ps1 it is a DRY planner.
|
|
#
|
|
# NON-NEGOTIABLES
|
|
# (1) OWNER-READ-ONLY : every budget number (lead cap + backfill target) is the
|
|
# OWNER's (chu-du-an) to set in memory-budget.json. This
|
|
# script ONLY reads them LIVE. It NEVER computes, writes,
|
|
# auto-tunes, or persists a budget number. (role_boundary_note)
|
|
# (2) DEFAULT 0 = OFF : if the optional crystallized_backfill block is ABSENT (or
|
|
# target<=0) the expected backfill is 0 (TAT / turned-off).
|
|
# Backfill only "opens" when owner sets target>0 AND measured
|
|
# headroom>0 (precondition). Standalone default = OFF.
|
|
# (3) NOT-FOR-SUB : this measures the LEAD hot-feed only. Sub packs are
|
|
# hand-distilled by em-main per task; blind-pouring backfill
|
|
# into a sub dilutes relevance. If invoked -Tier sub => WARN.
|
|
# (4) CALIBRATED RANGE: token headroom is a RANGE [bytes/4 .. bytes/3.0], NOT the
|
|
# false-precise byte/4. VN-diacritic hot-memory ~3.0-3.5
|
|
# byte/tok, so bytes/3.0 = worst-case (fewest tokens hidden
|
|
# behind the bytes). measured_headroom uses the worst case.
|
|
# (5) NO-API : Select-String + byte/file measure ONLY. NEVER calls a model.
|
|
# (6) PS 5.1 / ASCII : script BODY is ASCII-only (gotcha #30); any glyph via
|
|
# [char]0xXXXX code-point; target files read -Encoding UTF8.
|
|
# (7) exit 0 ALWAYS : measure-and-report, NOT a build-gate.
|
|
#
|
|
# Usage:
|
|
# powershell.exe -ExecutionPolicy Bypass -File scripts\crystallized-backfill.ps1
|
|
# powershell.exe -ExecutionPolicy Bypass -File scripts\crystallized-backfill.ps1 -Json
|
|
#
|
|
# See [TAILOR] block at the bottom for the honest simplifications vs the maximal spec.
|
|
|
|
param(
|
|
[string]$RepoRoot = "$PSScriptRoot\..",
|
|
[switch]$Json = $false,
|
|
[string]$Tier = "lead" # NOT-for-sub guard: anything but 'lead' => WARN (still measures lead)
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
|
|
# ---- glyphs as code-points (ASCII body, gotcha #30) -----------------------
|
|
$WARN = [char]0x26A0 # warning sign
|
|
$ARROW = [char]0x2192 # rightwards arrow (source-order display)
|
|
|
|
# ---- resolve paths --------------------------------------------------------
|
|
$RepoRoot = (Resolve-Path $RepoRoot).Path
|
|
$memRoot = Join-Path $RepoRoot '.claude\agent-memory'
|
|
$budgetPath = Join-Path $memRoot 'memory-budget.json'
|
|
|
|
if (-not (Test-Path $budgetPath)) { Write-Error "memory-budget.json not found: $budgetPath"; exit 1 }
|
|
|
|
# ---- load LIVE budget -----------------------------------------------------
|
|
$budget = Get-Content $budgetPath -Raw | ConvertFrom-Json
|
|
|
|
# (a) lead cap : LIVE-read, never hardcode (owner authority).
|
|
$tg = $budget.token_governor
|
|
if ($null -eq $tg -or $null -eq $tg.tier1_hotfeed_tokens) {
|
|
Write-Error "memory-budget.json missing token_governor.tier1_hotfeed_tokens (lead cap source)"; exit 1
|
|
}
|
|
$leadCap = [int]$tg.tier1_hotfeed_tokens.lead_tokens # LIVE. owner-set. read-only here.
|
|
|
|
# (b) crystallized_backfill : OPTIONAL block. Absent => SAFE DEFAULT (target=0=OFF,
|
|
# default hotload list). em-main adds the config block later; script runs
|
|
# standalone with defaults now.
|
|
$cb = $budget.crystallized_backfill
|
|
$defaultHotload = @(
|
|
'docs/STATUS.md',
|
|
'docs/HANDOFF.md',
|
|
'.claude/governance/ACTIVE-MARKS.md',
|
|
'docs/changelog/migration-todos.md'
|
|
)
|
|
if ($null -eq $cb) {
|
|
$target = 0 # DEFAULT OFF (TAT)
|
|
$hotloadSources = $defaultHotload
|
|
$cfgState = "ABSENT (safe default: target=0=OFF, default hotload list)"
|
|
} else {
|
|
# target : owner number. Read LIVE; default 0 if the field is missing.
|
|
if ($null -ne $cb.target) { $target = [int]$cb.target } else { $target = 0 }
|
|
# hotload_sources : owner list; fall back to default if missing/empty.
|
|
$hotloadSources = @()
|
|
if ($cb.hotload_sources) { foreach ($h in $cb.hotload_sources) { if ($h) { $hotloadSources += [string]$h } } }
|
|
if ($hotloadSources.Count -eq 0) { $hotloadSources = $defaultHotload }
|
|
$cfgState = "present (target + hotload_sources read LIVE)"
|
|
}
|
|
|
|
# ---- NOT-FOR-SUB guard (invariant 3) --------------------------------------
|
|
$subWarn = $false
|
|
if ($Tier -ne 'lead') {
|
|
$subWarn = $true
|
|
}
|
|
|
|
# ==========================================================================
|
|
# MEASURE : byte-count each LIVE hot-load source (UTF8), sum, derive RANGE.
|
|
# ==========================================================================
|
|
$srcRows = @()
|
|
$totalBytes = 0
|
|
$utf8 = New-Object System.Text.UTF8Encoding($false)
|
|
foreach ($rel in $hotloadSources) {
|
|
$full = Join-Path $RepoRoot ($rel -replace '/', '\')
|
|
if (Test-Path $full) {
|
|
# measure via UTF8 decode-length-in-bytes = on-disk byte size (explicit UTF8, gotcha #30)
|
|
$b = ([System.IO.File]::ReadAllBytes($full)).Length
|
|
$totalBytes += $b
|
|
$srcRows += [pscustomobject]@{ src = $rel; bytes = $b; state = 'ok' }
|
|
} else {
|
|
$srcRows += [pscustomobject]@{ src = $rel; bytes = 0; state = 'MISSING' }
|
|
}
|
|
}
|
|
|
|
# calibrated RANGE (invariant 4): tok_low = optimistic (bytes/4), tok_high = worst (bytes/3.0)
|
|
$tokLow = [int][math]::Round($totalBytes / 4.0)
|
|
$tokHigh = [int][math]::Round($totalBytes / 3.0)
|
|
|
|
# measured_headroom = cap - worst-case-load (can be NEGATIVE = hot-feed near full)
|
|
$measuredHeadroom = $leadCap - $tokHigh
|
|
|
|
# ---- expected_backfill (invariant 2 + precondition) -----------------------
|
|
# precondition: only OPEN backfill when headroom>0 AND target>0. Else 0.
|
|
if ($target -gt 0 -and $measuredHeadroom -gt 0) {
|
|
$expectedBackfill = [math]::Min($target, $measuredHeadroom)
|
|
$preconditionMet = $true
|
|
} else {
|
|
$expectedBackfill = 0
|
|
$preconditionMet = $false
|
|
}
|
|
|
|
# reason string (why backfill is what it is)
|
|
if ($target -le 0) {
|
|
$backReason = "target=0 => OFF (TAT, default until owner sets target>0)"
|
|
} elseif ($measuredHeadroom -le 0) {
|
|
$backReason = "measured_headroom<=0 => NO room (hot-feed at/over worst-case cap)"
|
|
} elseif ($expectedBackfill -eq $target) {
|
|
$backReason = "min(target,headroom)=target => full target fits under headroom"
|
|
} else {
|
|
$backReason = "min(target,headroom)=headroom => headroom binds (target exceeds room)"
|
|
}
|
|
|
|
# ==========================================================================
|
|
# -Json : one machine-readable line, then exit. (numbers all live-derived)
|
|
# ==========================================================================
|
|
if ($Json) {
|
|
$out = [ordered]@{
|
|
cap = $leadCap
|
|
bytes = $totalBytes
|
|
tok_low = $tokLow
|
|
tok_high = $tokHigh
|
|
headroom = $measuredHeadroom
|
|
target = $target
|
|
backfill = $expectedBackfill
|
|
}
|
|
Write-Output (($out | ConvertTo-Json -Compress))
|
|
exit 0
|
|
}
|
|
|
|
# ==========================================================================
|
|
# HUMAN REPORT
|
|
# ==========================================================================
|
|
$dash = [string]([char]45) # '-' as a value, never a bare '--' run (PS 5.1 decrement-op trap)
|
|
|
|
Write-Output "============================================================"
|
|
Write-Output " crystallized-backfill.ps1 - measure-then-fill (DRY planner)"
|
|
Write-Output " tier : $Tier"
|
|
Write-Output " budget.json : $budgetPath"
|
|
Write-Output " cb config : $cfgState"
|
|
Write-Output "============================================================"
|
|
|
|
if ($subWarn) {
|
|
Write-Output ""
|
|
Write-Output (" {0} NOT-FOR-SUB: -Tier '$Tier' requested. This planner measures the LEAD" -f $WARN)
|
|
Write-Output " hot-feed ONLY. Sub packs are hand-distilled by em-main per task; blind"
|
|
Write-Output " backfill into a sub dilutes relevance. Reporting LEAD numbers regardless."
|
|
}
|
|
|
|
# ---- source measure table -------------------------------------------------
|
|
Write-Output ""
|
|
Write-Output "### hot-load sources (LIVE byte-measure, UTF8)"
|
|
Write-Output ""
|
|
Write-Output ("{0,-44} {1,10} {2,8}" -f 'source', 'bytes', 'state')
|
|
Write-Output ("{0,-44} {1,10} {2,8}" -f ($dash*44), ($dash*10), ($dash*8))
|
|
foreach ($r in $srcRows) {
|
|
Write-Output ("{0,-44} {1,10} {2,8}" -f $r.src, $r.bytes, $r.state)
|
|
}
|
|
Write-Output ("{0,-44} {1,10} {2,8}" -f 'TOTAL', $totalBytes, '')
|
|
|
|
# ---- the budget number table (Upgrade-1) ----------------------------------
|
|
# ALL numbers read-live; ZERO hardcoded budget values.
|
|
Write-Output ""
|
|
Write-Output "### budget numbers (Upgrade-1) - ALL live-read, 0 hardcoded"
|
|
Write-Output ""
|
|
$hr = if ($measuredHeadroom -ge 0) { "$measuredHeadroom" } else { "$measuredHeadroom (NEG=near-full)" }
|
|
Write-Output ("{0,-22} : {1}" -f 'Tran tong (live cap)', "$leadCap tok")
|
|
Write-Output ("{0,-22} : {1}" -f 'hotload measured', "$totalBytes bytes")
|
|
Write-Output ("{0,-22} : {1}" -f 'tok RANGE [low..high]', "[$tokLow .. $tokHigh] tok (bytes/4 .. bytes/3.0)")
|
|
Write-Output ("{0,-22} : {1}" -f 'measured_headroom', "$hr tok (cap - tok_high, worst-case)")
|
|
Write-Output ("{0,-22} : {1}" -f 'target (owner)', "$target tok")
|
|
Write-Output ("{0,-22} : {1}" -f 'expected_backfill', "$expectedBackfill tok ($backReason)")
|
|
|
|
# ---- value-gated source-order plan (invariant, inert while target=0) ------
|
|
# When backfill>0 we would PULL crystallized content in this priority order,
|
|
# dedup vs the hot-load already counted. We NEVER actually pour it here (DRY,
|
|
# like archive-gate) - we only PRINT the order the fill WOULD follow.
|
|
Write-Output ""
|
|
Write-Output "### value-gated backfill source-order (planner / DRY - no pour)"
|
|
Write-Output ""
|
|
if ($preconditionMet) {
|
|
Write-Output (" precondition MET (target>0 AND headroom>0): would pull up to $expectedBackfill tok in order:")
|
|
Write-Output (" 1. gist {0} archive/*.gist.md (4-field distilled)" -f $ARROW)
|
|
Write-Output (" 2. value-marked {0} archive lines carrying gotcha# / anti-pattern / root-cause" -f $ARROW)
|
|
Write-Output (" 3. curated RAG {0} search_memory high-value hits" -f $ARROW)
|
|
Write-Output " (each stage DEDUP vs the hot-load sources above; stop at expected_backfill)"
|
|
} else {
|
|
Write-Output (" precondition NOT met -> source-order INERT (no pull). Opens only when target>0 AND headroom>0.")
|
|
Write-Output (" order (for reference): 1.gist {0} 2.value-marked-archive {0} 3.curated-RAG (dedup vs hot-load)" -f $ARROW)
|
|
}
|
|
|
|
# ---- honest caveat (invariant, always printed) ----------------------------
|
|
Write-Output ""
|
|
Write-Output "------------------------------------------------------------"
|
|
Write-Output " HONEST CAVEAT"
|
|
Write-Output " char/4 is NOT a real token count; VN-diacritic hot-memory ~3.0-3.5 byte/tok."
|
|
Write-Output " Headroom is a RANGE [bytes/4 .. bytes/3.0]; this planner uses bytes/3.0 (worst)."
|
|
Write-Output " backfill defaults to 0 = TAT until the OWNER sets target>0 AND measured headroom>0."
|
|
Write-Output " Every budget number is the OWNER's authority; this script is READ-ONLY (never tunes/writes)."
|
|
Write-Output " HEADROOM is measured over a STABLE FILE-FLOOR (the configured hotload_sources) = an UPPER BOUND."
|
|
Write-Output " The PEAK hot-feed also loads session-variable task-context NOT in this list; real headroom is SMALLER."
|
|
Write-Output "------------------------------------------------------------"
|
|
|
|
# measure-and-report; NEVER a build-gate.
|
|
exit 0
|
|
|
|
# ==========================================================================
|
|
# [TAILOR] honest simplifications vs the maximal spec:
|
|
# * headroom uses the WORST-CASE end of the RANGE (bytes/3.0 => fewest tokens
|
|
# hidden behind the bytes => smallest headroom => most conservative "open"
|
|
# decision). The optimistic bytes/4 is printed for context only.
|
|
# * byte-measure = [IO.File]::ReadAllBytes(...).Length (true on-disk size);
|
|
# NOT Get-Content length (which mangles VN/em-dash under PS 5.1 ANSI, gotcha
|
|
# #30) and NOT a re-encode estimate. Exact bytes, honest tokens-as-range.
|
|
# * the value-gated source-order is a PRINTED PLAN only (gist -> value-marked
|
|
# archive -> curated RAG, dedup vs hot-load). It performs NO pull and changes
|
|
# NO context - identical DRY-planner stance to memory-archive-gate.ps1.
|
|
# * crystallized_backfill block is OPTIONAL: absent => target=0 (OFF) + the 4
|
|
# default hotload sources, so the script is fully standalone before em-main
|
|
# adds the config. Present => target + hotload_sources are read LIVE.
|
|
# * NOT-FOR-SUB is a WARN (not a hard-exit): -Tier sub still prints LEAD numbers
|
|
# but flags that sub-packs are hand-distilled, never blind-backfilled.
|
|
# ==========================================================================
|