1e1c9a2433
All checks were successful
Deploy SOLUTION_ERP / build-deploy (push) Successful in 3m38s
- eval/runs/: baseline v1.1 final PASS after retrieval.py fix (vector search restored) - eval/trial-state-lock.json: quality_gate.pass=true, baseline=1.000, avg_rerank=0.847 - docs/gotchas.md: +gotcha #52 qdrant-client 1.18 removed search() silent AttributeError - docs/STATUS.md: S31 entry — RAG PASS, retrieval.py fix, CLI restart required - docs/HANDOFF.md: S31 brief + CRITICAL CLI restart note - docs/changelog/sessions/: S31 session log Root cause: qdrant-client 1.18 removed search() → vec_results always [] → BM25-only Fix: retrieval.py query_points().points (applied to AI_INFRA repo) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
55 lines
4.6 KiB
JSON
55 lines
4.6 KiB
JSON
{
|
|
"version": "v1.3",
|
|
"project_id": "solution_erp",
|
|
"framework_adopted": "2026-05-26",
|
|
"governance_path": "docs/governance/README.md",
|
|
"golden_set_version": "v1.1",
|
|
"spec_chosen": "A",
|
|
"baseline_note": "v1.0: 2026-05-26 recall@5=0.455 FAIL (vector broken). v1.1 FINAL: 2026-05-26 recall@5=1.000 PASS after fixing retrieval.py search()→query_points() (qdrant-client 1.18 removed search()).",
|
|
"quality_gate": {
|
|
"baseline_recall_at_5": 1.0,
|
|
"baseline_recall_at_5_note": "FINAL — v1.1 run post CLI restart + retrieval.py fix. 11/11 positive queries hit, avg_top1_rerank=0.847",
|
|
"baseline_avg_top1_rerank": 0.847,
|
|
"gate_threshold_recall": 0.7,
|
|
"gate_threshold_avg_rerank": 0.65,
|
|
"pass": true
|
|
},
|
|
"drift_monitor": {
|
|
"chunk_count_baseline": 2949,
|
|
"chunk_count_registry": 2949,
|
|
"chunk_count_note": "Anti #24 resolved: projects.json root_path fixed from AI_INFRA → SOLUTION_ERP. Bootstrap re-run 2026-05-26 correct.",
|
|
"drift_threshold_percent": 5,
|
|
"last_indexed_at_baseline": "2026-05-26T13:09:21.816262"
|
|
},
|
|
"trial_milestones": [
|
|
{"week": 0, "date": "2026-05-26", "status": "complete", "label": "Setup + v1.1 baseline PASS (11/11) — after fix qdrant-client search()→query_points()"},
|
|
{"week": 1, "date": "2026-06-02", "status": "skipped", "label": "Skipped — baseline already achieved week 0 with fix"},
|
|
{"week": 2, "date": "2026-06-09", "status": "pending", "label": "Monitor drift + verify MCP live after CLI restart"},
|
|
{"week": 3, "date": "2026-06-16", "status": "pending", "label": "Empirical chunk 512 vs 1500 retest (optional — current 1.0 recall may not need)"},
|
|
{"week": 4, "date": "2026-06-23", "status": "pending", "label": "Final trial evaluation + decide v1.3 stable confirm"}
|
|
],
|
|
"_decision_log": {
|
|
"spec_a_vs_b_resolution_chosen": "Spec A — Strict. SOLUTION_ERP chunks canonical + finite scope (51 gotchas, patterns, decisions) → strict retrieval test appropriate.",
|
|
"spec_chosen_date": "2026-05-26",
|
|
"anatomy_threshold_chosen": "6/6 STRICT per v1.3 §5.2 (corpus 2949 chunks mature)",
|
|
"governance_path_b_reason": "Path B delegation stub — no local customize needed at Phase 9 UAT stable stage. AI_INFRA canonical sufficient.",
|
|
"bootstrap_correct_command": "python D:\\Dropbox\\CONG_VIEC\\AI_INFRA\\claude-rag\\bootstrap.py --config D:\\Dropbox\\CONG_VIEC\\SOLUTION\\SOLUTION_ERP\\.claude\\rag.json",
|
|
"bootstrap_wrong_command": "python D:\\Dropbox\\CONG_VIEC\\AI_INFRA\\claude-rag\\bootstrap.py --project solution_erp (DO NOT USE — resolves from CWD, not project config)",
|
|
"retrieval_fix_applied": "retrieval.py vector_search(): search() → query_points() (qdrant-client 1.18 removed search()). Fixed 2026-05-26 S31. CLI restart required for MCP to pick up fix.",
|
|
"new_gotcha_52": "qdrant-client 1.18 removed QdrantClient.search() — use query_points() instead. Silent AttributeError swallowed by except Exception → vec_results always []. Symptom: BM25-only queries work, vector queries fail silently."
|
|
},
|
|
"_anti_patterns_observed": {
|
|
"anti_24_registry_drift": "projects.json had root_path=AI_INFRA for solution_erp entry. Fixed 2026-05-26. Caused 2 bad bootstraps (1351 AI_INFRA chunks written to proj_solution_erp collection).",
|
|
"anti_23_source_path": "Absolute Windows path D:\\Dropbox\\... in chunk payload. Low priority fix-forward.",
|
|
"mcp_reload_lesson": "Bootstrap.py clearing Qdrant collection + BM25 → MCP server must be restarted to pick up new data. Similar to agents/*.md hot-reload requiring CLI restart.",
|
|
"anti_qdrant_client_upgrade": "qdrant-client 1.18 silently removed search() method. retrieval.py had 'except Exception: continue' masking the AttributeError → vector search returned [] for ALL queries. Diagnosed by testing QdrantClient.search() directly and getting AttributeError. Fix: query_points().points. Lesson: pin qdrant-client version OR test search() in health check on startup."
|
|
},
|
|
"_lessons": [
|
|
"CRITICAL: --project flag overrides only collection_name, NOT project root. Always use --config for cross-project bootstrap.",
|
|
"projects.json root_path for solution_erp was wrong (AI_INFRA) — check ALL projects in registry before first bootstrap.",
|
|
"MCP server caches/stale after Qdrant collection replace → CLI restart needed for accurate baseline.",
|
|
"v1.0 baseline (11,922 chunk auto-reindex corpus) may have been from MCP auto-reindex picking up ALL files including HANDOFF.md + STATUS.md not in explicit corpus_paths.",
|
|
"SOLUTION_ERP failure mode: NOT Anti #9 keyword stacking (AI_INFRA lesson) but corpus gap (agents not indexed) + language mismatch (Vietnamese terms)."
|
|
]
|
|
}
|