vietreport-admin/solution-erp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[CLAUDE] Docs: S31 RAG v1.3 baseline PASS (11/11 recall@5=1.000) + gotcha #52
All checks were successful
Deploy SOLUTION_ERP / build-deploy (push) Successful in 3m38s
Details

Browse Source 
- eval/runs/: baseline v1.1 final PASS after retrieval.py fix (vector search restored)
- eval/trial-state-lock.json: quality_gate.pass=true, baseline=1.000, avg_rerank=0.847
- docs/gotchas.md: +gotcha #52 qdrant-client 1.18 removed search() silent AttributeError
- docs/STATUS.md: S31 entry — RAG PASS, retrieval.py fix, CLI restart required
- docs/HANDOFF.md: S31 brief + CRITICAL CLI restart note
- docs/changelog/sessions/: S31 session log

Root cause: qdrant-client 1.18 removed search() → vec_results always [] → BM25-only
Fix: retrieval.py query_points().points (applied to AI_INFRA repo)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
pqhuy1987
2026-05-26 13:42:04 +07:00
parent b223466ded
commit 1e1c9a2433
6 changed files with 201 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
eval/runs/2026-05-26-baseline-v1.1-final.json Normal file
View File

@ -0,0 +1,36 @@
{
"run_date": "2026-05-26",
"run_label": "v1.1 FINAL — after retrieval.py fix (query_points)",
"golden_set_version": "v1.1",
"spec": "A",
"status": "COMPLETE",
"recall_at_5": 1.0,
"hits": 11,
"positive_queries": 11,
"pass_gate": true,
"avg_top1_rerank": 0.847,
"pipeline_fix": "retrieval.py vector_search(): search() → query_points() (qdrant-client 1.18 removed search())",
"results": [
{"id":"q01","hit":true,"top1_source":"architecture.md","top1_rerank":0.8867,"note":"CI/CD gotcha #39 — BM25+vector hit"},
{"id":"q02","hit":true,"top1_source":"architecture.md","top1_rerank":0.9102,"note":"CI/CD gotcha #41 — BM25+vector hit"},
{"id":"q03","hit":true,"top1_source":"2026-05-22-s29-plan-ca-plan-b-contract-v2-wire.md","top1_rerank":0.8828,"note":"Session log — matches 'sessions' expected hint"},
{"id":"q04","hit":true,"top1_source":"ef-core-migration/SKILL.md","top1_rerank":0.8672,"note":"EF migration SKILL — vector retrieval required"},
{"id":"q05","hit":true,"top1_source":"gotchas.md","top1_rerank":0.9375,"note":"IIS gotcha #25 — vector retrieval required"},
{"id":"q06","hit":true,"top1_source":"reviewer.md","top1_rerank":0.7422,"note":"CQRS MediatR — vector retrieval, marginal rerank"},
{"id":"q07","hit":true,"top1_source":"reviewer.md","top1_rerank":0.7891,"note":"Smart Friend/Cognition — vector retrieval required"},
{"id":"q08","hit":true,"top1_source":"project_solution_erp.md","top1_rerank":0.8398,"note":"PE V2 ApprovalWorkflow — BM25 hit (was working before)"},
{"id":"q09","hit":true,"top1_source":"feedback_multi_agent_setup.md","top1_rerank":0.8047,"note":"Implementer worktree MAX_PATH — vector+memory hit"},
{"id":"q10","hit":true,"top1_source":"feedback_subagent_setup_pitfalls.md","top1_rerank":0.8086,"note":"Sub-agent S27 fix — vector+memory hit"},
{"id":"q11","hit":true,"top1_source":"gotchas.md","top1_rerank":0.8242,"note":"ApprovalWorkflow V1 V2 fallback — vector retrieval required"},
{"id":"q12","hit":true,"note":"CORRECT EXCLUSION — rerank 0.4336 < 0.7 (GraphQL not in project)"},
{"id":"q13","hit":true,"note":"CORRECT EXCLUSION — rerank 0.3789 < 0.7 (Redis not in project)"},
{"id":"q14","hit":true,"note":"CORRECT EXCLUSION — rerank 0.4277 < 0.7 (Kubernetes not in project)"}
],
"_fix_diagnosis": {
"root_cause": "qdrant-client 1.18.0 removed QdrantClient.search() method. retrieval.py used search() → AttributeError silently swallowed in except Exception clause → vec_results always empty. Only BM25 working, and BM25 strict FTS AND-match failed for multi-token queries.",
"fix": "retrieval.py vector_search(): _qdrant.search(query_vector=...) → _qdrant.query_points(query=...).points",
"impact": "Before fix: only BM25 queries worked (q01/q02/q03/q08). After fix: all 11 positive queries hit.",
"mcp_restart_required": "retrieval.py fix applied to AI_INFRA source. MCP server process still loaded old code — CLI restart required for live MCP to use fixed pipeline.",
"new_gotcha": "gotcha #52: qdrant-client 1.18 removed search() API — upgrade detection via 'except Exception: continue' masks error silently. Fix: use query_points(). Version bump: add dep pin or version check."
}
}