1e1c9a2433
All checks were successful
Deploy SOLUTION_ERP / build-deploy (push) Successful in 3m38s
- eval/runs/: baseline v1.1 final PASS after retrieval.py fix (vector search restored) - eval/trial-state-lock.json: quality_gate.pass=true, baseline=1.000, avg_rerank=0.847 - docs/gotchas.md: +gotcha #52 qdrant-client 1.18 removed search() silent AttributeError - docs/STATUS.md: S31 entry — RAG PASS, retrieval.py fix, CLI restart required - docs/HANDOFF.md: S31 brief + CRITICAL CLI restart note - docs/changelog/sessions/: S31 session log Root cause: qdrant-client 1.18 removed search() → vec_results always [] → BM25-only Fix: retrieval.py query_points().points (applied to AI_INFRA repo) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
37 lines
3.2 KiB
JSON
37 lines
3.2 KiB
JSON
{
|
|
"run_date": "2026-05-26",
|
|
"run_label": "v1.1 FINAL — after retrieval.py fix (query_points)",
|
|
"golden_set_version": "v1.1",
|
|
"spec": "A",
|
|
"status": "COMPLETE",
|
|
"recall_at_5": 1.0,
|
|
"hits": 11,
|
|
"positive_queries": 11,
|
|
"pass_gate": true,
|
|
"avg_top1_rerank": 0.847,
|
|
"pipeline_fix": "retrieval.py vector_search(): search() → query_points() (qdrant-client 1.18 removed search())",
|
|
"results": [
|
|
{"id":"q01","hit":true,"top1_source":"architecture.md","top1_rerank":0.8867,"note":"CI/CD gotcha #39 — BM25+vector hit"},
|
|
{"id":"q02","hit":true,"top1_source":"architecture.md","top1_rerank":0.9102,"note":"CI/CD gotcha #41 — BM25+vector hit"},
|
|
{"id":"q03","hit":true,"top1_source":"2026-05-22-s29-plan-ca-plan-b-contract-v2-wire.md","top1_rerank":0.8828,"note":"Session log — matches 'sessions' expected hint"},
|
|
{"id":"q04","hit":true,"top1_source":"ef-core-migration/SKILL.md","top1_rerank":0.8672,"note":"EF migration SKILL — vector retrieval required"},
|
|
{"id":"q05","hit":true,"top1_source":"gotchas.md","top1_rerank":0.9375,"note":"IIS gotcha #25 — vector retrieval required"},
|
|
{"id":"q06","hit":true,"top1_source":"reviewer.md","top1_rerank":0.7422,"note":"CQRS MediatR — vector retrieval, marginal rerank"},
|
|
{"id":"q07","hit":true,"top1_source":"reviewer.md","top1_rerank":0.7891,"note":"Smart Friend/Cognition — vector retrieval required"},
|
|
{"id":"q08","hit":true,"top1_source":"project_solution_erp.md","top1_rerank":0.8398,"note":"PE V2 ApprovalWorkflow — BM25 hit (was working before)"},
|
|
{"id":"q09","hit":true,"top1_source":"feedback_multi_agent_setup.md","top1_rerank":0.8047,"note":"Implementer worktree MAX_PATH — vector+memory hit"},
|
|
{"id":"q10","hit":true,"top1_source":"feedback_subagent_setup_pitfalls.md","top1_rerank":0.8086,"note":"Sub-agent S27 fix — vector+memory hit"},
|
|
{"id":"q11","hit":true,"top1_source":"gotchas.md","top1_rerank":0.8242,"note":"ApprovalWorkflow V1 V2 fallback — vector retrieval required"},
|
|
{"id":"q12","hit":true,"note":"CORRECT EXCLUSION — rerank 0.4336 < 0.7 (GraphQL not in project)"},
|
|
{"id":"q13","hit":true,"note":"CORRECT EXCLUSION — rerank 0.3789 < 0.7 (Redis not in project)"},
|
|
{"id":"q14","hit":true,"note":"CORRECT EXCLUSION — rerank 0.4277 < 0.7 (Kubernetes not in project)"}
|
|
],
|
|
"_fix_diagnosis": {
|
|
"root_cause": "qdrant-client 1.18.0 removed QdrantClient.search() method. retrieval.py used search() → AttributeError silently swallowed in except Exception clause → vec_results always empty. Only BM25 working, and BM25 strict FTS AND-match failed for multi-token queries.",
|
|
"fix": "retrieval.py vector_search(): _qdrant.search(query_vector=...) → _qdrant.query_points(query=...).points",
|
|
"impact": "Before fix: only BM25 queries worked (q01/q02/q03/q08). After fix: all 11 positive queries hit.",
|
|
"mcp_restart_required": "retrieval.py fix applied to AI_INFRA source. MCP server process still loaded old code — CLI restart required for live MCP to use fixed pipeline.",
|
|
"new_gotcha": "gotcha #52: qdrant-client 1.18 removed search() API — upgrade detection via 'except Exception: continue' masks error silently. Fix: use query_points(). Version bump: add dep pin or version check."
|
|
}
|
|
}
|