shinalok/youlbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 20: RAGAS evaluation suite

Browse Source 
- eval/run_ragas.py: collect contexts (RetrieverService) + answers (/chat API),
  evaluate with faithfulness / answer_relevancy / context_recall / context_precision
- eval/dataset.jsonl: 5 Korean Q&A pairs for initial evaluation
- eval/requirements.txt: ragas==0.2.9, datasets, langchain-google-vertexai
- Evaluator LLM priority: OpenAI > Anthropic > local Qwen3
- Runtime shim for ragas 0.2 / langchain-community 0.4+ vertexai incompatibility

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
sal
2026-06-01 17:11:00 +09:00
parent 589946ab36
commit 3faf8b09ce
4 changed files with 279 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
eval/requirements.txt
+3
View File
@@ -0,0 +1,3 @@
ragas==0.2.9
datasets>=2.14.0
langchain-google-vertexai>=2.0.0