diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f3ef90a..eec0ccf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,7 +7,7 @@ on: jobs: unit-tests: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 @@ -19,3 +19,18 @@ jobs: - run: python -m pytest -v env: RAG_TESTING: "1" + + engine-tests: + # The engine with its real libraries (ChromaDB, FAISS, sentence-transformers) and a small + # fake embedding model, so no model is downloaded. + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + cache: pip + cache-dependency-path: requirements.txt + - run: pip install torch --index-url https://download.pytorch.org/whl/cpu + - run: pip install -r requirements.txt faiss-cpu + - run: python -m pytest -v tests/test_embedders.py tests/test_bm25_tokens.py diff --git a/README.md b/README.md index 226bd96..5968769 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ repository. are kept as tables, 50 rows per passage with the header repeated. A passage that repeats, such as a page header, is kept at most twice. - **Finds the right passages in two steps:** - 1. Candidates come from two searches: by meaning (vector search with `all-MiniLM-L6-v2` in + 1. Candidates come from two searches: by meaning (vector search with `bge-small-en-v1.5` in ChromaDB; FAISS also works after `pip install faiss-cpu`) and by keywords (BM25). 2. A cross-encoder (`ms-marco-MiniLM-L-6-v2`) reads each question–passage pair and puts the candidates in order. Passages it is not confident about (below 0.3) are dropped. If nothing @@ -34,25 +34,65 @@ abstracts that support or contradict it. Every abstract is stored whole as one p app's own search engine (`src/rag_engine.py`); reading files and splitting them into the app's smaller passages is not part of this test. Laptop CPU (Apple M1), no GPU. +Choices below were made on SciFact's 809 training claims and then checked once on the 300 test +claims. That check is not fully independent: 196 of the test claims have an answering abstract +that also answers a training claim, and SciFact's test claims are part of the public MTEB +benchmark on which embedding models are usually compared. So the main choice was also checked +on a second dataset, FiQA, that played no part in it. FiQA is in MTEB too, so it checks this +project's choice, not the public rankings. + | Search set-up (10 results) | nDCG@10 | Recall@10 | Hit@3 | Time per claim | |---|---|---|---|---| -| Vector search only | 0.645 | 78.3% | 68.3% | 10 ms | +| Vector search only | 0.713 | 83.6% | 73.0% | 18 ms | | BM25 keyword search only | 0.652 | 77.4% | 69.3% | 10 ms | -| Vector search, then reranker (30 candidates) | 0.691 | 81.0% | 71.0% | 1.2 s | -| Vector + BM25, then reranker (30 + 30 candidates, no cut-off) | 0.696 | 83.1% | 72.3% | 1.9 s | +| Vector search, then reranker (30 candidates) | 0.704 | 83.5% | 72.3% | 1.1 s | +| Vector + BM25, then reranker (30 + 30 candidates, no cut-off) | 0.697 | 82.7% | 72.3% | 1.6 s | - **nDCG@10** is the usual score for this benchmark: 1.0 means the right abstracts are at the top of the first 10 results, 0 means none of them is there. **Recall@10** is the share of right abstracts found in the first 10. **Hit@3** is the share of claims with a right abstract - in the first 3. Time is the median per claim. -- **The reranker is what helps.** It lifts nDCG@10 by 0.046 over vector search alone (95% - interval +0.016 to +0.075, paired over the 300 claims), at about 1.2 s per claim on this CPU. -- **BM25 adds little on top of the reranker here:** +0.006 nDCG@10 (95% interval −0.009 to - +0.022), for about 0.6 s more per claim at 10 results. It may help more on other kinds of - documents; SciFact does not test that. + in the first 3. Time is the median per claim. All intervals below are 95% intervals from a + paired bootstrap: the claims are drawn again at random 5,000 times, and both set-ups are always + scored on the same claims. They are over the 300 test claims unless stated otherwise. +- **The embedding model was changed** from `all-MiniLM-L6-v2` to `bge-small-en-v1.5`. Three small + models were compared with vector search alone. The rule, fixed in advance: the best nDCG@10 on + SciFact's 809 training claims wins. + + | Embedding model | Training claims | Test claims | FiQA (648 questions) | + |---|---|---|---| + | `all-MiniLM-L6-v2` (the old one) | 0.660 | 0.645 | 0.369 | + | `e5-small-v2` | 0.708 | 0.687 | not run | + | **`bge-small-en-v1.5`** | **0.747** | **0.713** | **0.403** | + + nDCG@10 with vector search alone. On the SciFact test claims, the new model is +0.068 above the + old one (interval +0.041 to +0.094) and Hit@3 goes from 68.3% to 73.0% (+4.7 points, interval + +1.3 to +8.3). On [FiQA](https://huggingface.co/datasets/BeIR/fiqa) (57,600 financial forum + posts and everyday questions about money), which played no part in the choice, it is +0.035 + (interval +0.017 to +0.053) and Hit@3 goes from 51.2% to 54.3% (+3.1 points, interval +0.2 to + +6.2). The rule for FiQA, also fixed in advance, was to go back to the old model only if the new + one was clearly worse there. In order of events: `bge-small-en-v1.5` was first compared with the + old model in a quick check whose test numbers were seen; the written rule and the third model + came after that. The training-claim numbers are the same in both, so the choice does not change. +- **Part of the SciFact gain comes from text length.** The old model reads only the first 256 + tokens (word pieces) of a text, the new one up to 512, and 71% of SciFact abstracts are longer + than 256 tokens. For test claims whose answering abstract fits in 256 tokens, the gain is +0.030 + (interval −0.016 to +0.076, 86 claims); for longer ones, +0.083 (+0.052 to +0.116, 214 claims). + The app's passages are at most about 1,000 characters, mostly under 256 tokens, so the gain the + app sees is more likely near FiQA's +0.035, where only 19% of posts are longer than 256 tokens. +- **What it costs:** turning documents into vectors takes about 3 to 5 times longer (SciFact: + 6.9 minutes against 2.5; FiQA: 53 against 11, on this laptop), so uploads are slower. Each + passage is still stored as 384 numbers, so the index does not grow. The model is a larger + download (about 135 MB against 90 MB) and needs a short prefix before each question, which the + engine adds. Searching takes 18 ms per claim instead of 10 ms. +- **With the new model the reranker adds nothing here.** Vector search alone scores 0.009 above + vector search plus reranker (interval −0.018 to +0.035). With the old model the reranker + lifted nDCG@10 by 0.046 (interval +0.016 to +0.075). BM25 adds nothing on top either (−0.007, + interval −0.020 to +0.006). Both may help more on other kinds of documents; SciFact does not + test that. - **A bug found by this test.** BM25 used to split text on spaces only, so "cancer." and "cancer" were different words. Splitting into words raised BM25 alone from 0.560 to 0.652 (95% interval - +0.066 to +0.119); the full pipeline barely moved (0.694 to 0.696). The numbers before the fix + +0.066 to +0.119); the full pipeline, then with the old embedding model, barely moved (0.694 to + 0.696). The numbers before the fix are in commit `4350ee6`. The new splitting was chosen while looking at SciFact, so it was also checked on a dataset that played no part in that choice: on [FiQA](https://huggingface.co/datasets/BeIR/fiqa) (57,600 financial forum posts, 648 test @@ -60,31 +100,127 @@ smaller passages is not part of this test. Laptop CPU (Apple M1), no GPU. (`python -m evaluation.bm25_tokens_check --dataset fiqa`, results in `evaluation/bm25_tokens_check.json`). -**The app's default call** asks for 3 passages (9 vector + 9 BM25 candidates) and drops passages -with a reranker confidence below 0.3; when nothing is left, the app says it cannot answer. It takes -about 0.7 s per claim. +### The app's default call + +The app asks for 3 passages (9 vector + 9 BM25 candidates, put in order by the reranker) and +drops passages with a reranker confidence below 0.3; when nothing is left, it says it cannot +answer. It takes about 0.55 s per claim. | Default call | Hit@3 | Claims with no passage | |---|---|---| -| With the 0.3 cut-off (as the app runs) | 67.0% | 12.0% (36 of 300) | -| Without the cut-off | 72.7% | 0% | +| With the 0.3 cut-off (as the app runs) | 67.0% | 12.3% (37 of 300) | +| Without the cut-off | 72.3% | 0% | + +- **The new embedding model did not move this number.** With either model the app finds a right + abstract for 67.0% of the claims (difference 0, interval −1 to +1 point). Here the reranker + picks the final 3 passages, so a better vector search changes the candidates but not what the + app returns. Without the cut-off, the app's call (72.3%) and vector search alone (73.0%) are + level (+0.7 points for vector search, interval −3.0 to +4.3). +- **The cut-off costs 16 claims a right passage:** 9 that get no passage at all and 7 whose list + was cut to 1 or 2 passages. In all, 152 of the 300 claims get fewer than 3 passages. For the + other 28 claims with no passage, the 3 passages the app would have used were all wrong, so not + answering was the better outcome. + +Whole abstracts are longer than the app's passages, so the share of dropped passages and the +timings may differ on your own documents. + +### When should the app say "I cannot answer"? + +Every SciFact claim has an answer among the abstracts, so the test above shows only what the +cut-off costs. To see what it saves, the claims were also run on a second index without the +abstracts that SciFact marks as answering them: there, the right outcome is to give no passage. + +The reranker's confidence tells the two cases apart only weakly. On the test claims, a claim +with an answer in the index gets a higher top confidence than a claim without one 70% of the +time (AUROC 0.70; 0.5 would be chance). The median top confidence is 0.97 with an answer and +0.71 without, so most claims without an answer still get confident passages. + +| Cut-off | Hit@3 (answer in the index) | No passage given (no answer in the index) | +|---|---|---| +| none | 72.3% | 0% | +| 0.1 | 70.0% | 19.0% | +| **0.3 (the app)** | **67.0%** | **30.3%** | +| 0.5 | 64.0% | 36.0% | +| 0.7 | 58.7% | 49.3% | +| 0.9 | 52.0% | 67.0% | + +300 test claims on each index. Each step up refuses more claims without an answer, but also +loses claims that had one. + +Two things make the refusal shares look worse than they may be for real use. SciFact does not +mark every abstract that answers a claim, so some passages counted as wrong here may in fact +answer it: on the second index, 99 of the 300 claims still get a top passage with a confidence +of 0.9 or more. And the passages left there are on the same subject as the claim, which makes +refusing harder than for a question about something the documents do not cover at all. + +**The app keeps 0.3.** Before running, the rule for choosing the cut-off on the training claims +was set as the mean of the two columns. That rule picked 0.95, the top of the range tried, which +on test refuses 44% of the claims that do have an answer (Hit@3 47.7%). The rule itself is at +fault: refusing every claim scores 0.5 under it, more than answering every claim (0.38). So its +choice was not adopted. Which cut-off is right depends on how often users ask about things that +are not in their documents, and SciFact cannot say that. + +### Is the reranker still worth it? + +With the new embedding model, on SciFact, it does not clearly pay for itself. Here is vector +search alone (the 3 most similar abstracts, no reranker) against the app's call: + +| Hit@3 | Training claims (809) | Test claims (300) | +|---|---|---| +| No cut-off: vector search alone | 79.0% | 73.0% | +| No cut-off: the app's call | 75.9% | 72.3% | +| Difference | +3.1 (+0.5 to +5.6) | +0.7 (−3.0 to +4.3) | +| With a cut-off: vector search alone | 70.1% | 64.3% | +| With a cut-off: the app's call (0.3) | 68.9% | 67.0% | +| Difference | +1.2 (−1.5 to +4.0) | −2.7 (−6.7 to +1.3) | + +For vector search, the cut-off is a similarity score of 0.74 (1 means the same meaning). It was +chosen on the training claims so that it refuses as many claims without an answer as the app +does: on test, 30.0% against 30.3%. Its score tells the two cases apart about as well as the +reranker's confidence (AUROC 0.69 against 0.70). + +- Without a cut-off, vector search alone is better on the training claims and level on test. + With a cut-off, the two are level on both; on test, vector search found a right abstract a + little less often, but the interval includes no difference. +- Vector search alone takes 18 ms per claim, against about 550 ms for the app's call: 30 times + faster. + +**The reranker stays the default for now.** The evidence is mixed and comes from one dataset of +scientific claims, while the reranker was trained on everyday web questions, which may be closer +to what users ask. Whether to drop it is the next thing to test on a second dataset. + +**A larger reranker did not help either.** Two candidates went into the app's call without the +cut-off, on the 809 training claims. The rule, fixed before running: switch only if the paired +interval against the current reranker is above 0. + +| Reranker | Hit@3 (training claims) | Time per claim | +|---|---|---| +| `ms-marco-MiniLM-L-6-v2` (the app's) | 75.9% | 0.66 s | +| `ms-marco-MiniLM-L-12-v2` | 75.6% | 1.3 s | +| `bge-reranker-base` | 77.9% | 3.0 s | -The cut-off costs 17 claims a right passage: 10 that get no passage at all and 7 whose list was -cut to 1 or 2 passages. In all, 156 of the 300 claims get fewer than 3 passages. For the other -26 claims with no passage, the 3 passages the app would have used were all wrong, so not -answering was the better outcome. Every SciFact claim has an answer in the abstracts, so this test -shows what the cut-off costs, not how often it saves the app from answering a question the -documents cannot answer. Whole abstracts are longer than the app's passages, so the share of -dropped passages and the timings may differ on your own documents. +`bge-reranker-base` is 2.0 points better (interval −0.1 to +4.1 over the training claims): close, +but not clearly above 0, and 4.5 times slower. So the app's reranker stays, and the test claims +were not used for this. None of the three beats vector search alone on these claims (79.0%). +Timings come from separate runs, so the app's call shows 0.66 s here and 0.55 s above. -Reproduce (downloads the pinned benchmark and models, about 25 minutes on a laptop CPU): +Reproduce (downloads the pinned benchmark and models; times on a laptop CPU): ```bash pip install -r requirements-eval.txt -python -m evaluation.retrieval_eval --dataset scifact # results.json, per_question_scifact.csv -python -m evaluation.compare dense+rerank dense # paired difference with 95% interval +python -m evaluation.embedder_choice --dataset scifact # embedding model choice, about 16 min +python -m evaluation.embedder_choice --dataset fiqa # check on FiQA, about 65 min +python -m evaluation.retrieval_eval --dataset scifact # first table, about 25 min +python -m evaluation.compare dense dense+rerank # paired difference with 95% interval +python -m evaluation.compare dense dense --metric hit3 \ + --csv-b evaluation/per_question_scifact_all-MiniLM-L6-v2.csv # new against old embedding model +python -m evaluation.cutoff_tuning # cut-off table, about 40 min +python -m evaluation.dense_cutoff # vector search with a cut-off, about 8 min +python -m evaluation.reranker_check # larger rerankers, about 75 min ``` +Results for the old embedding model: `python -m evaluation.retrieval_eval --embedder all-MiniLM-L6-v2`. + ## Run it You need Python 3.10 or newer and [Ollama](https://ollama.com/). @@ -104,7 +240,14 @@ pip install -r requirements.txt uvicorn main:app --reload # http://localhost:8000, API page at /docs ``` -The first start downloads the embedding model and the reranker from Hugging Face (about 180 MB in all). +The first start downloads the embedding model and the reranker from Hugging Face (about 230 MB in all). + +**Upgrading from an earlier version:** the default embedding model is now `bge-small-en-v1.5`, and +each embedding model has its own index. The old index is kept but not used, and the app prints a +warning at start-up when the new index is empty but uploaded files exist. Upload your documents +again, or start the app with `RAG_EMBEDDER=all-MiniLM-L6-v2` to keep using the old index. A +document deleted while one model is in use stays in the other model's index. Each index records +the model that built it, and the app refuses to open an index built with a different model. Try it: @@ -136,12 +279,19 @@ curl -X POST http://localhost:8000/api/chat \ The unit tests check the main endpoints (chat, streaming, upload, documents, history, feedback) and input checks in a light test mode that does not load any model, so they run in about a -second. The tests of the search metrics run there too; the BM25 word test runs -when the full requirements are installed. +second. The tests of the search metrics run there too. + +The engine tests need the full requirements. They use the real ChromaDB and FAISS with a small +fake embedding model, so no model is downloaded. They check the question prefix, unit-length +vectors, one index per embedding model, the refusal of an index built with another model, and +the BM25 word splitting. CI runs both sets. ```bash pip install -r requirements-dev.txt -RAG_TESTING=1 python -m pytest +RAG_TESTING=1 python -m pytest # light tests + +pip install -r requirements.txt faiss-cpu +python -m pytest tests/test_embedders.py tests/test_bm25_tokens.py # engine tests ``` ## Scoring answers with judge models @@ -159,7 +309,7 @@ python tests/eval_judge.py --input my_questions.json --output judge_results.json | Setting | Where | Default | |---|---|---| | Models | `MODEL_REGISTRY` in `main.py` | `qwen3-1.7b`, `qwen3-4b-thinking` | -| Embedding model | `src/rag_engine.py` | `all-MiniLM-L6-v2` | +| Embedding model | `RAG_EMBEDDER` environment variable (`EMBEDDERS` in `src/rag_engine.py`) | `bge-small-en-v1.5` | | Reranker | `src/rag_engine.py` | `cross-encoder/ms-marco-MiniLM-L-6-v2` | | Passage size | `src/rag_engine.py` | about 500 characters, 100 overlap, joined up to 1,000 | | Confidence cut-off | `retrieve()` in `src/rag_engine.py` | 0.3 | diff --git a/docs/03_vector_layer.md b/docs/03_vector_layer.md index a87aaea..e215f0d 100644 --- a/docs/03_vector_layer.md +++ b/docs/03_vector_layer.md @@ -60,7 +60,7 @@ else: |------------|---------| | **ChromaDB** | Primary vector database | | **FAISS** | High-performance alternative | -| **sentence-transformers** | Embedding model (all-MiniLM-L6-v2) | +| **sentence-transformers** | Embedding model (bge-small-en-v1.5; all-MiniLM-L6-v2 before) | | **rank_bm25** | BM25 scoring for hybrid search | | **PyMuPDF, python-docx, openpyxl** | Document parsing | diff --git a/docs/06_literature_review.md b/docs/06_literature_review.md index c95d8cc..5b95604 100644 --- a/docs/06_literature_review.md +++ b/docs/06_literature_review.md @@ -77,13 +77,16 @@ Ollama abstracts away quantization, model management, and serving, enabling fast | Model | Dim | Speed | Quality | Decision | | ----------------------- | ---- | ------ | --------- | ----------- | -| **all-MiniLM-L6-v2** | 384 | Fast | Good | ✅ Default | +| all-MiniLM-L6-v2 | 384 | Fast | Good | First default | +| **bge-small-en-v1.5** | 384 | Medium | Better | ✅ Default (measured, see README) | | all-mpnet-base-v2 | 768 | Medium | Better | Alternative | | BGE-large | 1024 | Slow | Excellent | ❌ Memory | | OpenAI text-embedding-3 | 1536 | API | Excellent | ❌ Cloud | **Rationale** MiniLM provides an excellent speed-to-quality ratio and consistently ranks well on MTEB semantic retrieval tasks while remaining CPU-friendly. +It was replaced later by bge-small-en-v1.5, which has the same vector size and searched better in this +project's own measurement (README, "How well the search works"). [see here: mteb leadorboard](https://huggingface.co/spaces/mteb/leaderboard) --- diff --git a/evaluation/compare.py b/evaluation/compare.py index 002a0e2..52e87b2 100644 --- a/evaluation/compare.py +++ b/evaluation/compare.py @@ -8,8 +8,9 @@ python -m evaluation.compare app_default app_default_no_cutoff --metric hit3 # The same set-up in two runs, for example before and after a code change: - git show 4350ee6:evaluation/per_question_scifact.csv > before.csv - python -m evaluation.compare bm25 bm25 --csv-b before.csv + python -m evaluation.compare dense dense --csv-b evaluation/per_question_scifact_all-MiniLM-L6-v2.csv + git show 4350ee6:evaluation/per_question_scifact.csv > before.csv # before the BM25 fix + python -m evaluation.compare bm25 bm25 --csv evaluation/per_question_scifact_all-MiniLM-L6-v2.csv --csv-b before.csv """ import argparse @@ -18,7 +19,7 @@ import numpy as np import pandas as pd -DEFAULT_CSV = Path(__file__).resolve().parent / "per_question_scifact.csv" +DEFAULT_CSV = Path(__file__).resolve().parent / "per_question_scifact_bge-small-en-v1.5.csv" def paired_bootstrap(diff: np.ndarray, resamples: int = 5000, seed: int = 0) -> tuple[float, float, float]: diff --git a/evaluation/cutoff_bge-small-en-v1.5.json b/evaluation/cutoff_bge-small-en-v1.5.json new file mode 100644 index 0000000..f8e4aa7 --- /dev/null +++ b/evaluation/cutoff_bge-small-en-v1.5.json @@ -0,0 +1,358 @@ +{ + "embedder": "bge-small-en-v1.5", + "claims": { + "train": 809, + "test": 300 + }, + "chosen_cutoff_on_train": 0.95, + "test_at_chosen": { + "cutoff": 0.95, + "hit@3_answerable": 0.4767, + "answered_answerable": 0.5633, + "no_passage_unanswerable": 0.74, + "score": 0.6083 + }, + "test_at_old_0.3": { + "cutoff": 0.3, + "hit@3_answerable": 0.67, + "answered_answerable": 0.8767, + "no_passage_unanswerable": 0.3033, + "score": 0.4867 + }, + "test_without_cutoff": { + "cutoff": 0.0, + "hit@3_answerable": 0.7233, + "answered_answerable": 1.0, + "no_passage_unanswerable": 0.0, + "score": 0.3617 + }, + "sweep": { + "train": [ + { + "cutoff": 0.0, + "hit@3_answerable": 0.759, + "answered_answerable": 1.0, + "no_passage_unanswerable": 0.0, + "score": 0.3795 + }, + { + "cutoff": 0.01, + "hit@3_answerable": 0.7528, + "answered_answerable": 0.9802, + "no_passage_unanswerable": 0.0606, + "score": 0.4067 + }, + { + "cutoff": 0.02, + "hit@3_answerable": 0.7417, + "answered_answerable": 0.974, + "no_passage_unanswerable": 0.0754, + "score": 0.4085 + }, + { + "cutoff": 0.03, + "hit@3_answerable": 0.7379, + "answered_answerable": 0.9654, + "no_passage_unanswerable": 0.0878, + "score": 0.4129 + }, + { + "cutoff": 0.05, + "hit@3_answerable": 0.7305, + "answered_answerable": 0.953, + "no_passage_unanswerable": 0.1211, + "score": 0.4258 + }, + { + "cutoff": 0.1, + "hit@3_answerable": 0.7206, + "answered_answerable": 0.9308, + "no_passage_unanswerable": 0.1805, + "score": 0.4506 + }, + { + "cutoff": 0.15, + "hit@3_answerable": 0.7157, + "answered_answerable": 0.9122, + "no_passage_unanswerable": 0.2287, + "score": 0.4722 + }, + { + "cutoff": 0.2, + "hit@3_answerable": 0.712, + "answered_answerable": 0.9036, + "no_passage_unanswerable": 0.2571, + "score": 0.4845 + }, + { + "cutoff": 0.25, + "hit@3_answerable": 0.6996, + "answered_answerable": 0.8949, + "no_passage_unanswerable": 0.2769, + "score": 0.4883 + }, + { + "cutoff": 0.3, + "hit@3_answerable": 0.6885, + "answered_answerable": 0.8826, + "no_passage_unanswerable": 0.3066, + "score": 0.4975 + }, + { + "cutoff": 0.35, + "hit@3_answerable": 0.6724, + "answered_answerable": 0.8603, + "no_passage_unanswerable": 0.3399, + "score": 0.5062 + }, + { + "cutoff": 0.4, + "hit@3_answerable": 0.6675, + "answered_answerable": 0.8566, + "no_passage_unanswerable": 0.3511, + "score": 0.5093 + }, + { + "cutoff": 0.45, + "hit@3_answerable": 0.6613, + "answered_answerable": 0.848, + "no_passage_unanswerable": 0.382, + "score": 0.5216 + }, + { + "cutoff": 0.5, + "hit@3_answerable": 0.6489, + "answered_answerable": 0.8368, + "no_passage_unanswerable": 0.3993, + "score": 0.5241 + }, + { + "cutoff": 0.55, + "hit@3_answerable": 0.6415, + "answered_answerable": 0.8257, + "no_passage_unanswerable": 0.4227, + "score": 0.5321 + }, + { + "cutoff": 0.6, + "hit@3_answerable": 0.6378, + "answered_answerable": 0.8084, + "no_passage_unanswerable": 0.4524, + "score": 0.5451 + }, + { + "cutoff": 0.65, + "hit@3_answerable": 0.6267, + "answered_answerable": 0.7948, + "no_passage_unanswerable": 0.471, + "score": 0.5488 + }, + { + "cutoff": 0.7, + "hit@3_answerable": 0.6119, + "answered_answerable": 0.7738, + "no_passage_unanswerable": 0.4969, + "score": 0.5544 + }, + { + "cutoff": 0.75, + "hit@3_answerable": 0.5946, + "answered_answerable": 0.7515, + "no_passage_unanswerable": 0.5216, + "score": 0.5581 + }, + { + "cutoff": 0.8, + "hit@3_answerable": 0.581, + "answered_answerable": 0.7268, + "no_passage_unanswerable": 0.5538, + "score": 0.5674 + }, + { + "cutoff": 0.85, + "hit@3_answerable": 0.5661, + "answered_answerable": 0.6934, + "no_passage_unanswerable": 0.6007, + "score": 0.5834 + }, + { + "cutoff": 0.9, + "hit@3_answerable": 0.5389, + "answered_answerable": 0.6601, + "no_passage_unanswerable": 0.6465, + "score": 0.5927 + }, + { + "cutoff": 0.95, + "hit@3_answerable": 0.4759, + "answered_answerable": 0.5773, + "no_passage_unanswerable": 0.7268, + "score": 0.6014 + } + ], + "test": [ + { + "cutoff": 0.0, + "hit@3_answerable": 0.7233, + "answered_answerable": 1.0, + "no_passage_unanswerable": 0.0, + "score": 0.3617 + }, + { + "cutoff": 0.01, + "hit@3_answerable": 0.7133, + "answered_answerable": 0.96, + "no_passage_unanswerable": 0.0567, + "score": 0.385 + }, + { + "cutoff": 0.02, + "hit@3_answerable": 0.7133, + "answered_answerable": 0.9533, + "no_passage_unanswerable": 0.0867, + "score": 0.4 + }, + { + "cutoff": 0.03, + "hit@3_answerable": 0.7133, + "answered_answerable": 0.95, + "no_passage_unanswerable": 0.1, + "score": 0.4067 + }, + { + "cutoff": 0.05, + "hit@3_answerable": 0.71, + "answered_answerable": 0.94, + "no_passage_unanswerable": 0.1267, + "score": 0.4183 + }, + { + "cutoff": 0.1, + "hit@3_answerable": 0.7, + "answered_answerable": 0.9233, + "no_passage_unanswerable": 0.19, + "score": 0.445 + }, + { + "cutoff": 0.15, + "hit@3_answerable": 0.6933, + "answered_answerable": 0.8967, + "no_passage_unanswerable": 0.24, + "score": 0.4667 + }, + { + "cutoff": 0.2, + "hit@3_answerable": 0.68, + "answered_answerable": 0.8833, + "no_passage_unanswerable": 0.2767, + "score": 0.4783 + }, + { + "cutoff": 0.25, + "hit@3_answerable": 0.6733, + "answered_answerable": 0.8833, + "no_passage_unanswerable": 0.29, + "score": 0.4817 + }, + { + "cutoff": 0.3, + "hit@3_answerable": 0.67, + "answered_answerable": 0.8767, + "no_passage_unanswerable": 0.3033, + "score": 0.4867 + }, + { + "cutoff": 0.35, + "hit@3_answerable": 0.66, + "answered_answerable": 0.8633, + "no_passage_unanswerable": 0.33, + "score": 0.495 + }, + { + "cutoff": 0.4, + "hit@3_answerable": 0.6567, + "answered_answerable": 0.8633, + "no_passage_unanswerable": 0.3367, + "score": 0.4967 + }, + { + "cutoff": 0.45, + "hit@3_answerable": 0.6433, + "answered_answerable": 0.85, + "no_passage_unanswerable": 0.3467, + "score": 0.495 + }, + { + "cutoff": 0.5, + "hit@3_answerable": 0.64, + "answered_answerable": 0.8433, + "no_passage_unanswerable": 0.36, + "score": 0.5 + }, + { + "cutoff": 0.55, + "hit@3_answerable": 0.6267, + "answered_answerable": 0.82, + "no_passage_unanswerable": 0.4, + "score": 0.5133 + }, + { + "cutoff": 0.6, + "hit@3_answerable": 0.6167, + "answered_answerable": 0.8, + "no_passage_unanswerable": 0.4467, + "score": 0.5317 + }, + { + "cutoff": 0.65, + "hit@3_answerable": 0.5967, + "answered_answerable": 0.7867, + "no_passage_unanswerable": 0.4533, + "score": 0.525 + }, + { + "cutoff": 0.7, + "hit@3_answerable": 0.5867, + "answered_answerable": 0.7667, + "no_passage_unanswerable": 0.4933, + "score": 0.54 + }, + { + "cutoff": 0.75, + "hit@3_answerable": 0.5767, + "answered_answerable": 0.74, + "no_passage_unanswerable": 0.5333, + "score": 0.555 + }, + { + "cutoff": 0.8, + "hit@3_answerable": 0.5667, + "answered_answerable": 0.7133, + "no_passage_unanswerable": 0.5867, + "score": 0.5767 + }, + { + "cutoff": 0.85, + "hit@3_answerable": 0.5567, + "answered_answerable": 0.6867, + "no_passage_unanswerable": 0.6267, + "score": 0.5917 + }, + { + "cutoff": 0.9, + "hit@3_answerable": 0.52, + "answered_answerable": 0.6367, + "no_passage_unanswerable": 0.67, + "score": 0.595 + }, + { + "cutoff": 0.95, + "hit@3_answerable": 0.4767, + "answered_answerable": 0.5633, + "no_passage_unanswerable": 0.74, + "score": 0.6083 + } + ] + }, + "minutes": 40.4 +} diff --git a/evaluation/cutoff_per_claim_bge-small-en-v1.5.jsonl b/evaluation/cutoff_per_claim_bge-small-en-v1.5.jsonl new file mode 100644 index 0000000..a1a918f --- /dev/null +++ b/evaluation/cutoff_per_claim_bge-small-en-v1.5.jsonl @@ -0,0 +1,2218 @@ +{"split":"train","index":"answerable","query_id":"0","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"2","confidence":[0.606,0.069,0.043],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"4","confidence":[0.982,0.777,0.612],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"6","confidence":[0.997,0.838,0.543],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"9","confidence":[0.999,0.047,0.03],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"10","confidence":[0.707,0.045,0.011],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"11","confidence":[0.797,0.054,0.018],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"12","confidence":[0.611,0.393,0.131],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"14","confidence":[0.919,0.597,0.363],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"15","confidence":[0.572,0.381,0.315],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"17","confidence":[0.739,0.509,0.169],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"18","confidence":[0.981,0.799,0.73],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"19","confidence":[0.313,0.11,0.094],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"20","confidence":[0.311,0.089,0.087],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"21","confidence":[0.592,0.389,0.202],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"22","confidence":[0.948,0.499,0.462],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"24","confidence":[0.998,0.884,0.864],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"25","confidence":[0.999,0.834,0.611],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"26","confidence":[0.052,0.046,0.018],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"27","confidence":[0.083,0.076,0.038],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"28","confidence":[0.999,0.829,0.654],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"30","confidence":[0.992,0.982,0.906],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"32","confidence":[0.982,0.008,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"34","confidence":[0.989,0.984,0.977],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"35","confidence":[0.99,0.986,0.941],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"37","confidence":[0.986,0.98,0.948],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"39","confidence":[0.279,0.003,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"40","confidence":[0.348,0.002,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"41","confidence":[0.999,0.3,0.188],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"43","confidence":[0.962,0.845,0.248],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"44","confidence":[0.692,0.235,0.126],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"45","confidence":[0.705,0.208,0.154],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"46","confidence":[0.999,0.404,0.311],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"47","confidence":[0.999,0.936,0.242],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"52","confidence":[0.985,0.975,0.957],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"55","confidence":[0.999,0.981,0.98],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"58","confidence":[0.992,0.832,0.652],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"60","confidence":[0.967,0.924,0.916],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"61","confidence":[0.994,0.979,0.965],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"62","confidence":[0.803,0.189,0.029],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"63","confidence":[0.992,0.985,0.972],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"64","confidence":[0.983,0.982,0.968],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"66","confidence":[0.445,0.017,0.006],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"67","confidence":[1.0,0.988,0.987],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"68","confidence":[1.0,0.998,0.996],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"69","confidence":[0.998,0.996,0.996],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"71","confidence":[0.99,0.904,0.743],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"73","confidence":[0.732,0.037,0.009],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"74","confidence":[0.507,0.24,0.177],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"76","confidence":[0.999,0.979,0.24],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"77","confidence":[1.0,0.414,0.059],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"78","confidence":[0.999,0.704,0.587],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"79","confidence":[0.995,0.984,0.919],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"80","confidence":[0.343,0.314,0.195],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"81","confidence":[0.742,0.697,0.041],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"82","confidence":[0.689,0.4,0.283],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"85","confidence":[0.904,0.894,0.77],"relevant":[false,true,true]} +{"split":"train","index":"answerable","query_id":"86","confidence":[0.904,0.894,0.77],"relevant":[false,true,true]} +{"split":"train","index":"answerable","query_id":"88","confidence":[0.999,0.921,0.914],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"89","confidence":[0.962,0.957,0.945],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"90","confidence":[0.962,0.957,0.945],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"91","confidence":[0.417,0.091,0.013],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"92","confidence":[0.596,0.143,0.078],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"93","confidence":[0.996,0.993,0.959],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"95","confidence":[0.083,0.005,0.003],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"96","confidence":[0.91,0.585,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"98","confidence":[0.99,0.016,0.015],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"104","confidence":[0.394,0.006,0.003],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"105","confidence":[0.916,0.017,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"106","confidence":[0.999,0.998,0.968],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"108","confidence":[1.0,1.0,1.0],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"109","confidence":[0.983,0.948,0.668],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"111","confidence":[0.999,0.998,0.997],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"112","confidence":[0.999,0.974,0.972],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"114","confidence":[0.281,0.0,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"116","confidence":[0.619,0.0,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"119","confidence":[0.958,0.113,0.099],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"120","confidence":[0.957,0.807,0.311],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"121","confidence":[0.116,0.093,0.05],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"122","confidence":[0.209,0.203,0.139],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"123","confidence":[0.992,0.95,0.71],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"126","confidence":[0.754,0.681,0.496],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"134","confidence":[0.821,0.651,0.397],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"138","confidence":[0.999,0.117,0.101],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"139","confidence":[0.151,0.04,0.022],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"144","confidence":[0.997,0.785,0.636],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"149","confidence":[0.995,0.892,0.618],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"152","confidence":[0.997,0.497,0.408],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"153","confidence":[0.971,0.92,0.906],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"154","confidence":[0.974,0.914,0.905],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"155","confidence":[0.988,0.974,0.968],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"156","confidence":[0.996,0.991,0.982],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"157","confidence":[0.999,0.997,0.979],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"159","confidence":[0.999,0.995,0.995],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"160","confidence":[0.944,0.817,0.73],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"161","confidence":[0.305,0.0,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"164","confidence":[0.326,0.251,0.126],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"165","confidence":[0.602,0.511,0.408],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"166","confidence":[0.998,0.997,0.958],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"167","confidence":[0.999,0.998,0.984],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"168","confidence":[0.446,0.343,0.278],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"169","confidence":[0.678,0.441,0.428],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"172","confidence":[1.0,0.996,0.995],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"173","confidence":[0.874,0.797,0.052],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"174","confidence":[0.046,0.035,0.033],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"175","confidence":[0.13,0.111,0.1],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"176","confidence":[0.941,0.239,0.058],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"177","confidence":[1.0,0.998,0.978],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"178","confidence":[0.993,0.985,0.983],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"181","confidence":[0.989,0.801,0.074],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"182","confidence":[0.986,0.914,0.824],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"184","confidence":[0.9,0.181,0.094],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"186","confidence":[0.017,0.007,0.006],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"187","confidence":[0.016,0.009,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"189","confidence":[0.997,0.981,0.977],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"196","confidence":[0.998,0.752,0.644],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"197","confidence":[0.984,0.982,0.98],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"199","confidence":[0.14,0.069,0.056],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"200","confidence":[0.991,0.635,0.631],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"201","confidence":[1.0,0.937,0.916],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"203","confidence":[1.0,0.707,0.675],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"204","confidence":[0.978,0.886,0.557],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"205","confidence":[0.964,0.791,0.121],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"209","confidence":[0.848,0.028,0.006],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"210","confidence":[0.997,0.054,0.05],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"211","confidence":[0.087,0.056,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"214","confidence":[0.997,0.997,0.969],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"220","confidence":[1.0,0.115,0.042],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"221","confidence":[1.0,0.984,0.958],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"222","confidence":[1.0,0.99,0.927],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"223","confidence":[0.996,0.955,0.801],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"224","confidence":[1.0,1.0,0.999],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"225","confidence":[0.998,0.997,0.994],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"226","confidence":[0.999,0.999,0.999],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"227","confidence":[0.999,0.951,0.932],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"228","confidence":[0.997,0.408,0.406],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"229","confidence":[0.762,0.297,0.186],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"235","confidence":[0.932,0.008,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"241","confidence":[0.397,0.074,0.062],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"242","confidence":[0.247,0.03,0.029],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"243","confidence":[0.997,0.585,0.559],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"244","confidence":[0.436,0.081,0.054],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"245","confidence":[0.984,0.984,0.962],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"246","confidence":[0.986,0.971,0.83],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"247","confidence":[0.91,0.882,0.801],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"250","confidence":[1.0,0.305,0.269],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"251","confidence":[1.0,0.83,0.768],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"253","confidence":[0.98,0.913,0.261],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"254","confidence":[0.986,0.925,0.324],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"255","confidence":[0.83,0.803,0.04],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"256","confidence":[0.994,0.955,0.183],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"258","confidence":[0.878,0.796,0.794],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"259","confidence":[0.998,0.166,0.087],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"262","confidence":[0.97,0.939,0.927],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"263","confidence":[0.988,0.968,0.896],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"264","confidence":[0.994,0.975,0.973],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"265","confidence":[1.0,0.671,0.531],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"266","confidence":[0.474,0.106,0.006],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"267","confidence":[0.992,0.373,0.037],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"272","confidence":[0.997,0.808,0.299],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"277","confidence":[0.991,0.726,0.549],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"278","confidence":[0.993,0.554,0.333],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"280","confidence":[0.997,0.424,0.136],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"281","confidence":[0.828,0.384,0.342],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"283","confidence":[0.999,0.998,0.914],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"285","confidence":[0.039,0.024,0.009],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"286","confidence":[0.994,0.94,0.547],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"287","confidence":[0.995,0.964,0.733],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"290","confidence":[0.993,0.474,0.304],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"292","confidence":[0.992,0.128,0.042],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"293","confidence":[0.457,0.177,0.114],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"296","confidence":[0.97,0.958,0.75],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"299","confidence":[0.996,0.488,0.332],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"301","confidence":[0.436,0.261,0.214],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"304","confidence":[0.913,0.843,0.702],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"305","confidence":[0.992,0.953,0.924],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"306","confidence":[0.65,0.437,0.193],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"308","confidence":[0.991,0.963,0.634],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"309","confidence":[0.775,0.655,0.625],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"310","confidence":[0.563,0.51,0.089],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"313","confidence":[0.592,0.481,0.474],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"315","confidence":[0.919,0.312,0.129],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"316","confidence":[0.744,0.718,0.163],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"317","confidence":[0.934,0.277,0.172],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"323","confidence":[0.032,0.03,0.028],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"325","confidence":[0.183,0.072,0.066],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"326","confidence":[0.79,0.426,0.139],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"330","confidence":[0.879,0.087,0.057],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"331","confidence":[0.311,0.17,0.009],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"332","confidence":[1.0,0.984,0.521],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"333","confidence":[1.0,0.983,0.502],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"334","confidence":[0.719,0.256,0.06],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"335","confidence":[0.95,0.833,0.346],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"336","confidence":[0.324,0.026,0.014],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"337","confidence":[0.269,0.034,0.017],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"339","confidence":[0.998,0.964,0.741],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"340","confidence":[0.996,0.033,0.013],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"341","confidence":[0.997,0.08,0.015],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"342","confidence":[0.957,0.926,0.883],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"345","confidence":[0.817,0.658,0.569],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"346","confidence":[0.893,0.81,0.469],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"347","confidence":[0.749,0.402,0.312],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"349","confidence":[0.748,0.009,0.008],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"351","confidence":[0.927,0.619,0.159],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"352","confidence":[0.004,0.003,0.003],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"355","confidence":[0.814,0.326,0.141],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"356","confidence":[0.998,0.889,0.791],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"357","confidence":[0.99,0.989,0.385],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"358","confidence":[0.992,0.988,0.326],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"361","confidence":[0.97,0.794,0.359],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"363","confidence":[0.796,0.575,0.252],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"364","confidence":[0.32,0.103,0.019],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"365","confidence":[0.811,0.056,0.041],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"366","confidence":[1.0,0.212,0.106],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"367","confidence":[0.999,0.988,0.902],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"368","confidence":[0.999,0.995,0.857],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"369","confidence":[0.998,0.992,0.988],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"370","confidence":[1.0,0.987,0.97],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"371","confidence":[1.0,0.989,0.988],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"372","confidence":[0.995,0.137,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"375","confidence":[1.0,0.67,0.52],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"376","confidence":[0.426,0.247,0.09],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"377","confidence":[0.631,0.113,0.112],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"378","confidence":[0.582,0.541,0.256],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"379","confidence":[0.994,0.513,0.057],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"381","confidence":[0.99,0.976,0.933],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"382","confidence":[0.516,0.348,0.099],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"383","confidence":[0.801,0.687,0.67],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"389","confidence":[0.083,0.032,0.006],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"390","confidence":[0.059,0.046,0.021],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"391","confidence":[0.094,0.074,0.054],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"392","confidence":[0.037,0.01,0.007],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"393","confidence":[0.071,0.012,0.009],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"394","confidence":[0.981,0.789,0.789],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"396","confidence":[0.967,0.111,0.073],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"397","confidence":[0.938,0.062,0.025],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"398","confidence":[0.982,0.327,0.208],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"400","confidence":[0.999,0.302,0.206],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"401","confidence":[0.715,0.498,0.251],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"403","confidence":[0.091,0.047,0.047],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"404","confidence":[0.107,0.049,0.04],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"406","confidence":[0.581,0.518,0.421],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"407","confidence":[0.967,0.564,0.425],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"413","confidence":[0.98,0.966,0.957],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"414","confidence":[0.991,0.99,0.986],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"416","confidence":[0.768,0.742,0.662],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"417","confidence":[0.696,0.695,0.568],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"418","confidence":[0.956,0.776,0.602],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"420","confidence":[1.0,0.959,0.956],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"422","confidence":[0.133,0.006,0.002],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"423","confidence":[0.984,0.958,0.874],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"425","confidence":[0.9,0.504,0.033],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"426","confidence":[0.999,0.984,0.974],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"428","confidence":[0.999,0.994,0.984],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"429","confidence":[0.966,0.612,0.459],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"430","confidence":[0.955,0.947,0.922],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"432","confidence":[0.999,0.938,0.686],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"434","confidence":[0.993,0.983,0.964],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"435","confidence":[0.989,0.986,0.967],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"441","confidence":[0.992,0.963,0.903],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"444","confidence":[0.998,0.882,0.846],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"445","confidence":[0.985,0.902,0.869],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"447","confidence":[0.999,0.998,0.996],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"448","confidence":[0.999,0.998,0.997],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"449","confidence":[0.996,0.991,0.948],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"453","confidence":[0.994,0.212,0.042],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"454","confidence":[0.989,0.019,0.012],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"455","confidence":[0.974,0.919,0.875],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"456","confidence":[0.943,0.803,0.719],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"458","confidence":[0.999,0.972,0.958],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"461","confidence":[0.908,0.007,0.003],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"463","confidence":[0.944,0.902,0.858],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"466","confidence":[0.981,0.962,0.946],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"469","confidence":[0.295,0.009,0.007],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"470","confidence":[0.998,0.982,0.969],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"472","confidence":[0.72,0.555,0.523],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"473","confidence":[0.351,0.023,0.018],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"474","confidence":[0.13,0.074,0.068],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"479","confidence":[0.994,0.833,0.503],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"480","confidence":[0.997,0.884,0.473],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"481","confidence":[0.682,0.563,0.452],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"482","confidence":[0.998,0.947,0.943],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"483","confidence":[0.996,0.99,0.872],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"484","confidence":[0.974,0.887,0.882],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"485","confidence":[0.967,0.962,0.894],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"486","confidence":[0.95,0.823,0.733],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"487","confidence":[0.952,0.946,0.883],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"488","confidence":[0.989,0.498,0.229],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"489","confidence":[0.917,0.793,0.764],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"490","confidence":[0.999,0.5,0.364],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"492","confidence":[0.981,0.656,0.604],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"493","confidence":[0.343,0.164,0.047],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"494","confidence":[0.003,0.002,0.002],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"495","confidence":[0.821,0.075,0.012],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"498","confidence":[0.848,0.086,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"499","confidence":[0.966,0.502,0.135],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"500","confidence":[0.984,0.681,0.123],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"504","confidence":[0.23,0.054,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"505","confidence":[1.0,0.926,0.46],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"506","confidence":[0.55,0.262,0.218],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"509","confidence":[0.522,0.163,0.036],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"515","confidence":[0.999,0.963,0.951],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"523","confidence":[0.986,0.92,0.47],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"524","confidence":[0.987,0.927,0.525],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"526","confidence":[0.836,0.093,0.044],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"529","confidence":[1.0,1.0,0.999],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"530","confidence":[0.994,0.994,0.992],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"531","confidence":[0.999,0.999,0.998],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"537","confidence":[0.932,0.2,0.046],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"541","confidence":[0.574,0.321,0.321],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"542","confidence":[0.968,0.046,0.005],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"545","confidence":[0.49,0.149,0.103],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"547","confidence":[1.0,0.999,0.991],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"548","confidence":[0.978,0.951,0.892],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"550","confidence":[0.901,0.38,0.179],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"553","confidence":[0.998,0.881,0.732],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"555","confidence":[0.936,0.866,0.611],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"557","confidence":[0.803,0.614,0.609],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"559","confidence":[0.548,0.541,0.087],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"562","confidence":[0.88,0.005,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"563","confidence":[0.974,0.643,0.494],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"564","confidence":[0.971,0.679,0.541],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"565","confidence":[0.998,0.194,0.073],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"566","confidence":[0.999,0.108,0.052],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"568","confidence":[0.989,0.961,0.959],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"570","confidence":[1.0,0.941,0.879],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"571","confidence":[1.0,0.913,0.432],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"572","confidence":[0.112,0.099,0.066],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"573","confidence":[0.708,0.134,0.017],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"574","confidence":[0.751,0.114,0.048],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"576","confidence":[0.991,0.975,0.822],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"579","confidence":[0.977,0.976,0.892],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"580","confidence":[0.975,0.933,0.808],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"582","confidence":[0.994,0.901,0.131],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"584","confidence":[0.996,0.972,0.964],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"585","confidence":[0.596,0.02,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"588","confidence":[0.644,0.2,0.148],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"590","confidence":[1.0,0.33,0.177],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"591","confidence":[1.0,0.574,0.488],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"592","confidence":[1.0,0.579,0.494],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"594","confidence":[0.998,0.792,0.438],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"595","confidence":[1.0,0.389,0.086],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"600","confidence":[0.983,0.96,0.864],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"601","confidence":[0.995,0.964,0.919],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"602","confidence":[0.96,0.555,0.335],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"603","confidence":[0.987,0.955,0.743],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"606","confidence":[0.873,0.612,0.189],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"607","confidence":[0.975,0.442,0.374],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"609","confidence":[0.116,0.095,0.072],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"610","confidence":[0.662,0.108,0.069],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"611","confidence":[0.718,0.633,0.356],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"612","confidence":[1.0,0.047,0.046],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"614","confidence":[1.0,0.561,0.126],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"615","confidence":[1.0,0.729,0.154],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"616","confidence":[0.104,0.042,0.036],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"617","confidence":[0.05,0.049,0.015],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"618","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"620","confidence":[0.065,0.028,0.011],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"621","confidence":[0.999,0.866,0.209],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"622","confidence":[0.988,0.955,0.684],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"624","confidence":[0.999,0.973,0.955],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"625","confidence":[0.999,0.995,0.99],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"626","confidence":[1.0,0.665,0.615],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"631","confidence":[1.0,0.981,0.892],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"632","confidence":[0.942,0.79,0.068],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"633","confidence":[0.897,0.253,0.057],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"635","confidence":[0.999,0.925,0.802],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"638","confidence":[0.296,0.075,0.058],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"640","confidence":[0.738,0.619,0.161],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"642","confidence":[0.982,0.319,0.028],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"643","confidence":[0.312,0.134,0.097],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"645","confidence":[0.983,0.974,0.973],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"646","confidence":[0.988,0.988,0.981],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"647","confidence":[0.897,0.159,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"648","confidence":[0.976,0.153,0.011],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"650","confidence":[0.013,0.001,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"651","confidence":[0.94,0.808,0.785],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"652","confidence":[0.965,0.859,0.798],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"653","confidence":[0.999,0.999,0.955],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"654","confidence":[0.037,0.032,0.023],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"655","confidence":[0.045,0.025,0.024],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"657","confidence":[0.684,0.084,0.033],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"658","confidence":[0.608,0.302,0.253],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"661","confidence":[0.997,0.299,0.177],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"662","confidence":[0.99,0.042,0.033],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"663","confidence":[0.992,0.93,0.855],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"665","confidence":[1.0,0.925,0.764],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"666","confidence":[1.0,0.941,0.916],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"667","confidence":[0.998,0.997,0.984],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"668","confidence":[0.998,0.976,0.974],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"669","confidence":[0.985,0.974,0.97],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"670","confidence":[0.977,0.34,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"671","confidence":[0.979,0.311,0.08],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"672","confidence":[0.997,0.985,0.974],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"673","confidence":[1.0,0.898,0.729],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"676","confidence":[0.486,0.451,0.345],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"677","confidence":[0.46,0.354,0.346],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"679","confidence":[0.983,0.97,0.835],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"680","confidence":[0.963,0.959,0.903],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"681","confidence":[0.997,0.951,0.94],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"682","confidence":[0.987,0.983,0.955],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"683","confidence":[0.988,0.944,0.875],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"685","confidence":[0.926,0.905,0.867],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"686","confidence":[0.088,0.053,0.036],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"687","confidence":[0.168,0.115,0.092],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"688","confidence":[0.764,0.356,0.31],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"689","confidence":[0.623,0.259,0.207],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"694","confidence":[0.994,0.951,0.917],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"696","confidence":[0.81,0.635,0.029],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"698","confidence":[0.902,0.782,0.731],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"703","confidence":[0.948,0.665,0.211],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"704","confidence":[1.0,0.987,0.927],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"705","confidence":[0.965,0.944,0.112],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"709","confidence":[0.996,0.908,0.072],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"710","confidence":[0.97,0.878,0.36],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"713","confidence":[0.851,0.542,0.46],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"714","confidence":[0.94,0.637,0.472],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"717","confidence":[0.274,0.235,0.099],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"724","confidence":[0.999,0.102,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"726","confidence":[0.982,0.886,0.569],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"730","confidence":[0.974,0.962,0.939],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"732","confidence":[0.266,0.132,0.088],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"733","confidence":[0.207,0.203,0.059],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"734","confidence":[1.0,0.995,0.818],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"736","confidence":[0.995,0.852,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"737","confidence":[0.986,0.946,0.778],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"738","confidence":[0.995,0.982,0.932],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"739","confidence":[0.209,0.126,0.086],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"740","confidence":[1.0,0.993,0.987],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"745","confidence":[0.923,0.614,0.122],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"746","confidence":[0.994,0.978,0.735],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"747","confidence":[0.912,0.009,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"748","confidence":[0.986,0.021,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"749","confidence":[0.984,0.975,0.973],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"751","confidence":[0.999,0.561,0.307],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"752","confidence":[0.996,0.458,0.358],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"753","confidence":[0.083,0.061,0.055],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"755","confidence":[0.852,0.586,0.418],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"757","confidence":[0.949,0.748,0.274],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"758","confidence":[0.994,0.798,0.022],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"760","confidence":[0.994,0.981,0.977],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"761","confidence":[0.997,0.988,0.986],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"762","confidence":[0.809,0.712,0.346],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"764","confidence":[0.983,0.952,0.49],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"765","confidence":[0.983,0.811,0.327],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"766","confidence":[0.968,0.8,0.247],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"767","confidence":[0.79,0.34,0.164],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"771","confidence":[0.776,0.744,0.425],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"772","confidence":[0.986,0.017,0.017],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"774","confidence":[0.953,0.65,0.347],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"776","confidence":[0.945,0.606,0.205],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"777","confidence":[0.968,0.861,0.634],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"778","confidence":[1.0,0.993,0.979],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"779","confidence":[1.0,0.997,0.979],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"780","confidence":[0.999,0.998,0.99],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"782","confidence":[0.992,0.987,0.986],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"787","confidence":[0.999,0.072,0.07],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"788","confidence":[1.0,0.325,0.243],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"789","confidence":[0.959,0.942,0.921],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"790","confidence":[0.836,0.835,0.683],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"791","confidence":[0.996,0.983,0.978],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"792","confidence":[0.999,0.572,0.203],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"795","confidence":[0.99,0.901,0.9],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"797","confidence":[0.998,0.992,0.988],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"798","confidence":[0.771,0.674,0.142],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"799","confidence":[0.581,0.442,0.221],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"801","confidence":[1.0,0.089,0.064],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"802","confidence":[1.0,0.954,0.937],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"803","confidence":[1.0,0.152,0.052],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"804","confidence":[1.0,0.832,0.714],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"807","confidence":[0.976,0.042,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"810","confidence":[0.567,0.545,0.085],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"812","confidence":[0.963,0.086,0.079],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"813","confidence":[0.999,0.502,0.435],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"815","confidence":[0.997,0.499,0.163],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"816","confidence":[0.994,0.855,0.213],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"817","confidence":[0.005,0.003,0.002],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"818","confidence":[0.025,0.018,0.015],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"822","confidence":[1.0,0.999,0.98],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"825","confidence":[0.999,0.997,0.954],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"826","confidence":[0.973,0.913,0.755],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"828","confidence":[0.897,0.79,0.438],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"835","confidence":[0.911,0.612,0.439],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"838","confidence":[0.487,0.475,0.403],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"840","confidence":[0.908,0.297,0.168],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"841","confidence":[0.924,0.74,0.101],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"844","confidence":[0.999,0.981,0.978],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"846","confidence":[0.853,0.538,0.344],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"848","confidence":[0.969,0.88,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"853","confidence":[0.978,0.289,0.054],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"854","confidence":[0.998,0.622,0.407],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"855","confidence":[0.998,0.982,0.829],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"856","confidence":[0.968,0.523,0.517],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"857","confidence":[0.713,0.661,0.475],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"858","confidence":[0.84,0.513,0.293],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"860","confidence":[0.97,0.211,0.088],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"861","confidence":[0.955,0.126,0.05],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"863","confidence":[0.998,0.998,0.977],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"866","confidence":[0.997,0.962,0.849],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"867","confidence":[0.98,0.065,0.045],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"871","confidence":[0.896,0.345,0.334],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"876","confidence":[0.811,0.668,0.335],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"877","confidence":[0.244,0.113,0.076],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"881","confidence":[0.992,0.117,0.04],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"883","confidence":[0.997,0.176,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"884","confidence":[0.996,0.135,0.043],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"885","confidence":[0.999,0.977,0.403],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"886","confidence":[0.996,0.991,0.891],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"890","confidence":[0.31,0.09,0.013],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"891","confidence":[0.264,0.099,0.013],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"893","confidence":[0.998,0.898,0.473],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"894","confidence":[0.916,0.305,0.262],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"895","confidence":[0.008,0.001,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"896","confidence":[0.088,0.084,0.042],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"897","confidence":[0.132,0.11,0.09],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"898","confidence":[0.989,0.703,0.651],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"899","confidence":[0.989,0.482,0.479],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"900","confidence":[0.977,0.908,0.278],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"901","confidence":[0.999,0.882,0.873],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"902","confidence":[0.999,0.954,0.927],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"908","confidence":[1.0,0.972,0.966],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"909","confidence":[0.95,0.276,0.167],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"910","confidence":[0.964,0.836,0.125],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"912","confidence":[0.989,0.987,0.643],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"916","confidence":[0.001,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"917","confidence":[1.0,0.963,0.751],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"919","confidence":[0.999,0.981,0.878],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"923","confidence":[1.0,0.075,0.036],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"925","confidence":[0.998,0.225,0.137],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"926","confidence":[0.906,0.268,0.005],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"927","confidence":[0.925,0.358,0.004],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"928","confidence":[0.986,0.882,0.646],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"929","confidence":[0.992,0.905,0.821],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"930","confidence":[0.977,0.062,0.025],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"933","confidence":[0.96,0.014,0.003],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"934","confidence":[0.036,0.005,0.003],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"935","confidence":[0.975,0.689,0.688],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"938","confidence":[0.859,0.802,0.798],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"939","confidence":[0.977,0.954,0.881],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"940","confidence":[0.922,0.441,0.031],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"941","confidence":[0.977,0.284,0.015],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"942","confidence":[0.916,0.607,0.459],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"944","confidence":[1.0,0.72,0.696],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"945","confidence":[0.998,0.721,0.143],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"946","confidence":[0.999,0.808,0.271],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"949","confidence":[0.048,0.029,0.028],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"951","confidence":[0.981,0.954,0.943],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"952","confidence":[1.0,0.997,0.631],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"953","confidence":[1.0,0.998,0.851],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"954","confidence":[1.0,0.998,0.76],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"955","confidence":[0.984,0.975,0.928],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"959","confidence":[0.989,0.046,0.022],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"962","confidence":[0.997,0.991,0.919],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"963","confidence":[0.968,0.929,0.898],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"964","confidence":[0.986,0.951,0.95],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"965","confidence":[0.997,0.355,0.354],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"969","confidence":[0.936,0.889,0.721],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"970","confidence":[0.848,0.824,0.682],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"972","confidence":[0.998,0.998,0.997],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"973","confidence":[0.998,0.997,0.996],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"976","confidence":[0.825,0.7,0.564],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"977","confidence":[0.985,0.864,0.774],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"978","confidence":[0.882,0.579,0.444],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"979","confidence":[1.0,0.978,0.969],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"980","confidence":[0.879,0.82,0.304],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"984","confidence":[0.997,0.982,0.958],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"988","confidence":[0.765,0.39,0.364],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"989","confidence":[0.924,0.25,0.109],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"990","confidence":[0.999,0.731,0.45],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"992","confidence":[0.538,0.06,0.029],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"994","confidence":[0.992,0.952,0.678],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"996","confidence":[0.793,0.387,0.117],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"997","confidence":[0.985,0.829,0.543],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"998","confidence":[0.988,0.573,0.437],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"999","confidence":[0.998,0.896,0.579],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1000","confidence":[0.986,0.982,0.315],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1001","confidence":[0.923,0.311,0.077],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1002","confidence":[0.956,0.935,0.832],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1003","confidence":[1.0,0.998,0.99],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"1004","confidence":[0.997,0.996,0.989],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"1005","confidence":[1.0,0.999,0.996],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"1006","confidence":[1.0,0.541,0.481],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1008","confidence":[0.06,0.005,0.003],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1009","confidence":[0.996,0.757,0.489],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1011","confidence":[1.0,0.942,0.901],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1015","confidence":[0.984,0.058,0.043],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1016","confidence":[0.923,0.04,0.03],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1018","confidence":[0.986,0.976,0.449],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1023","confidence":[0.998,0.844,0.755],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1025","confidence":[0.668,0.28,0.204],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1026","confidence":[0.996,0.987,0.689],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1027","confidence":[0.996,0.944,0.918],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1028","confidence":[1.0,0.993,0.969],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1030","confidence":[0.999,0.994,0.968],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1031","confidence":[0.975,0.239,0.14],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1032","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1033","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1034","confidence":[0.99,0.969,0.867],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1035","confidence":[0.994,0.98,0.916],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1036","confidence":[0.999,0.994,0.946],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1037","confidence":[0.999,0.983,0.977],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1038","confidence":[0.999,0.97,0.957],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1040","confidence":[0.998,0.987,0.986],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1042","confidence":[0.929,0.739,0.38],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1043","confidence":[1.0,0.985,0.977],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1044","confidence":[0.998,0.94,0.648],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1045","confidence":[0.997,0.83,0.638],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1046","confidence":[0.922,0.863,0.615],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1047","confidence":[0.832,0.795,0.6],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1048","confidence":[0.131,0.066,0.047],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1050","confidence":[0.999,0.998,0.965],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1052","confidence":[0.311,0.082,0.009],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1053","confidence":[0.131,0.084,0.009],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1054","confidence":[0.985,0.786,0.624],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1055","confidence":[0.997,0.141,0.136],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1056","confidence":[0.996,0.25,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1058","confidence":[0.539,0.064,0.053],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1065","confidence":[0.992,0.091,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1067","confidence":[0.183,0.095,0.074],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1068","confidence":[0.116,0.101,0.067],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1069","confidence":[0.991,0.016,0.012],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1070","confidence":[0.071,0.041,0.002],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1072","confidence":[0.999,0.63,0.428],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1073","confidence":[0.998,0.607,0.566],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1074","confidence":[0.843,0.319,0.307],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1075","confidence":[0.588,0.405,0.298],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1081","confidence":[0.693,0.411,0.389],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1084","confidence":[0.913,0.83,0.416],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1085","confidence":[0.893,0.869,0.497],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1087","confidence":[0.999,0.256,0.008],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1090","confidence":[0.998,0.94,0.101],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1091","confidence":[0.878,0.352,0.338],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1096","confidence":[0.999,0.998,0.998],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1097","confidence":[0.941,0.405,0.355],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1098","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1101","confidence":[0.533,0.086,0.082],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1102","confidence":[0.666,0.252,0.072],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1103","confidence":[0.999,0.696,0.277],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1105","confidence":[0.995,0.008,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1106","confidence":[0.982,0.005,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1109","confidence":[0.001,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1111","confidence":[0.26,0.179,0.088],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1112","confidence":[0.637,0.235,0.136],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1114","confidence":[0.934,0.003,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1115","confidence":[0.999,0.99,0.174],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1118","confidence":[0.473,0.218,0.022],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1119","confidence":[0.999,0.999,0.997],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"1120","confidence":[0.999,0.999,0.998],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"1125","confidence":[0.975,0.083,0.027],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1126","confidence":[0.995,0.41,0.064],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1127","confidence":[0.968,0.826,0.475],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1128","confidence":[0.993,0.992,0.989],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1133","confidence":[0.999,0.129,0.129],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1134","confidence":[0.994,0.92,0.61],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1135","confidence":[0.996,0.947,0.914],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1136","confidence":[0.997,0.98,0.869],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1138","confidence":[0.908,0.843,0.8],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1139","confidence":[0.313,0.25,0.059],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1141","confidence":[0.891,0.172,0.029],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1142","confidence":[0.897,0.156,0.149],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1145","confidence":[0.998,0.983,0.061],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1148","confidence":[0.025,0.023,0.019],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1153","confidence":[1.0,0.901,0.285],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1156","confidence":[0.463,0.06,0.034],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1157","confidence":[0.946,0.004,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1158","confidence":[0.994,0.321,0.045],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1159","confidence":[0.766,0.095,0.059],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1161","confidence":[0.13,0.098,0.093],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1162","confidence":[0.992,0.003,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1164","confidence":[0.962,0.952,0.865],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1165","confidence":[0.939,0.88,0.801],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1166","confidence":[0.673,0.453,0.1],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1168","confidence":[0.603,0.492,0.077],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1169","confidence":[0.981,0.963,0.96],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1170","confidence":[0.998,0.966,0.925],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1171","confidence":[1.0,0.887,0.82],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1173","confidence":[0.998,0.294,0.288],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1174","confidence":[0.024,0.003,0.002],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1176","confidence":[1.0,0.541,0.485],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1177","confidence":[1.0,0.475,0.256],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1178","confidence":[0.73,0.063,0.061],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1181","confidence":[0.998,0.997,0.992],"relevant":[true,true,true]} +{"split":"train","index":"answerable","query_id":"1182","confidence":[0.998,0.998,0.994],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1183","confidence":[0.946,0.52,0.46],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1184","confidence":[0.911,0.118,0.01],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1186","confidence":[0.028,0.012,0.009],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1188","confidence":[0.807,0.689,0.188],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1190","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1193","confidence":[0.339,0.263,0.259],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1195","confidence":[0.722,0.018,0.014],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1205","confidence":[0.995,0.414,0.321],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1206","confidence":[0.756,0.053,0.009],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1208","confidence":[0.308,0.014,0.012],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1209","confidence":[0.781,0.774,0.155],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1210","confidence":[0.904,0.091,0.055],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1211","confidence":[0.798,0.096,0.05],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1212","confidence":[0.65,0.558,0.546],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1214","confidence":[0.691,0.622,0.521],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1215","confidence":[0.861,0.7,0.2],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1218","confidence":[0.993,0.012,0.006],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1219","confidence":[0.995,0.993,0.935],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"1220","confidence":[0.923,0.882,0.765],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1223","confidence":[0.173,0.045,0.029],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1224","confidence":[0.997,0.997,0.997],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1227","confidence":[0.995,0.109,0.043],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1228","confidence":[0.994,0.021,0.016],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1229","confidence":[0.863,0.339,0.239],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1230","confidence":[0.997,0.593,0.054],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1231","confidence":[0.995,0.047,0.047],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1234","confidence":[0.986,0.435,0.381],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1235","confidence":[0.84,0.416,0.367],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1236","confidence":[0.916,0.78,0.748],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1237","confidence":[0.53,0.412,0.38],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1238","confidence":[0.295,0.185,0.102],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1239","confidence":[0.964,0.691,0.684],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1244","confidence":[0.096,0.091,0.038],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1246","confidence":[0.001,0.001,0.0],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1247","confidence":[0.341,0.015,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1248","confidence":[0.998,0.335,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1249","confidence":[0.963,0.003,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1253","confidence":[0.179,0.044,0.02],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1254","confidence":[0.902,0.232,0.081],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1255","confidence":[0.197,0.156,0.105],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1257","confidence":[0.535,0.286,0.132],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1258","confidence":[0.028,0.018,0.005],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1260","confidence":[0.942,0.68,0.501],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1261","confidence":[0.915,0.912,0.816],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1263","confidence":[0.998,0.994,0.818],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1265","confidence":[0.995,0.964,0.897],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1268","confidence":[0.988,0.715,0.672],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1269","confidence":[0.998,0.17,0.018],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1275","confidence":[0.997,0.988,0.973],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1276","confidence":[0.99,0.937,0.794],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1284","confidence":[0.997,0.058,0.004],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1288","confidence":[0.995,0.324,0.154],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1289","confidence":[0.806,0.014,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1291","confidence":[0.94,0.895,0.697],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1293","confidence":[0.989,0.362,0.114],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1294","confidence":[0.881,0.881,0.856],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1295","confidence":[0.731,0.007,0.007],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1297","confidence":[0.992,0.891,0.202],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1300","confidence":[0.111,0.019,0.003],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1302","confidence":[0.039,0.012,0.007],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1304","confidence":[0.005,0.003,0.001],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1305","confidence":[0.017,0.0,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1306","confidence":[0.999,0.998,0.998],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1307","confidence":[0.566,0.452,0.412],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1308","confidence":[0.655,0.497,0.212],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1309","confidence":[0.098,0.069,0.016],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1310","confidence":[0.201,0.153,0.073],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1311","confidence":[0.889,0.596,0.535],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1312","confidence":[0.99,0.966,0.943],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1314","confidence":[0.025,0.021,0.004],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1315","confidence":[0.021,0.017,0.004],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1322","confidence":[0.962,0.4,0.399],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1323","confidence":[0.989,0.415,0.307],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1324","confidence":[0.964,0.433,0.016],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1325","confidence":[0.751,0.617,0.294],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1327","confidence":[0.637,0.512,0.14],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1328","confidence":[0.987,0.958,0.226],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1330","confidence":[0.917,0.642,0.55],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1331","confidence":[0.69,0.657,0.542],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1333","confidence":[1.0,0.288,0.187],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1334","confidence":[0.99,0.61,0.561],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1340","confidence":[0.995,0.001,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1341","confidence":[0.998,0.001,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1342","confidence":[0.334,0.074,0.046],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1345","confidence":[0.975,0.926,0.916],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1346","confidence":[0.993,0.984,0.974],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1347","confidence":[0.997,0.991,0.958],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1348","confidence":[0.999,0.997,0.96],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1349","confidence":[0.988,0.795,0.404],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1350","confidence":[0.993,0.971,0.641],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1351","confidence":[0.861,0.614,0.59],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1353","confidence":[0.987,0.948,0.142],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1355","confidence":[0.965,0.832,0.802],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1356","confidence":[0.996,0.016,0.006],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1360","confidence":[0.998,0.65,0.6],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1361","confidence":[0.969,0.955,0.907],"relevant":[true,false,true]} +{"split":"train","index":"answerable","query_id":"1364","confidence":[0.002,0.002,0.0],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1366","confidence":[0.476,0.026,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1367","confidence":[0.927,0.804,0.641],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1371","confidence":[0.997,0.535,0.433],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1372","confidence":[0.995,0.009,0.001],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1373","confidence":[0.996,0.007,0.003],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1374","confidence":[0.013,0.009,0.002],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1375","confidence":[0.009,0.006,0.0],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1376","confidence":[0.993,0.983,0.971],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1378","confidence":[0.772,0.41,0.174],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1380","confidence":[0.977,0.892,0.833],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1381","confidence":[0.997,0.782,0.702],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1383","confidence":[0.928,0.7,0.69],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1386","confidence":[0.973,0.533,0.117],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1387","confidence":[0.687,0.623,0.383],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1390","confidence":[1.0,0.986,0.127],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1391","confidence":[0.998,0.09,0.056],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1392","confidence":[1.0,0.97,0.956],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1393","confidence":[0.897,0.654,0.592],"relevant":[false,true,false]} +{"split":"train","index":"answerable","query_id":"1394","confidence":[0.989,0.952,0.904],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1397","confidence":[0.043,0.022,0.021],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1398","confidence":[0.99,0.987,0.985],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1400","confidence":[0.947,0.946,0.089],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1401","confidence":[0.89,0.779,0.657],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1402","confidence":[0.443,0.14,0.028],"relevant":[false,false,false]} +{"split":"train","index":"answerable","query_id":"1403","confidence":[0.99,0.897,0.735],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1404","confidence":[0.994,0.91,0.703],"relevant":[true,true,false]} +{"split":"train","index":"answerable","query_id":"1405","confidence":[0.82,0.329,0.262],"relevant":[false,false,true]} +{"split":"train","index":"answerable","query_id":"1406","confidence":[0.991,0.3,0.005],"relevant":[true,false,false]} +{"split":"train","index":"answerable","query_id":"1407","confidence":[0.815,0.593,0.465],"relevant":[false,false,true]} +{"split":"train","index":"unanswerable","query_id":"0","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"2","confidence":[0.043,0.022,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"4","confidence":[0.982,0.777,0.612],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"6","confidence":[0.838,0.57,0.543],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"9","confidence":[0.03,0.008,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"10","confidence":[0.707,0.045,0.011],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"11","confidence":[0.797,0.054,0.018],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"12","confidence":[0.114,0.025,0.019],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"14","confidence":[0.919,0.597,0.018],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"15","confidence":[0.381,0.315,0.183],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"17","confidence":[0.739,0.509,0.169],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"18","confidence":[0.981,0.799,0.73],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"19","confidence":[0.313,0.11,0.094],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"20","confidence":[0.311,0.089,0.087],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"21","confidence":[0.389,0.202,0.053],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"22","confidence":[0.499,0.456,0.349],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"24","confidence":[0.884,0.774,0.647],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"25","confidence":[0.834,0.736,0.611],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"26","confidence":[0.052,0.046,0.018],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"27","confidence":[0.083,0.076,0.038],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"28","confidence":[0.829,0.74,0.654],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"30","confidence":[0.982,0.906,0.629],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"32","confidence":[0.008,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"34","confidence":[0.989,0.975,0.97],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"35","confidence":[0.986,0.941,0.901],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"37","confidence":[0.98,0.948,0.853],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"39","confidence":[0.003,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"40","confidence":[0.002,0.002,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"41","confidence":[0.3,0.188,0.035],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"43","confidence":[0.962,0.248,0.233],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"44","confidence":[0.235,0.126,0.116],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"45","confidence":[0.208,0.154,0.12],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"46","confidence":[0.404,0.311,0.302],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"47","confidence":[0.242,0.081,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"52","confidence":[0.975,0.973,0.957],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"55","confidence":[0.999,0.981,0.98],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"58","confidence":[0.832,0.652,0.352],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"60","confidence":[0.967,0.924,0.911],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"61","confidence":[0.994,0.965,0.924],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"62","confidence":[0.803,0.189,0.01],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"63","confidence":[0.992,0.985,0.868],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"64","confidence":[0.983,0.982,0.91],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"66","confidence":[0.445,0.006,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"67","confidence":[0.987,0.917,0.841],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"68","confidence":[0.998,0.996,0.88],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"69","confidence":[0.998,0.996,0.996],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"71","confidence":[0.904,0.743,0.342],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"73","confidence":[0.732,0.009,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"74","confidence":[0.24,0.177,0.149],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"76","confidence":[0.979,0.24,0.165],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"77","confidence":[0.414,0.059,0.029],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"78","confidence":[0.704,0.587,0.195],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"79","confidence":[0.995,0.919,0.86],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"80","confidence":[0.343,0.195,0.051],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"81","confidence":[0.742,0.697,0.041],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"82","confidence":[0.4,0.261,0.213],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"85","confidence":[0.514,0.435,0.266],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"86","confidence":[0.514,0.435,0.266],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"88","confidence":[0.921,0.914,0.469],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"89","confidence":[0.962,0.957,0.728],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"90","confidence":[0.962,0.957,0.728],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"91","confidence":[0.417,0.091,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"92","confidence":[0.596,0.143,0.078],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"93","confidence":[0.996,0.949,0.946],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"95","confidence":[0.083,0.005,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"96","confidence":[0.585,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"98","confidence":[0.016,0.015,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"104","confidence":[0.006,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"105","confidence":[0.017,0.002,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"106","confidence":[0.968,0.962,0.946],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"108","confidence":[1.0,0.623,0.454],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"109","confidence":[0.983,0.668,0.568],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"111","confidence":[0.999,0.998,0.997],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"112","confidence":[0.974,0.972,0.805],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"114","confidence":[0.0,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"116","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"119","confidence":[0.099,0.095,0.074],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"120","confidence":[0.311,0.014,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"121","confidence":[0.093,0.05,0.033],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"122","confidence":[0.209,0.203,0.086],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"123","confidence":[0.95,0.71,0.563],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"126","confidence":[0.754,0.496,0.135],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"134","confidence":[0.821,0.397,0.118],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"138","confidence":[0.117,0.101,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"139","confidence":[0.151,0.04,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"144","confidence":[0.785,0.55,0.327],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"149","confidence":[0.892,0.618,0.579],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"152","confidence":[0.363,0.225,0.139],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"153","confidence":[0.92,0.906,0.904],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"154","confidence":[0.914,0.905,0.903],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"155","confidence":[0.988,0.968,0.967],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"156","confidence":[0.991,0.982,0.967],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"157","confidence":[0.997,0.979,0.976],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"159","confidence":[0.995,0.995,0.989],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"160","confidence":[0.817,0.73,0.16],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"161","confidence":[0.0,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"164","confidence":[0.098,0.092,0.074],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"165","confidence":[0.408,0.357,0.247],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"166","confidence":[0.997,0.958,0.943],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"167","confidence":[0.998,0.984,0.958],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"168","confidence":[0.343,0.272,0.248],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"169","confidence":[0.678,0.427,0.311],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"172","confidence":[0.996,0.995,0.961],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"173","confidence":[0.874,0.797,0.052],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"174","confidence":[0.035,0.033,0.014],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"175","confidence":[0.13,0.111,0.042],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"176","confidence":[0.941,0.058,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"177","confidence":[0.998,0.978,0.974],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"178","confidence":[0.967,0.907,0.69],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"181","confidence":[0.801,0.074,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"182","confidence":[0.986,0.914,0.824],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"184","confidence":[0.181,0.094,0.073],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"186","confidence":[0.002,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"187","confidence":[0.002,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"189","confidence":[0.981,0.977,0.828],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"196","confidence":[0.752,0.644,0.318],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"197","confidence":[0.984,0.98,0.955],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"199","confidence":[0.14,0.069,0.056],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"200","confidence":[0.635,0.631,0.37],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"201","confidence":[0.937,0.916,0.709],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"203","confidence":[0.707,0.685,0.675],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"204","confidence":[0.557,0.13,0.091],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"205","confidence":[0.451,0.19,0.094],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"209","confidence":[0.848,0.028,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"210","confidence":[0.214,0.054,0.05],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"211","confidence":[0.056,0.01,0.008],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"214","confidence":[0.997,0.997,0.969],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"220","confidence":[0.042,0.017,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"221","confidence":[0.984,0.958,0.953],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"222","confidence":[0.99,0.927,0.825],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"223","confidence":[0.972,0.801,0.691],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"224","confidence":[1.0,1.0,0.999],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"225","confidence":[0.998,0.997,0.994],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"226","confidence":[0.999,0.999,0.999],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"227","confidence":[0.951,0.932,0.913],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"228","confidence":[0.042,0.035,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"229","confidence":[0.297,0.278,0.186],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"235","confidence":[0.015,0.008,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"241","confidence":[0.074,0.062,0.045],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"242","confidence":[0.03,0.029,0.026],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"243","confidence":[0.692,0.585,0.559],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"244","confidence":[0.436,0.054,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"245","confidence":[0.984,0.962,0.922],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"246","confidence":[0.83,0.76,0.673],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"247","confidence":[0.882,0.792,0.555],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"250","confidence":[0.305,0.269,0.175],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"251","confidence":[0.83,0.768,0.509],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"253","confidence":[0.913,0.026,0.01],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"254","confidence":[0.925,0.025,0.017],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"255","confidence":[0.803,0.04,0.03],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"256","confidence":[0.955,0.183,0.067],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"258","confidence":[0.878,0.794,0.602],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"259","confidence":[0.998,0.166,0.087],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"262","confidence":[0.97,0.939,0.927],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"263","confidence":[0.896,0.678,0.671],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"264","confidence":[0.911,0.716,0.662],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"265","confidence":[0.671,0.531,0.255],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"266","confidence":[0.106,0.006,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"267","confidence":[0.373,0.037,0.033],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"272","confidence":[0.808,0.299,0.171],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"277","confidence":[0.726,0.549,0.038],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"278","confidence":[0.554,0.333,0.011],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"280","confidence":[0.424,0.136,0.127],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"281","confidence":[0.828,0.571,0.299],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"283","confidence":[0.998,0.678,0.497],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"285","confidence":[0.039,0.009,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"286","confidence":[0.94,0.547,0.068],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"287","confidence":[0.964,0.733,0.353],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"290","confidence":[0.474,0.216,0.073],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"292","confidence":[0.128,0.042,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"293","confidence":[0.177,0.114,0.111],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"296","confidence":[0.958,0.75,0.596],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"299","confidence":[0.488,0.332,0.317],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"301","confidence":[0.214,0.089,0.073],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"304","confidence":[0.702,0.575,0.418],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"305","confidence":[0.924,0.822,0.76],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"306","confidence":[0.65,0.193,0.163],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"308","confidence":[0.963,0.634,0.501],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"309","confidence":[0.775,0.655,0.625],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"310","confidence":[0.563,0.51,0.089],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"313","confidence":[0.592,0.481,0.474],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"315","confidence":[0.919,0.312,0.129],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"316","confidence":[0.744,0.718,0.163],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"317","confidence":[0.277,0.172,0.068],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"323","confidence":[0.03,0.028,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"325","confidence":[0.183,0.066,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"326","confidence":[0.426,0.12,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"330","confidence":[0.087,0.057,0.019],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"331","confidence":[0.17,0.009,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"332","confidence":[0.984,0.865,0.708],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"333","confidence":[0.983,0.881,0.755],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"334","confidence":[0.123,0.06,0.05],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"335","confidence":[0.833,0.346,0.111],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"336","confidence":[0.026,0.007,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"337","confidence":[0.034,0.008,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"339","confidence":[0.964,0.741,0.672],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"340","confidence":[0.013,0.006,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"341","confidence":[0.015,0.013,0.01],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"342","confidence":[0.957,0.926,0.809],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"345","confidence":[0.817,0.569,0.42],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"346","confidence":[0.81,0.469,0.353],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"347","confidence":[0.402,0.275,0.254],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"349","confidence":[0.009,0.008,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"351","confidence":[0.927,0.619,0.159],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"352","confidence":[0.004,0.003,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"355","confidence":[0.326,0.141,0.008],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"356","confidence":[0.889,0.791,0.665],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"357","confidence":[0.989,0.385,0.366],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"358","confidence":[0.992,0.326,0.319],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"361","confidence":[0.359,0.182,0.096],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"363","confidence":[0.796,0.575,0.43],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"364","confidence":[0.103,0.019,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"365","confidence":[0.056,0.041,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"366","confidence":[0.212,0.106,0.067],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"367","confidence":[0.988,0.902,0.88],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"368","confidence":[0.995,0.857,0.818],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"369","confidence":[0.992,0.988,0.987],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"370","confidence":[0.987,0.97,0.969],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"371","confidence":[0.989,0.988,0.988],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"372","confidence":[0.137,0.064,0.047],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"375","confidence":[0.67,0.52,0.111],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"376","confidence":[0.426,0.09,0.059],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"377","confidence":[0.113,0.112,0.049],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"378","confidence":[0.541,0.256,0.101],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"379","confidence":[0.513,0.047,0.042],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"381","confidence":[0.872,0.778,0.405],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"382","confidence":[0.516,0.348,0.099],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"383","confidence":[0.801,0.67,0.302],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"389","confidence":[0.083,0.032,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"390","confidence":[0.059,0.046,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"391","confidence":[0.094,0.074,0.054],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"392","confidence":[0.037,0.01,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"393","confidence":[0.071,0.012,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"394","confidence":[0.789,0.789,0.555],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"396","confidence":[0.111,0.073,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"397","confidence":[0.062,0.028,0.025],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"398","confidence":[0.982,0.327,0.208],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"400","confidence":[0.302,0.206,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"401","confidence":[0.498,0.251,0.249],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"403","confidence":[0.091,0.047,0.047],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"404","confidence":[0.107,0.049,0.04],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"406","confidence":[0.581,0.421,0.326],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"407","confidence":[0.564,0.425,0.347],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"413","confidence":[0.98,0.966,0.943],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"414","confidence":[0.991,0.99,0.984],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"416","confidence":[0.768,0.742,0.662],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"417","confidence":[0.696,0.695,0.568],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"418","confidence":[0.956,0.776,0.602],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"420","confidence":[0.959,0.956,0.937],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"422","confidence":[0.133,0.002,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"423","confidence":[0.874,0.864,0.727],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"425","confidence":[0.504,0.033,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"426","confidence":[0.984,0.974,0.944],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"428","confidence":[0.994,0.984,0.968],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"429","confidence":[0.612,0.459,0.368],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"430","confidence":[0.955,0.947,0.922],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"432","confidence":[0.938,0.686,0.561],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"434","confidence":[0.993,0.983,0.964],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"435","confidence":[0.989,0.986,0.967],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"441","confidence":[0.992,0.903,0.899],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"444","confidence":[0.882,0.846,0.729],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"445","confidence":[0.902,0.869,0.848],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"447","confidence":[0.996,0.993,0.979],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"448","confidence":[0.997,0.991,0.979],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"449","confidence":[0.991,0.948,0.942],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"453","confidence":[0.212,0.042,0.029],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"454","confidence":[0.019,0.012,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"455","confidence":[0.974,0.919,0.561],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"456","confidence":[0.943,0.803,0.719],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"458","confidence":[0.972,0.949,0.896],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"461","confidence":[0.007,0.003,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"463","confidence":[0.944,0.902,0.858],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"466","confidence":[0.962,0.946,0.945],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"469","confidence":[0.295,0.007,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"470","confidence":[0.982,0.969,0.964],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"472","confidence":[0.523,0.112,0.096],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"473","confidence":[0.351,0.023,0.018],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"474","confidence":[0.194,0.13,0.074],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"479","confidence":[0.833,0.503,0.228],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"480","confidence":[0.884,0.473,0.075],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"481","confidence":[0.563,0.452,0.3],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"482","confidence":[0.947,0.943,0.877],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"483","confidence":[0.99,0.872,0.865],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"484","confidence":[0.974,0.887,0.882],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"485","confidence":[0.967,0.73,0.647],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"486","confidence":[0.95,0.823,0.733],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"487","confidence":[0.946,0.883,0.817],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"488","confidence":[0.498,0.229,0.131],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"489","confidence":[0.917,0.793,0.764],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"490","confidence":[0.5,0.364,0.342],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"492","confidence":[0.751,0.656,0.604],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"493","confidence":[0.343,0.047,0.03],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"494","confidence":[0.003,0.002,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"495","confidence":[0.075,0.012,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"498","confidence":[0.086,0.007,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"499","confidence":[0.502,0.135,0.133],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"500","confidence":[0.681,0.123,0.07],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"504","confidence":[0.054,0.01,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"505","confidence":[0.926,0.46,0.286],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"506","confidence":[0.262,0.218,0.102],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"509","confidence":[0.522,0.163,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"515","confidence":[0.963,0.951,0.94],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"523","confidence":[0.986,0.47,0.292],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"524","confidence":[0.987,0.525,0.335],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"526","confidence":[0.836,0.236,0.093],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"529","confidence":[1.0,1.0,0.999],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"530","confidence":[0.994,0.994,0.992],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"531","confidence":[0.998,0.998,0.995],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"537","confidence":[0.2,0.046,0.034],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"541","confidence":[0.574,0.202,0.089],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"542","confidence":[0.046,0.006,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"545","confidence":[0.49,0.149,0.073],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"547","confidence":[0.999,0.991,0.99],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"548","confidence":[0.892,0.762,0.54],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"550","confidence":[0.901,0.38,0.161],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"553","confidence":[0.881,0.732,0.66],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"555","confidence":[0.936,0.866,0.375],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"557","confidence":[0.609,0.427,0.096],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"559","confidence":[0.252,0.087,0.074],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"562","confidence":[0.005,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"563","confidence":[0.643,0.382,0.279],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"564","confidence":[0.679,0.541,0.306],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"565","confidence":[0.194,0.073,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"566","confidence":[0.108,0.052,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"568","confidence":[0.961,0.924,0.714],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"570","confidence":[0.941,0.879,0.6],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"571","confidence":[0.913,0.432,0.171],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"572","confidence":[0.112,0.066,0.059],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"573","confidence":[0.134,0.017,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"574","confidence":[0.114,0.048,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"576","confidence":[0.975,0.822,0.689],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"579","confidence":[0.977,0.892,0.41],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"580","confidence":[0.933,0.808,0.552],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"582","confidence":[0.901,0.131,0.09],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"584","confidence":[0.972,0.964,0.122],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"585","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"588","confidence":[0.644,0.148,0.111],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"590","confidence":[0.177,0.089,0.078],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"591","confidence":[0.574,0.488,0.409],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"592","confidence":[0.579,0.494,0.375],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"594","confidence":[0.792,0.438,0.131],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"595","confidence":[0.389,0.086,0.071],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"600","confidence":[0.96,0.864,0.622],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"601","confidence":[0.964,0.919,0.632],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"602","confidence":[0.96,0.555,0.335],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"603","confidence":[0.955,0.743,0.635],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"606","confidence":[0.873,0.612,0.189],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"607","confidence":[0.442,0.374,0.187],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"609","confidence":[0.116,0.095,0.072],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"610","confidence":[0.662,0.108,0.069],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"611","confidence":[0.633,0.356,0.339],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"612","confidence":[0.047,0.046,0.027],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"614","confidence":[0.561,0.126,0.06],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"615","confidence":[0.729,0.154,0.11],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"616","confidence":[0.104,0.042,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"617","confidence":[0.05,0.049,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"618","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"620","confidence":[0.065,0.028,0.011],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"621","confidence":[0.866,0.091,0.082],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"622","confidence":[0.988,0.955,0.684],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"624","confidence":[0.973,0.955,0.951],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"625","confidence":[0.995,0.99,0.977],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"626","confidence":[0.665,0.615,0.056],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"631","confidence":[0.981,0.892,0.827],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"632","confidence":[0.79,0.068,0.056],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"633","confidence":[0.253,0.056,0.037],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"635","confidence":[0.925,0.865,0.802],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"638","confidence":[0.296,0.058,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"640","confidence":[0.738,0.263,0.161],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"642","confidence":[0.319,0.028,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"643","confidence":[0.312,0.097,0.033],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"645","confidence":[0.983,0.943,0.907],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"646","confidence":[0.988,0.936,0.889],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"647","confidence":[0.159,0.01,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"648","confidence":[0.153,0.012,0.011],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"650","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"651","confidence":[0.94,0.808,0.785],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"652","confidence":[0.965,0.859,0.798],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"653","confidence":[0.955,0.938,0.934],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"654","confidence":[0.037,0.032,0.03],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"655","confidence":[0.045,0.025,0.024],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"657","confidence":[0.084,0.033,0.03],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"658","confidence":[0.608,0.036,0.034],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"661","confidence":[0.299,0.177,0.063],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"662","confidence":[0.042,0.033,0.02],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"663","confidence":[0.93,0.206,0.154],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"665","confidence":[0.925,0.744,0.609],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"666","confidence":[0.941,0.916,0.886],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"667","confidence":[0.984,0.978,0.975],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"668","confidence":[0.974,0.956,0.952],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"669","confidence":[0.97,0.944,0.856],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"670","confidence":[0.34,0.047,0.029],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"671","confidence":[0.311,0.08,0.023],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"672","confidence":[0.985,0.974,0.521],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"673","confidence":[0.898,0.729,0.562],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"676","confidence":[0.486,0.018,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"677","confidence":[0.346,0.024,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"679","confidence":[0.983,0.97,0.897],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"680","confidence":[0.903,0.696,0.66],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"681","confidence":[0.951,0.94,0.927],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"682","confidence":[0.955,0.915,0.9],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"683","confidence":[0.944,0.875,0.748],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"685","confidence":[0.926,0.867,0.289],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"686","confidence":[0.088,0.053,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"687","confidence":[0.168,0.115,0.073],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"688","confidence":[0.764,0.356,0.171],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"689","confidence":[0.259,0.207,0.13],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"694","confidence":[0.951,0.917,0.541],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"696","confidence":[0.81,0.029,0.014],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"698","confidence":[0.782,0.731,0.59],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"703","confidence":[0.948,0.665,0.211],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"704","confidence":[1.0,0.987,0.927],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"705","confidence":[0.944,0.112,0.065],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"709","confidence":[0.908,0.072,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"710","confidence":[0.878,0.36,0.136],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"713","confidence":[0.675,0.542,0.46],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"714","confidence":[0.637,0.472,0.415],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"717","confidence":[0.274,0.099,0.068],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"724","confidence":[0.102,0.013,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"726","confidence":[0.569,0.5,0.448],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"730","confidence":[0.974,0.939,0.465],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"732","confidence":[0.132,0.024,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"733","confidence":[0.034,0.024,0.018],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"734","confidence":[0.818,0.667,0.652],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"736","confidence":[0.852,0.047,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"737","confidence":[0.986,0.778,0.643],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"738","confidence":[0.995,0.932,0.602],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"739","confidence":[0.209,0.126,0.086],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"740","confidence":[0.993,0.987,0.986],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"745","confidence":[0.614,0.122,0.072],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"746","confidence":[0.978,0.735,0.185],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"747","confidence":[0.009,0.007,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"748","confidence":[0.021,0.01,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"749","confidence":[0.984,0.975,0.973],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"751","confidence":[0.561,0.307,0.221],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"752","confidence":[0.458,0.358,0.344],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"753","confidence":[0.083,0.061,0.055],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"755","confidence":[0.852,0.586,0.418],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"757","confidence":[0.949,0.165,0.065],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"758","confidence":[0.994,0.798,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"760","confidence":[0.981,0.977,0.97],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"761","confidence":[0.997,0.988,0.986],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"762","confidence":[0.712,0.346,0.098],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"764","confidence":[0.983,0.49,0.19],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"765","confidence":[0.983,0.327,0.178],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"766","confidence":[0.968,0.247,0.046],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"767","confidence":[0.34,0.164,0.156],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"771","confidence":[0.776,0.425,0.176],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"772","confidence":[0.017,0.017,0.013],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"774","confidence":[0.953,0.65,0.285],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"776","confidence":[0.945,0.606,0.205],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"777","confidence":[0.968,0.861,0.634],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"778","confidence":[0.993,0.978,0.973],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"779","confidence":[0.997,0.979,0.97],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"780","confidence":[0.99,0.969,0.966],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"782","confidence":[0.987,0.953,0.952],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"787","confidence":[0.072,0.07,0.069],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"788","confidence":[0.325,0.243,0.241],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"789","confidence":[0.959,0.942,0.609],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"790","confidence":[0.836,0.835,0.683],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"791","confidence":[0.996,0.983,0.67],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"792","confidence":[0.572,0.203,0.2],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"795","confidence":[0.901,0.786,0.729],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"797","confidence":[0.998,0.988,0.92],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"798","confidence":[0.771,0.142,0.065],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"799","confidence":[0.581,0.301,0.064],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"801","confidence":[0.115,0.089,0.064],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"802","confidence":[0.954,0.937,0.278],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"803","confidence":[0.152,0.092,0.052],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"804","confidence":[0.832,0.714,0.552],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"807","confidence":[0.042,0.002,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"810","confidence":[0.567,0.545,0.085],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"812","confidence":[0.111,0.086,0.079],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"813","confidence":[0.502,0.376,0.298],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"815","confidence":[0.499,0.163,0.057],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"816","confidence":[0.855,0.213,0.106],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"817","confidence":[0.005,0.003,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"818","confidence":[0.025,0.018,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"822","confidence":[1.0,0.98,0.376],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"825","confidence":[0.999,0.954,0.661],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"826","confidence":[0.973,0.913,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"828","confidence":[0.79,0.438,0.368],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"835","confidence":[0.439,0.262,0.228],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"838","confidence":[0.487,0.475,0.037],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"840","confidence":[0.297,0.238,0.168],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"841","confidence":[0.74,0.346,0.101],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"844","confidence":[0.902,0.857,0.857],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"846","confidence":[0.344,0.152,0.091],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"848","confidence":[0.88,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"853","confidence":[0.289,0.054,0.04],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"854","confidence":[0.622,0.543,0.407],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"855","confidence":[0.982,0.829,0.405],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"856","confidence":[0.968,0.523,0.517],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"857","confidence":[0.713,0.661,0.475],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"858","confidence":[0.84,0.293,0.237],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"860","confidence":[0.211,0.088,0.084],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"861","confidence":[0.126,0.05,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"863","confidence":[0.977,0.974,0.929],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"866","confidence":[0.997,0.849,0.585],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"867","confidence":[0.065,0.045,0.043],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"871","confidence":[0.896,0.345,0.334],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"876","confidence":[0.811,0.668,0.335],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"877","confidence":[0.113,0.076,0.057],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"881","confidence":[0.117,0.04,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"883","confidence":[0.176,0.047,0.034],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"884","confidence":[0.135,0.043,0.008],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"885","confidence":[0.977,0.403,0.278],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"886","confidence":[0.991,0.219,0.065],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"890","confidence":[0.09,0.013,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"891","confidence":[0.099,0.013,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"893","confidence":[0.898,0.703,0.473],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"894","confidence":[0.916,0.305,0.262],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"895","confidence":[0.001,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"896","confidence":[0.088,0.032,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"897","confidence":[0.132,0.11,0.048],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"898","confidence":[0.703,0.394,0.344],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"899","confidence":[0.482,0.479,0.102],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"900","confidence":[0.977,0.908,0.278],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"901","confidence":[0.882,0.873,0.428],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"902","confidence":[0.927,0.904,0.87],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"908","confidence":[0.972,0.966,0.888],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"909","confidence":[0.276,0.167,0.084],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"910","confidence":[0.836,0.125,0.048],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"912","confidence":[0.987,0.643,0.347],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"916","confidence":[0.002,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"917","confidence":[0.963,0.735,0.7],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"919","confidence":[0.981,0.878,0.865],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"923","confidence":[0.075,0.041,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"925","confidence":[0.225,0.137,0.092],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"926","confidence":[0.268,0.005,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"927","confidence":[0.358,0.004,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"928","confidence":[0.882,0.646,0.468],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"929","confidence":[0.905,0.821,0.672],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"930","confidence":[0.048,0.025,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"933","confidence":[0.002,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"934","confidence":[0.005,0.003,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"935","confidence":[0.689,0.688,0.306],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"938","confidence":[0.859,0.802,0.704],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"939","confidence":[0.977,0.954,0.881],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"940","confidence":[0.441,0.031,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"941","confidence":[0.284,0.015,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"942","confidence":[0.607,0.252,0.065],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"944","confidence":[0.72,0.696,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"945","confidence":[0.998,0.143,0.135],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"946","confidence":[0.999,0.271,0.046],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"949","confidence":[0.048,0.029,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"951","confidence":[0.954,0.943,0.9],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"952","confidence":[0.997,0.094,0.081],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"953","confidence":[0.998,0.851,0.719],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"954","confidence":[0.998,0.168,0.166],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"955","confidence":[0.984,0.975,0.928],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"959","confidence":[0.046,0.035,0.031],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"962","confidence":[0.991,0.835,0.276],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"963","confidence":[0.968,0.929,0.866],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"964","confidence":[0.986,0.95,0.896],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"965","confidence":[0.997,0.355,0.159],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"969","confidence":[0.936,0.889,0.189],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"970","confidence":[0.848,0.824,0.621],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"972","confidence":[0.991,0.983,0.964],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"973","confidence":[0.993,0.988,0.969],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"976","confidence":[0.825,0.7,0.564],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"977","confidence":[0.985,0.864,0.774],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"978","confidence":[0.882,0.579,0.444],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"979","confidence":[0.969,0.957,0.811],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"980","confidence":[0.304,0.07,0.042],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"984","confidence":[0.982,0.958,0.938],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"988","confidence":[0.765,0.39,0.364],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"989","confidence":[0.292,0.109,0.093],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"990","confidence":[0.731,0.45,0.101],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"992","confidence":[0.06,0.029,0.02],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"994","confidence":[0.952,0.678,0.528],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"996","confidence":[0.755,0.117,0.109],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"997","confidence":[0.543,0.511,0.385],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"998","confidence":[0.573,0.437,0.326],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"999","confidence":[0.896,0.579,0.489],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1000","confidence":[0.982,0.315,0.168],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1001","confidence":[0.311,0.077,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1002","confidence":[0.956,0.935,0.832],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1003","confidence":[0.998,0.99,0.988],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1004","confidence":[0.435,0.435,0.277],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1005","confidence":[0.76,0.711,0.703],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1006","confidence":[0.541,0.481,0.442],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1008","confidence":[0.005,0.003,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1009","confidence":[0.949,0.757,0.489],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1011","confidence":[0.942,0.901,0.853],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1015","confidence":[0.058,0.043,0.023],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1016","confidence":[0.04,0.03,0.014],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1018","confidence":[0.976,0.449,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1023","confidence":[0.755,0.418,0.409],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1025","confidence":[0.28,0.204,0.145],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1026","confidence":[0.996,0.689,0.41],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1027","confidence":[0.996,0.918,0.769],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1028","confidence":[0.969,0.956,0.929],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1030","confidence":[0.994,0.968,0.968],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1031","confidence":[0.239,0.14,0.076],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1032","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1033","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1034","confidence":[0.99,0.867,0.739],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1035","confidence":[0.994,0.916,0.807],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1036","confidence":[0.999,0.946,0.927],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1037","confidence":[0.983,0.977,0.976],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1038","confidence":[0.97,0.957,0.938],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1040","confidence":[0.998,0.987,0.986],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1042","confidence":[0.398,0.38,0.351],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1043","confidence":[0.985,0.977,0.724],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1044","confidence":[0.94,0.648,0.509],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1045","confidence":[0.83,0.638,0.442],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1046","confidence":[0.922,0.615,0.615],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1047","confidence":[0.832,0.527,0.134],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1048","confidence":[0.131,0.066,0.047],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1050","confidence":[0.999,0.965,0.081],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1052","confidence":[0.009,0.009,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1053","confidence":[0.009,0.007,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1054","confidence":[0.786,0.624,0.125],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1055","confidence":[0.181,0.136,0.112],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1056","confidence":[0.25,0.047,0.024],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1058","confidence":[0.064,0.053,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1065","confidence":[0.01,0.008,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1067","confidence":[0.183,0.095,0.074],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1068","confidence":[0.116,0.067,0.061],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1069","confidence":[0.016,0.012,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1070","confidence":[0.071,0.041,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1072","confidence":[0.63,0.228,0.189],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1073","confidence":[0.566,0.332,0.235],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1074","confidence":[0.843,0.319,0.306],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1075","confidence":[0.588,0.405,0.145],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1081","confidence":[0.411,0.389,0.271],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1084","confidence":[0.29,0.276,0.23],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1085","confidence":[0.893,0.497,0.222],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1087","confidence":[0.256,0.008,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1090","confidence":[0.94,0.101,0.093],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1091","confidence":[0.878,0.482,0.352],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1096","confidence":[0.999,0.998,0.998],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1097","confidence":[0.405,0.355,0.067],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1098","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1101","confidence":[0.086,0.082,0.07],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1102","confidence":[0.252,0.072,0.058],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1103","confidence":[0.696,0.277,0.267],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1105","confidence":[0.008,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1106","confidence":[0.005,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1109","confidence":[0.001,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1111","confidence":[0.179,0.088,0.019],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1112","confidence":[0.235,0.136,0.032],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1114","confidence":[0.003,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1115","confidence":[0.99,0.174,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1118","confidence":[0.473,0.022,0.015],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1119","confidence":[0.999,0.961,0.039],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1120","confidence":[0.998,0.642,0.12],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1125","confidence":[0.083,0.027,0.026],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1126","confidence":[0.41,0.198,0.064],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1127","confidence":[0.826,0.483,0.475],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1128","confidence":[0.993,0.989,0.975],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1133","confidence":[0.129,0.129,0.078],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1134","confidence":[0.92,0.61,0.383],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1135","confidence":[0.914,0.358,0.325],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1136","confidence":[0.98,0.869,0.584],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1138","confidence":[0.908,0.843,0.624],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1139","confidence":[0.313,0.059,0.014],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1141","confidence":[0.172,0.029,0.025],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1142","confidence":[0.156,0.149,0.093],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1145","confidence":[0.983,0.061,0.058],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1148","confidence":[0.04,0.019,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1153","confidence":[0.901,0.285,0.18],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1156","confidence":[0.034,0.015,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1157","confidence":[0.002,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1158","confidence":[0.034,0.028,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1159","confidence":[0.095,0.059,0.023],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1161","confidence":[0.13,0.098,0.093],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1162","confidence":[0.003,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1164","confidence":[0.962,0.952,0.865],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1165","confidence":[0.939,0.88,0.801],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1166","confidence":[0.673,0.07,0.054],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1168","confidence":[0.603,0.077,0.047],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1169","confidence":[0.96,0.918,0.9],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1170","confidence":[0.925,0.905,0.887],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1171","confidence":[0.887,0.817,0.761],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1173","confidence":[0.288,0.238,0.237],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1174","confidence":[0.024,0.003,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1176","confidence":[0.541,0.009,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1177","confidence":[0.256,0.032,0.02],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1178","confidence":[0.73,0.063,0.061],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1181","confidence":[0.63,0.289,0.251],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1182","confidence":[0.998,0.994,0.99],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1183","confidence":[0.52,0.46,0.027],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1184","confidence":[0.118,0.01,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1186","confidence":[0.028,0.012,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1188","confidence":[0.689,0.188,0.069],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1190","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1193","confidence":[0.096,0.032,0.025],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1195","confidence":[0.064,0.018,0.014],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1205","confidence":[0.414,0.321,0.211],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1206","confidence":[0.053,0.009,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1208","confidence":[0.308,0.014,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1209","confidence":[0.781,0.774,0.007],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1210","confidence":[0.091,0.055,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1211","confidence":[0.096,0.05,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1212","confidence":[0.558,0.153,0.125],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1214","confidence":[0.521,0.464,0.436],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1215","confidence":[0.861,0.2,0.043],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1218","confidence":[0.993,0.012,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1219","confidence":[0.993,0.799,0.717],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1220","confidence":[0.923,0.765,0.678],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1223","confidence":[0.054,0.045,0.029],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1224","confidence":[0.997,0.997,0.99],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1227","confidence":[0.109,0.017,0.009],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1228","confidence":[0.021,0.013,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1229","confidence":[0.339,0.239,0.238],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1230","confidence":[0.593,0.1,0.085],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1231","confidence":[0.047,0.047,0.038],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1234","confidence":[0.435,0.381,0.375],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1235","confidence":[0.84,0.416,0.367],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1236","confidence":[0.916,0.78,0.748],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1237","confidence":[0.53,0.412,0.38],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1238","confidence":[0.185,0.102,0.081],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1239","confidence":[0.691,0.684,0.625],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1244","confidence":[0.096,0.091,0.038],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1246","confidence":[0.003,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1247","confidence":[0.015,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1248","confidence":[0.335,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1249","confidence":[0.003,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1253","confidence":[0.179,0.044,0.011],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1254","confidence":[0.066,0.057,0.036],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1255","confidence":[0.197,0.105,0.1],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1257","confidence":[0.535,0.132,0.121],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1258","confidence":[0.028,0.018,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1260","confidence":[0.68,0.501,0.176],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1261","confidence":[0.912,0.816,0.677],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1263","confidence":[0.994,0.818,0.486],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1265","confidence":[0.964,0.897,0.634],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1268","confidence":[0.988,0.715,0.342],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1269","confidence":[0.17,0.018,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1275","confidence":[0.973,0.972,0.964],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1276","confidence":[0.99,0.794,0.77],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1284","confidence":[0.058,0.004,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1288","confidence":[0.324,0.154,0.085],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1289","confidence":[0.014,0.007,0.005],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1291","confidence":[0.94,0.751,0.697],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1293","confidence":[0.114,0.095,0.024],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1294","confidence":[0.856,0.854,0.659],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1295","confidence":[0.007,0.007,0.006],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1297","confidence":[0.202,0.012,0.01],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1300","confidence":[0.111,0.019,0.002],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1302","confidence":[0.043,0.039,0.012],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1304","confidence":[0.023,0.005,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1305","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1306","confidence":[0.999,0.998,0.996],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1307","confidence":[0.452,0.412,0.07],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1308","confidence":[0.655,0.497,0.212],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1309","confidence":[0.098,0.069,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1310","confidence":[0.153,0.073,0.024],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1311","confidence":[0.889,0.596,0.535],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1312","confidence":[0.966,0.682,0.679],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1314","confidence":[0.025,0.004,0.003],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1315","confidence":[0.021,0.004,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1322","confidence":[0.4,0.399,0.188],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1323","confidence":[0.415,0.307,0.094],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1324","confidence":[0.433,0.022,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1325","confidence":[0.751,0.617,0.294],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1327","confidence":[0.14,0.11,0.017],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1328","confidence":[0.987,0.218,0.136],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1330","confidence":[0.917,0.642,0.55],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1331","confidence":[0.69,0.657,0.542],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1333","confidence":[0.288,0.187,0.034],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1334","confidence":[0.561,0.468,0.243],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1340","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1341","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1342","confidence":[0.334,0.148,0.046],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1345","confidence":[0.975,0.926,0.924],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1346","confidence":[0.993,0.984,0.91],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1347","confidence":[0.991,0.958,0.927],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1348","confidence":[0.997,0.96,0.954],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1349","confidence":[0.795,0.404,0.146],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1350","confidence":[0.971,0.641,0.605],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1351","confidence":[0.614,0.59,0.549],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1353","confidence":[0.142,0.093,0.016],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1355","confidence":[0.832,0.802,0.505],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1356","confidence":[0.016,0.006,0.004],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1360","confidence":[0.65,0.6,0.244],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1361","confidence":[0.879,0.864,0.705],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1364","confidence":[0.002,0.002,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1366","confidence":[0.002,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1367","confidence":[0.804,0.641,0.54],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1371","confidence":[0.535,0.433,0.357],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1372","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1373","confidence":[0.007,0.001,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1374","confidence":[0.009,0.002,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1375","confidence":[0.006,0.0,0.0],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1376","confidence":[0.993,0.971,0.873],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1378","confidence":[0.41,0.174,0.137],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1380","confidence":[0.892,0.446,0.255],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1381","confidence":[0.782,0.702,0.658],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1383","confidence":[0.928,0.7,0.69],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1386","confidence":[0.117,0.037,0.02],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1387","confidence":[0.687,0.445,0.383],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1390","confidence":[0.986,0.127,0.022],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1391","confidence":[0.09,0.056,0.054],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1392","confidence":[0.97,0.956,0.929],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1393","confidence":[0.897,0.592,0.373],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1394","confidence":[0.989,0.952,0.904],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1397","confidence":[0.043,0.022,0.021],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1398","confidence":[0.99,0.987,0.914],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1400","confidence":[0.946,0.089,0.085],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1401","confidence":[0.779,0.657,0.398],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1402","confidence":[0.443,0.14,0.028],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1403","confidence":[0.735,0.439,0.198],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1404","confidence":[0.703,0.416,0.225],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1405","confidence":[0.82,0.329,0.257],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1406","confidence":[0.3,0.005,0.001],"relevant":[false,false,false]} +{"split":"train","index":"unanswerable","query_id":"1407","confidence":[0.815,0.593,0.367],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1","confidence":[0.004,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"3","confidence":[0.952,0.814,0.617],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"5","confidence":[0.569,0.085,0.007],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"13","confidence":[0.848,0.454,0.211],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"36","confidence":[0.977,0.961,0.949],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"42","confidence":[0.997,0.121,0.061],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"48","confidence":[0.328,0.3,0.23],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"49","confidence":[0.995,0.978,0.085],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"50","confidence":[0.999,0.559,0.549],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"51","confidence":[0.997,0.995,0.869],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"53","confidence":[0.995,0.985,0.984],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"54","confidence":[0.999,0.986,0.974],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"56","confidence":[0.997,0.72,0.112],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"57","confidence":[0.991,0.3,0.119],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"70","confidence":[0.997,0.996,0.995],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"72","confidence":[0.092,0.017,0.003],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"75","confidence":[0.54,0.172,0.161],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"94","confidence":[0.002,0.002,0.002],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"99","confidence":[0.493,0.055,0.037],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"100","confidence":[0.998,0.543,0.313],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"113","confidence":[0.998,0.993,0.983],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"115","confidence":[0.577,0.001,0.0],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"118","confidence":[1.0,0.998,0.837],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"124","confidence":[0.995,0.983,0.771],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"127","confidence":[0.835,0.358,0.113],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"128","confidence":[0.006,0.001,0.001],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"129","confidence":[0.999,0.001,0.001],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"130","confidence":[0.999,0.002,0.002],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"132","confidence":[0.958,0.882,0.85],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"133","confidence":[0.832,0.589,0.499],"relevant":[false,true,true]} +{"split":"test","index":"answerable","query_id":"137","confidence":[0.999,0.058,0.034],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"141","confidence":[0.997,0.751,0.679],"relevant":[true,false,true]} +{"split":"test","index":"answerable","query_id":"142","confidence":[0.997,0.697,0.683],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"143","confidence":[0.997,0.777,0.621],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"146","confidence":[0.998,0.678,0.591],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"148","confidence":[0.999,0.946,0.683],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"163","confidence":[0.999,0.999,0.963],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"171","confidence":[0.999,0.993,0.991],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"179","confidence":[0.998,0.998,0.997],"relevant":[true,true,false]} +{"split":"test","index":"answerable","query_id":"180","confidence":[0.991,0.774,0.15],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"183","confidence":[0.711,0.588,0.346],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"185","confidence":[0.987,0.945,0.916],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"198","confidence":[0.074,0.055,0.038],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"208","confidence":[0.998,0.971,0.95],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"212","confidence":[0.892,0.742,0.612],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"213","confidence":[0.999,0.997,0.989],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"216","confidence":[0.999,0.998,0.944],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"217","confidence":[1.0,0.979,0.953],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"218","confidence":[1.0,0.994,0.992],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"219","confidence":[1.0,0.997,0.995],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"230","confidence":[0.185,0.008,0.005],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"232","confidence":[0.999,0.977,0.694],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"233","confidence":[0.907,0.019,0.01],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"236","confidence":[0.961,0.048,0.019],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"237","confidence":[0.918,0.882,0.758],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"238","confidence":[0.517,0.162,0.142],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"239","confidence":[0.308,0.104,0.047],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"248","confidence":[0.999,0.216,0.173],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"249","confidence":[0.995,0.179,0.046],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"261","confidence":[0.997,0.977,0.971],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"268","confidence":[0.87,0.644,0.105],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"269","confidence":[0.873,0.084,0.005],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"274","confidence":[0.997,0.895,0.621],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"275","confidence":[0.998,0.998,0.971],"relevant":[true,false,true]} +{"split":"test","index":"answerable","query_id":"279","confidence":[0.997,0.571,0.57],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"294","confidence":[0.687,0.236,0.178],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"295","confidence":[0.996,0.98,0.968],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"298","confidence":[0.999,0.85,0.685],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"300","confidence":[0.447,0.301,0.068],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"303","confidence":[0.952,0.774,0.141],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"312","confidence":[0.728,0.644,0.112],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"314","confidence":[0.67,0.49,0.14],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"324","confidence":[0.196,0.049,0.014],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"327","confidence":[1.0,0.533,0.444],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"338","confidence":[0.999,0.99,0.654],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"343","confidence":[0.986,0.91,0.897],"relevant":[false,true,true]} +{"split":"test","index":"answerable","query_id":"350","confidence":[0.963,0.312,0.129],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"354","confidence":[0.997,0.514,0.284],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"362","confidence":[0.951,0.535,0.355],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"380","confidence":[0.996,0.411,0.277],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"384","confidence":[0.808,0.452,0.328],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"385","confidence":[0.999,0.96,0.856],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"386","confidence":[0.984,0.079,0.052],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"388","confidence":[0.069,0.009,0.005],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"399","confidence":[1.0,0.699,0.467],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"410","confidence":[0.996,0.68,0.629],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"411","confidence":[0.976,0.582,0.389],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"415","confidence":[0.987,0.985,0.978],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"421","confidence":[0.114,0.004,0.002],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"431","confidence":[0.948,0.925,0.871],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"436","confidence":[0.996,0.263,0.203],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"437","confidence":[0.728,0.715,0.604],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"439","confidence":[0.343,0.304,0.174],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"440","confidence":[0.447,0.267,0.185],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"443","confidence":[1.0,0.985,0.953],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"452","confidence":[0.786,0.53,0.478],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"475","confidence":[0.117,0.089,0.034],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"478","confidence":[0.537,0.079,0.057],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"491","confidence":[0.426,0.111,0.041],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"501","confidence":[0.95,0.522,0.053],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"502","confidence":[0.002,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"507","confidence":[0.999,0.988,0.941],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"508","confidence":[0.762,0.133,0.042],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"513","confidence":[0.987,0.908,0.476],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"514","confidence":[0.794,0.556,0.518],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"516","confidence":[0.998,0.986,0.985],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"517","confidence":[0.748,0.548,0.219],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"521","confidence":[0.904,0.17,0.057],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"525","confidence":[0.993,0.664,0.427],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"527","confidence":[0.531,0.123,0.1],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"528","confidence":[0.925,0.507,0.421],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"532","confidence":[0.729,0.03,0.018],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"533","confidence":[0.884,0.071,0.06],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"535","confidence":[0.755,0.707,0.687],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"536","confidence":[0.972,0.794,0.232],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"539","confidence":[0.999,0.705,0.522],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"540","confidence":[0.891,0.749,0.624],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"544","confidence":[0.67,0.3,0.116],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"549","confidence":[0.954,0.73,0.506],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"551","confidence":[0.938,0.458,0.195],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"552","confidence":[0.995,0.543,0.393],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"554","confidence":[0.823,0.807,0.75],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"560","confidence":[0.985,0.982,0.974],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"569","confidence":[0.993,0.992,0.926],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"575","confidence":[0.639,0.121,0.023],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"577","confidence":[0.116,0.046,0.038],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"578","confidence":[0.984,0.659,0.262],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"587","confidence":[0.561,0.143,0.103],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"589","confidence":[1.0,0.213,0.111],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"593","confidence":[0.998,0.674,0.149],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"597","confidence":[0.991,0.897,0.859],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"598","confidence":[0.991,0.987,0.969],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"613","confidence":[1.0,0.033,0.026],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"619","confidence":[0.031,0.023,0.02],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"623","confidence":[0.997,0.969,0.932],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"628","confidence":[0.958,0.908,0.353],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"636","confidence":[0.965,0.92,0.792],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"637","confidence":[0.589,0.109,0.101],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"641","confidence":[0.992,0.489,0.062],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"644","confidence":[0.988,0.57,0.151],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"649","confidence":[0.008,0.004,0.001],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"659","confidence":[0.004,0.002,0.001],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"660","confidence":[0.073,0.014,0.01],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"674","confidence":[0.996,0.82,0.595],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"684","confidence":[0.948,0.8,0.573],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"690","confidence":[0.005,0.003,0.0],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"691","confidence":[0.928,0.863,0.531],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"692","confidence":[0.86,0.023,0.007],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"693","confidence":[0.983,0.011,0.002],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"700","confidence":[0.719,0.241,0.141],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"702","confidence":[0.886,0.63,0.112],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"715","confidence":[0.903,0.682,0.664],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"716","confidence":[0.596,0.407,0.242],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"718","confidence":[0.191,0.152,0.046],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"721","confidence":[0.93,0.088,0.03],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"723","confidence":[0.999,0.15,0.124],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"727","confidence":[0.914,0.796,0.295],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"728","confidence":[0.959,0.846,0.548],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"729","confidence":[0.933,0.564,0.558],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"742","confidence":[0.943,0.176,0.049],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"743","confidence":[0.967,0.784,0.49],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"744","confidence":[1.0,0.954,0.857],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"756","confidence":[0.997,0.996,0.951],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"759","confidence":[0.998,0.994,0.993],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"768","confidence":[0.736,0.087,0.005],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"770","confidence":[0.854,0.708,0.458],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"775","confidence":[0.973,0.892,0.742],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"781","confidence":[0.998,0.997,0.99],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"783","confidence":[0.996,0.987,0.982],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"784","confidence":[0.986,0.949,0.881],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"785","confidence":[0.502,0.038,0.009],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"793","confidence":[0.676,0.456,0.113],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"800","confidence":[0.964,0.957,0.922],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"805","confidence":[1.0,0.946,0.715],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"808","confidence":[0.948,0.038,0.005],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"811","confidence":[0.887,0.566,0.316],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"814","confidence":[0.999,0.643,0.412],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"820","confidence":[0.049,0.028,0.011],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"821","confidence":[0.011,0.007,0.005],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"823","confidence":[0.998,0.992,0.911],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"830","confidence":[0.317,0.153,0.142],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"831","confidence":[0.263,0.219,0.064],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"832","confidence":[0.999,0.943,0.818],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"834","confidence":[0.91,0.021,0.009],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"837","confidence":[0.895,0.855,0.479],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"839","confidence":[0.998,0.96,0.748],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"845","confidence":[1.0,1.0,0.999],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"847","confidence":[0.979,0.571,0.261],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"852","confidence":[0.926,0.868,0.65],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"859","confidence":[0.968,0.803,0.755],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"870","confidence":[0.896,0.345,0.334],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"873","confidence":[0.521,0.519,0.487],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"879","confidence":[0.997,0.092,0.015],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"880","confidence":[0.934,0.018,0.01],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"882","confidence":[0.989,0.089,0.027],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"887","confidence":[0.008,0.007,0.002],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"903","confidence":[0.999,0.928,0.879],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"904","confidence":[1.0,0.358,0.039],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"907","confidence":[1.0,0.984,0.962],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"911","confidence":[0.986,0.841,0.713],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"913","confidence":[0.971,0.959,0.931],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"914","confidence":[0.968,0.831,0.815],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"921","confidence":[0.999,0.398,0.302],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"922","confidence":[1.0,0.068,0.04],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"936","confidence":[0.987,0.221,0.172],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"956","confidence":[0.979,0.797,0.741],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"957","confidence":[0.843,0.028,0.028],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"960","confidence":[0.996,0.085,0.046],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"967","confidence":[0.989,0.724,0.558],"relevant":[true,false,true]} +{"split":"test","index":"answerable","query_id":"971","confidence":[0.998,0.998,0.998],"relevant":[true,true,false]} +{"split":"test","index":"answerable","query_id":"975","confidence":[0.973,0.891,0.789],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"982","confidence":[0.983,0.188,0.029],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"985","confidence":[0.984,0.981,0.959],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"993","confidence":[0.935,0.881,0.425],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1012","confidence":[1.0,0.94,0.906],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1014","confidence":[0.622,0.053,0.029],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1019","confidence":[0.969,0.829,0.052],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1020","confidence":[0.496,0.309,0.094],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1021","confidence":[0.543,0.242,0.072],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1024","confidence":[0.941,0.301,0.221],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1029","confidence":[0.998,0.997,0.934],"relevant":[true,true,false]} +{"split":"test","index":"answerable","query_id":"1041","confidence":[0.998,0.985,0.984],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1049","confidence":[0.112,0.046,0.027],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1062","confidence":[0.999,0.734,0.715],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1086","confidence":[1.0,0.28,0.018],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1088","confidence":[0.987,0.956,0.887],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1089","confidence":[0.999,0.968,0.256],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1099","confidence":[0.996,0.995,0.928],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1100","confidence":[0.99,0.941,0.885],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1104","confidence":[0.998,0.689,0.228],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1107","confidence":[0.993,0.756,0.727],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1110","confidence":[0.024,0.011,0.003],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1121","confidence":[0.989,0.969,0.962],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1130","confidence":[1.0,0.96,0.954],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1132","confidence":[0.997,0.995,0.995],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1137","confidence":[0.997,0.977,0.968],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1140","confidence":[0.285,0.267,0.081],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1144","confidence":[0.995,0.973,0.099],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1146","confidence":[0.999,0.022,0.019],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1150","confidence":[0.999,0.734,0.668],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1163","confidence":[0.99,0.931,0.131],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1175","confidence":[0.123,0.095,0.019],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1179","confidence":[0.959,0.066,0.046],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1180","confidence":[0.997,0.988,0.948],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1185","confidence":[0.959,0.131,0.016],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1187","confidence":[0.888,0.823,0.507],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1191","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1194","confidence":[0.75,0.199,0.108],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1196","confidence":[0.049,0.029,0.024],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1197","confidence":[0.109,0.018,0.011],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1199","confidence":[0.978,0.824,0.082],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1200","confidence":[0.127,0.072,0.017],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1202","confidence":[0.944,0.909,0.5],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1204","confidence":[0.445,0.234,0.184],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"1207","confidence":[0.684,0.01,0.005],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1213","confidence":[0.619,0.584,0.572],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1216","confidence":[0.999,0.076,0.059],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1221","confidence":[0.193,0.148,0.103],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1225","confidence":[0.623,0.519,0.279],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1226","confidence":[0.783,0.031,0.03],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1232","confidence":[0.996,0.078,0.058],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1241","confidence":[0.991,0.464,0.317],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1245","confidence":[0.008,0.005,0.001],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1259","confidence":[0.956,0.821,0.603],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1262","confidence":[1.0,0.99,0.967],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1266","confidence":[0.995,0.992,0.989],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"1270","confidence":[0.998,0.143,0.022],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1271","confidence":[0.996,0.069,0.059],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1272","confidence":[0.11,0.054,0.037],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1273","confidence":[1.0,0.809,0.63],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1274","confidence":[0.995,0.989,0.983],"relevant":[true,true,false]} +{"split":"test","index":"answerable","query_id":"1278","confidence":[0.896,0.165,0.136],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1279","confidence":[0.848,0.197,0.131],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1280","confidence":[0.994,0.964,0.889],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1281","confidence":[0.771,0.117,0.079],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1282","confidence":[0.852,0.018,0.009],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1290","confidence":[0.993,0.105,0.1],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1292","confidence":[0.94,0.895,0.697],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1298","confidence":[1.0,0.044,0.006],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1303","confidence":[0.009,0.007,0.002],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1316","confidence":[0.585,0.124,0.116],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1319","confidence":[0.838,0.589,0.52],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1320","confidence":[0.907,0.752,0.514],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1332","confidence":[0.997,0.988,0.986],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1335","confidence":[0.998,0.975,0.961],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1336","confidence":[0.996,0.987,0.957],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1337","confidence":[0.953,0.95,0.685],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1339","confidence":[0.984,0.0,0.0],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1344","confidence":[0.791,0.765,0.754],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"1352","confidence":[0.998,0.59,0.3],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1359","confidence":[0.998,0.675,0.577],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1362","confidence":[0.003,0.001,0.001],"relevant":[false,false,false]} +{"split":"test","index":"answerable","query_id":"1363","confidence":[0.01,0.005,0.001],"relevant":[false,false,true]} +{"split":"test","index":"answerable","query_id":"1368","confidence":[0.724,0.556,0.487],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1370","confidence":[0.651,0.413,0.236],"relevant":[false,true,false]} +{"split":"test","index":"answerable","query_id":"1379","confidence":[0.989,0.97,0.954],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1382","confidence":[0.889,0.868,0.609],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1385","confidence":[0.992,0.532,0.149],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1389","confidence":[0.997,0.977,0.881],"relevant":[true,false,false]} +{"split":"test","index":"answerable","query_id":"1395","confidence":[0.055,0.046,0.022],"relevant":[true,false,false]} +{"split":"test","index":"unanswerable","query_id":"1","confidence":[0.004,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"3","confidence":[0.617,0.595,0.433],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"5","confidence":[0.085,0.007,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"13","confidence":[0.848,0.454,0.144],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"36","confidence":[0.977,0.961,0.949],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"42","confidence":[0.121,0.061,0.027],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"48","confidence":[0.328,0.3,0.01],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"49","confidence":[0.978,0.085,0.057],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"50","confidence":[0.559,0.549,0.531],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"51","confidence":[0.997,0.869,0.8],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"53","confidence":[0.985,0.984,0.931],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"54","confidence":[0.999,0.986,0.974],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"56","confidence":[0.72,0.112,0.034],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"57","confidence":[0.3,0.119,0.076],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"70","confidence":[0.997,0.996,0.995],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"72","confidence":[0.017,0.003,0.003],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"75","confidence":[0.172,0.161,0.149],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"94","confidence":[0.002,0.002,0.002],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"99","confidence":[0.055,0.037,0.026],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"100","confidence":[0.543,0.313,0.28],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"113","confidence":[0.993,0.983,0.924],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"115","confidence":[0.001,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"118","confidence":[0.998,0.837,0.705],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"124","confidence":[0.983,0.771,0.553],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"127","confidence":[0.835,0.113,0.091],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"128","confidence":[0.006,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"129","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"130","confidence":[0.002,0.002,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"132","confidence":[0.958,0.882,0.85],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"133","confidence":[0.832,0.069,0.063],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"137","confidence":[0.058,0.034,0.028],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"141","confidence":[0.751,0.167,0.034],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"142","confidence":[0.697,0.683,0.55],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"143","confidence":[0.777,0.621,0.327],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"146","confidence":[0.678,0.591,0.347],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"148","confidence":[0.946,0.683,0.291],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"163","confidence":[0.999,0.963,0.934],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"171","confidence":[0.993,0.991,0.975],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"179","confidence":[0.989,0.97,0.834],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"180","confidence":[0.774,0.15,0.015],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"183","confidence":[0.346,0.312,0.196],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"185","confidence":[0.987,0.916,0.723],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"198","confidence":[0.074,0.055,0.038],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"208","confidence":[0.971,0.95,0.801],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"212","confidence":[0.892,0.742,0.513],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"213","confidence":[0.999,0.997,0.989],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"216","confidence":[0.998,0.944,0.905],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"217","confidence":[0.979,0.953,0.953],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"218","confidence":[0.994,0.992,0.988],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"219","confidence":[0.997,0.995,0.992],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"230","confidence":[0.008,0.005,0.004],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"232","confidence":[0.999,0.694,0.003],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"233","confidence":[0.01,0.004,0.003],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"236","confidence":[0.048,0.019,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"237","confidence":[0.918,0.882,0.758],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"238","confidence":[0.517,0.162,0.142],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"239","confidence":[0.308,0.104,0.047],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"248","confidence":[0.216,0.173,0.094],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"249","confidence":[0.179,0.046,0.016],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"261","confidence":[0.997,0.971,0.929],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"268","confidence":[0.87,0.018,0.013],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"269","confidence":[0.873,0.005,0.004],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"274","confidence":[0.895,0.621,0.537],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"275","confidence":[0.998,0.959,0.951],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"279","confidence":[0.571,0.57,0.026],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"294","confidence":[0.236,0.178,0.107],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"295","confidence":[0.98,0.968,0.84],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"298","confidence":[0.85,0.753,0.685],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"300","confidence":[0.447,0.052,0.052],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"303","confidence":[0.952,0.774,0.141],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"312","confidence":[0.728,0.644,0.112],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"314","confidence":[0.67,0.49,0.14],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"324","confidence":[0.196,0.049,0.014],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"327","confidence":[0.533,0.444,0.107],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"338","confidence":[0.99,0.654,0.43],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"343","confidence":[0.986,0.947,0.869],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"350","confidence":[0.312,0.129,0.055],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"354","confidence":[0.514,0.284,0.017],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"362","confidence":[0.535,0.355,0.042],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"380","confidence":[0.411,0.277,0.119],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"384","confidence":[0.808,0.452,0.328],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"385","confidence":[0.999,0.856,0.693],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"386","confidence":[0.079,0.052,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"388","confidence":[0.069,0.009,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"399","confidence":[0.699,0.467,0.141],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"410","confidence":[0.68,0.629,0.224],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"411","confidence":[0.582,0.389,0.159],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"415","confidence":[0.987,0.985,0.976],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"421","confidence":[0.114,0.004,0.002],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"431","confidence":[0.948,0.925,0.871],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"436","confidence":[0.595,0.263,0.203],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"437","confidence":[0.728,0.715,0.604],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"439","confidence":[0.304,0.174,0.134],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"440","confidence":[0.267,0.185,0.178],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"443","confidence":[0.985,0.953,0.875],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"452","confidence":[0.786,0.478,0.396],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"475","confidence":[0.117,0.089,0.034],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"478","confidence":[0.079,0.057,0.041],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"491","confidence":[0.111,0.048,0.041],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"501","confidence":[0.522,0.053,0.051],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"502","confidence":[0.002,0.001,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"507","confidence":[0.999,0.941,0.838],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"508","confidence":[0.762,0.133,0.042],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"513","confidence":[0.987,0.476,0.23],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"514","confidence":[0.794,0.556,0.221],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"516","confidence":[0.986,0.985,0.969],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"517","confidence":[0.748,0.548,0.512],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"521","confidence":[0.17,0.057,0.054],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"525","confidence":[0.664,0.427,0.33],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"527","confidence":[0.531,0.123,0.1],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"528","confidence":[0.925,0.002,0.002],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"532","confidence":[0.03,0.018,0.008],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"533","confidence":[0.071,0.06,0.048],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"535","confidence":[0.755,0.707,0.518],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"536","confidence":[0.794,0.232,0.096],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"539","confidence":[0.705,0.663,0.522],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"540","confidence":[0.891,0.749,0.213],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"544","confidence":[0.67,0.3,0.116],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"549","confidence":[0.73,0.506,0.426],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"551","confidence":[0.938,0.458,0.135],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"552","confidence":[0.543,0.393,0.212],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"554","confidence":[0.823,0.807,0.75],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"560","confidence":[0.985,0.982,0.974],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"569","confidence":[0.992,0.926,0.878],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"575","confidence":[0.121,0.045,0.023],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"577","confidence":[0.116,0.046,0.038],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"578","confidence":[0.659,0.262,0.245],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"587","confidence":[0.561,0.103,0.079],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"589","confidence":[0.213,0.111,0.088],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"593","confidence":[0.674,0.152,0.149],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"597","confidence":[0.897,0.859,0.559],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"598","confidence":[0.987,0.949,0.918],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"613","confidence":[0.033,0.026,0.018],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"619","confidence":[0.023,0.02,0.019],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"623","confidence":[0.997,0.969,0.932],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"628","confidence":[0.908,0.353,0.328],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"636","confidence":[0.92,0.792,0.704],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"637","confidence":[0.589,0.101,0.026],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"641","confidence":[0.489,0.062,0.061],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"644","confidence":[0.57,0.151,0.141],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"649","confidence":[0.004,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"659","confidence":[0.004,0.002,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"660","confidence":[0.073,0.014,0.01],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"674","confidence":[0.82,0.235,0.134],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"684","confidence":[0.948,0.8,0.573],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"690","confidence":[0.003,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"691","confidence":[0.928,0.531,0.407],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"692","confidence":[0.023,0.007,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"693","confidence":[0.011,0.002,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"700","confidence":[0.719,0.241,0.141],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"702","confidence":[0.886,0.63,0.112],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"715","confidence":[0.903,0.682,0.664],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"716","confidence":[0.596,0.407,0.242],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"718","confidence":[0.079,0.046,0.043],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"721","confidence":[0.088,0.043,0.039],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"723","confidence":[0.15,0.124,0.062],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"727","confidence":[0.796,0.295,0.252],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"728","confidence":[0.846,0.548,0.382],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"729","confidence":[0.564,0.558,0.514],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"742","confidence":[0.176,0.049,0.048],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"743","confidence":[0.784,0.49,0.126],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"744","confidence":[1.0,0.857,0.098],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"756","confidence":[0.997,0.996,0.961],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"759","confidence":[0.994,0.993,0.99],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"768","confidence":[0.736,0.087,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"770","confidence":[0.708,0.458,0.291],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"775","confidence":[0.973,0.892,0.742],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"781","confidence":[0.998,0.99,0.954],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"783","confidence":[0.996,0.987,0.961],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"784","confidence":[0.949,0.896,0.881],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"785","confidence":[0.502,0.038,0.009],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"793","confidence":[0.456,0.113,0.095],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"800","confidence":[0.964,0.922,0.857],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"805","confidence":[0.946,0.715,0.456],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"808","confidence":[0.038,0.005,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"811","confidence":[0.566,0.316,0.202],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"814","confidence":[0.19,0.159,0.107],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"820","confidence":[0.049,0.028,0.011],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"821","confidence":[0.011,0.007,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"823","confidence":[0.992,0.911,0.56],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"830","confidence":[0.153,0.142,0.112],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"831","confidence":[0.263,0.064,0.048],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"832","confidence":[0.943,0.818,0.806],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"834","confidence":[0.91,0.021,0.009],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"837","confidence":[0.895,0.479,0.358],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"839","confidence":[0.96,0.748,0.713],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"845","confidence":[1.0,0.999,0.999],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"847","confidence":[0.571,0.261,0.052],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"852","confidence":[0.926,0.65,0.165],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"859","confidence":[0.968,0.803,0.465],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"870","confidence":[0.896,0.345,0.334],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"873","confidence":[0.521,0.519,0.487],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"879","confidence":[0.092,0.015,0.012],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"880","confidence":[0.018,0.01,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"882","confidence":[0.089,0.027,0.005],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"887","confidence":[0.008,0.007,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"903","confidence":[0.928,0.879,0.771],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"904","confidence":[0.358,0.03,0.019],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"907","confidence":[0.984,0.962,0.763],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"911","confidence":[0.841,0.713,0.438],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"913","confidence":[0.971,0.959,0.931],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"914","confidence":[0.968,0.831,0.815],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"921","confidence":[0.398,0.302,0.059],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"922","confidence":[0.068,0.04,0.031],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"936","confidence":[0.221,0.172,0.132],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"956","confidence":[0.797,0.741,0.656],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"957","confidence":[0.028,0.023,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"960","confidence":[0.085,0.046,0.035],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"967","confidence":[0.724,0.318,0.019],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"971","confidence":[0.998,0.994,0.991],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"975","confidence":[0.973,0.891,0.789],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"982","confidence":[0.188,0.029,0.025],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"985","confidence":[0.981,0.959,0.95],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"993","confidence":[0.881,0.425,0.104],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1012","confidence":[0.94,0.906,0.799],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1014","confidence":[0.053,0.029,0.01],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1019","confidence":[0.969,0.052,0.029],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1020","confidence":[0.496,0.094,0.047],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1021","confidence":[0.543,0.072,0.05],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1024","confidence":[0.301,0.221,0.17],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1029","confidence":[0.934,0.751,0.611],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1041","confidence":[0.998,0.985,0.984],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1049","confidence":[0.112,0.046,0.027],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1062","confidence":[0.734,0.715,0.393],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1086","confidence":[0.28,0.018,0.004],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1088","confidence":[0.956,0.887,0.487],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1089","confidence":[0.968,0.256,0.131],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1099","confidence":[0.996,0.995,0.928],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1100","confidence":[0.99,0.941,0.885],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1104","confidence":[0.689,0.176,0.166],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1107","confidence":[0.756,0.727,0.008],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1110","confidence":[0.024,0.011,0.003],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1121","confidence":[0.969,0.962,0.961],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1130","confidence":[0.96,0.954,0.888],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1132","confidence":[0.995,0.995,0.994],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1137","confidence":[0.977,0.968,0.873],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1140","confidence":[0.267,0.081,0.019],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1144","confidence":[0.973,0.099,0.03],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1146","confidence":[0.019,0.006,0.002],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1150","confidence":[0.734,0.668,0.657],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1163","confidence":[0.931,0.131,0.014],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1175","confidence":[0.123,0.095,0.019],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1179","confidence":[0.959,0.066,0.046],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1180","confidence":[0.988,0.948,0.914],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1185","confidence":[0.131,0.016,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1187","confidence":[0.823,0.507,0.346],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1191","confidence":[0.001,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1194","confidence":[0.199,0.108,0.015],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1196","confidence":[0.049,0.029,0.024],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1197","confidence":[0.109,0.018,0.011],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1199","confidence":[0.978,0.824,0.236],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1200","confidence":[0.127,0.072,0.017],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1202","confidence":[0.944,0.5,0.355],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1204","confidence":[0.445,0.234,0.15],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1207","confidence":[0.01,0.005,0.004],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1213","confidence":[0.619,0.584,0.572],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1216","confidence":[0.076,0.059,0.055],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1221","confidence":[0.193,0.148,0.103],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1225","confidence":[0.519,0.279,0.231],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1226","confidence":[0.783,0.031,0.03],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1232","confidence":[0.078,0.058,0.053],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1241","confidence":[0.464,0.317,0.314],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1245","confidence":[0.008,0.005,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1259","confidence":[0.821,0.603,0.132],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1262","confidence":[0.99,0.967,0.955],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1266","confidence":[0.995,0.988,0.973],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1270","confidence":[0.143,0.022,0.014],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1271","confidence":[0.069,0.059,0.023],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1272","confidence":[0.11,0.054,0.007],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1273","confidence":[0.809,0.63,0.51],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1274","confidence":[0.983,0.978,0.974],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1278","confidence":[0.896,0.264,0.165],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1279","confidence":[0.848,0.197,0.131],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1280","confidence":[0.994,0.964,0.889],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1281","confidence":[0.771,0.117,0.079],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1282","confidence":[0.018,0.009,0.008],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1290","confidence":[0.105,0.1,0.024],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1292","confidence":[0.94,0.697,0.654],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1298","confidence":[0.044,0.006,0.004],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1303","confidence":[0.007,0.002,0.001],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1316","confidence":[0.585,0.124,0.116],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1319","confidence":[0.589,0.52,0.503],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1320","confidence":[0.752,0.514,0.412],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1332","confidence":[0.997,0.988,0.986],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1335","confidence":[0.975,0.961,0.844],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1336","confidence":[0.987,0.957,0.863],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1337","confidence":[0.953,0.685,0.477],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1339","confidence":[0.0,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1344","confidence":[0.791,0.765,0.692],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1352","confidence":[0.3,0.085,0.05],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1359","confidence":[0.675,0.577,0.259],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1362","confidence":[0.003,0.001,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1363","confidence":[0.01,0.0,0.0],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1368","confidence":[0.556,0.487,0.379],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1370","confidence":[0.651,0.236,0.19],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1379","confidence":[0.954,0.699,0.591],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1382","confidence":[0.868,0.609,0.327],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1385","confidence":[0.149,0.109,0.025],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1389","confidence":[0.977,0.93,0.881],"relevant":[false,false,false]} +{"split":"test","index":"unanswerable","query_id":"1395","confidence":[0.046,0.022,0.021],"relevant":[false,false,false]} diff --git a/evaluation/cutoff_tuning.py b/evaluation/cutoff_tuning.py new file mode 100644 index 0000000..4ef2241 --- /dev/null +++ b/evaluation/cutoff_tuning.py @@ -0,0 +1,133 @@ +"""Choose the app's confidence cut-off on SciFact training claims, then check it on the test claims. + +The cut-off decides when the app says "I cannot answer". To measure both sides of that choice, +each split (train, test) is run on two indexes: + answerable all 5,183 abstracts: every claim has a right abstract in the index; + unanswerable all abstracts except the ones that answer this split's claims: no claim has one. + +Each claim goes through the app's default call (3 passages, 9 vector + 9 BM25 candidates, +reranker) with the cut-off switched off, and the reranker confidence of each kept passage is +recorded. Passages are kept in order of confidence, so the result for any cut-off t is simply the +passages with confidence >= t. + +Score of a cut-off = mean of + (a) Hit@3 on the answerable index (a claim with no passage counts as a miss), and + (b) the share of claims with no passage on the unanswerable index (saying "I cannot answer"). +The cut-off with the best score on the training claims is chosen (ties: the lower one) and then +reported on the test claims, next to the app's old 0.3. + +This score turned out to be a poor rule: refusing every claim scores 0.5, answering every claim +about 0.38, so it favours refusing. It chose 0.95, the top of the range; the app keeps 0.3. The +useful output is the whole trade-off (`sweep` in the results file), shown in the README. + + python -m evaluation.cutoff_tuning --embedder bge-small-en-v1.5 +""" + +import argparse +import json +import sys +import tempfile +import time +from pathlib import Path + +import numpy as np +import pandas as pd +from sentence_transformers import SentenceTransformer + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "src")) + +from rag_engine import EMBEDDERS, RAGEngine, load_reranker # noqa: E402 + +from evaluation.retrieval_eval import load # noqa: E402 + +CUTOFFS = [0.0, 0.01, 0.02, 0.03] + [round(0.05 * i, 2) for i in range(1, 20)] # 0 ... 0.95 +OLD_CUTOFF = 0.3 +OUT_DIR = Path(__file__).resolve().parent + + +def add_passages(engine: RAGEngine, ids: list[str], texts: list[str], emb: np.ndarray) -> None: + for start in range(0, len(ids), 2000): + engine.collection.add(ids=ids[start:start + 2000], documents=texts[start:start + 2000], + embeddings=emb[start:start + 2000].tolist(), + metadatas=[{"source": i} for i in ids[start:start + 2000]]) + + +def run_claims(engine: RAGEngine, questions: dict, relevant: dict, split: str, index: str) -> list[dict]: + rows = [] + for qid, q in questions.items(): + _, metas = engine.retrieve(q, confidence_threshold=0.0) # the app's default call, no cut-off + rows.append({"split": split, "index": index, "query_id": qid, + "confidence": [m["confidence"] for m in metas], + "relevant": [relevant[qid].get(m["source"], 0) > 0 for m in metas]}) + return rows + + +def at_cutoff(rows: list[dict], t: float) -> dict: + """Hit@3 on answerable, share with no passage on unanswerable, and their mean.""" + ans = [r for r in rows if r["index"] == "answerable"] + una = [r for r in rows if r["index"] == "unanswerable"] + hit = np.mean([any(rel for c, rel in zip(r["confidence"], r["relevant"]) if c >= t) for r in ans]) + answered = np.mean([any(c >= t for c in r["confidence"]) for r in ans]) + abstain = np.mean([not any(c >= t for c in r["confidence"]) for r in una]) + return {"cutoff": t, "hit@3_answerable": round(float(hit), 4), "answered_answerable": round(float(answered), 4), + "no_passage_unanswerable": round(float(abstain), 4), "score": round(float((hit + abstain) / 2), 4)} + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("--embedder", choices=sorted(EMBEDDERS), default="bge-small-en-v1.5") + args = parser.parse_args() + + corpus, test_q, test_rel = load("scifact", "test") + _, train_q, train_rel = load("scifact", "train") + texts = [(t + "\n" + b).strip() for t, b in zip(corpus["title"].fillna("").astype(str), + corpus["text"].fillna("").astype(str))] + ids = corpus["_id"].astype(str).tolist() + + spec = EMBEDDERS[args.embedder] + embedder = SentenceTransformer(spec["repo"], revision=spec["revision"], device="cpu") + reranker = load_reranker(device="cpu") + + rows = [] + t_start = time.time() + with tempfile.TemporaryDirectory() as tmp: + def engine(name): + return RAGEngine(db_path=tmp, collection_name=name, embedding_model=embedder, + reranker=reranker, embedder=args.embedder) + + full = engine("all") + emb = full.embed_passages(texts) # once; the other indexes reuse these vectors + add_passages(full, ids, texts, emb) + print(f"index built in {time.time() - t_start:.0f} s", flush=True) + for split, questions, relevant in (("train", train_q, train_rel), ("test", test_q, test_rel)): + answers = {d for rel in relevant.values() for d, g in rel.items() if g > 0} + keep = [i for i, d in enumerate(ids) if d not in answers] + without = engine(f"without-{split}") + add_passages(without, [ids[i] for i in keep], [texts[i] for i in keep], emb[keep]) + rows += run_claims(full, questions, relevant, split, "answerable") + rows += run_claims(without, questions, relevant, split, "unanswerable") + print(f"{split}: {len(questions)} claims, {len(answers)} answering abstracts removed for the " + f"unanswerable index; {time.time() - t_start:.0f} s", flush=True) + + sweep = {split: [at_cutoff([r for r in rows if r["split"] == split], t) for t in CUTOFFS] + for split in ("train", "test")} + best = max(sweep["train"], key=lambda x: (x["score"], -x["cutoff"])) + chosen = best["cutoff"] + result = { + "embedder": args.embedder, + "claims": {"train": len(train_q), "test": len(test_q)}, + "chosen_cutoff_on_train": chosen, + "test_at_chosen": next(x for x in sweep["test"] if x["cutoff"] == chosen), + "test_at_old_0.3": next(x for x in sweep["test"] if x["cutoff"] == OLD_CUTOFF), + "test_without_cutoff": next(x for x in sweep["test"] if x["cutoff"] == 0.0), + "sweep": sweep, + "minutes": round((time.time() - t_start) / 60, 1), + } + (OUT_DIR / f"cutoff_{args.embedder}.json").write_text(json.dumps(result, indent=2) + "\n") + pd.DataFrame(rows).to_json(OUT_DIR / f"cutoff_per_claim_{args.embedder}.jsonl", orient="records", lines=True) + print(json.dumps({k: v for k, v in result.items() if k != "sweep"}, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/evaluation/dense_cutoff.py b/evaluation/dense_cutoff.py new file mode 100644 index 0000000..971b0f4 --- /dev/null +++ b/evaluation/dense_cutoff.py @@ -0,0 +1,205 @@ +"""Vector search alone with a similarity cut-off, against the reranker path with its confidence cut-off. + +Run after `evaluation.cutoff_tuning` (it reads that script's per-claim results for the +reranker path). Same claims, same two indexes per split: + answerable all 5,183 abstracts; + unanswerable all abstracts except the ones that answer this split's claims. + +Vector path: the 3 nearest passages by cosine similarity (ChromaDB, the chosen embedding model), +no keyword search, no reranker. A passage is kept if its similarity is >= the cut-off. + +1. How well each path's score tells answerable from unanswerable claims, without any cut-off: + AUROC of the best kept passage's score (0.5 = no better than chance, 1 = perfect). +2. The vector path's cut-off is chosen on the training claims so that it refuses at least as + many unanswerable training claims as the reranker path at the app's old cut-off (0.3). + Both are then compared on the test claims: Hit@3 on the answerable index, share of claims + with no passage on the unanswerable index, each with a paired 95% bootstrap interval. +3. For the record, the same paired differences on the training claims too, and without any + cut-off on both paths (`difference_vector_minus_reranker`). + + python -m evaluation.dense_cutoff --embedder bge-small-en-v1.5 + python -m evaluation.dense_cutoff --saved # recompute from the saved per-claim results +""" + +import argparse +import json +import statistics +import sys +import tempfile +import time +from pathlib import Path + +import numpy as np +from sentence_transformers import SentenceTransformer + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "src")) + +from rag_engine import EMBEDDERS, RAGEngine # noqa: E402 + +from evaluation.compare import paired_bootstrap # noqa: E402 +from evaluation.cutoff_tuning import OLD_CUTOFF, add_passages # noqa: E402 +from evaluation.retrieval_eval import load # noqa: E402 + +OUT_DIR = Path(__file__).resolve().parent +TOP_K = 3 + + +def auroc(pos: list[float], neg: list[float]) -> float: + """Chance that a random answerable claim scores higher than a random unanswerable one.""" + p, n = np.asarray(pos)[:, None], np.asarray(neg)[None, :] + return float((p > n).mean() + 0.5 * (p == n).mean()) + + +def kept(row: dict, t: float, key: str) -> list[bool]: + return [s >= t for s in row[key]] + + +def hit(row: dict, t: float, key: str) -> bool: + return any(k and rel for k, rel in zip(kept(row, t, key), row["relevant"])) + + +def refused(row: dict, t: float, key: str) -> bool: + return not any(kept(row, t, key)) + + +def summary(rows: list[dict], t: float, key: str) -> dict: + ans = [r for r in rows if r["index"] == "answerable"] + una = [r for r in rows if r["index"] == "unanswerable"] + return {"cutoff": t, + "hit@3_answerable": round(float(np.mean([hit(r, t, key) for r in ans])), 4), + "answered_answerable": round(float(np.mean([not refused(r, t, key) for r in ans])), 4), + "no_passage_unanswerable": round(float(np.mean([refused(r, t, key) for r in una])), 4)} + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("--embedder", choices=sorted(EMBEDDERS), default="bge-small-en-v1.5") + parser.add_argument("--saved", action="store_true", + help="reuse the saved per-claim results instead of searching again " + "(timings are kept from the run that searched)") + args = parser.parse_args() + + rerank_file = OUT_DIR / f"cutoff_per_claim_{args.embedder}.jsonl" + rerank_rows = [json.loads(line) for line in open(rerank_file, encoding="utf-8")] + + corpus, test_q, test_rel = load("scifact", "test") + _, train_q, train_rel = load("scifact", "train") + out_json = OUT_DIR / f"dense_cutoff_{args.embedder}.json" + out_rows = OUT_DIR / f"dense_per_claim_{args.embedder}.jsonl" + if args.saved: + rows = [json.loads(line) for line in open(out_rows, encoding="utf-8")] + before = json.loads(out_json.read_text()) + report(args.embedder, rows, rerank_rows, len(train_q), len(test_q), + before["vector_path_ms_per_claim_median"], before["minutes"], out_json) + return + texts = [(t + "\n" + b).strip() for t, b in zip(corpus["title"].fillna("").astype(str), + corpus["text"].fillna("").astype(str))] + ids = corpus["_id"].astype(str).tolist() + position = {d: i for i, d in enumerate(ids)} + spec = EMBEDDERS[args.embedder] + embedder = SentenceTransformer(spec["repo"], revision=spec["revision"], device="cpu") + + rows, ms = [], [] + t_start = time.time() + with tempfile.TemporaryDirectory() as tmp: + def engine(name): + return RAGEngine(db_path=tmp, collection_name=name, embedding_model=embedder, + reranker=object(), embedder=args.embedder) + + full = engine("all") + emb = full.embed_passages(texts) + add_passages(full, ids, texts, emb) + print(f"index built in {time.time() - t_start:.0f} s", flush=True) + for split, questions, relevant in (("train", train_q, train_rel), ("test", test_q, test_rel)): + answers = {d for rel in relevant.values() for d, g in rel.items() if g > 0} + keep = [i for i, d in enumerate(ids) if d not in answers] + without = engine(f"without-{split}") + add_passages(without, [ids[i] for i in keep], [texts[i] for i in keep], emb[keep]) + for index_name, eng in (("answerable", full), ("unanswerable", without)): + for qid, q in questions.items(): + t0 = time.perf_counter() + q_emb = eng.embed_query(q) + res = eng.collection.query(query_embeddings=q_emb.tolist(), n_results=TOP_K, + include=["distances"]) + ms.append((time.perf_counter() - t0) * 1000) + found = res["ids"][0] + sims = [1 - d / 2 for d in res["distances"][0]] # squared L2 on unit vectors + exact = emb[[position[f] for f in found]] @ q_emb[0] + assert np.allclose(sims, exact, atol=1e-3), (qid, sims, exact) + rows.append({"split": split, "index": index_name, "query_id": qid, "similarity": sims, + "relevant": [relevant[qid].get(f, 0) > 0 for f in found]}) + print(f"{split}: {len(questions)} claims done; {time.time() - t_start:.0f} s", flush=True) + + with open(out_rows, "w", encoding="utf-8") as f: + for r in rows: + f.write(json.dumps(r) + "\n") + report(args.embedder, rows, rerank_rows, len(train_q), len(test_q), + round(statistics.median(ms), 1), round((time.time() - t_start) / 60, 1), out_json) + + +def report(embedder: str, rows: list[dict], rerank_rows: list[dict], n_train: int, n_test: int, + ms_median: float, minutes: float, out_json: Path) -> None: + def part(rs, split): + return [r for r in rs if r["split"] == split] + + def top(r, key): + return max(r[key], default=0.0) + + separation = {} + for split in ("train", "test"): + separation[split] = {} + for name, rs, key in (("vector_similarity", rows, "similarity"), ("reranker_confidence", rerank_rows, "confidence")): + rs = part(rs, split) + separation[split][name] = round(auroc([top(r, key) for r in rs if r["index"] == "answerable"], + [top(r, key) for r in rs if r["index"] == "unanswerable"]), 4) + + # Cut-off chosen on train: refuse at least as many unanswerable claims as the reranker path at 0.3. + target = summary(part(rerank_rows, "train"), OLD_CUTOFF, "confidence")["no_passage_unanswerable"] + una_train = sorted(top(r, "similarity") for r in part(rows, "train") if r["index"] == "unanswerable") + candidates = sorted({round(s, 4) for s in una_train} | {round(s + 1e-4, 4) for s in una_train}) + chosen = next(t for t in candidates + if summary(part(rows, "train"), t, "similarity")["no_passage_unanswerable"] >= target) + + vec_test, rr_test = part(rows, "test"), part(rerank_rows, "test") + + def differences(split, t_vec, t_rr, measures): + by_id = {(r["index"], r["query_id"]): r for r in part(rerank_rows, split)} + out = {} + for index_name, fn in measures: + v = [r for r in part(rows, split) if r["index"] == index_name] + diff = np.array([float(fn(r, t_vec, "similarity")) + - float(fn(by_id[(index_name, r["query_id"])], t_rr, "confidence")) for r in v]) + mean, low, high = paired_bootstrap(diff) + out["hit@3_answerable" if index_name == "answerable" else "no_passage_unanswerable"] = { + "difference": round(mean, 4), "interval_95": [round(low, 4), round(high, 4)], "claims": len(diff)} + return out + + both = (("answerable", hit), ("unanswerable", refused)) + paired = differences("test", chosen, OLD_CUTOFF, both) + every = {split: {"cutoffs_matched": differences(split, chosen, OLD_CUTOFF, both), + "no_cutoff": differences(split, 0.0, 0.0, (("answerable", hit),))} + for split in ("train", "test")} + + result = { + "embedder": embedder, + "claims": {"train": n_train, "test": n_test}, + "auroc_answerable_vs_unanswerable": separation, + "reranker_path_old_cutoff": {"cutoff": OLD_CUTOFF, + "train": summary(part(rerank_rows, "train"), OLD_CUTOFF, "confidence"), + "test": summary(rr_test, OLD_CUTOFF, "confidence")}, + "vector_path_chosen_on_train": {"cutoff": chosen, + "train": summary(part(rows, "train"), chosen, "similarity"), + "test": summary(vec_test, chosen, "similarity")}, + "vector_path_without_cutoff_test": summary(vec_test, 0.0, "similarity"), + "test_difference_vector_minus_reranker": paired, + "difference_vector_minus_reranker": every, + "vector_path_ms_per_claim_median": ms_median, + "minutes": minutes, + } + out_json.write_text(json.dumps(result, indent=2) + "\n") + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/evaluation/dense_cutoff_bge-small-en-v1.5.json b/evaluation/dense_cutoff_bge-small-en-v1.5.json new file mode 100644 index 0000000..7e0cb32 --- /dev/null +++ b/evaluation/dense_cutoff_bge-small-en-v1.5.json @@ -0,0 +1,135 @@ +{ + "embedder": "bge-small-en-v1.5", + "claims": { + "train": 809, + "test": 300 + }, + "auroc_answerable_vs_unanswerable": { + "train": { + "vector_similarity": 0.7018, + "reranker_confidence": 0.7135 + }, + "test": { + "vector_similarity": 0.6905, + "reranker_confidence": 0.7044 + } + }, + "reranker_path_old_cutoff": { + "cutoff": 0.3, + "train": { + "cutoff": 0.3, + "hit@3_answerable": 0.6885, + "answered_answerable": 0.8826, + "no_passage_unanswerable": 0.3066 + }, + "test": { + "cutoff": 0.3, + "hit@3_answerable": 0.67, + "answered_answerable": 0.8767, + "no_passage_unanswerable": 0.3033 + } + }, + "vector_path_chosen_on_train": { + "cutoff": 0.7406, + "train": { + "cutoff": 0.7406, + "hit@3_answerable": 0.7009, + "answered_answerable": 0.8603, + "no_passage_unanswerable": 0.3066 + }, + "test": { + "cutoff": 0.7406, + "hit@3_answerable": 0.6433, + "answered_answerable": 0.87, + "no_passage_unanswerable": 0.3 + } + }, + "vector_path_without_cutoff_test": { + "cutoff": 0.0, + "hit@3_answerable": 0.73, + "answered_answerable": 1.0, + "no_passage_unanswerable": 0.0 + }, + "test_difference_vector_minus_reranker": { + "hit@3_answerable": { + "difference": -0.0267, + "interval_95": [ + -0.0667, + 0.0133 + ], + "claims": 300 + }, + "no_passage_unanswerable": { + "difference": -0.0033, + "interval_95": [ + -0.05, + 0.0467 + ], + "claims": 300 + } + }, + "difference_vector_minus_reranker": { + "train": { + "cutoffs_matched": { + "hit@3_answerable": { + "difference": 0.0124, + "interval_95": [ + -0.0148, + 0.0396 + ], + "claims": 809 + }, + "no_passage_unanswerable": { + "difference": 0.0, + "interval_95": [ + -0.0334, + 0.0321 + ], + "claims": 809 + } + }, + "no_cutoff": { + "hit@3_answerable": { + "difference": 0.0309, + "interval_95": [ + 0.0049, + 0.0556 + ], + "claims": 809 + } + } + }, + "test": { + "cutoffs_matched": { + "hit@3_answerable": { + "difference": -0.0267, + "interval_95": [ + -0.0667, + 0.0133 + ], + "claims": 300 + }, + "no_passage_unanswerable": { + "difference": -0.0033, + "interval_95": [ + -0.05, + 0.0467 + ], + "claims": 300 + } + }, + "no_cutoff": { + "hit@3_answerable": { + "difference": 0.0067, + "interval_95": [ + -0.03, + 0.0433 + ], + "claims": 300 + } + } + } + }, + "vector_path_ms_per_claim_median": 17.9, + "minutes": 8.2 +} diff --git a/evaluation/dense_per_claim_bge-small-en-v1.5.jsonl b/evaluation/dense_per_claim_bge-small-en-v1.5.jsonl new file mode 100644 index 0000000..4e827b0 --- /dev/null +++ b/evaluation/dense_per_claim_bge-small-en-v1.5.jsonl @@ -0,0 +1,2218 @@ +{"split": "train", "index": "answerable", "query_id": "0", "similarity": [0.6743555963039398, 0.6661394834518433, 0.6493954658508301], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "2", "similarity": [0.7202525734901428, 0.6952096521854401, 0.69093257188797], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "4", "similarity": [0.7936265915632248, 0.7816746234893799, 0.7523935288190842], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "6", "similarity": [0.831746444106102, 0.8078370690345764, 0.7642121315002441], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "9", "similarity": [0.8696698844432831, 0.7408380806446075, 0.7341804504394531], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "10", "similarity": [0.7664345651865005, 0.7504134923219681, 0.7442342936992645], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "11", "similarity": [0.7722457051277161, 0.7426230907440186, 0.7314457297325134], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "12", "similarity": [0.8610618710517883, 0.8206847608089447, 0.8153871446847916], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "14", "similarity": [0.7969818413257599, 0.7521674484014511, 0.7139106392860413], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "15", "similarity": [0.7333955466747284, 0.7319148182868958, 0.7297512888908386], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "17", "similarity": [0.773045539855957, 0.7697790265083313, 0.7599047124385834], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "18", "similarity": [0.803011104464531, 0.770806223154068, 0.7480430603027344], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "19", "similarity": [0.7531127482652664, 0.7527973353862762, 0.7511198222637177], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "20", "similarity": [0.7518254369497299, 0.7516311407089233, 0.7503232061862946], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "21", "similarity": [0.7406105101108551, 0.6753512620925903, 0.6730741858482361], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "22", "similarity": [0.8394749015569687, 0.7302623689174652, 0.7275663912296295], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "24", "similarity": [0.8221776634454727, 0.8075330853462219, 0.7869404703378677], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "25", "similarity": [0.8330990076065063, 0.8125565648078918, 0.7638443857431412], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "26", "similarity": [0.7605648636817932, 0.7454597353935242, 0.7449705004692078], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "27", "similarity": [0.7591478228569031, 0.7437950372695923, 0.7427179217338562], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "28", "similarity": [0.7986528277397156, 0.7876337766647339, 0.7743129730224609], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "30", "similarity": [0.8559183478355408, 0.8279979526996613, 0.797067403793335], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "32", "similarity": [0.7593181729316711, 0.6801932454109192, 0.6662082076072693], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "34", "similarity": [0.8263355195522308, 0.8215991407632828, 0.8111550807952881], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "35", "similarity": [0.8150556683540344, 0.8109435439109802, 0.8099926859140396], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "37", "similarity": [0.8113171756267548, 0.8074645102024078, 0.8001414090394974], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "39", "similarity": [0.7644672244787216, 0.7283899188041687, 0.7024332880973816], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "40", "similarity": [0.7909461855888367, 0.7205093801021576, 0.7106808722019196], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "41", "similarity": [0.8546374440193176, 0.7950724959373474, 0.7617624253034592], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "43", "similarity": [0.8047029227018356, 0.7557521462440491, 0.7376760840415955], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "44", "similarity": [0.7949952185153961, 0.736034095287323, 0.7342734038829803], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "45", "similarity": [0.7885264903306961, 0.7364977598190308, 0.7362096309661865], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "46", "similarity": [0.917203888297081, 0.7303387522697449, 0.7209568023681641], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "47", "similarity": [0.8881632164120674, 0.8022752702236176, 0.773116260766983], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "52", "similarity": [0.8654983192682266, 0.7389041781425476, 0.7365627288818359], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "55", "similarity": [0.8827374801039696, 0.8379458338022232, 0.8255658596754074], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "58", "similarity": [0.862905278801918, 0.8113747835159302, 0.794045940041542], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "60", "similarity": [0.710111528635025, 0.6817617118358612, 0.6649458408355713], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "61", "similarity": [0.7855395376682281, 0.747697114944458, 0.7468201816082001], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "62", "similarity": [0.7509466409683228, 0.7099989056587219, 0.7022122740745544], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "63", "similarity": [0.7915595769882202, 0.7913290709257126, 0.7798593938350677], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "64", "similarity": [0.7805306017398834, 0.7749602645635605, 0.7735163867473602], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "66", "similarity": [0.6906289160251617, 0.6856237649917603, 0.6849557757377625], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "67", "similarity": [0.9175116419792175, 0.7949395179748535, 0.779300644993782], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "68", "similarity": [0.8550709336996078, 0.7901908457279205, 0.7732623368501663], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "69", "similarity": [0.7696284353733063, 0.7629066556692123, 0.755319207906723], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "71", "similarity": [0.83976711332798, 0.7771604359149933, 0.7642926871776581], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "73", "similarity": [0.653090238571167, 0.6530661284923553, 0.6521689891815186], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "74", "similarity": [0.7724331170320511, 0.7719530165195465, 0.7713121622800827], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "76", "similarity": [0.8420679867267609, 0.7291402816772461, 0.7247202396392822], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "77", "similarity": [0.8528269827365875, 0.7382997274398804, 0.7269737124443054], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "78", "similarity": [0.8838930502533913, 0.7621912062168121, 0.7576114684343338], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "79", "similarity": [0.8251993805170059, 0.7859862148761749, 0.7722349464893341], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "80", "similarity": [0.7131943702697754, 0.7057628333568573, 0.7019301056861877], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "81", "similarity": [0.773992508649826, 0.7323450446128845, 0.7306736409664154], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "82", "similarity": [0.7933057993650436, 0.7868746519088745, 0.7698315531015396], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "85", "similarity": [0.799657553434372, 0.7912176251411438, 0.7729697376489639], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "86", "similarity": [0.799657553434372, 0.7912176251411438, 0.7729697376489639], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "88", "similarity": [0.8433793187141418, 0.7727448493242264, 0.7711000740528107], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "89", "similarity": [0.829504519701004, 0.7971605062484741, 0.7943258285522461], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "90", "similarity": [0.829504519701004, 0.7971605062484741, 0.7943258285522461], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "91", "similarity": [0.7364961504936218, 0.7274278998374939, 0.7168539762496948], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "92", "similarity": [0.7494840025901794, 0.7345928251743317, 0.7327717542648315], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "93", "similarity": [0.8077520877122879, 0.7975044101476669, 0.7852956801652908], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "95", "similarity": [0.692511260509491, 0.6869924664497375, 0.6639431416988373], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "96", "similarity": [0.7421402633190155, 0.6947263479232788, 0.64202880859375], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "98", "similarity": [0.8007678687572479, 0.7177221179008484, 0.7088478803634644], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "104", "similarity": [0.8142558336257935, 0.7158631980419159, 0.7117479741573334], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "105", "similarity": [0.6939563453197479, 0.6639677286148071, 0.6420066952705383], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "106", "similarity": [0.8279110640287399, 0.7935451865196228, 0.728417843580246], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "108", "similarity": [0.9129747450351715, 0.8696193397045135, 0.8431127965450287], "relevant": [true, false, true]} +{"split": "train", "index": "answerable", "query_id": "109", "similarity": [0.7910209596157074, 0.7664453089237213, 0.7661898583173752], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "111", "similarity": [0.8494295626878738, 0.8229000717401505, 0.8078939020633698], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "112", "similarity": [0.8462062180042267, 0.7969379276037216, 0.7762596011161804], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "114", "similarity": [0.7585528343915939, 0.5982098579406738, 0.5942170023918152], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "116", "similarity": [0.7657574117183685, 0.6569671630859375, 0.6355636119842529], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "119", "similarity": [0.7836718708276749, 0.75462606549263, 0.7423906922340393], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "120", "similarity": [0.7958637475967407, 0.7693542540073395, 0.74610635638237], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "121", "similarity": [0.711250901222229, 0.686120331287384, 0.6793652772903442], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "122", "similarity": [0.6895979046821594, 0.6871328949928284, 0.6857678294181824], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "123", "similarity": [0.8816559389233589, 0.8500168174505234, 0.7779911458492279], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "126", "similarity": [0.7914946973323822, 0.7466034293174744, 0.7456600666046143], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "134", "similarity": [0.7376936674118042, 0.7360779941082001, 0.7198386490345001], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "138", "similarity": [0.8586550056934357, 0.7229597866535187, 0.7226295471191406], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "139", "similarity": [0.7608333826065063, 0.7111278176307678, 0.7087642252445221], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "144", "similarity": [0.8298124223947525, 0.8155592530965805, 0.7670129686594009], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "149", "similarity": [0.8607482761144638, 0.827512726187706, 0.8057051599025726], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "152", "similarity": [0.8099904507398605, 0.7442683279514313, 0.7431421875953674], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "153", "similarity": [0.8067291378974915, 0.7835433781147003, 0.7780543863773346], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "154", "similarity": [0.8084442913532257, 0.7863014042377472, 0.7776831537485123], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "155", "similarity": [0.8375220149755478, 0.7899214625358582, 0.7714906334877014], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "156", "similarity": [0.8122534453868866, 0.8100514113903046, 0.7867436707019806], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "157", "similarity": [0.8372163474559784, 0.7683407664299011, 0.750388041138649], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "159", "similarity": [0.8998213186860085, 0.842844620347023, 0.8309024721384048], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "160", "similarity": [0.8069299608469009, 0.7890415638685226, 0.7043436169624329], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "161", "similarity": [0.6847791075706482, 0.6744487881660461, 0.672580897808075], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "164", "similarity": [0.7938051223754883, 0.7745620310306549, 0.7705127596855164], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "165", "similarity": [0.8316074311733246, 0.797494500875473, 0.7903923094272614], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "166", "similarity": [0.8474060744047165, 0.8140323013067245, 0.7988773882389069], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "167", "similarity": [0.8471693396568298, 0.832714393734932, 0.814598023891449], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "168", "similarity": [0.7981133610010147, 0.7668047547340393, 0.7632088959217072], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "169", "similarity": [0.8160370886325836, 0.7931871116161346, 0.7842923551797867], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "172", "similarity": [0.8545302748680115, 0.7758011221885681, 0.75599405169487], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "173", "similarity": [0.7178650796413422, 0.7063077092170715, 0.7018230259418488], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "174", "similarity": [0.746100664138794, 0.7011155188083649, 0.6987755298614502], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "175", "similarity": [0.734713613986969, 0.7130447924137115, 0.7091350555419922], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "176", "similarity": [0.7473401427268982, 0.7099269330501556, 0.7098615765571594], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "177", "similarity": [0.9261213913559914, 0.8139075040817261, 0.7937134504318237], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "178", "similarity": [0.8628402203321457, 0.8436847031116486, 0.841600775718689], "relevant": [true, true, true]} +{"split": "train", "index": "answerable", "query_id": "181", "similarity": [0.798283264040947, 0.7591406255960464, 0.7417440116405487], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "182", "similarity": [0.7346388101577759, 0.7318356037139893, 0.7308130860328674], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "184", "similarity": [0.7673879563808441, 0.7627602219581604, 0.751897782087326], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "186", "similarity": [0.7164493799209595, 0.6415234804153442, 0.6314411759376526], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "187", "similarity": [0.7136515974998474, 0.6479557454586029, 0.6103510856628418], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "189", "similarity": [0.8012084513902664, 0.7760195136070251, 0.7699835449457169], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "196", "similarity": [0.7912807464599609, 0.7194398939609528, 0.7191303372383118], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "197", "similarity": [0.7531260102987289, 0.7163334488868713, 0.7116860747337341], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "199", "similarity": [0.7120475172996521, 0.6992188692092896, 0.6981609761714935], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "200", "similarity": [0.8058499395847321, 0.7953193336725235, 0.7777671217918396], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "201", "similarity": [0.83592788875103, 0.8340784907341003, 0.792444720864296], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "203", "similarity": [0.8691299855709076, 0.7780070304870605, 0.7734811902046204], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "204", "similarity": [0.820356696844101, 0.7791834771633148, 0.7356718182563782], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "205", "similarity": [0.7976561486721039, 0.763868123292923, 0.7265685200691223], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "209", "similarity": [0.7107541561126709, 0.6859710812568665, 0.682780921459198], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "210", "similarity": [0.8005596697330475, 0.6958027482032776, 0.6934970021247864], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "211", "similarity": [0.6868328750133514, 0.6759992837905884, 0.6610491871833801], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "214", "similarity": [0.8390742838382721, 0.8375431299209595, 0.779787689447403], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "220", "similarity": [0.8667274415493011, 0.786771148443222, 0.7833092212677002], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "221", "similarity": [0.8296740651130676, 0.7577261477708817, 0.7466467916965485], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "222", "similarity": [0.8209195733070374, 0.7552010267972946, 0.7240142226219177], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "223", "similarity": [0.7966649681329727, 0.7896290123462677, 0.769658550620079], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "224", "similarity": [0.9544168412685394, 0.8999269157648087, 0.8750919103622437], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "225", "similarity": [0.9002565070986748, 0.8864379301667213, 0.8500816226005554], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "226", "similarity": [0.9112038984894753, 0.8909343555569649, 0.868279442191124], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "227", "similarity": [0.8247604817152023, 0.7859774827957153, 0.751044899225235], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "228", "similarity": [0.8487570434808731, 0.7534330487251282, 0.7363096475601196], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "229", "similarity": [0.7725884020328522, 0.7429361045360565, 0.7313541173934937], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "235", "similarity": [0.8610649704933167, 0.7732241898775101, 0.7712610363960266], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "241", "similarity": [0.7905431836843491, 0.7877479195594788, 0.7824418097734451], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "242", "similarity": [0.7711471915245056, 0.7684162557125092, 0.7652585208415985], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "243", "similarity": [0.8268251419067383, 0.7777058184146881, 0.7772357761859894], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "244", "similarity": [0.7675657272338867, 0.751402273774147, 0.7336758673191071], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "245", "similarity": [0.802569791674614, 0.7824402153491974, 0.7724698781967163], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "246", "similarity": [0.7993142008781433, 0.7802673131227493, 0.7489398717880249], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "247", "similarity": [0.7978346943855286, 0.7726992666721344, 0.7589726150035858], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "250", "similarity": [0.8504604697227478, 0.7416687607765198, 0.7377647757530212], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "251", "similarity": [0.8618159890174866, 0.7682634890079498, 0.7430208027362823], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "253", "similarity": [0.853800967335701, 0.838812530040741, 0.7999542206525803], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "254", "similarity": [0.8514477610588074, 0.8421264141798019, 0.7960283309221268], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "255", "similarity": [0.8205214142799377, 0.7803323119878769, 0.7633304297924042], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "256", "similarity": [0.8510240912437439, 0.8055005371570587, 0.7534071803092957], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "258", "similarity": [0.7787579596042633, 0.7783207446336746, 0.7699795812368393], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "259", "similarity": [0.8224280774593353, 0.7673878371715546, 0.7468484938144684], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "262", "similarity": [0.819299042224884, 0.7846173644065857, 0.7762330621480942], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "263", "similarity": [0.7869455814361572, 0.7806810140609741, 0.7799376547336578], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "264", "similarity": [0.8133974373340607, 0.8095715790987015, 0.7918643653392792], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "265", "similarity": [0.9000475108623505, 0.7665570080280304, 0.7454635500907898], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "266", "similarity": [0.7554333508014679, 0.7455405294895172, 0.7410260438919067], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "267", "similarity": [0.801408439874649, 0.7059914469718933, 0.70528244972229], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "272", "similarity": [0.8789187073707581, 0.8176601231098175, 0.7572607398033142], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "277", "similarity": [0.8454149812459946, 0.8045091480016708, 0.7943011522293091], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "278", "similarity": [0.835475891828537, 0.7804588675498962, 0.7553446739912033], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "280", "similarity": [0.831778883934021, 0.7816796451807022, 0.755294606089592], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "281", "similarity": [0.7648773193359375, 0.7617455273866653, 0.7416425347328186], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "283", "similarity": [0.8963152319192886, 0.8512682318687439, 0.7989916205406189], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "285", "similarity": [0.7767836451530457, 0.7662868201732635, 0.7399886846542358], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "286", "similarity": [0.8448923081159592, 0.7636047303676605, 0.7617461532354355], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "287", "similarity": [0.8488821387290955, 0.7656799256801605, 0.7554545104503632], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "290", "similarity": [0.7362960577011108, 0.6973913908004761, 0.6931456327438354], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "292", "similarity": [0.7065927088260651, 0.6784105896949768, 0.665621280670166], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "293", "similarity": [0.7751148343086243, 0.7501926720142365, 0.7489355802536011], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "296", "similarity": [0.8146794736385345, 0.781454473733902, 0.7734027355909348], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "299", "similarity": [0.7754642963409424, 0.7704409062862396, 0.754976212978363], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "301", "similarity": [0.7666088789701462, 0.739851713180542, 0.7366050481796265], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "304", "similarity": [0.7390015721321106, 0.7276181578636169, 0.7250238060951233], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "305", "similarity": [0.7663225829601288, 0.7489531636238098, 0.7457069158554077], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "306", "similarity": [0.7945036590099335, 0.7227696776390076, 0.7201881408691406], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "308", "similarity": [0.8183186054229736, 0.7733506858348846, 0.7671698033809662], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "309", "similarity": [0.7889255285263062, 0.725557267665863, 0.7170924544334412], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "310", "similarity": [0.7474476099014282, 0.74595907330513, 0.7235594987869263], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "313", "similarity": [0.7606395781040192, 0.7572888284921646, 0.7267559468746185], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "315", "similarity": [0.8346422016620636, 0.7842191457748413, 0.7781549394130707], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "316", "similarity": [0.7668445706367493, 0.7295786142349243, 0.7290200293064117], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "317", "similarity": [0.774908035993576, 0.7549627870321274, 0.7418418228626251], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "323", "similarity": [0.7036052942276001, 0.6943826079368591, 0.691929280757904], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "325", "similarity": [0.7472081780433655, 0.7349664270877838, 0.7285302579402924], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "326", "similarity": [0.7694805860519409, 0.7604025304317474, 0.7574030160903931], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "330", "similarity": [0.6589972972869873, 0.6064719259738922, 0.6042637526988983], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "331", "similarity": [0.6369708776473999, 0.6003256440162659, 0.598145067691803], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "332", "similarity": [0.9221823960542679, 0.8330013751983643, 0.7760626077651978], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "333", "similarity": [0.9280330315232277, 0.8280045688152313, 0.7722956538200378], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "334", "similarity": [0.783937931060791, 0.7318542003631592, 0.7287138104438782], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "335", "similarity": [0.8452212512493134, 0.732795238494873, 0.7277260422706604], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "336", "similarity": [0.7469155490398407, 0.7262393534183502, 0.7219460010528564], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "337", "similarity": [0.7437011003494263, 0.7223957777023315, 0.7206664681434631], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "339", "similarity": [0.8153382539749146, 0.7766936123371124, 0.7339113354682922], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "340", "similarity": [0.8768865540623665, 0.7472544610500336, 0.7228391468524933], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "341", "similarity": [0.8792873919010162, 0.7394987940788269, 0.7221116721630096], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "342", "similarity": [0.8364132642745972, 0.8003194630146027, 0.7861088365316391], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "345", "similarity": [0.7858441174030304, 0.7354811429977417, 0.727349042892456], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "346", "similarity": [0.7770395427942276, 0.7604358643293381, 0.7488808333873749], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "347", "similarity": [0.7794892191886902, 0.7618178129196167, 0.7548521161079407], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "349", "similarity": [0.7884018570184708, 0.7299363911151886, 0.7123643457889557], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "351", "similarity": [0.748631477355957, 0.7352218925952911, 0.7307652533054352], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "352", "similarity": [0.6983654201030731, 0.6983175873756409, 0.6888350546360016], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "355", "similarity": [0.786814734339714, 0.7739812135696411, 0.7636400014162064], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "356", "similarity": [0.7966925501823425, 0.7809641361236572, 0.7620969712734222], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "357", "similarity": [0.8106293380260468, 0.7753032743930817, 0.7554234564304352], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "358", "similarity": [0.8103315979242325, 0.7777228355407715, 0.7552768588066101], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "361", "similarity": [0.8031536787748337, 0.7963901907205582, 0.794131338596344], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "363", "similarity": [0.7316132187843323, 0.7301232218742371, 0.7261697053909302], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "364", "similarity": [0.7466318607330322, 0.7199723124504089, 0.7199244797229767], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "365", "similarity": [0.7892120331525803, 0.7508397400379181, 0.7445455193519592], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "366", "similarity": [0.8891923129558563, 0.7459114193916321, 0.7380953431129456], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "367", "similarity": [0.8916175290942192, 0.8452742397785187, 0.8318142890930176], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "368", "similarity": [0.8895446956157684, 0.833355575799942, 0.8266983032226562], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "369", "similarity": [0.8387098759412766, 0.820567786693573, 0.8127890974283218], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "370", "similarity": [0.8667483478784561, 0.7539039552211761, 0.7513227462768555], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "371", "similarity": [0.8768894448876381, 0.758394181728363, 0.7573489844799042], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "372", "similarity": [0.8250362575054169, 0.7404491901397705, 0.732106626033783], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "375", "similarity": [0.8377140462398529, 0.7374249696731567, 0.7292406558990479], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "376", "similarity": [0.7570200562477112, 0.7498992681503296, 0.748746395111084], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "377", "similarity": [0.7054826021194458, 0.6991153359413147, 0.6967560350894928], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "378", "similarity": [0.7866958379745483, 0.7807696908712387, 0.7799125611782074], "relevant": [true, true, true]} +{"split": "train", "index": "answerable", "query_id": "379", "similarity": [0.7877171337604523, 0.7657669335603714, 0.7540490478277206], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "381", "similarity": [0.8249915242195129, 0.7817505896091461, 0.7773790061473846], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "382", "similarity": [0.7188830077648163, 0.717323899269104, 0.7171142995357513], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "383", "similarity": [0.7449514269828796, 0.7441080808639526, 0.725062370300293], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "389", "similarity": [0.7454147934913635, 0.7250365316867828, 0.7057153284549713], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "390", "similarity": [0.7346757650375366, 0.7147364020347595, 0.7117703557014465], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "391", "similarity": [0.74649578332901, 0.7358254790306091, 0.7076335251331329], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "392", "similarity": [0.7431449890136719, 0.7303867340087891, 0.7172231376171112], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "393", "similarity": [0.7478915154933929, 0.7439828515052795, 0.7106620967388153], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "394", "similarity": [0.7990979105234146, 0.7837949246168137, 0.7814426124095917], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "396", "similarity": [0.8159323930740356, 0.771126389503479, 0.7630386054515839], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "397", "similarity": [0.8086576461791992, 0.7674490809440613, 0.7655914276838303], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "398", "similarity": [0.8309390991926193, 0.7598005533218384, 0.7343215346336365], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "400", "similarity": [0.8701051622629166, 0.76393261551857, 0.7553676664829254], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "401", "similarity": [0.7513352036476135, 0.6983528137207031, 0.696308046579361], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "403", "similarity": [0.7576313614845276, 0.7338289022445679, 0.7310670614242554], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "404", "similarity": [0.7404117286205292, 0.735698401927948, 0.7284148931503296], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "406", "similarity": [0.7949658781290054, 0.7934073507785797, 0.7796372026205063], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "407", "similarity": [0.799221932888031, 0.7907141447067261, 0.7820418924093246], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "413", "similarity": [0.8334698528051376, 0.8332948535680771, 0.8116761744022369], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "414", "similarity": [0.8389291763305664, 0.8383483290672302, 0.8165377676486969], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "416", "similarity": [0.8031213283538818, 0.802703320980072, 0.7974547445774078], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "417", "similarity": [0.8058040291070938, 0.8039175271987915, 0.8008300960063934], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "418", "similarity": [0.7442044019699097, 0.7435019910335541, 0.740953803062439], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "420", "similarity": [0.9071099162101746, 0.8119044452905655, 0.7832479178905487], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "422", "similarity": [0.7426939606666565, 0.7366817593574524, 0.7124320268630981], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "423", "similarity": [0.8621344268321991, 0.8210036158561707, 0.820083498954773], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "425", "similarity": [0.8182313144207001, 0.7959323972463608, 0.7225145101547241], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "426", "similarity": [0.8137271106243134, 0.7787235081195831, 0.7783945798873901], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "428", "similarity": [0.7796604931354523, 0.773147851228714, 0.7622964382171631], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "429", "similarity": [0.7836914658546448, 0.7564771175384521, 0.7535692602396011], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "430", "similarity": [0.7720558345317841, 0.7553471028804779, 0.7523102909326553], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "432", "similarity": [0.8645133674144745, 0.7852071523666382, 0.7720938920974731], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "434", "similarity": [0.808689147233963, 0.786549910902977, 0.777647852897644], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "435", "similarity": [0.8076968789100647, 0.7848390340805054, 0.78081876039505], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "441", "similarity": [0.7660639733076096, 0.7605568617582321, 0.7211592793464661], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "444", "similarity": [0.8809879124164581, 0.8067832887172699, 0.7917380332946777], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "445", "similarity": [0.8584003001451492, 0.7844165116548538, 0.7805034071207047], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "447", "similarity": [0.8678804486989975, 0.8350792527198792, 0.8219677954912186], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "448", "similarity": [0.8687124252319336, 0.8430374562740326, 0.8129747211933136], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "449", "similarity": [0.8152835667133331, 0.8051509261131287, 0.758063405752182], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "453", "similarity": [0.7795686572790146, 0.7139587998390198, 0.693849503993988], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "454", "similarity": [0.8124266415834427, 0.717708557844162, 0.7018381953239441], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "455", "similarity": [0.7912134230136871, 0.76301209628582, 0.758814811706543], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "456", "similarity": [0.7246090471744537, 0.7148322761058807, 0.7125334143638611], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "458", "similarity": [0.8852721303701401, 0.7911069691181183, 0.7825569212436676], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "461", "similarity": [0.7937905341386795, 0.7339238226413727, 0.7337647378444672], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "463", "similarity": [0.7522185742855072, 0.7470166087150574, 0.7431104481220245], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "466", "similarity": [0.8235295116901398, 0.8224178850650787, 0.811815932393074], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "469", "similarity": [0.78164342045784, 0.7758375108242035, 0.7617533951997757], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "470", "similarity": [0.8145812302827835, 0.8022085428237915, 0.7864704430103302], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "472", "similarity": [0.8148061782121658, 0.8017789423465729, 0.7937802374362946], "relevant": [true, false, true]} +{"split": "train", "index": "answerable", "query_id": "473", "similarity": [0.744611382484436, 0.7319713234901428, 0.7266983985900879], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "474", "similarity": [0.7574525773525238, 0.7275685966014862, 0.7140130400657654], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "479", "similarity": [0.8417339324951172, 0.7246934175491333, 0.7241749167442322], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "480", "similarity": [0.843607559800148, 0.7477181553840637, 0.7233226299285889], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "481", "similarity": [0.7557999640703201, 0.7274763882160187, 0.7225792109966278], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "482", "similarity": [0.8834361135959625, 0.7940271198749542, 0.7699302583932877], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "483", "similarity": [0.8354425877332687, 0.8171805143356323, 0.80718132853508], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "484", "similarity": [0.7796555459499359, 0.7695381492376328, 0.760710671544075], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "485", "similarity": [0.813699871301651, 0.7721941620111465, 0.755938395857811], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "486", "similarity": [0.7833331227302551, 0.7762202471494675, 0.7661490142345428], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "487", "similarity": [0.8288094252347946, 0.7928760349750519, 0.761897623538971], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "488", "similarity": [0.912215918302536, 0.7641455829143524, 0.7589432150125504], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "489", "similarity": [0.7852666974067688, 0.7643639147281647, 0.7546597719192505], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "490", "similarity": [0.8339539468288422, 0.7284731268882751, 0.7187975645065308], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "492", "similarity": [0.7732836306095123, 0.7570655196905136, 0.751641646027565], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "493", "similarity": [0.7341592907905579, 0.7325196862220764, 0.7237964868545532], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "494", "similarity": [0.679891049861908, 0.6732895970344543, 0.6708688735961914], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "495", "similarity": [0.7579257041215897, 0.7166905403137207, 0.6959202289581299], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "498", "similarity": [0.7473151981830597, 0.7328599095344543, 0.7109872698783875], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "499", "similarity": [0.8914960473775864, 0.7663809359073639, 0.7443576455116272], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "500", "similarity": [0.8171446621417999, 0.7617614418268204, 0.7532487064599991], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "504", "similarity": [0.7798389494419098, 0.7673095613718033, 0.7218812108039856], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "505", "similarity": [0.9073010683059692, 0.835754320025444, 0.7940398752689362], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "506", "similarity": [0.784526139497757, 0.77230004966259, 0.7568075209856033], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "509", "similarity": [0.7415445446968079, 0.7138466238975525, 0.7010796070098877], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "515", "similarity": [0.8576897829771042, 0.8207335472106934, 0.7716128826141357], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "523", "similarity": [0.7998229414224625, 0.7938956618309021, 0.7474994957447052], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "524", "similarity": [0.7987575829029083, 0.793286070227623, 0.74582439661026], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "526", "similarity": [0.7823859006166458, 0.7629523575305939, 0.759398564696312], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "529", "similarity": [0.8197894394397736, 0.8177839517593384, 0.8069619834423065], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "530", "similarity": [0.8244608938694, 0.8049950450658798, 0.8035381138324738], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "531", "similarity": [0.8303274512290955, 0.8289750665426254, 0.8228576183319092], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "537", "similarity": [0.7840682566165924, 0.7105554044246674, 0.709848165512085], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "541", "similarity": [0.7666011452674866, 0.7657915353775024, 0.765053778886795], "relevant": [true, false, true]} +{"split": "train", "index": "answerable", "query_id": "542", "similarity": [0.7992426455020905, 0.7211527824401855, 0.7073899507522583], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "545", "similarity": [0.7380990386009216, 0.7182808518409729, 0.7129628360271454], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "547", "similarity": [0.8196718096733093, 0.8143540620803833, 0.7799928188323975], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "548", "similarity": [0.8279155194759369, 0.8007048964500427, 0.7983205765485764], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "550", "similarity": [0.7913141995668411, 0.7609764933586121, 0.760685607790947], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "553", "similarity": [0.899651899933815, 0.8298218548297882, 0.7860737144947052], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "555", "similarity": [0.773094654083252, 0.7582191675901413, 0.7467481791973114], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "557", "similarity": [0.7722592204809189, 0.7533663809299469, 0.7475010752677917], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "559", "similarity": [0.7534452378749847, 0.7234956622123718, 0.7163050174713135], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "562", "similarity": [0.6765826642513275, 0.6150684952735901, 0.6147632896900177], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "563", "similarity": [0.8156133890151978, 0.7361018657684326, 0.7356517016887665], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "564", "similarity": [0.8084435164928436, 0.7681157439947128, 0.7475987374782562], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "565", "similarity": [0.8612872064113617, 0.7497756779193878, 0.7439849376678467], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "566", "similarity": [0.8515060544013977, 0.7523233741521835, 0.7404789924621582], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "568", "similarity": [0.7840147614479065, 0.7802299410104752, 0.7780882716178894], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "570", "similarity": [0.9032429829239845, 0.7418638169765472, 0.7373995184898376], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "571", "similarity": [0.8931726515293121, 0.748290479183197, 0.7409789562225342], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "572", "similarity": [0.7271565794944763, 0.7260324358940125, 0.7259914875030518], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "573", "similarity": [0.794885665178299, 0.7398793697357178, 0.7319880127906799], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "574", "similarity": [0.7933594137430191, 0.7446029186248779, 0.7423139810562134], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "576", "similarity": [0.8632538914680481, 0.8294616043567657, 0.7883845269680023], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "579", "similarity": [0.8020156919956207, 0.7567082047462463, 0.7179262936115265], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "580", "similarity": [0.8176627904176712, 0.7648082375526428, 0.7543285936117172], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "582", "similarity": [0.8261767029762268, 0.7758013755083084, 0.7452141642570496], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "584", "similarity": [0.8497199416160583, 0.8188939839601517, 0.8161137998104095], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "585", "similarity": [0.7958589047193527, 0.6724662482738495, 0.6700088679790497], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "588", "similarity": [0.7974539697170258, 0.7451485991477966, 0.7325600385665894], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "590", "similarity": [0.9190879464149475, 0.7425821125507355, 0.7413307428359985], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "591", "similarity": [0.7937461584806442, 0.7451080977916718, 0.7179076075553894], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "592", "similarity": [0.7971246242523193, 0.7378025054931641, 0.7313423752784729], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "594", "similarity": [0.839431956410408, 0.7714495956897736, 0.7501881122589111], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "595", "similarity": [0.8362892866134644, 0.8021578788757324, 0.7774516195058823], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "600", "similarity": [0.8096437305212021, 0.7779613584280014, 0.7265678942203522], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "601", "similarity": [0.8204912841320038, 0.787810355424881, 0.7575357258319855], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "602", "similarity": [0.8267884254455566, 0.7810387015342712, 0.7691070139408112], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "603", "similarity": [0.8232736736536026, 0.7783141434192657, 0.7781451493501663], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "606", "similarity": [0.750757709145546, 0.727537989616394, 0.7224106192588806], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "607", "similarity": [0.7759218513965607, 0.7543666362762451, 0.7409155368804932], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "609", "similarity": [0.7347753047943115, 0.7279931604862213, 0.7273638844490051], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "610", "similarity": [0.7627751231193542, 0.740424394607544, 0.7374442219734192], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "611", "similarity": [0.7679294645786285, 0.7610408067703247, 0.7533142566680908], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "612", "similarity": [0.8821331560611725, 0.7518459856510162, 0.7477491796016693], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "614", "similarity": [0.9017741531133652, 0.7809935510158539, 0.769224613904953], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "615", "similarity": [0.9029609560966492, 0.7768963277339935, 0.7558552920818329], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "616", "similarity": [0.7907443940639496, 0.7437035739421844, 0.7339965999126434], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "617", "similarity": [0.7813747525215149, 0.7397114932537079, 0.7262601256370544], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "618", "similarity": [0.7099215388298035, 0.6962088942527771, 0.6869371831417084], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "620", "similarity": [0.7218672633171082, 0.713262289762497, 0.710917592048645], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "621", "similarity": [0.8684546947479248, 0.7828826904296875, 0.7382872700691223], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "622", "similarity": [0.8168721199035645, 0.7997741997241974, 0.796265035867691], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "624", "similarity": [0.8063040524721146, 0.7288092374801636, 0.7231293618679047], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "625", "similarity": [0.8009082227945328, 0.7466286718845367, 0.7459809184074402], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "626", "similarity": [0.8750921785831451, 0.7665399610996246, 0.7632743716239929], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "631", "similarity": [0.9076938480138779, 0.7678332328796387, 0.7570955157279968], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "632", "similarity": [0.8071456253528595, 0.7632428705692291, 0.7487988471984863], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "633", "similarity": [0.7999032586812973, 0.7367984652519226, 0.7341923713684082], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "635", "similarity": [0.8385961949825287, 0.786901593208313, 0.7725078165531158], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "638", "similarity": [0.6789764165878296, 0.6755789220333099, 0.6722744107246399], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "640", "similarity": [0.7534099072217941, 0.7460885047912598, 0.7407481670379639], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "642", "similarity": [0.7686727494001389, 0.7598511725664139, 0.7500396817922592], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "643", "similarity": [0.8023082762956619, 0.7840522229671478, 0.7824215292930603], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "645", "similarity": [0.8432025164365768, 0.8281956613063812, 0.8235684931278229], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "646", "similarity": [0.8676139265298843, 0.8573531210422516, 0.8404635637998581], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "647", "similarity": [0.6973212361335754, 0.6805143356323242, 0.6725983023643494], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "648", "similarity": [0.7645771652460098, 0.6937139928340912, 0.6803711652755737], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "650", "similarity": [0.6678709387779236, 0.6376749873161316, 0.5976994335651398], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "651", "similarity": [0.8172009587287903, 0.8056614845991135, 0.7761200368404388], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "652", "similarity": [0.8082528710365295, 0.8027731776237488, 0.7648601233959198], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "653", "similarity": [0.8917482793331146, 0.8256288468837738, 0.8155095428228378], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "654", "similarity": [0.7850146293640137, 0.7766844034194946, 0.7738217711448669], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "655", "similarity": [0.7805096209049225, 0.7765169143676758, 0.7662976384162903], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "657", "similarity": [0.7509233951568604, 0.7275756597518921, 0.7268141210079193], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "658", "similarity": [0.7854704260826111, 0.7521674185991287, 0.7449151873588562], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "661", "similarity": [0.8348927199840546, 0.6992660164833069, 0.694811999797821], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "662", "similarity": [0.8133942484855652, 0.7032884955406189, 0.6966816186904907], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "663", "similarity": [0.8211402893066406, 0.7778372913599014, 0.77754345536232], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "665", "similarity": [0.8601456433534622, 0.7736488282680511, 0.7557117342948914], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "666", "similarity": [0.850131094455719, 0.8273941427469254, 0.8168899416923523], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "667", "similarity": [0.8547398149967194, 0.83725206553936, 0.8234689831733704], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "668", "similarity": [0.8687056601047516, 0.8168555200099945, 0.8088171035051346], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "669", "similarity": [0.8288978338241577, 0.7917438000440598, 0.7844952791929245], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "670", "similarity": [0.8095563352108002, 0.7996747642755508, 0.7213427424430847], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "671", "similarity": [0.8276043087244034, 0.8094975352287292, 0.7245045602321625], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "672", "similarity": [0.8181031048297882, 0.7700462192296982, 0.750457227230072], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "673", "similarity": [0.8063361346721649, 0.7888921797275543, 0.7827435731887817], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "676", "similarity": [0.6965290904045105, 0.6903015673160553, 0.6889573335647583], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "677", "similarity": [0.6898801624774933, 0.6805703043937683, 0.6778800785541534], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "679", "similarity": [0.7249082326889038, 0.7236112058162689, 0.721375048160553], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "680", "similarity": [0.7997374683618546, 0.7717077881097794, 0.7678134739398956], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "681", "similarity": [0.8670855462551117, 0.775579571723938, 0.7660472691059113], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "682", "similarity": [0.8022426068782806, 0.7774588614702225, 0.7740497142076492], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "683", "similarity": [0.8619808852672577, 0.7740795463323593, 0.7719045728445053], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "685", "similarity": [0.7971104681491852, 0.7687287777662277, 0.7643282860517502], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "686", "similarity": [0.7107270658016205, 0.6759070158004761, 0.6719257235527039], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "687", "similarity": [0.7150942087173462, 0.7148874998092651, 0.7043114304542542], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "688", "similarity": [0.7393090724945068, 0.7322761416435242, 0.7178425788879395], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "689", "similarity": [0.7502168118953705, 0.7420904636383057, 0.7388607859611511], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "694", "similarity": [0.7964365035295486, 0.7477402091026306, 0.7390053868293762], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "696", "similarity": [0.7539165169000626, 0.7513799667358398, 0.7485659420490265], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "698", "similarity": [0.7706145346164703, 0.7626806199550629, 0.7322136461734772], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "703", "similarity": [0.8383234143257141, 0.8285835385322571, 0.7760291397571564], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "704", "similarity": [0.8679492175579071, 0.7932506203651428, 0.7798978686332703], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "705", "similarity": [0.8103006333112717, 0.7877953350543976, 0.7712800353765488], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "709", "similarity": [0.8274313658475876, 0.808691993355751, 0.7522976696491241], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "710", "similarity": [0.793739378452301, 0.7818190306425095, 0.7746629863977432], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "713", "similarity": [0.8032367527484894, 0.7988155484199524, 0.7749946415424347], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "714", "similarity": [0.8032543063163757, 0.7982801496982574, 0.766046553850174], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "717", "similarity": [0.7884392738342285, 0.7464303374290466, 0.7396201193332672], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "724", "similarity": [0.8808983266353607, 0.7285520136356354, 0.7126335501670837], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "726", "similarity": [0.8255225419998169, 0.7318083941936493, 0.7306639552116394], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "730", "similarity": [0.8437144160270691, 0.7694779336452484, 0.7631988525390625], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "732", "similarity": [0.7163704037666321, 0.7059911787509918, 0.7032646238803864], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "733", "similarity": [0.723460853099823, 0.7224564552307129, 0.7188536524772644], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "734", "similarity": [0.809464305639267, 0.7673287391662598, 0.7547140419483185], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "736", "similarity": [0.8916817381978035, 0.7714681029319763, 0.7662953287363052], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "737", "similarity": [0.8389841318130493, 0.828109934926033, 0.7513737380504608], "relevant": [true, false, true]} +{"split": "train", "index": "answerable", "query_id": "738", "similarity": [0.838642805814743, 0.8261473774909973, 0.7524771094322205], "relevant": [true, false, true]} +{"split": "train", "index": "answerable", "query_id": "739", "similarity": [0.7571854889392853, 0.7462649345397949, 0.7445719838142395], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "740", "similarity": [0.8653978109359741, 0.7451772093772888, 0.7164890170097351], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "745", "similarity": [0.808016300201416, 0.718946099281311, 0.7087541818618774], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "746", "similarity": [0.8268809020519257, 0.7101989686489105, 0.7062446475028992], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "747", "similarity": [0.791101723909378, 0.7334296405315399, 0.7222139835357666], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "748", "similarity": [0.7989277243614197, 0.7393054366111755, 0.7259688973426819], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "749", "similarity": [0.7922426760196686, 0.7773850858211517, 0.775256335735321], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "751", "similarity": [0.8420079797506332, 0.7374675869941711, 0.7271431386470795], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "752", "similarity": [0.8191435933113098, 0.7226511836051941, 0.7153142094612122], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "753", "similarity": [0.7550563514232635, 0.7225726246833801, 0.701239138841629], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "755", "similarity": [0.7480941712856293, 0.7324312329292297, 0.7269082963466644], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "757", "similarity": [0.8117633014917374, 0.751469150185585, 0.744934618473053], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "758", "similarity": [0.7959991991519928, 0.7608086466789246, 0.7320583462715149], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "760", "similarity": [0.8503089100122452, 0.8409194052219391, 0.8280619978904724], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "761", "similarity": [0.858815386891365, 0.8527800291776657, 0.8352398574352264], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "762", "similarity": [0.7354868650436401, 0.7309287190437317, 0.7140659987926483], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "764", "similarity": [0.7521709203720093, 0.7208198010921478, 0.7144337594509125], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "765", "similarity": [0.7601679563522339, 0.7600177824497223, 0.7245045304298401], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "766", "similarity": [0.7802191376686096, 0.7599180489778519, 0.73366978764534], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "767", "similarity": [0.8074373155832291, 0.7394118010997772, 0.7236438095569611], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "771", "similarity": [0.8278285562992096, 0.7688468098640442, 0.7353808879852295], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "772", "similarity": [0.7928536236286163, 0.705479621887207, 0.7006737589836121], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "774", "similarity": [0.801011711359024, 0.7747075855731964, 0.7512310892343521], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "776", "similarity": [0.7953521758317947, 0.7570894956588745, 0.7455210387706757], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "777", "similarity": [0.7973565459251404, 0.7642276883125305, 0.7485178112983704], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "778", "similarity": [0.9178216457366943, 0.8503748178482056, 0.8077827990055084], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "779", "similarity": [0.9268618747591972, 0.8608434200286865, 0.8049572110176086], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "780", "similarity": [0.885567232966423, 0.8712374716997147, 0.8651392757892609], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "782", "similarity": [0.804231584072113, 0.7975089102983475, 0.7886494100093842], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "787", "similarity": [0.827848270535469, 0.7391039133071899, 0.7356665730476379], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "788", "similarity": [0.8392334878444672, 0.7314905822277069, 0.731465220451355], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "789", "similarity": [0.8147610425949097, 0.7837952077388763, 0.7794433385133743], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "790", "similarity": [0.781557098031044, 0.7701910138130188, 0.7692896127700806], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "791", "similarity": [0.8338236808776855, 0.806316688656807, 0.7977703809738159], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "792", "similarity": [0.802955836057663, 0.722818911075592, 0.6735801100730896], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "795", "similarity": [0.8036012351512909, 0.7890514135360718, 0.7715860605239868], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "797", "similarity": [0.7855600714683533, 0.7282567620277405, 0.7234041392803192], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "798", "similarity": [0.7859642654657364, 0.7715334594249725, 0.7249912023544312], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "799", "similarity": [0.7948629856109619, 0.7678020149469376, 0.7627875804901123], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "801", "similarity": [0.8794440999627113, 0.74568310379982, 0.7446142435073853], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "802", "similarity": [0.8762379437685013, 0.774838924407959, 0.7600924372673035], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "803", "similarity": [0.8853499889373779, 0.7417340874671936, 0.7377223372459412], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "804", "similarity": [0.8521168828010559, 0.7611710727214813, 0.7499198317527771], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "807", "similarity": [0.7402983903884888, 0.665814995765686, 0.6612484455108643], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "810", "similarity": [0.7357175648212433, 0.7168549597263336, 0.7068305015563965], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "812", "similarity": [0.7837216705083847, 0.7189671695232391, 0.7178508043289185], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "813", "similarity": [0.9006145671010017, 0.738783597946167, 0.7369035482406616], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "815", "similarity": [0.8102766126394272, 0.7517403066158295, 0.7349398732185364], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "816", "similarity": [0.7854543626308441, 0.7516177296638489, 0.726654589176178], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "817", "similarity": [0.7073197066783905, 0.6925304532051086, 0.677672803401947], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "818", "similarity": [0.7172259092330933, 0.6964311301708221, 0.6933293640613556], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "822", "similarity": [0.8458169400691986, 0.8161345720291138, 0.7886572182178497], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "825", "similarity": [0.8486387431621552, 0.8172162175178528, 0.799265593290329], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "826", "similarity": [0.7844380885362625, 0.7831602394580841, 0.7684875875711441], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "828", "similarity": [0.7271105647087097, 0.7246025800704956, 0.7244063913822174], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "835", "similarity": [0.7438382506370544, 0.7418850958347321, 0.736731082201004], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "838", "similarity": [0.74385204911232, 0.7426156997680664, 0.7411195635795593], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "840", "similarity": [0.8153374791145325, 0.7709458768367767, 0.7594282478094101], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "841", "similarity": [0.7739625126123428, 0.7611455470323563, 0.7525176703929901], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "844", "similarity": [0.8557151108980179, 0.8277643322944641, 0.8133211731910706], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "846", "similarity": [0.7558400928974152, 0.7320988178253174, 0.7194762229919434], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "848", "similarity": [0.770084023475647, 0.7428392171859741, 0.6545960903167725], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "853", "similarity": [0.8019897639751434, 0.7934530973434448, 0.7656076848506927], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "854", "similarity": [0.8139833807945251, 0.7630944848060608, 0.7405787110328674], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "855", "similarity": [0.8406430780887604, 0.7670384347438812, 0.730956494808197], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "856", "similarity": [0.7850127518177032, 0.7185004353523254, 0.7154105007648468], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "857", "similarity": [0.7663156092166901, 0.7039053738117218, 0.7011780738830566], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "858", "similarity": [0.7456038296222687, 0.7421378195285797, 0.7375540137290955], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "860", "similarity": [0.7589690983295441, 0.7520494312047958, 0.7216998338699341], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "861", "similarity": [0.7752079516649246, 0.7495481371879578, 0.7231614589691162], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "863", "similarity": [0.8315854072570801, 0.8272912204265594, 0.8014323115348816], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "866", "similarity": [0.8281862735748291, 0.814848318696022, 0.7921213805675507], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "867", "similarity": [0.8329892456531525, 0.791371539235115, 0.7571572959423065], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "871", "similarity": [0.7546938359737396, 0.7390194535255432, 0.7333924770355225], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "876", "similarity": [0.7293016314506531, 0.7270817756652832, 0.7093775570392609], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "877", "similarity": [0.7948048710823059, 0.7594139873981476, 0.7554209232330322], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "881", "similarity": [0.7943063527345657, 0.7260026633739471, 0.7122210264205933], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "883", "similarity": [0.7989579439163208, 0.7183352708816528, 0.7178047299385071], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "884", "similarity": [0.8039573431015015, 0.7369839549064636, 0.728388249874115], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "885", "similarity": [0.797065943479538, 0.6671206951141357, 0.6624412536621094], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "886", "similarity": [0.7904758602380753, 0.6707760095596313, 0.6673046350479126], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "890", "similarity": [0.7382434606552124, 0.7280863523483276, 0.7073297798633575], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "891", "similarity": [0.7383506000041962, 0.7285906672477722, 0.7105085253715515], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "893", "similarity": [0.8810287266969681, 0.7605312764644623, 0.7510315626859665], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "894", "similarity": [0.7405959963798523, 0.7389715015888214, 0.7307273149490356], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "895", "similarity": [0.7081639170646667, 0.7069540023803711, 0.6881061494350433], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "896", "similarity": [0.7296015024185181, 0.7113987803459167, 0.7109827995300293], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "897", "similarity": [0.7439222931861877, 0.7278873026371002, 0.7202640473842621], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "898", "similarity": [0.8008821159601212, 0.7568112313747406, 0.7271930575370789], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "899", "similarity": [0.8020182251930237, 0.7410225868225098, 0.7304472625255585], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "900", "similarity": [0.7926431596279144, 0.7624494284391403, 0.7592556178569794], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "901", "similarity": [0.8331272304058075, 0.7988950908184052, 0.7599821984767914], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "902", "similarity": [0.8181126117706299, 0.754242792725563, 0.7515291273593903], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "908", "similarity": [0.8869540691375732, 0.8216206729412079, 0.7639961540699005], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "909", "similarity": [0.7554930448532104, 0.7477361261844635, 0.7326869368553162], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "910", "similarity": [0.7786519825458527, 0.7411478757858276, 0.7343249917030334], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "912", "similarity": [0.79078209400177, 0.7405704855918884, 0.7262744903564453], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "916", "similarity": [0.7372337579727173, 0.7156241238117218, 0.7152287065982819], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "917", "similarity": [0.8440102338790894, 0.7535901367664337, 0.7528033256530762], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "919", "similarity": [0.8820896744728088, 0.8336595296859741, 0.7799831181764603], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "923", "similarity": [0.8153675347566605, 0.7315139174461365, 0.723044216632843], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "925", "similarity": [0.8488390445709229, 0.7362455725669861, 0.7185382544994354], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "926", "similarity": [0.7875040769577026, 0.7576473653316498, 0.7433277070522308], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "927", "similarity": [0.792409136891365, 0.772347092628479, 0.75074303150177], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "928", "similarity": [0.8094804286956787, 0.800159752368927, 0.7986780852079391], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "929", "similarity": [0.8042383790016174, 0.7948403060436249, 0.79262974858284], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "930", "similarity": [0.7571748197078705, 0.7268863618373871, 0.7213022708892822], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "933", "similarity": [0.8050578981637955, 0.7525765746831894, 0.7321294546127319], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "934", "similarity": [0.6902641952037811, 0.6845209002494812, 0.6780474185943604], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "935", "similarity": [0.7512212693691254, 0.7509172856807709, 0.7490985989570618], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "938", "similarity": [0.7546387910842896, 0.7499330937862396, 0.7391771078109741], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "939", "similarity": [0.7883596420288086, 0.7528213113546371, 0.7513607442378998], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "940", "similarity": [0.8239181935787201, 0.7444093227386475, 0.7400923073291779], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "941", "similarity": [0.8261829018592834, 0.7442225217819214, 0.7132763266563416], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "942", "similarity": [0.7328991889953613, 0.7305219173431396, 0.6998265981674194], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "944", "similarity": [0.8683575689792633, 0.7956591546535492, 0.7476993799209595], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "945", "similarity": [0.7882467210292816, 0.7663561105728149, 0.7511470019817352], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "946", "similarity": [0.806108757853508, 0.7840558737516403, 0.7724235355854034], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "949", "similarity": [0.7103632986545563, 0.7096782326698303, 0.7036439478397369], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "951", "similarity": [0.8019553869962692, 0.8013897240161896, 0.7830590009689331], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "952", "similarity": [0.8573231995105743, 0.8381377458572388, 0.7413673996925354], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "953", "similarity": [0.8676624894142151, 0.8387051522731781, 0.7562851309776306], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "954", "similarity": [0.8754015266895294, 0.8503528237342834, 0.7382790744304657], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "955", "similarity": [0.8131008297204971, 0.79011370241642, 0.7848748564720154], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "959", "similarity": [0.8270676881074905, 0.7443468272686005, 0.7440708875656128], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "962", "similarity": [0.8503915965557098, 0.7883048057556152, 0.7748455107212067], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "963", "similarity": [0.7990561723709106, 0.7883995771408081, 0.7707941234111786], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "964", "similarity": [0.8029077053070068, 0.7875388711690903, 0.7798086106777191], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "965", "similarity": [0.7984504103660583, 0.7854383885860443, 0.7432243227958679], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "969", "similarity": [0.8385936915874481, 0.827216774225235, 0.8219519257545471], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "970", "similarity": [0.8360921442508698, 0.8248811960220337, 0.8193785697221756], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "972", "similarity": [0.8610276579856873, 0.8584420531988144, 0.8578550219535828], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "973", "similarity": [0.8530328869819641, 0.8462266474962234, 0.8416590988636017], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "976", "similarity": [0.7509341537952423, 0.7386078536510468, 0.7278020083904266], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "977", "similarity": [0.7669423222541809, 0.7504903376102448, 0.7484643459320068], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "978", "similarity": [0.7630377858877182, 0.7561012208461761, 0.7439178824424744], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "979", "similarity": [0.8307358920574188, 0.7730239629745483, 0.7715601325035095], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "980", "similarity": [0.8261099457740784, 0.7415502369403839, 0.7357895076274872], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "984", "similarity": [0.8345543742179871, 0.7448683381080627, 0.7440845668315887], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "988", "similarity": [0.7116624712944031, 0.7023470103740692, 0.7021383345127106], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "989", "similarity": [0.7582820355892181, 0.7571038603782654, 0.7426852881908417], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "990", "similarity": [0.8265350461006165, 0.7603559195995331, 0.7601180225610733], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "992", "similarity": [0.7383913993835449, 0.7301704287528992, 0.7271365821361542], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "994", "similarity": [0.7990368008613586, 0.7530112713575363, 0.730272114276886], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "996", "similarity": [0.7329501509666443, 0.717174619436264, 0.7112597525119781], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "997", "similarity": [0.7517288625240326, 0.7381479740142822, 0.7332055568695068], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "998", "similarity": [0.7831122726202011, 0.7494730055332184, 0.7357588112354279], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "999", "similarity": [0.792180985212326, 0.758182555437088, 0.7264441549777985], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1000", "similarity": [0.8493355512619019, 0.8399275839328766, 0.7744677662849426], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1001", "similarity": [0.7983088195323944, 0.7674045264720917, 0.7636613547801971], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1002", "similarity": [0.7267246544361115, 0.7244704365730286, 0.7237834334373474], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1003", "similarity": [0.8820209205150604, 0.835933655500412, 0.796100452542305], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1004", "similarity": [0.8462212532758713, 0.8353467583656311, 0.8296568989753723], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1005", "similarity": [0.8486182540655136, 0.8376973271369934, 0.8353819102048874], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1006", "similarity": [0.8449366390705109, 0.7500372231006622, 0.7445518374443054], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1008", "similarity": [0.6656312346458435, 0.663727343082428, 0.6626081466674805], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1009", "similarity": [0.8328834474086761, 0.7500120997428894, 0.7477516531944275], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1011", "similarity": [0.910947747528553, 0.8499154150485992, 0.8197534084320068], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1015", "similarity": [0.8629602789878845, 0.781498908996582, 0.7700998932123184], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1016", "similarity": [0.8017535507678986, 0.7651303112506866, 0.7339853346347809], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1018", "similarity": [0.772017240524292, 0.7632372081279755, 0.7133059799671173], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1023", "similarity": [0.8266031742095947, 0.7599268853664398, 0.7471069097518921], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1025", "similarity": [0.7785683870315552, 0.7660516202449799, 0.7590863704681396], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1026", "similarity": [0.7960101664066315, 0.7804818451404572, 0.7407881617546082], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1027", "similarity": [0.8178786188364029, 0.7849875390529633, 0.76154425740242], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1028", "similarity": [0.8618821203708649, 0.8164515197277069, 0.8011361658573151], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1030", "similarity": [0.8531309366226196, 0.8160981088876724, 0.7991757392883301], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1031", "similarity": [0.8142156600952148, 0.7488709092140198, 0.7317734360694885], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1032", "similarity": [0.7087749540805817, 0.7012769281864166, 0.6856142282485962], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1033", "similarity": [0.7090161144733429, 0.6935286521911621, 0.6827674508094788], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1034", "similarity": [0.8256979137659073, 0.7857226133346558, 0.7829702496528625], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1035", "similarity": [0.8358866572380066, 0.7901736497879028, 0.7776326090097427], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1036", "similarity": [0.8109264671802521, 0.7740311622619629, 0.76955246925354], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1037", "similarity": [0.8399792611598969, 0.7608411312103271, 0.7573316991329193], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1038", "similarity": [0.8337345123291016, 0.7520299851894379, 0.7510363757610321], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1040", "similarity": [0.8726554065942764, 0.8556417673826218, 0.850968673825264], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1042", "similarity": [0.8412312865257263, 0.7702829986810684, 0.7624458223581314], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1043", "similarity": [0.8966350182890892, 0.8200756013393402, 0.7680630385875702], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1044", "similarity": [0.8465864658355713, 0.813885286450386, 0.8026711046695709], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1045", "similarity": [0.8694534599781036, 0.822735607624054, 0.821233719587326], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1046", "similarity": [0.7657424509525299, 0.7479530572891235, 0.7423962652683258], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1047", "similarity": [0.7629158347845078, 0.757500410079956, 0.7398114800453186], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1048", "similarity": [0.7588323801755905, 0.7287680506706238, 0.6919870376586914], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1050", "similarity": [0.8843087777495384, 0.8502438366413116, 0.8017205893993378], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1052", "similarity": [0.7777526825666428, 0.7481845021247864, 0.7435440421104431], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1053", "similarity": [0.7769794464111328, 0.7546882778406143, 0.7515071332454681], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1054", "similarity": [0.865217998623848, 0.8237361460924149, 0.7692524790763855], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1055", "similarity": [0.798666849732399, 0.6842359900474548, 0.6785028278827667], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1056", "similarity": [0.7759224474430084, 0.7117655873298645, 0.688514232635498], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1058", "similarity": [0.7264112532138824, 0.7066402435302734, 0.7035789489746094], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1065", "similarity": [0.8260906040668488, 0.7627216726541519, 0.736193835735321], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1067", "similarity": [0.7473624050617218, 0.7451508939266205, 0.7323092222213745], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1068", "similarity": [0.7470147013664246, 0.745197981595993, 0.7378225028514862], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1069", "similarity": [0.8090501725673676, 0.7103495895862579, 0.6829668879508972], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1070", "similarity": [0.6938135027885437, 0.6659897863864899, 0.6651551127433777], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1072", "similarity": [0.7945558875799179, 0.7812539786100388, 0.7764405608177185], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1073", "similarity": [0.8084756433963776, 0.7994497418403625, 0.7712922990322113], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1074", "similarity": [0.7475562989711761, 0.7301547527313232, 0.7293511629104614], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1075", "similarity": [0.7420191764831543, 0.7169780433177948, 0.7144778370857239], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1081", "similarity": [0.8287108540534973, 0.7761693894863129, 0.7586312890052795], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1084", "similarity": [0.8181868344545364, 0.7633379995822906, 0.7518914937973022], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1085", "similarity": [0.815987691283226, 0.7605259418487549, 0.7570421397686005], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1087", "similarity": [0.8820889666676521, 0.7020715475082397, 0.6810908317565918], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1090", "similarity": [0.8258684277534485, 0.7787383943796158, 0.7380810081958771], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1091", "similarity": [0.7622055560350418, 0.761833906173706, 0.7609482109546661], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1096", "similarity": [0.785455733537674, 0.7795210927724838, 0.7736160159111023], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1097", "similarity": [0.7523523420095444, 0.7249884605407715, 0.7198195457458496], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1098", "similarity": [0.6665371656417847, 0.6557440757751465, 0.6538390517234802], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1101", "similarity": [0.8245618939399719, 0.7778476774692535, 0.7572317272424698], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1102", "similarity": [0.8209609687328339, 0.7645393908023834, 0.7643821835517883], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1103", "similarity": [0.8376265317201614, 0.7583572566509247, 0.746524453163147], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1105", "similarity": [0.8159435391426086, 0.760391116142273, 0.6958648264408112], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1106", "similarity": [0.8044769763946533, 0.7671150118112564, 0.6840510964393616], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1109", "similarity": [0.7628272622823715, 0.7528801560401917, 0.7480077147483826], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1111", "similarity": [0.7216687798500061, 0.6979867219924927, 0.6941133439540863], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1112", "similarity": [0.7096224427223206, 0.7045440673828125, 0.702228307723999], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1114", "similarity": [0.7347972691059113, 0.6514759659767151, 0.6476187407970428], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1115", "similarity": [0.8722987025976181, 0.7905796617269516, 0.6888029277324677], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1118", "similarity": [0.7995619028806686, 0.7838832437992096, 0.7648280262947083], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1119", "similarity": [0.8540148437023163, 0.8475401997566223, 0.8463270962238312], "relevant": [false, true, true]} +{"split": "train", "index": "answerable", "query_id": "1120", "similarity": [0.8461803644895554, 0.8348343670368195, 0.8314347565174103], "relevant": [true, true, true]} +{"split": "train", "index": "answerable", "query_id": "1125", "similarity": [0.8379108011722565, 0.7444025278091431, 0.736393928527832], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1126", "similarity": [0.8475892543792725, 0.7454472184181213, 0.7411778569221497], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1127", "similarity": [0.7624961286783218, 0.7576964199542999, 0.7551779001951218], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1128", "similarity": [0.809174045920372, 0.7994157373905182, 0.7760883271694183], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1133", "similarity": [0.8535109907388687, 0.7224153876304626, 0.7152541875839233], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1134", "similarity": [0.8044715523719788, 0.7197994589805603, 0.7145183682441711], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1135", "similarity": [0.8157930970191956, 0.7291499078273773, 0.7220125496387482], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1136", "similarity": [0.8134805262088776, 0.7303658127784729, 0.7241285443305969], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1138", "similarity": [0.7958751767873764, 0.7644109427928925, 0.7591228783130646], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1139", "similarity": [0.7547314763069153, 0.7343789339065552, 0.733781635761261], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1141", "similarity": [0.8150985240936279, 0.7722900807857513, 0.7397933602333069], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1142", "similarity": [0.7614957690238953, 0.74363973736763, 0.7318915128707886], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1145", "similarity": [0.8662234842777252, 0.7915880382061005, 0.7898464649915695], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1148", "similarity": [0.7866864055395126, 0.753456711769104, 0.7442952990531921], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1153", "similarity": [0.8706367313861847, 0.7474693655967712, 0.7442169785499573], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1156", "similarity": [0.7300963997840881, 0.7056097686290741, 0.7043372392654419], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1157", "similarity": [0.6726245880126953, 0.5982733368873596, 0.593912661075592], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1158", "similarity": [0.659281849861145, 0.5902798175811768, 0.5840377807617188], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1159", "similarity": [0.6810646653175354, 0.6175147891044617, 0.6108599007129669], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1161", "similarity": [0.7528535276651382, 0.7502782940864563, 0.74137082695961], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1162", "similarity": [0.7883584201335907, 0.700724720954895, 0.6957291662693024], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1164", "similarity": [0.7746501564979553, 0.765903890132904, 0.7517634630203247], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1165", "similarity": [0.7524697780609131, 0.7461255788803101, 0.7410194873809814], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1166", "similarity": [0.7503015398979187, 0.7366451025009155, 0.7279684245586395], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1168", "similarity": [0.7335118651390076, 0.7306457161903381, 0.7169109284877777], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1169", "similarity": [0.7866526246070862, 0.7739813327789307, 0.7720394134521484], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1170", "similarity": [0.8472492396831512, 0.7779661417007446, 0.7646422982215881], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1171", "similarity": [0.8656387776136398, 0.7709612548351288, 0.7576873600482941], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1173", "similarity": [0.8036873042583466, 0.7346437275409698, 0.7202604413032532], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1174", "similarity": [0.6582577228546143, 0.6580683588981628, 0.657436341047287], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1176", "similarity": [0.8480446934700012, 0.7428351044654846, 0.7140675783157349], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1177", "similarity": [0.8203848600387573, 0.7359096705913544, 0.7323784530162811], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1178", "similarity": [0.7163034081459045, 0.7125367522239685, 0.7062589824199677], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1181", "similarity": [0.8589077889919281, 0.8268034607172012, 0.8262852430343628], "relevant": [true, true, true]} +{"split": "train", "index": "answerable", "query_id": "1182", "similarity": [0.8527604639530182, 0.8400330692529678, 0.8351099491119385], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1183", "similarity": [0.7923339605331421, 0.7281382381916046, 0.7229679822921753], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1184", "similarity": [0.8007275015115738, 0.7912818342447281, 0.7819187045097351], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1186", "similarity": [0.673378050327301, 0.6619417071342468, 0.6604689955711365], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1188", "similarity": [0.781534269452095, 0.7472604811191559, 0.7417531609535217], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1190", "similarity": [0.6671825647354126, 0.6607522666454315, 0.6426924169063568], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1193", "similarity": [0.7800511717796326, 0.7623727470636368, 0.738323986530304], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1195", "similarity": [0.7594172358512878, 0.7592852860689163, 0.7534511089324951], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1205", "similarity": [0.813304215669632, 0.7447939217090607, 0.7312864363193512], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1206", "similarity": [0.7686936408281326, 0.729873776435852, 0.7278819978237152], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1208", "similarity": [0.7604275196790695, 0.7371688485145569, 0.7309963703155518], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1209", "similarity": [0.7677058428525925, 0.7436337471008301, 0.7260614931583405], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1210", "similarity": [0.7999733984470367, 0.7232895195484161, 0.7214600145816803], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1211", "similarity": [0.7742965966463089, 0.7075820565223694, 0.7058398723602295], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1212", "similarity": [0.7483029067516327, 0.7473549544811249, 0.7287211418151855], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1214", "similarity": [0.7469580173492432, 0.742387056350708, 0.7354199290275574], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1215", "similarity": [0.7505136281251907, 0.7504944205284119, 0.7435985207557678], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1218", "similarity": [0.8152779042720795, 0.7790800929069519, 0.7519600540399551], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1219", "similarity": [0.8199833184480667, 0.8055682182312012, 0.7823396325111389], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1220", "similarity": [0.8084553778171539, 0.7902956008911133, 0.7741862237453461], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1223", "similarity": [0.7494103908538818, 0.740333616733551, 0.7306410372257233], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1224", "similarity": [0.881818138062954, 0.8791269138455391, 0.8737442493438721], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1227", "similarity": [0.7959557622671127, 0.7897576987743378, 0.7047947645187378], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1228", "similarity": [0.8123592138290405, 0.7884754836559296, 0.7267667055130005], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1229", "similarity": [0.7712590098381042, 0.7600196599960327, 0.7564376443624496], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1230", "similarity": [0.8513919860124588, 0.744617223739624, 0.7262831926345825], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1231", "similarity": [0.8142222315073013, 0.7326832115650177, 0.7202796339988708], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1234", "similarity": [0.8126256316900253, 0.7359524965286255, 0.7220979928970337], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1235", "similarity": [0.8214781284332275, 0.8057977855205536, 0.7886789888143539], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1236", "similarity": [0.8246312141418457, 0.8154988437891006, 0.7961212992668152], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1237", "similarity": [0.7711710035800934, 0.7214123606681824, 0.716849684715271], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1238", "similarity": [0.7738404721021652, 0.7468968033790588, 0.7160899937152863], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1239", "similarity": [0.7917342334985733, 0.7713826447725296, 0.7634824216365814], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1244", "similarity": [0.7506600618362427, 0.7473787665367126, 0.7231443524360657], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1246", "similarity": [0.640532910823822, 0.636176586151123, 0.62366783618927], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1247", "similarity": [0.6834089756011963, 0.6610089838504791, 0.6561354100704193], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1248", "similarity": [0.8238555490970612, 0.7539302557706833, 0.7211357355117798], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1249", "similarity": [0.8148635476827621, 0.7270060777664185, 0.7114799916744232], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1253", "similarity": [0.7178986668586731, 0.7093921303749084, 0.7079170346260071], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1254", "similarity": [0.7873894274234772, 0.7576692253351212, 0.7421499788761139], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1255", "similarity": [0.7403502464294434, 0.7311533093452454, 0.7203367352485657], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1257", "similarity": [0.7321399450302124, 0.7156320214271545, 0.7138335108757019], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1258", "similarity": [0.7088688313961029, 0.6965362429618835, 0.6946381628513336], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1260", "similarity": [0.8460547626018524, 0.8246602714061737, 0.7736005485057831], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1261", "similarity": [0.8185825645923615, 0.7888357937335968, 0.7758171707391739], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1263", "similarity": [0.8345031142234802, 0.8077424913644791, 0.7986447513103485], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1265", "similarity": [0.8582984358072281, 0.8229336440563202, 0.8150872439146042], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1268", "similarity": [0.799137532711029, 0.786033108830452, 0.7806877046823502], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1269", "similarity": [0.8007786720991135, 0.6971688866615295, 0.68408203125], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1275", "similarity": [0.8725337088108063, 0.8324843347072601, 0.8272392749786377], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1276", "similarity": [0.8011693954467773, 0.798363983631134, 0.7305687069892883], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1284", "similarity": [0.8416735678911209, 0.7255374193191528, 0.7018266916275024], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1288", "similarity": [0.8902101367712021, 0.7815888822078705, 0.744818925857544], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1289", "similarity": [0.716447502374649, 0.664328932762146, 0.6507286727428436], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1291", "similarity": [0.7539853304624557, 0.7216142117977142, 0.7145121097564697], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1293", "similarity": [0.8150130808353424, 0.7115666270256042, 0.6991639137268066], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1294", "similarity": [0.790450781583786, 0.7655034065246582, 0.762460857629776], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1295", "similarity": [0.7149604856967926, 0.6616585552692413, 0.6586233377456665], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1297", "similarity": [0.8229981064796448, 0.7124563753604889, 0.7096561789512634], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1300", "similarity": [0.7198328971862793, 0.7113838791847229, 0.6955954432487488], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1302", "similarity": [0.6787577271461487, 0.6753587126731873, 0.6747417449951172], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1304", "similarity": [0.707335352897644, 0.7020653188228607, 0.6945831775665283], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1305", "similarity": [0.7259074747562408, 0.6627052426338196, 0.6617379188537598], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1306", "similarity": [0.7809685617685318, 0.7752000838518143, 0.773018091917038], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1307", "similarity": [0.7127151191234589, 0.7036095857620239, 0.6968755424022675], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1308", "similarity": [0.7415428161621094, 0.7366088032722473, 0.735999196767807], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1309", "similarity": [0.7172083258628845, 0.7164248526096344, 0.7068226337432861], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1310", "similarity": [0.6478099524974823, 0.6344208717346191, 0.6077778935432434], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1311", "similarity": [0.7730823755264282, 0.7485357224941254, 0.7339045405387878], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1312", "similarity": [0.7527627497911453, 0.747255802154541, 0.7461741268634796], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1314", "similarity": [0.7867420464754105, 0.7822768986225128, 0.7614134848117828], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1315", "similarity": [0.7863766998052597, 0.7827333509922028, 0.7602973729372025], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1322", "similarity": [0.8209279179573059, 0.7736445814371109, 0.7264952063560486], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1323", "similarity": [0.780267983675003, 0.7577110677957535, 0.740676999092102], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1324", "similarity": [0.7727006375789642, 0.7205967307090759, 0.7202180624008179], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1325", "similarity": [0.679835706949234, 0.6762182116508484, 0.675610214471817], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1327", "similarity": [0.7782852798700333, 0.7612844407558441, 0.7493210136890411], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1328", "similarity": [0.8514085710048676, 0.8345562219619751, 0.7611636519432068], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1330", "similarity": [0.7765937894582748, 0.7759107947349548, 0.7532674819231033], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1331", "similarity": [0.7698952853679657, 0.7495735883712769, 0.7459672093391418], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1333", "similarity": [0.8801332116127014, 0.7396338582038879, 0.733180582523346], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1334", "similarity": [0.7718786895275116, 0.76320680975914, 0.7454763650894165], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1340", "similarity": [0.7790743410587311, 0.6786149740219116, 0.6680285930633545], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1341", "similarity": [0.7892820239067078, 0.6939012110233307, 0.6747142970561981], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1342", "similarity": [0.7583879828453064, 0.7543212622404099, 0.7538482248783112], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1345", "similarity": [0.7714502811431885, 0.7707046866416931, 0.7612437009811401], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1346", "similarity": [0.7958939224481583, 0.7892387807369232, 0.7847459316253662], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1347", "similarity": [0.8692599385976791, 0.8296005129814148, 0.7950258553028107], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1348", "similarity": [0.868915468454361, 0.8483973741531372, 0.7981808483600616], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1349", "similarity": [0.8417955189943314, 0.7638999223709106, 0.7270225286483765], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1350", "similarity": [0.8473409563302994, 0.7717137783765793, 0.7288267314434052], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1351", "similarity": [0.7992516905069351, 0.7825589179992676, 0.7719734907150269], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1353", "similarity": [0.8011661767959595, 0.7724070996046066, 0.7641543447971344], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1355", "similarity": [0.7431687116622925, 0.727251261472702, 0.7209344506263733], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1356", "similarity": [0.834776371717453, 0.7363413572311401, 0.7292818427085876], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1360", "similarity": [0.8653843700885773, 0.7980111241340637, 0.744180291891098], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1361", "similarity": [0.8294092565774918, 0.798822432756424, 0.7552832514047623], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1364", "similarity": [0.6510841548442841, 0.6478641033172607, 0.6439041793346405], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1366", "similarity": [0.7602935135364532, 0.7224477827548981, 0.6970880031585693], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1367", "similarity": [0.8272363543510437, 0.7751749455928802, 0.7610020041465759], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1371", "similarity": [0.8687934279441833, 0.7912206947803497, 0.7911145687103271], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1372", "similarity": [0.8069006204605103, 0.699622631072998, 0.6874270141124725], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1373", "similarity": [0.8201972842216492, 0.6991589069366455, 0.689741849899292], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1374", "similarity": [0.6819762587547302, 0.6497040688991547, 0.6419709324836731], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1375", "similarity": [0.6826499700546265, 0.6507920622825623, 0.6493946313858032], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1376", "similarity": [0.8089888095855713, 0.8028970062732697, 0.7914849519729614], "relevant": [false, false, true]} +{"split": "train", "index": "answerable", "query_id": "1378", "similarity": [0.8150003105401993, 0.7386638820171356, 0.7299855053424835], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1380", "similarity": [0.8549450486898422, 0.8500075340270996, 0.8483098745346069], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1381", "similarity": [0.8308640122413635, 0.7885770350694656, 0.7860381305217743], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1383", "similarity": [0.7906708121299744, 0.7504098117351532, 0.7479522824287415], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1386", "similarity": [0.7590246200561523, 0.728066086769104, 0.7057912945747375], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1387", "similarity": [0.7810551673173904, 0.7675543129444122, 0.7537312507629395], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1390", "similarity": [0.7684580236673355, 0.7657101154327393, 0.740533709526062], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1391", "similarity": [0.7897912561893463, 0.712019145488739, 0.7030500769615173], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1392", "similarity": [0.7945204377174377, 0.7319477498531342, 0.7295149564743042], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1393", "similarity": [0.7745547890663147, 0.7605706453323364, 0.7597777247428894], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1394", "similarity": [0.8155699819326401, 0.7866444885730743, 0.7840911746025085], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1397", "similarity": [0.7752045542001724, 0.7446959018707275, 0.735956072807312], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1398", "similarity": [0.7656487226486206, 0.7373034060001373, 0.7372013330459595], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1400", "similarity": [0.7747121751308441, 0.7667337357997894, 0.7394778728485107], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1401", "similarity": [0.7735694646835327, 0.7468122243881226, 0.7248751521110535], "relevant": [false, true, false]} +{"split": "train", "index": "answerable", "query_id": "1402", "similarity": [0.7279670834541321, 0.7278309762477875, 0.7188257575035095], "relevant": [false, false, false]} +{"split": "train", "index": "answerable", "query_id": "1403", "similarity": [0.8084254264831543, 0.7615374624729156, 0.7486324310302734], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1404", "similarity": [0.8182784914970398, 0.771178126335144, 0.7514962553977966], "relevant": [true, true, false]} +{"split": "train", "index": "answerable", "query_id": "1405", "similarity": [0.784251943230629, 0.7468580007553101, 0.7451568245887756], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1406", "similarity": [0.7386927604675293, 0.6826524436473846, 0.6781324744224548], "relevant": [true, false, false]} +{"split": "train", "index": "answerable", "query_id": "1407", "similarity": [0.7519550621509552, 0.7481494545936584, 0.7462581396102905], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "0", "similarity": [0.6743555963039398, 0.6661394834518433, 0.6351119875907898], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "2", "similarity": [0.7202525734901428, 0.6952096521854401, 0.6807268261909485], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "4", "similarity": [0.7936265915632248, 0.7816746234893799, 0.7523935288190842], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "6", "similarity": [0.8078370690345764, 0.7642121315002441, 0.7531138360500336], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "9", "similarity": [0.7408380806446075, 0.7341804504394531, 0.7070913314819336], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "10", "similarity": [0.7664345651865005, 0.7504134923219681, 0.7402638792991638], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "11", "similarity": [0.7722457051277161, 0.7426230907440186, 0.7263981699943542], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "12", "similarity": [0.8153871446847916, 0.7706069052219391, 0.7657620310783386], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "14", "similarity": [0.7969818413257599, 0.7070271968841553, 0.698044091463089], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "15", "similarity": [0.7319148182868958, 0.7297512888908386, 0.7294167280197144], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "17", "similarity": [0.773045539855957, 0.7599047124385834, 0.7567489743232727], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "18", "similarity": [0.803011104464531, 0.770806223154068, 0.7480430603027344], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "19", "similarity": [0.7531127482652664, 0.7511198222637177, 0.7463624477386475], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "20", "similarity": [0.7516311407089233, 0.7503232061862946, 0.7477818131446838], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "21", "similarity": [0.6730573773384094, 0.6586595177650452, 0.6576074957847595], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "22", "similarity": [0.7302623689174652, 0.7275663912296295, 0.7241469919681549], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "24", "similarity": [0.8075330853462219, 0.7869404703378677, 0.7519597709178925], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "25", "similarity": [0.8125565648078918, 0.7638443857431412, 0.7521581947803497], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "26", "similarity": [0.7605648636817932, 0.7454597353935242, 0.7449705004692078], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "27", "similarity": [0.7591478228569031, 0.7437950372695923, 0.7427179217338562], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "28", "similarity": [0.7876337766647339, 0.7743129730224609, 0.7687095105648041], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "30", "similarity": [0.8279979526996613, 0.797067403793335, 0.7970393896102905], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "32", "similarity": [0.6801932454109192, 0.6662082076072693, 0.6527736485004425], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "34", "similarity": [0.8263355195522308, 0.8111550807952881, 0.7993204444646835], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "35", "similarity": [0.8109435439109802, 0.7897277325391769, 0.7644847333431244], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "37", "similarity": [0.8001414090394974, 0.7902652621269226, 0.7773182988166809], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "39", "similarity": [0.7283899188041687, 0.7024332880973816, 0.6936116218566895], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "40", "similarity": [0.7205093801021576, 0.7106808722019196, 0.701792448759079], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "41", "similarity": [0.7950724959373474, 0.7617624253034592, 0.7448392808437347], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "43", "similarity": [0.7557521462440491, 0.7376760840415955, 0.730961412191391], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "44", "similarity": [0.736034095287323, 0.7342734038829803, 0.7278039157390594], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "45", "similarity": [0.7364977598190308, 0.7362096309661865, 0.7218332290649414], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "46", "similarity": [0.7303387522697449, 0.7209568023681641, 0.7207206189632416], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "47", "similarity": [0.773116260766983, 0.7382197976112366, 0.7288577258586884], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "52", "similarity": [0.7389041781425476, 0.7365627288818359, 0.7353872656822205], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "55", "similarity": [0.8379458338022232, 0.8255658596754074, 0.8195455521345139], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "58", "similarity": [0.8113747835159302, 0.794045940041542, 0.7918629348278046], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "60", "similarity": [0.6817617118358612, 0.6649458408355713, 0.6623082756996155], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "61", "similarity": [0.747697114944458, 0.7468201816082001, 0.7382126450538635], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "62", "similarity": [0.7509466409683228, 0.7099989056587219, 0.6901913583278656], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "63", "similarity": [0.7915595769882202, 0.7913290709257126, 0.7756114900112152], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "64", "similarity": [0.7735163867473602, 0.7733036577701569, 0.7690877914428711], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "66", "similarity": [0.6906289160251617, 0.6856237649917603, 0.6737444996833801], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "67", "similarity": [0.7949395179748535, 0.779300644993782, 0.77044577896595], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "68", "similarity": [0.7901908457279205, 0.7732623368501663, 0.74782395362854], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "69", "similarity": [0.7696284353733063, 0.7629066556692123, 0.7549821138381958], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "71", "similarity": [0.7771604359149933, 0.7642926871776581, 0.7497268915176392], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "73", "similarity": [0.653090238571167, 0.6530661284923553, 0.6459601521492004], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "74", "similarity": [0.7719530165195465, 0.7713121622800827, 0.7536675781011581], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "76", "similarity": [0.7247202396392822, 0.7166720032691956, 0.7162139415740967], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "77", "similarity": [0.7269737124443054, 0.7231139838695526, 0.7199029326438904], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "78", "similarity": [0.7621912062168121, 0.7576114684343338, 0.7517169415950775], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "79", "similarity": [0.8251993805170059, 0.7722349464893341, 0.7517187595367432], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "80", "similarity": [0.7131943702697754, 0.7057628333568573, 0.7019301056861877], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "81", "similarity": [0.773992508649826, 0.7323450446128845, 0.7306736409664154], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "82", "similarity": [0.7698315531015396, 0.7660190016031265, 0.7628508359193802], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "85", "similarity": [0.7553403526544571, 0.7343727946281433, 0.7330269813537598], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "86", "similarity": [0.7553403526544571, 0.7343727946281433, 0.7330269813537598], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "88", "similarity": [0.7567655444145203, 0.7490067481994629, 0.7258516550064087], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "89", "similarity": [0.829504519701004, 0.7943258285522461, 0.7574195861816406], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "90", "similarity": [0.829504519701004, 0.7943258285522461, 0.7574195861816406], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "91", "similarity": [0.7364961504936218, 0.7274278998374939, 0.7168539762496948], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "92", "similarity": [0.7494840025901794, 0.7345928251743317, 0.7327717542648315], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "93", "similarity": [0.8077520877122879, 0.7975044101476669, 0.767619714140892], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "95", "similarity": [0.692511260509491, 0.6869924664497375, 0.6639431416988373], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "96", "similarity": [0.6947263479232788, 0.64202880859375, 0.6083950996398926], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "98", "similarity": [0.7088478803634644, 0.7058668434619904, 0.7022247910499573], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "104", "similarity": [0.7117479741573334, 0.7067061960697174, 0.6921114921569824], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "105", "similarity": [0.6639677286148071, 0.6420066952705383, 0.6364181041717529], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "106", "similarity": [0.728417843580246, 0.7185796499252319, 0.7040062546730042], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "108", "similarity": [0.8696193397045135, 0.7527582347393036, 0.7110337615013123], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "109", "similarity": [0.7664453089237213, 0.7661898583173752, 0.7636139690876007], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "111", "similarity": [0.8229000717401505, 0.8078939020633698, 0.7946768403053284], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "112", "similarity": [0.7969379276037216, 0.7762596011161804, 0.7743415981531143], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "114", "similarity": [0.5942170023918152, 0.5932196378707886, 0.5900485217571259], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "116", "similarity": [0.6569671630859375, 0.6271463632583618, 0.6270986795425415], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "119", "similarity": [0.7423906922340393, 0.7338029146194458, 0.7256566286087036], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "120", "similarity": [0.74610635638237, 0.7402622997760773, 0.7129555642604828], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "121", "similarity": [0.686120331287384, 0.6793652772903442, 0.6792430877685547], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "122", "similarity": [0.6871328949928284, 0.6857678294181824, 0.6730887293815613], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "123", "similarity": [0.8500168174505234, 0.7735156118869781, 0.7721417248249054], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "126", "similarity": [0.7466034293174744, 0.7291136384010315, 0.728581428527832], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "134", "similarity": [0.7376936674118042, 0.7198386490345001, 0.717206597328186], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "138", "similarity": [0.7229597866535187, 0.7226295471191406, 0.7115136384963989], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "139", "similarity": [0.7608333826065063, 0.7111278176307678, 0.7087642252445221], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "144", "similarity": [0.7670129686594009, 0.7446509599685669, 0.7352847754955292], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "149", "similarity": [0.827512726187706, 0.8057051599025726, 0.777649387717247], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "152", "similarity": [0.7431421875953674, 0.7230039536952972, 0.7128510177135468], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "153", "similarity": [0.7835433781147003, 0.7780543863773346, 0.7765051573514938], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "154", "similarity": [0.7863014042377472, 0.7776831537485123, 0.7740245461463928], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "155", "similarity": [0.8375220149755478, 0.7714906334877014, 0.7568400502204895], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "156", "similarity": [0.8122534453868866, 0.7867436707019806, 0.7757784724235535], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "157", "similarity": [0.7683407664299011, 0.750388041138649, 0.7397310435771942], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "159", "similarity": [0.842844620347023, 0.8309024721384048, 0.8169040083885193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "160", "similarity": [0.7890415638685226, 0.7043436169624329, 0.6781620383262634], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "161", "similarity": [0.6847791075706482, 0.6744487881660461, 0.672580897808075], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "164", "similarity": [0.7745620310306549, 0.7705127596855164, 0.7645782381296158], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "165", "similarity": [0.8316074311733246, 0.797494500875473, 0.7768280804157257], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "166", "similarity": [0.8140323013067245, 0.7988773882389069, 0.7643180936574936], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "167", "similarity": [0.832714393734932, 0.814598023891449, 0.7689167559146881], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "168", "similarity": [0.7668047547340393, 0.7632088959217072, 0.760837510228157], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "169", "similarity": [0.8160370886325836, 0.7842923551797867, 0.7831099033355713], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "172", "similarity": [0.7758011221885681, 0.75599405169487, 0.7422971129417419], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "173", "similarity": [0.7178650796413422, 0.7063077092170715, 0.7018230259418488], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "174", "similarity": [0.746100664138794, 0.7011155188083649, 0.6987755298614502], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "175", "similarity": [0.734713613986969, 0.7130447924137115, 0.700180172920227], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "176", "similarity": [0.7473401427268982, 0.6962134838104248, 0.6947311162948608], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "177", "similarity": [0.8139075040817261, 0.7937134504318237, 0.7920214384794235], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "178", "similarity": [0.8031602501869202, 0.7765081226825714, 0.7738562822341919], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "181", "similarity": [0.7591406255960464, 0.7417440116405487, 0.7359500527381897], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "182", "similarity": [0.7346388101577759, 0.7318356037139893, 0.7308130860328674], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "184", "similarity": [0.7627602219581604, 0.751897782087326, 0.7481544017791748], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "186", "similarity": [0.6415234804153442, 0.6314411759376526, 0.5994780659675598], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "187", "similarity": [0.6479557454586029, 0.6087019741535187, 0.5960513353347778], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "189", "similarity": [0.7760195136070251, 0.7699835449457169, 0.7300823330879211], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "196", "similarity": [0.7194398939609528, 0.7191303372383118, 0.7184400856494904], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "197", "similarity": [0.7531260102987289, 0.7163334488868713, 0.7074184417724609], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "199", "similarity": [0.7120475172996521, 0.6992188692092896, 0.6981609761714935], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "200", "similarity": [0.7777671217918396, 0.7767321914434433, 0.7696073055267334], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "201", "similarity": [0.792444720864296, 0.7789599299430847, 0.7741325199604034], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "203", "similarity": [0.7780070304870605, 0.7734811902046204, 0.7592159360647202], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "204", "similarity": [0.7356718182563782, 0.7115798592567444, 0.7079093158245087], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "205", "similarity": [0.7265685200691223, 0.7011190950870514, 0.6891753673553467], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "209", "similarity": [0.7107541561126709, 0.6859710812568665, 0.6783170700073242], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "210", "similarity": [0.6934970021247864, 0.693207323551178, 0.6929682493209839], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "211", "similarity": [0.6759992837905884, 0.6610491871833801, 0.6584566831588745], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "214", "similarity": [0.8390742838382721, 0.8375431299209595, 0.779787689447403], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "220", "similarity": [0.7833092212677002, 0.7570644468069077, 0.7570329010486603], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "221", "similarity": [0.7577261477708817, 0.7466467916965485, 0.7275600135326385], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "222", "similarity": [0.7552010267972946, 0.7240142226219177, 0.7133079469203949], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "223", "similarity": [0.7896290123462677, 0.769658550620079, 0.7647881805896759], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "224", "similarity": [0.9544168412685394, 0.8999269157648087, 0.8750919103622437], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "225", "similarity": [0.9002565070986748, 0.8864379301667213, 0.8500816226005554], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "226", "similarity": [0.9112038984894753, 0.8909343555569649, 0.868279442191124], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "227", "similarity": [0.7441785037517548, 0.7428009212017059, 0.7401016354560852], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "228", "similarity": [0.7248694002628326, 0.7230833768844604, 0.7186226844787598], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "229", "similarity": [0.7429361045360565, 0.7313541173934937, 0.7271401286125183], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "235", "similarity": [0.7732241898775101, 0.7712610363960266, 0.7481139004230499], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "241", "similarity": [0.7905431836843491, 0.7689495980739594, 0.7656590640544891], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "242", "similarity": [0.7711471915245056, 0.7652585208415985, 0.7556554973125458], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "243", "similarity": [0.7777058184146881, 0.7772357761859894, 0.7764475494623184], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "244", "similarity": [0.751402273774147, 0.7336758673191071, 0.7301607429981232], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "245", "similarity": [0.7824402153491974, 0.7626601159572601, 0.7560359388589859], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "246", "similarity": [0.7489398717880249, 0.7466635704040527, 0.7459060251712799], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "247", "similarity": [0.7978346943855286, 0.7726992666721344, 0.7589726150035858], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "250", "similarity": [0.7416687607765198, 0.7377647757530212, 0.7359915673732758], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "251", "similarity": [0.7682634890079498, 0.7430208027362823, 0.7420796751976013], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "253", "similarity": [0.838812530040741, 0.7822377234697342, 0.7628329694271088], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "254", "similarity": [0.8421264141798019, 0.7811212241649628, 0.7636570483446121], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "255", "similarity": [0.7803323119878769, 0.7633304297924042, 0.7577584385871887], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "256", "similarity": [0.8055005371570587, 0.7534071803092957, 0.7498359382152557], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "258", "similarity": [0.7783207446336746, 0.7656569927930832, 0.7492944002151489], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "259", "similarity": [0.8224280774593353, 0.7673878371715546, 0.7276119291782379], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "262", "similarity": [0.819299042224884, 0.7846173644065857, 0.7762330621480942], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "263", "similarity": [0.7869455814361572, 0.7643697261810303, 0.7575117647647858], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "264", "similarity": [0.7778819501399994, 0.7631869316101074, 0.754646897315979], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "265", "similarity": [0.7665570080280304, 0.7454635500907898, 0.7454483509063721], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "266", "similarity": [0.7554333508014679, 0.7410260438919067, 0.7397151291370392], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "267", "similarity": [0.7059914469718933, 0.70528244972229, 0.7042689919471741], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "272", "similarity": [0.8176601231098175, 0.7572607398033142, 0.7473922073841095], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "277", "similarity": [0.8045091480016708, 0.7943011522293091, 0.7209595143795013], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "278", "similarity": [0.7804588675498962, 0.7553446739912033, 0.7381503283977509], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "280", "similarity": [0.7816796451807022, 0.755294606089592, 0.7439766824245453], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "281", "similarity": [0.7617455273866653, 0.7416425347328186, 0.7380707263946533], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "283", "similarity": [0.8512682318687439, 0.7989916205406189, 0.7946998775005341], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "285", "similarity": [0.7767836451530457, 0.7399886846542358, 0.7090740203857422], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "286", "similarity": [0.7636047303676605, 0.7617461532354355, 0.7588805854320526], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "287", "similarity": [0.7656799256801605, 0.7554545104503632, 0.7534299045801163], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "290", "similarity": [0.6973913908004761, 0.6931456327438354, 0.6919934153556824], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "292", "similarity": [0.6784105896949768, 0.665621280670166, 0.6416434049606323], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "293", "similarity": [0.7751148343086243, 0.7501926720142365, 0.7467072606086731], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "296", "similarity": [0.781454473733902, 0.7734027355909348, 0.7611364424228668], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "299", "similarity": [0.7704409062862396, 0.754976212978363, 0.7436389327049255], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "301", "similarity": [0.739851713180542, 0.7309857308864594, 0.7257967591285706], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "304", "similarity": [0.7276181578636169, 0.7201343178749084, 0.7175693809986115], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "305", "similarity": [0.7457069158554077, 0.7402111291885376, 0.7394351065158844], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "306", "similarity": [0.7227696776390076, 0.7170228064060211, 0.7165250778198242], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "308", "similarity": [0.7733506858348846, 0.7671698033809662, 0.7486645579338074], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "309", "similarity": [0.7170924544334412, 0.7170799970626831, 0.7150355875492096], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "310", "similarity": [0.7474476099014282, 0.74595907330513, 0.7235594987869263], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "313", "similarity": [0.7606395781040192, 0.7572888284921646, 0.7267559468746185], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "315", "similarity": [0.8346422016620636, 0.7842191457748413, 0.7781549394130707], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "316", "similarity": [0.7668445706367493, 0.7290200293064117, 0.7283411622047424], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "317", "similarity": [0.7549627870321274, 0.7418418228626251, 0.7385295033454895], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "323", "similarity": [0.7036052942276001, 0.6943826079368591, 0.6894238591194153], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "325", "similarity": [0.7472081780433655, 0.7277472913265228, 0.7273038625717163], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "326", "similarity": [0.7604025304317474, 0.7574030160903931, 0.739469051361084], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "330", "similarity": [0.6064719259738922, 0.6042637526988983, 0.5971347689628601], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "331", "similarity": [0.6003256440162659, 0.598145067691803, 0.5977296233177185], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "332", "similarity": [0.8330013751983643, 0.7760626077651978, 0.7733902931213379], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "333", "similarity": [0.8280045688152313, 0.7722956538200378, 0.772090882062912], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "334", "similarity": [0.7318542003631592, 0.7287138104438782, 0.7278897166252136], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "335", "similarity": [0.7277260422706604, 0.7253406047821045, 0.7249982953071594], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "336", "similarity": [0.7219460010528564, 0.7078419327735901, 0.6910052001476288], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "337", "similarity": [0.7206664681434631, 0.7111210227012634, 0.6981758773326874], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "339", "similarity": [0.7766936123371124, 0.7339113354682922, 0.7326063513755798], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "340", "similarity": [0.7472544610500336, 0.7228391468524933, 0.7190118432044983], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "341", "similarity": [0.7394987940788269, 0.7221116721630096, 0.7165306806564331], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "342", "similarity": [0.7861088365316391, 0.7800518274307251, 0.7698532044887543], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "345", "similarity": [0.7858441174030304, 0.7354811429977417, 0.7263548970222473], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "346", "similarity": [0.7770395427942276, 0.7488808333873749, 0.7484247982501984], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "347", "similarity": [0.7794892191886902, 0.7618178129196167, 0.7522100955247879], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "349", "similarity": [0.7299363911151886, 0.7123643457889557, 0.7095620036125183], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "351", "similarity": [0.748631477355957, 0.7352218925952911, 0.7307652533054352], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "352", "similarity": [0.6983654201030731, 0.6983175873756409, 0.6888350546360016], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "355", "similarity": [0.7636400014162064, 0.7381472587585449, 0.7273174524307251], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "356", "similarity": [0.7809641361236572, 0.7620969712734222, 0.7158841788768768], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "357", "similarity": [0.7753032743930817, 0.7554234564304352, 0.7518662214279175], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "358", "similarity": [0.7777228355407715, 0.7552768588066101, 0.7504657804965973], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "361", "similarity": [0.8031536787748337, 0.7963901907205582, 0.7757875025272369], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "363", "similarity": [0.7316132187843323, 0.7301232218742371, 0.7261697053909302], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "364", "similarity": [0.7199723124504089, 0.7199244797229767, 0.714759111404419], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "365", "similarity": [0.7508397400379181, 0.7445455193519592, 0.7417135238647461], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "366", "similarity": [0.7459114193916321, 0.7380953431129456, 0.7270409464836121], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "367", "similarity": [0.8452742397785187, 0.8318142890930176, 0.794167548418045], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "368", "similarity": [0.833355575799942, 0.8266983032226562, 0.776526689529419], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "369", "similarity": [0.820567786693573, 0.8127890974283218, 0.8075849860906601], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "370", "similarity": [0.7539039552211761, 0.7513227462768555, 0.7486061453819275], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "371", "similarity": [0.758394181728363, 0.7573489844799042, 0.754201740026474], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "372", "similarity": [0.7404491901397705, 0.732106626033783, 0.7209745645523071], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "375", "similarity": [0.7374249696731567, 0.7292406558990479, 0.7215518951416016], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "376", "similarity": [0.7498992681503296, 0.748746395111084, 0.7349723875522614], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "377", "similarity": [0.6991153359413147, 0.6967560350894928, 0.6935345530509949], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "378", "similarity": [0.7629097998142242, 0.7557880729436874, 0.7464435696601868], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "379", "similarity": [0.7657669335603714, 0.7540490478277206, 0.7526653409004211], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "381", "similarity": [0.763795018196106, 0.7584256231784821, 0.756153404712677], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "382", "similarity": [0.7188830077648163, 0.717323899269104, 0.7143609523773193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "383", "similarity": [0.7449514269828796, 0.725062370300293, 0.720287948846817], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "389", "similarity": [0.7250365316867828, 0.7057153284549713, 0.6979816257953644], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "390", "similarity": [0.7147364020347595, 0.7117703557014465, 0.6958765685558319], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "391", "similarity": [0.7358254790306091, 0.7076335251331329, 0.7007665038108826], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "392", "similarity": [0.7303867340087891, 0.7172231376171112, 0.7084306180477142], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "393", "similarity": [0.7439828515052795, 0.7106620967388153, 0.7024618983268738], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "394", "similarity": [0.7990979105234146, 0.7837949246168137, 0.7798386067152023], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "396", "similarity": [0.771126389503479, 0.7630386054515839, 0.7514984309673309], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "397", "similarity": [0.7674490809440613, 0.7655914276838303, 0.7588768750429153], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "398", "similarity": [0.8309390991926193, 0.7598005533218384, 0.7343215346336365], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "400", "similarity": [0.76393261551857, 0.7427095174789429, 0.7414951622486115], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "401", "similarity": [0.6983528137207031, 0.696308046579361, 0.6957342326641083], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "403", "similarity": [0.7338289022445679, 0.7310670614242554, 0.7273328304290771], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "404", "similarity": [0.7404117286205292, 0.735698401927948, 0.7251268029212952], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "406", "similarity": [0.7934073507785797, 0.7796372026205063, 0.7750330567359924], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "407", "similarity": [0.7907141447067261, 0.7820418924093246, 0.7778795510530472], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "413", "similarity": [0.8334698528051376, 0.8332948535680771, 0.8116761744022369], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "414", "similarity": [0.8389291763305664, 0.8383483290672302, 0.8165377676486969], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "416", "similarity": [0.8031213283538818, 0.7974547445774078, 0.7871087491512299], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "417", "similarity": [0.8058040291070938, 0.8008300960063934, 0.7833793014287949], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "418", "similarity": [0.7442044019699097, 0.7435019910335541, 0.740953803062439], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "420", "similarity": [0.8119044452905655, 0.7832479178905487, 0.760636568069458], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "422", "similarity": [0.7426939606666565, 0.7366817593574524, 0.7124320268630981], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "423", "similarity": [0.8621344268321991, 0.820083498954773, 0.8058556318283081], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "425", "similarity": [0.7959323972463608, 0.7137619256973267, 0.7073786854743958], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "426", "similarity": [0.8137271106243134, 0.746715247631073, 0.7432023584842682], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "428", "similarity": [0.773147851228714, 0.7622964382171631, 0.7534873485565186], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "429", "similarity": [0.7564771175384521, 0.7535692602396011, 0.739320695400238], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "430", "similarity": [0.7720558345317841, 0.7553471028804779, 0.7523102909326553], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "432", "similarity": [0.7852071523666382, 0.7720938920974731, 0.7696898579597473], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "434", "similarity": [0.808689147233963, 0.786549910902977, 0.777647852897644], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "435", "similarity": [0.8076968789100647, 0.7848390340805054, 0.78081876039505], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "441", "similarity": [0.7660639733076096, 0.7211592793464661, 0.7171247899532318], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "444", "similarity": [0.8067832887172699, 0.7695930600166321, 0.7683048993349075], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "445", "similarity": [0.7844165116548538, 0.764088898897171, 0.7627144157886505], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "447", "similarity": [0.8350792527198792, 0.8219677954912186, 0.7970877438783646], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "448", "similarity": [0.8430374562740326, 0.8129747211933136, 0.7954511344432831], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "449", "similarity": [0.8152835667133331, 0.758063405752182, 0.7437411546707153], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "453", "similarity": [0.7139587998390198, 0.693849503993988, 0.6790202558040619], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "454", "similarity": [0.717708557844162, 0.6865943968296051, 0.6858850419521332], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "455", "similarity": [0.7912134230136871, 0.756369948387146, 0.746599018573761], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "456", "similarity": [0.7246090471744537, 0.7148322761058807, 0.7125334143638611], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "458", "similarity": [0.7911069691181183, 0.7825569212436676, 0.775775820016861], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "461", "similarity": [0.7337647378444672, 0.7328118085861206, 0.7298890948295593], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "463", "similarity": [0.7522185742855072, 0.7470166087150574, 0.7431104481220245], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "466", "similarity": [0.8235295116901398, 0.8224178850650787, 0.7450781464576721], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "469", "similarity": [0.7758375108242035, 0.7617533951997757, 0.7087381184101105], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "470", "similarity": [0.8145812302827835, 0.8022085428237915, 0.7642954140901566], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "472", "similarity": [0.8017789423465729, 0.7936558425426483, 0.7669838070869446], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "473", "similarity": [0.744611382484436, 0.7319713234901428, 0.7266983985900879], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "474", "similarity": [0.7574525773525238, 0.7275685966014862, 0.7140130400657654], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "479", "similarity": [0.7241749167442322, 0.7196911573410034, 0.7188668847084045], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "480", "similarity": [0.7233226299285889, 0.7179554104804993, 0.716350644826889], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "481", "similarity": [0.7274763882160187, 0.7225792109966278, 0.7181436717510223], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "482", "similarity": [0.7940271198749542, 0.7699302583932877, 0.7594105005264282], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "483", "similarity": [0.8354425877332687, 0.8171805143356323, 0.80718132853508], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "484", "similarity": [0.7796555459499359, 0.7695381492376328, 0.7510660737752914], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "485", "similarity": [0.7721941620111465, 0.755938395857811, 0.7543618977069855], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "486", "similarity": [0.7833331227302551, 0.7661490142345428, 0.758819192647934], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "487", "similarity": [0.7928760349750519, 0.761897623538971, 0.7535775154829025], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "488", "similarity": [0.7641455829143524, 0.7589432150125504, 0.74277663230896], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "489", "similarity": [0.7852666974067688, 0.7643639147281647, 0.7546597719192505], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "490", "similarity": [0.7284731268882751, 0.7187975645065308, 0.7151761054992676], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "492", "similarity": [0.7570655196905136, 0.751641646027565, 0.7484045922756195], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "493", "similarity": [0.7325196862220764, 0.7237964868545532, 0.7141005992889404], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "494", "similarity": [0.6732895970344543, 0.6708688735961914, 0.6654185652732849], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "495", "similarity": [0.7166905403137207, 0.6944868564605713, 0.6915726661682129], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "498", "similarity": [0.7328599095344543, 0.7109872698783875, 0.7065249681472778], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "499", "similarity": [0.7663809359073639, 0.7443576455116272, 0.7429574131965637], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "500", "similarity": [0.7617614418268204, 0.7532487064599991, 0.7467074990272522], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "504", "similarity": [0.7673095613718033, 0.7218812108039856, 0.7133083641529083], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "505", "similarity": [0.835754320025444, 0.7940398752689362, 0.7782316505908966], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "506", "similarity": [0.77230004966259, 0.7568075209856033, 0.7311896681785583], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "509", "similarity": [0.7415445446968079, 0.7138466238975525, 0.7010796070098877], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "515", "similarity": [0.8207335472106934, 0.7716128826141357, 0.756532147526741], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "523", "similarity": [0.7998229414224625, 0.7474994957447052, 0.7433373630046844], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "524", "similarity": [0.7987575829029083, 0.74582439661026, 0.7411057651042938], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "526", "similarity": [0.7823859006166458, 0.7629523575305939, 0.7518898695707321], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "529", "similarity": [0.8197894394397736, 0.8177839517593384, 0.7835080027580261], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "530", "similarity": [0.8244608938694, 0.8049950450658798, 0.7896179407835007], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "531", "similarity": [0.8303274512290955, 0.8289750665426254, 0.8090966939926147], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "537", "similarity": [0.7105554044246674, 0.709848165512085, 0.709653377532959], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "541", "similarity": [0.7550123184919357, 0.7490641474723816, 0.748663455247879], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "542", "similarity": [0.7211527824401855, 0.7073899507522583, 0.7025124430656433], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "545", "similarity": [0.7380990386009216, 0.7182808518409729, 0.7129628360271454], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "547", "similarity": [0.8143540620803833, 0.7799928188323975, 0.7776795029640198], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "548", "similarity": [0.8007048964500427, 0.752267986536026, 0.7493774890899658], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "550", "similarity": [0.7913141995668411, 0.760685607790947, 0.7604869157075882], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "553", "similarity": [0.8298218548297882, 0.7860737144947052, 0.7640659809112549], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "555", "similarity": [0.773094654083252, 0.7463534474372864, 0.7397324442863464], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "557", "similarity": [0.720844030380249, 0.7176457643508911, 0.7158521413803101], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "559", "similarity": [0.7534452378749847, 0.7163050174713135, 0.7162930369377136], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "562", "similarity": [0.6150684952735901, 0.6147632896900177, 0.6091788113117218], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "563", "similarity": [0.7361018657684326, 0.7356517016887665, 0.734354168176651], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "564", "similarity": [0.7681157439947128, 0.7475987374782562, 0.745546817779541], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "565", "similarity": [0.7497756779193878, 0.7439849376678467, 0.7353810667991638], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "566", "similarity": [0.7523233741521835, 0.7404789924621582, 0.740138977766037], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "568", "similarity": [0.7802299410104752, 0.7744504511356354, 0.769679844379425], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "570", "similarity": [0.7418638169765472, 0.7373995184898376, 0.7373107671737671], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "571", "similarity": [0.748290479183197, 0.7409789562225342, 0.7334262132644653], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "572", "similarity": [0.7271565794944763, 0.7260324358940125, 0.7244478464126587], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "573", "similarity": [0.7398793697357178, 0.7319880127906799, 0.7266956865787506], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "574", "similarity": [0.7446029186248779, 0.7423139810562134, 0.7287477850914001], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "576", "similarity": [0.8294616043567657, 0.7883845269680023, 0.7881584018468857], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "579", "similarity": [0.7567082047462463, 0.7179262936115265, 0.7147670388221741], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "580", "similarity": [0.7648082375526428, 0.7543285936117172, 0.7476340234279633], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "582", "similarity": [0.7758013755083084, 0.7452141642570496, 0.7383712530136108], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "584", "similarity": [0.8188939839601517, 0.8161137998104095, 0.7854167520999908], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "585", "similarity": [0.6700088679790497, 0.6656413376331329, 0.6652300953865051], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "588", "similarity": [0.7974539697170258, 0.7451485991477966, 0.7308088541030884], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "590", "similarity": [0.7425821125507355, 0.7413307428359985, 0.7387865781784058], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "591", "similarity": [0.7451080977916718, 0.7179076075553894, 0.7035652995109558], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "592", "similarity": [0.7378025054931641, 0.7313423752784729, 0.7036163210868835], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "594", "similarity": [0.7714495956897736, 0.7501881122589111, 0.7252589166164398], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "595", "similarity": [0.8021578788757324, 0.7774516195058823, 0.7524197399616241], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "600", "similarity": [0.7779613584280014, 0.7265678942203522, 0.7217729687690735], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "601", "similarity": [0.787810355424881, 0.7575357258319855, 0.745840847492218], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "602", "similarity": [0.8267884254455566, 0.7810387015342712, 0.7691070139408112], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "603", "similarity": [0.8232736736536026, 0.7783141434192657, 0.7781451493501663], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "606", "similarity": [0.750757709145546, 0.727537989616394, 0.719590961933136], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "607", "similarity": [0.7543666362762451, 0.7409155368804932, 0.7366493344306946], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "609", "similarity": [0.7347753047943115, 0.7279931604862213, 0.7273638844490051], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "610", "similarity": [0.7627751231193542, 0.740424394607544, 0.7374442219734192], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "611", "similarity": [0.7610408067703247, 0.7533142566680908, 0.7521534115076065], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "612", "similarity": [0.7518459856510162, 0.7477491796016693, 0.7406472563743591], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "614", "similarity": [0.7809935510158539, 0.769224613904953, 0.7507791817188263], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "615", "similarity": [0.7768963277339935, 0.7558552920818329, 0.7538757771253586], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "616", "similarity": [0.7907443940639496, 0.7437035739421844, 0.7339965999126434], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "617", "similarity": [0.7813747525215149, 0.7397114932537079, 0.7262601256370544], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "618", "similarity": [0.6962088942527771, 0.6869371831417084, 0.6815714836120605], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "620", "similarity": [0.7218672633171082, 0.710917592048645, 0.7073909342288971], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "621", "similarity": [0.7828826904296875, 0.7382872700691223, 0.7241593599319458], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "622", "similarity": [0.8168721199035645, 0.796265035867691, 0.7954546958208084], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "624", "similarity": [0.7288092374801636, 0.7231293618679047, 0.7210270166397095], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "625", "similarity": [0.7466286718845367, 0.7459809184074402, 0.7448886334896088], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "626", "similarity": [0.7665399610996246, 0.7632743716239929, 0.7507507652044296], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "631", "similarity": [0.7678332328796387, 0.7570955157279968, 0.7425901889801025], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "632", "similarity": [0.7632428705692291, 0.7487988471984863, 0.7475325465202332], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "633", "similarity": [0.7367984652519226, 0.7341923713684082, 0.7311365604400635], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "635", "similarity": [0.786901593208313, 0.7725078165531158, 0.7720777988433838], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "638", "similarity": [0.6789764165878296, 0.6722744107246399, 0.6655920743942261], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "640", "similarity": [0.7534099072217941, 0.7407481670379639, 0.7238442599773407], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "642", "similarity": [0.7686727494001389, 0.7500396817922592, 0.749942421913147], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "643", "similarity": [0.7824215292930603, 0.7786211967468262, 0.7714829444885254], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "645", "similarity": [0.8432025164365768, 0.8192502707242966, 0.8141939342021942], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "646", "similarity": [0.8573531210422516, 0.8384974002838135, 0.8169087022542953], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "647", "similarity": [0.6805143356323242, 0.6725983023643494, 0.6663665771484375], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "648", "similarity": [0.6937139928340912, 0.6803711652755737, 0.6694743633270264], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "650", "similarity": [0.5976994335651398, 0.5835953950881958, 0.5801671743392944], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "651", "similarity": [0.8056614845991135, 0.7761200368404388, 0.7468160390853882], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "652", "similarity": [0.8027731776237488, 0.7648601233959198, 0.7599263936281204], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "653", "similarity": [0.8155095428228378, 0.7770617604255676, 0.7683222591876984], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "654", "similarity": [0.7766844034194946, 0.7738217711448669, 0.760888934135437], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "655", "similarity": [0.7765169143676758, 0.7662976384162903, 0.7573173344135284], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "657", "similarity": [0.7275756597518921, 0.7268141210079193, 0.7196104526519775], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "658", "similarity": [0.7449151873588562, 0.7426898181438446, 0.739071249961853], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "661", "similarity": [0.6992660164833069, 0.694811999797821, 0.6947508752346039], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "662", "similarity": [0.7032884955406189, 0.6966816186904907, 0.6932341456413269], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "663", "similarity": [0.77754345536232, 0.7496978044509888, 0.7467899322509766], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "665", "similarity": [0.7736488282680511, 0.7557117342948914, 0.7525823712348938], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "666", "similarity": [0.8168899416923523, 0.7957369685173035, 0.7861902415752411], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "667", "similarity": [0.8204390555620193, 0.8097603172063828, 0.8039198815822601], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "668", "similarity": [0.8088171035051346, 0.7978813350200653, 0.7967068701982498], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "669", "similarity": [0.776995375752449, 0.7719022482633591, 0.7691591084003448], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "670", "similarity": [0.8095563352108002, 0.7213427424430847, 0.7121243476867676], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "671", "similarity": [0.8094975352287292, 0.7172150015830994, 0.7145721316337585], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "672", "similarity": [0.7700462192296982, 0.750457227230072, 0.7491567432880402], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "673", "similarity": [0.8063361346721649, 0.7827435731887817, 0.7788608223199844], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "676", "similarity": [0.6965290904045105, 0.6903015673160553, 0.6830572485923767], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "677", "similarity": [0.6898801624774933, 0.6805703043937683, 0.6778800785541534], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "679", "similarity": [0.7249082326889038, 0.7236112058162689, 0.721375048160553], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "680", "similarity": [0.7717077881097794, 0.7674135565757751, 0.7584739774465561], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "681", "similarity": [0.7660472691059113, 0.7652904391288757, 0.7639086842536926], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "682", "similarity": [0.7774588614702225, 0.7740497142076492, 0.7731667757034302], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "683", "similarity": [0.7740795463323593, 0.7705885469913483, 0.7670879065990448], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "685", "similarity": [0.7687287777662277, 0.7643282860517502, 0.7134677171707153], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "686", "similarity": [0.7107270658016205, 0.6759070158004761, 0.6719257235527039], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "687", "similarity": [0.7150942087173462, 0.7148874998092651, 0.6935938596725464], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "688", "similarity": [0.7393090724945068, 0.7178425788879395, 0.6668052673339844], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "689", "similarity": [0.7502168118953705, 0.7420904636383057, 0.7388607859611511], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "694", "similarity": [0.7477402091026306, 0.7390053868293762, 0.7208278179168701], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "696", "similarity": [0.7539165169000626, 0.7485659420490265, 0.7467148005962372], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "698", "similarity": [0.7706145346164703, 0.7626806199550629, 0.6951538622379303], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "703", "similarity": [0.8285835385322571, 0.7760291397571564, 0.769644245505333], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "704", "similarity": [0.8679492175579071, 0.7932506203651428, 0.7798978686332703], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "705", "similarity": [0.7877953350543976, 0.7712800353765488, 0.7612424045801163], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "709", "similarity": [0.808691993355751, 0.7522976696491241, 0.7453892230987549], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "710", "similarity": [0.793739378452301, 0.7746629863977432, 0.7688981592655182], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "713", "similarity": [0.7988155484199524, 0.7749946415424347, 0.7642834484577179], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "714", "similarity": [0.7982801496982574, 0.766046553850174, 0.7619592696428299], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "717", "similarity": [0.7396201193332672, 0.7345797419548035, 0.7341039776802063], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "724", "similarity": [0.7285520136356354, 0.7126335501670837, 0.7086562514305115], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "726", "similarity": [0.7318083941936493, 0.7306639552116394, 0.7299075722694397], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "730", "similarity": [0.7694779336452484, 0.7631988525390625, 0.727477490901947], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "732", "similarity": [0.7163704037666321, 0.7032646238803864, 0.7003713846206665], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "733", "similarity": [0.723460853099823, 0.7188536524772644, 0.7098270654678345], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "734", "similarity": [0.7547140419483185, 0.7475535571575165, 0.7475035786628723], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "736", "similarity": [0.7714681029319763, 0.7662953287363052, 0.7593231648206711], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "737", "similarity": [0.828109934926033, 0.7431663274765015, 0.7138491868972778], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "738", "similarity": [0.8261473774909973, 0.7334741950035095, 0.7052523791790009], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "739", "similarity": [0.7571854889392853, 0.7462649345397949, 0.7445719838142395], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "740", "similarity": [0.7451772093772888, 0.7164890170097351, 0.7159836292266846], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "745", "similarity": [0.718946099281311, 0.708685040473938, 0.7027754783630371], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "746", "similarity": [0.7101989686489105, 0.7062446475028992, 0.7037193179130554], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "747", "similarity": [0.7334296405315399, 0.7222139835357666, 0.7030145227909088], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "748", "similarity": [0.7393054366111755, 0.7259688973426819, 0.7089735865592957], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "749", "similarity": [0.7922426760196686, 0.7773850858211517, 0.775256335735321], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "751", "similarity": [0.7374675869941711, 0.7271431386470795, 0.7260202169418335], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "752", "similarity": [0.7226511836051941, 0.7153142094612122, 0.7139240801334381], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "753", "similarity": [0.7550563514232635, 0.7225726246833801, 0.701239138841629], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "755", "similarity": [0.7480941712856293, 0.7324312329292297, 0.7269082963466644], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "757", "similarity": [0.751469150185585, 0.744934618473053, 0.7424089908599854], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "758", "similarity": [0.7959991991519928, 0.7608086466789246, 0.7320583462715149], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "760", "similarity": [0.8409194052219391, 0.8280619978904724, 0.8040884435176849], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "761", "similarity": [0.858815386891365, 0.8527800291776657, 0.8352398574352264], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "762", "similarity": [0.7309287190437317, 0.7140659987926483, 0.7108508348464966], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "764", "similarity": [0.7208198010921478, 0.7044726610183716, 0.6999868750572205], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "765", "similarity": [0.7600177824497223, 0.7224271595478058, 0.707382321357727], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "766", "similarity": [0.7802191376686096, 0.73366978764534, 0.7253309488296509], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "767", "similarity": [0.7394118010997772, 0.7236438095569611, 0.7118019759654999], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "771", "similarity": [0.7688468098640442, 0.735121101140976, 0.7350068092346191], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "772", "similarity": [0.705479621887207, 0.7006737589836121, 0.6947582066059113], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "774", "similarity": [0.801011711359024, 0.7747075855731964, 0.7512310892343521], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "776", "similarity": [0.7953521758317947, 0.7570894956588745, 0.7455210387706757], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "777", "similarity": [0.7973565459251404, 0.7642276883125305, 0.7485178112983704], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "778", "similarity": [0.8503748178482056, 0.8077827990055084, 0.7714016735553741], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "779", "similarity": [0.8608434200286865, 0.8049572110176086, 0.7746706008911133], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "780", "similarity": [0.8651392757892609, 0.8347466886043549, 0.7894943356513977], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "782", "similarity": [0.804231584072113, 0.774621456861496, 0.7658772170543671], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "787", "similarity": [0.7391039133071899, 0.7356665730476379, 0.7322947084903717], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "788", "similarity": [0.7314905822277069, 0.731465220451355, 0.7314170897006989], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "789", "similarity": [0.7837952077388763, 0.7794433385133743, 0.7722109258174896], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "790", "similarity": [0.7701910138130188, 0.7692896127700806, 0.7467285394668579], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "791", "similarity": [0.806316688656807, 0.7977703809738159, 0.7681101262569427], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "792", "similarity": [0.722818911075592, 0.6735801100730896, 0.6699383556842804], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "795", "similarity": [0.7890514135360718, 0.7346176505088806, 0.7334099113941193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "797", "similarity": [0.7282567620277405, 0.7234041392803192, 0.7184463739395142], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "798", "similarity": [0.7859642654657364, 0.7249912023544312, 0.7192050516605377], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "799", "similarity": [0.7627875804901123, 0.753158763051033, 0.7521714568138123], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "801", "similarity": [0.74568310379982, 0.7446142435073853, 0.7284365892410278], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "802", "similarity": [0.774838924407959, 0.7600924372673035, 0.7525270879268646], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "803", "similarity": [0.7417340874671936, 0.7377223372459412, 0.7232614159584045], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "804", "similarity": [0.7611710727214813, 0.7499198317527771, 0.7487348914146423], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "807", "similarity": [0.665814995765686, 0.6612484455108643, 0.6478817760944366], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "810", "similarity": [0.7168549597263336, 0.7068305015563965, 0.7042608857154846], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "812", "similarity": [0.7189671695232391, 0.7178508043289185, 0.7138408422470093], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "813", "similarity": [0.738783597946167, 0.7369035482406616, 0.732652872800827], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "815", "similarity": [0.7517403066158295, 0.7349398732185364, 0.7219379544258118], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "816", "similarity": [0.7516177296638489, 0.726654589176178, 0.725712239742279], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "817", "similarity": [0.7073197066783905, 0.677672803401947, 0.6765538454055786], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "818", "similarity": [0.7172259092330933, 0.6858574151992798, 0.6699628233909607], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "822", "similarity": [0.8458169400691986, 0.7886572182178497, 0.7741573005914688], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "825", "similarity": [0.8486387431621552, 0.799265593290329, 0.781707301735878], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "826", "similarity": [0.7844380885362625, 0.7831602394580841, 0.7273702323436737], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "828", "similarity": [0.7246025800704956, 0.7244063913822174, 0.7205868363380432], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "835", "similarity": [0.7418850958347321, 0.736731082201004, 0.7317321598529816], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "838", "similarity": [0.74385204911232, 0.7426156997680664, 0.7411195635795593], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "840", "similarity": [0.7709458768367767, 0.7594282478094101, 0.7508112043142319], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "841", "similarity": [0.7611455470323563, 0.7525176703929901, 0.7369663715362549], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "844", "similarity": [0.8277643322944641, 0.7973887473344803, 0.794764369726181], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "846", "similarity": [0.7320988178253174, 0.7194762229919434, 0.719196081161499], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "848", "similarity": [0.7428392171859741, 0.6545960903167725, 0.6230416893959045], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "853", "similarity": [0.7934530973434448, 0.7656076848506927, 0.7575658410787582], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "854", "similarity": [0.7405787110328674, 0.7252097129821777, 0.6956942677497864], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "855", "similarity": [0.7670384347438812, 0.730956494808197, 0.7245295643806458], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "856", "similarity": [0.7850127518177032, 0.7154105007648468, 0.7012171745300293], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "857", "similarity": [0.7663156092166901, 0.7039053738117218, 0.7011780738830566], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "858", "similarity": [0.7456038296222687, 0.7421378195285797, 0.732568234205246], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "860", "similarity": [0.7520494312047958, 0.7216998338699341, 0.7165025472640991], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "861", "similarity": [0.7495481371879578, 0.7231614589691162, 0.7131150960922241], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "863", "similarity": [0.8315854072570801, 0.8014323115348816, 0.7895929217338562], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "866", "similarity": [0.8281862735748291, 0.7921213805675507, 0.7717942595481873], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "867", "similarity": [0.791371539235115, 0.7571572959423065, 0.7351617813110352], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "871", "similarity": [0.7546938359737396, 0.7390194535255432, 0.7298319935798645], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "876", "similarity": [0.7270817756652832, 0.7093775570392609, 0.7084465026855469], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "877", "similarity": [0.7594139873981476, 0.7554209232330322, 0.730657160282135], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "881", "similarity": [0.7260026633739471, 0.7122210264205933, 0.7102192640304565], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "883", "similarity": [0.7183352708816528, 0.7178047299385071, 0.7140618562698364], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "884", "similarity": [0.7369839549064636, 0.728388249874115, 0.7193517088890076], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "885", "similarity": [0.6671206951141357, 0.6494053602218628, 0.6459172070026398], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "886", "similarity": [0.6673046350479126, 0.6635119318962097, 0.6581628620624542], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "890", "similarity": [0.7073297798633575, 0.6993963122367859, 0.6976543366909027], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "891", "similarity": [0.7105085253715515, 0.6994269788265228, 0.6986293792724609], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "893", "similarity": [0.7605312764644623, 0.7510315626859665, 0.7436894178390503], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "894", "similarity": [0.7405959963798523, 0.7389715015888214, 0.7307273149490356], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "895", "similarity": [0.7081639170646667, 0.7069540023803711, 0.6881061494350433], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "896", "similarity": [0.7296015024185181, 0.7113987803459167, 0.7021191120147705], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "897", "similarity": [0.7439222931861877, 0.7278873026371002, 0.7202640473842621], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "898", "similarity": [0.7568112313747406, 0.7228044867515564, 0.7214011549949646], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "899", "similarity": [0.7410225868225098, 0.7304472625255585, 0.7286701202392578], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "900", "similarity": [0.7926431596279144, 0.7624494284391403, 0.7592556178569794], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "901", "similarity": [0.7988950908184052, 0.7599821984767914, 0.7344325184822083], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "902", "similarity": [0.7515291273593903, 0.7506524324417114, 0.7442839741706848], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "908", "similarity": [0.8216206729412079, 0.7639961540699005, 0.7500873655080795], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "909", "similarity": [0.7477361261844635, 0.7326869368553162, 0.7274015843868256], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "910", "similarity": [0.7411478757858276, 0.7343249917030334, 0.727816641330719], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "912", "similarity": [0.7405704855918884, 0.721457839012146, 0.7158391177654266], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "916", "similarity": [0.7372337579727173, 0.7156241238117218, 0.7152287065982819], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "917", "similarity": [0.7535901367664337, 0.7524543404579163, 0.7403921186923981], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "919", "similarity": [0.8336595296859741, 0.7799831181764603, 0.7695037722587585], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "923", "similarity": [0.7315139174461365, 0.723044216632843, 0.7176604270935059], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "925", "similarity": [0.7362455725669861, 0.7185382544994354, 0.7115289568901062], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "926", "similarity": [0.7576473653316498, 0.7433277070522308, 0.7309505343437195], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "927", "similarity": [0.772347092628479, 0.75074303150177, 0.743353009223938], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "928", "similarity": [0.800159752368927, 0.7986780852079391, 0.7965593636035919], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "929", "similarity": [0.7948403060436249, 0.79262974858284, 0.7876076400279999], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "930", "similarity": [0.7268863618373871, 0.7120380997657776, 0.7112936973571777], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "933", "similarity": [0.7321294546127319, 0.7311786711215973, 0.7245190143585205], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "934", "similarity": [0.6902641952037811, 0.6845209002494812, 0.6724686026573181], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "935", "similarity": [0.7512212693691254, 0.7509172856807709, 0.7490985989570618], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "938", "similarity": [0.7499330937862396, 0.7391771078109741, 0.732935220003128], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "939", "similarity": [0.7883596420288086, 0.7528213113546371, 0.7513607442378998], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "940", "similarity": [0.7444093227386475, 0.7400923073291779, 0.7135135531425476], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "941", "similarity": [0.7442225217819214, 0.7132763266563416, 0.7093216776847839], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "942", "similarity": [0.7305219173431396, 0.6998265981674194, 0.691872775554657], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "944", "similarity": [0.7956591546535492, 0.7476993799209595, 0.7444208264350891], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "945", "similarity": [0.7882467210292816, 0.7511470019817352, 0.7382878661155701], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "946", "similarity": [0.806108757853508, 0.7683953046798706, 0.7639212757349014], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "949", "similarity": [0.7103632986545563, 0.7096782326698303, 0.7036439478397369], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "951", "similarity": [0.8013897240161896, 0.7830590009689331, 0.7316949367523193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "952", "similarity": [0.8381377458572388, 0.7413673996925354, 0.7400474548339844], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "953", "similarity": [0.8387051522731781, 0.7541421055793762, 0.7540779709815979], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "954", "similarity": [0.8503528237342834, 0.7381626963615417, 0.7349614500999451], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "955", "similarity": [0.8131008297204971, 0.79011370241642, 0.7481876015663147], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "959", "similarity": [0.7443468272686005, 0.7436552047729492, 0.7430962324142456], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "962", "similarity": [0.8503915965557098, 0.7700242102146149, 0.7348827123641968], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "963", "similarity": [0.7990561723709106, 0.7707941234111786, 0.7688368111848831], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "964", "similarity": [0.8029077053070068, 0.7798086106777191, 0.765897125005722], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "965", "similarity": [0.7854383885860443, 0.7432243227958679, 0.7232396602630615], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "969", "similarity": [0.827216774225235, 0.8219519257545471, 0.7578043937683105], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "970", "similarity": [0.8248811960220337, 0.8193785697221756, 0.7644977420568466], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "972", "similarity": [0.854307621717453, 0.8535270392894745, 0.839334174990654], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "973", "similarity": [0.8530328869819641, 0.8462266474962234, 0.8247256278991699], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "976", "similarity": [0.7509341537952423, 0.7386078536510468, 0.7278020083904266], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "977", "similarity": [0.7669423222541809, 0.7504903376102448, 0.7484643459320068], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "978", "similarity": [0.7630377858877182, 0.7561012208461761, 0.7439178824424744], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "979", "similarity": [0.7730239629745483, 0.7715601325035095, 0.7513041198253632], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "980", "similarity": [0.7415502369403839, 0.7357895076274872, 0.7238962650299072], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "984", "similarity": [0.7440845668315887, 0.7438970804214478, 0.7407447695732117], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "988", "similarity": [0.7116624712944031, 0.7023470103740692, 0.7021383345127106], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "989", "similarity": [0.7582820355892181, 0.7426852881908417, 0.7396976351737976], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "990", "similarity": [0.7603559195995331, 0.7601180225610733, 0.7540798038244247], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "992", "similarity": [0.7301704287528992, 0.7271365821361542, 0.7243022322654724], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "994", "similarity": [0.7530112713575363, 0.730272114276886, 0.7248201370239258], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "996", "similarity": [0.7112597525119781, 0.7109620571136475, 0.7082840800285339], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "997", "similarity": [0.7332055568695068, 0.7331060171127319, 0.7122666835784912], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "998", "similarity": [0.7494730055332184, 0.7357588112354279, 0.7263544797897339], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "999", "similarity": [0.758182555437088, 0.7264441549777985, 0.720836877822876], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1000", "similarity": [0.8399275839328766, 0.7744677662849426, 0.7614753246307373], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1001", "similarity": [0.7674045264720917, 0.7636613547801971, 0.76311194896698], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1002", "similarity": [0.7267246544361115, 0.7244704365730286, 0.7237834334373474], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1003", "similarity": [0.835933655500412, 0.796100452542305, 0.7680663764476776], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1004", "similarity": [0.7743880450725555, 0.7684497833251953, 0.7531827688217163], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1005", "similarity": [0.7706953883171082, 0.7670604735612869, 0.7392837703227997], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1006", "similarity": [0.7500372231006622, 0.7445518374443054, 0.7391430139541626], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1008", "similarity": [0.6656312346458435, 0.6626081466674805, 0.6589396297931671], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1009", "similarity": [0.7500120997428894, 0.7477516531944275, 0.7321408689022064], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1011", "similarity": [0.8499154150485992, 0.8197534084320068, 0.777554452419281], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1015", "similarity": [0.781498908996582, 0.7700998932123184, 0.7477116584777832], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1016", "similarity": [0.7651303112506866, 0.7339853346347809, 0.7317227125167847], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1018", "similarity": [0.772017240524292, 0.7133059799671173, 0.7116091847419739], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1023", "similarity": [0.7599268853664398, 0.7471069097518921, 0.7300037741661072], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1025", "similarity": [0.7660516202449799, 0.7590863704681396, 0.7578324377536774], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1026", "similarity": [0.7960101664066315, 0.7407881617546082, 0.7368924915790558], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1027", "similarity": [0.8178786188364029, 0.76154425740242, 0.7382799386978149], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1028", "similarity": [0.8011361658573151, 0.7950483560562134, 0.7848707437515259], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1030", "similarity": [0.8160981088876724, 0.7991757392883301, 0.792784571647644], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1031", "similarity": [0.7488709092140198, 0.7317734360694885, 0.7311144769191742], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1032", "similarity": [0.7087749540805817, 0.6856142282485962, 0.6850295066833496], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1033", "similarity": [0.6935286521911621, 0.6827674508094788, 0.6786627769470215], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1034", "similarity": [0.8256979137659073, 0.7829702496528625, 0.779874712228775], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1035", "similarity": [0.8358866572380066, 0.7776326090097427, 0.7683584094047546], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1036", "similarity": [0.8109264671802521, 0.7740311622619629, 0.7599045187234879], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1037", "similarity": [0.7608411312103271, 0.7573316991329193, 0.756858691573143], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1038", "similarity": [0.7520299851894379, 0.7510363757610321, 0.7449999451637268], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1040", "similarity": [0.8726554065942764, 0.8556417673826218, 0.850968673825264], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1042", "similarity": [0.7624458223581314, 0.7531585395336151, 0.7531282603740692], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1043", "similarity": [0.8200756013393402, 0.7680630385875702, 0.7586776316165924], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1044", "similarity": [0.813885286450386, 0.8026711046695709, 0.7955123484134674], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1045", "similarity": [0.822735607624054, 0.821233719587326, 0.7965040504932404], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1046", "similarity": [0.7479530572891235, 0.7423962652683258, 0.7388684451580048], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1047", "similarity": [0.7629158347845078, 0.7398114800453186, 0.7346190810203552], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1048", "similarity": [0.7588323801755905, 0.7287680506706238, 0.6919870376586914], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1050", "similarity": [0.8502438366413116, 0.8017205893993378, 0.7649603188037872], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1052", "similarity": [0.7481845021247864, 0.7435440421104431, 0.72843998670578], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1053", "similarity": [0.7546882778406143, 0.7515071332454681, 0.7208357751369476], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1054", "similarity": [0.8237361460924149, 0.7692524790763855, 0.7644208669662476], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1055", "similarity": [0.6842359900474548, 0.6785028278827667, 0.6723746657371521], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1056", "similarity": [0.7117655873298645, 0.688514232635498, 0.6862346529960632], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1058", "similarity": [0.7066402435302734, 0.7035789489746094, 0.696299135684967], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1065", "similarity": [0.7627216726541519, 0.736193835735321, 0.7317436039447784], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1067", "similarity": [0.7473624050617218, 0.7451508939266205, 0.7323092222213745], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1068", "similarity": [0.7470147013664246, 0.745197981595993, 0.7378225028514862], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1069", "similarity": [0.7103495895862579, 0.6754537224769592, 0.6725934445858002], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1070", "similarity": [0.6938135027885437, 0.6659897863864899, 0.6626788973808289], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1072", "similarity": [0.7812539786100388, 0.7764405608177185, 0.7725322246551514], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1073", "similarity": [0.7994497418403625, 0.7712922990322113, 0.7561406493186951], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1074", "similarity": [0.7475562989711761, 0.7301547527313232, 0.7293511629104614], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1075", "similarity": [0.7420191764831543, 0.7144778370857239, 0.7137627899646759], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1081", "similarity": [0.7761693894863129, 0.7586312890052795, 0.7517067790031433], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1084", "similarity": [0.7398973107337952, 0.7367467880249023, 0.7358432114124298], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1085", "similarity": [0.7605259418487549, 0.7570421397686005, 0.7336769700050354], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1087", "similarity": [0.7020715475082397, 0.6810908317565918, 0.6739609241485596], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1090", "similarity": [0.7787383943796158, 0.7380810081958771, 0.7365565896034241], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1091", "similarity": [0.7622055560350418, 0.761833906173706, 0.7609482109546661], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1096", "similarity": [0.785455733537674, 0.7795210927724838, 0.7736160159111023], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1097", "similarity": [0.7198195457458496, 0.719269722700119, 0.7186437845230103], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1098", "similarity": [0.6665371656417847, 0.6557440757751465, 0.6538390517234802], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1101", "similarity": [0.7778476774692535, 0.7572317272424698, 0.7480395138263702], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1102", "similarity": [0.7645393908023834, 0.7643821835517883, 0.7448614835739136], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1103", "similarity": [0.7583572566509247, 0.746524453163147, 0.7452169060707092], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1105", "similarity": [0.760391116142273, 0.6958648264408112, 0.644305944442749], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1106", "similarity": [0.7671150118112564, 0.6840510964393616, 0.6580237746238708], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1109", "similarity": [0.7628272622823715, 0.7528801560401917, 0.7480077147483826], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1111", "similarity": [0.7216687798500061, 0.6979867219924927, 0.6941133439540863], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1112", "similarity": [0.7096224427223206, 0.7045440673828125, 0.702228307723999], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1114", "similarity": [0.6514759659767151, 0.6476187407970428, 0.6449723541736603], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1115", "similarity": [0.7905796617269516, 0.6665708124637604, 0.6609759330749512], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1118", "similarity": [0.7838832437992096, 0.7648280262947083, 0.7377362251281738], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1119", "similarity": [0.8540148437023163, 0.7958435118198395, 0.7152791619300842], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1120", "similarity": [0.8276244848966599, 0.7793287038803101, 0.7322081029415131], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1125", "similarity": [0.7444025278091431, 0.736393928527832, 0.7271469235420227], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1126", "similarity": [0.7454472184181213, 0.7411778569221497, 0.7405459880828857], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1127", "similarity": [0.7624961286783218, 0.7551779001951218, 0.7542207390069962], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1128", "similarity": [0.809174045920372, 0.7994157373905182, 0.7760883271694183], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1133", "similarity": [0.7224153876304626, 0.7152541875839233, 0.7104299664497375], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1134", "similarity": [0.7197994589805603, 0.7144677937030792, 0.7123459279537201], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1135", "similarity": [0.7220125496387482, 0.7209006547927856, 0.7157452702522278], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1136", "similarity": [0.7303658127784729, 0.723516196012497, 0.7213810384273529], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1138", "similarity": [0.7958751767873764, 0.7591228783130646, 0.7488352954387665], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1139", "similarity": [0.7343789339065552, 0.733781635761261, 0.7313639521598816], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1141", "similarity": [0.7722900807857513, 0.7397933602333069, 0.738625705242157], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1142", "similarity": [0.74363973736763, 0.7318915128707886, 0.7226588428020477], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1145", "similarity": [0.7915880382061005, 0.7898464649915695, 0.7634410560131073], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1148", "similarity": [0.753456711769104, 0.7442952990531921, 0.734516829252243], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1153", "similarity": [0.7442169785499573, 0.742701917886734, 0.7420384287834167], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1156", "similarity": [0.7056097686290741, 0.7043372392654419, 0.6852452456951141], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1157", "similarity": [0.5982733368873596, 0.5953760743141174, 0.593912661075592], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1158", "similarity": [0.5902798175811768, 0.5840377807617188, 0.580047070980072], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1159", "similarity": [0.6175147891044617, 0.6088502109050751, 0.6025124192237854], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1161", "similarity": [0.7528535276651382, 0.7502782940864563, 0.74137082695961], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1162", "similarity": [0.6957291662693024, 0.6931494474411011, 0.6896323561668396], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1164", "similarity": [0.7746501564979553, 0.765903890132904, 0.7489967942237854], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1165", "similarity": [0.7524697780609131, 0.7461255788803101, 0.7410194873809814], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1166", "similarity": [0.7503015398979187, 0.7366451025009155, 0.7264215350151062], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1168", "similarity": [0.7335118651390076, 0.7306457161903381, 0.696422815322876], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1169", "similarity": [0.7739813327789307, 0.7720394134521484, 0.7692136019468307], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1170", "similarity": [0.7779661417007446, 0.7646422982215881, 0.7644343078136444], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1171", "similarity": [0.7709612548351288, 0.7576873600482941, 0.7530582100152969], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1173", "similarity": [0.7346437275409698, 0.7189045548439026, 0.7140614986419678], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1174", "similarity": [0.6580683588981628, 0.657436341047287, 0.6524739563465118], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1176", "similarity": [0.7428351044654846, 0.7140675783157349, 0.6971994042396545], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1177", "similarity": [0.7359096705913544, 0.7285256385803223, 0.7265502214431763], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1178", "similarity": [0.7163034081459045, 0.7125367522239685, 0.7062589824199677], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1181", "similarity": [0.7758880257606506, 0.750955268740654, 0.7276442646980286], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1182", "similarity": [0.8400330692529678, 0.8351099491119385, 0.834029883146286], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1183", "similarity": [0.7281382381916046, 0.7229679822921753, 0.7226250171661377], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1184", "similarity": [0.7912818342447281, 0.7819187045097351, 0.7609924525022507], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1186", "similarity": [0.673378050327301, 0.6619417071342468, 0.660442590713501], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1188", "similarity": [0.7472604811191559, 0.7417531609535217, 0.7289049327373505], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1190", "similarity": [0.6671825647354126, 0.6607522666454315, 0.6426924169063568], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1193", "similarity": [0.7800511717796326, 0.737825334072113, 0.7252154350280762], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1195", "similarity": [0.7594172358512878, 0.7534511089324951, 0.7441295981407166], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1205", "similarity": [0.7447939217090607, 0.7292021214962006, 0.7252877354621887], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1206", "similarity": [0.729873776435852, 0.7278819978237152, 0.7238543033599854], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1208", "similarity": [0.7371688485145569, 0.7309963703155518, 0.7274534106254578], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1209", "similarity": [0.7677058428525925, 0.7436337471008301, 0.7260614931583405], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1210", "similarity": [0.7232895195484161, 0.7214600145816803, 0.7186303734779358], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1211", "similarity": [0.7075820565223694, 0.7058398723602295, 0.7023976743221283], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1212", "similarity": [0.7473549544811249, 0.7287211418151855, 0.7272055149078369], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1214", "similarity": [0.742387056350708, 0.7354199290275574, 0.7319941520690918], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1215", "similarity": [0.7505136281251907, 0.7504944205284119, 0.7435985207557678], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1218", "similarity": [0.8152779042720795, 0.7790800929069519, 0.7481930553913116], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1219", "similarity": [0.8055682182312012, 0.7823396325111389, 0.7760923206806183], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1220", "similarity": [0.7902956008911133, 0.7556643038988113, 0.7556587010622025], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1223", "similarity": [0.740333616733551, 0.7306410372257233, 0.7274071574211121], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1224", "similarity": [0.881818138062954, 0.8737442493438721, 0.8347122222185135], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1227", "similarity": [0.7959557622671127, 0.7047513723373413, 0.7026788890361786], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1228", "similarity": [0.8123592138290405, 0.7267667055130005, 0.7084674835205078], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1229", "similarity": [0.7712590098381042, 0.7564376443624496, 0.7541534006595612], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1230", "similarity": [0.744617223739624, 0.7262831926345825, 0.7235401272773743], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1231", "similarity": [0.7326832115650177, 0.7202796339988708, 0.71685391664505], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1234", "similarity": [0.7359524965286255, 0.7220979928970337, 0.7001237869262695], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1235", "similarity": [0.8214781284332275, 0.8057977855205536, 0.7797394096851349], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1236", "similarity": [0.8246312141418457, 0.8154988437891006, 0.7860249876976013], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1237", "similarity": [0.7214123606681824, 0.716849684715271, 0.7121708989143372], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1238", "similarity": [0.7468968033790588, 0.7160899937152863, 0.7139918506145477], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1239", "similarity": [0.7713826447725296, 0.7634824216365814, 0.7606399059295654], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1244", "similarity": [0.7506600618362427, 0.7473787665367126, 0.7207642793655396], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1246", "similarity": [0.640532910823822, 0.636176586151123, 0.6200881600379944], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1247", "similarity": [0.6834089756011963, 0.6610089838504791, 0.6561354100704193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1248", "similarity": [0.7539302557706833, 0.7211357355117798, 0.7169204354286194], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1249", "similarity": [0.7270060777664185, 0.7114799916744232, 0.6926114857196808], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1253", "similarity": [0.7178986668586731, 0.7093921303749084, 0.707351565361023], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1254", "similarity": [0.7576692253351212, 0.7421499788761139, 0.7402649521827698], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1255", "similarity": [0.7403502464294434, 0.7311533093452454, 0.7112830281257629], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1257", "similarity": [0.7321399450302124, 0.7156320214271545, 0.7138335108757019], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1258", "similarity": [0.7211444675922394, 0.7088688313961029, 0.7010284066200256], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1260", "similarity": [0.8246602714061737, 0.7736005485057831, 0.7643870562314987], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1261", "similarity": [0.7888357937335968, 0.7676584869623184, 0.7595033943653107], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1263", "similarity": [0.8077424913644791, 0.7986447513103485, 0.7795458137989044], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1265", "similarity": [0.7967823147773743, 0.7916865050792694, 0.7894431352615356], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1268", "similarity": [0.799137532711029, 0.786033108830452, 0.7672653496265411], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1269", "similarity": [0.6971688866615295, 0.68408203125, 0.675427258014679], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1275", "similarity": [0.8272392749786377, 0.8040930479764938, 0.7886626273393631], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1276", "similarity": [0.8011693954467773, 0.7305687069892883, 0.727100133895874], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1284", "similarity": [0.7255374193191528, 0.7018266916275024, 0.6967397034168243], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1288", "similarity": [0.7815888822078705, 0.744818925857544, 0.7416953444480896], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1289", "similarity": [0.664328932762146, 0.6507286727428436, 0.6498554944992065], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1291", "similarity": [0.7216142117977142, 0.7145121097564697, 0.7138743102550507], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1293", "similarity": [0.6991639137268066, 0.6989210844039917, 0.696602463722229], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1294", "similarity": [0.790450781583786, 0.762460857629776, 0.7238372266292572], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1295", "similarity": [0.6616585552692413, 0.6504712402820587, 0.64973583817482], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1297", "similarity": [0.7124563753604889, 0.7096561789512634, 0.702076256275177], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1300", "similarity": [0.7113838791847229, 0.6955954432487488, 0.6937891244888306], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1302", "similarity": [0.6787577271461487, 0.6737949252128601, 0.6737038493156433], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1304", "similarity": [0.707335352897644, 0.6945831775665283, 0.6881603598594666], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1305", "similarity": [0.6627052426338196, 0.6617379188537598, 0.661604255437851], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1306", "similarity": [0.773018091917038, 0.7555280923843384, 0.7470069825649261], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1307", "similarity": [0.7127151191234589, 0.7036095857620239, 0.6968755424022675], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1308", "similarity": [0.7415428161621094, 0.7366088032722473, 0.735999196767807], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1309", "similarity": [0.7172083258628845, 0.7164248526096344, 0.7068226337432861], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1310", "similarity": [0.6344208717346191, 0.6077778935432434, 0.6071881353855133], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1311", "similarity": [0.7730823755264282, 0.7485357224941254, 0.7339045405387878], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1312", "similarity": [0.7527627497911453, 0.747255802154541, 0.7318343222141266], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1314", "similarity": [0.7867420464754105, 0.7614134848117828, 0.7585773169994354], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1315", "similarity": [0.7863766998052597, 0.7602973729372025, 0.758441835641861], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1322", "similarity": [0.7736445814371109, 0.7264952063560486, 0.7254515290260315], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1323", "similarity": [0.7577110677957535, 0.7289695739746094, 0.7145530581474304], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1324", "similarity": [0.7205967307090759, 0.7202180624008179, 0.7122567296028137], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1325", "similarity": [0.679835706949234, 0.675610214471817, 0.6740062236785889], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1327", "similarity": [0.7493210136890411, 0.7055773138999939, 0.7038706541061401], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1328", "similarity": [0.8345562219619751, 0.7611636519432068, 0.7269318103790283], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1330", "similarity": [0.7765937894582748, 0.7759107947349548, 0.7532674819231033], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1331", "similarity": [0.7698952853679657, 0.7495735883712769, 0.7419137060642242], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1333", "similarity": [0.7396338582038879, 0.733180582523346, 0.7305769622325897], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1334", "similarity": [0.76320680975914, 0.7454763650894165, 0.7427091598510742], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1340", "similarity": [0.6786149740219116, 0.6680285930633545, 0.6590788960456848], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1341", "similarity": [0.6939012110233307, 0.6747142970561981, 0.662995457649231], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1342", "similarity": [0.7583879828453064, 0.7543212622404099, 0.7538482248783112], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1345", "similarity": [0.7714502811431885, 0.7707046866416931, 0.7612437009811401], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1346", "similarity": [0.7892387807369232, 0.7847459316253662, 0.7758897691965103], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1347", "similarity": [0.8296005129814148, 0.7950258553028107, 0.7856966108083725], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1348", "similarity": [0.8483973741531372, 0.7981808483600616, 0.7815340459346771], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1349", "similarity": [0.7638999223709106, 0.7270225286483765, 0.722736120223999], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1350", "similarity": [0.7717137783765793, 0.7288267314434052, 0.7137937843799591], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1351", "similarity": [0.7825589179992676, 0.7719734907150269, 0.7660543322563171], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1353", "similarity": [0.7641543447971344, 0.7320875525474548, 0.7311726808547974], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1355", "similarity": [0.7431687116622925, 0.7209344506263733, 0.7193658351898193], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1356", "similarity": [0.7363413572311401, 0.7292818427085876, 0.7226618826389313], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1360", "similarity": [0.7980111241340637, 0.744180291891098, 0.7404682040214539], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1361", "similarity": [0.7552832514047623, 0.7518760114908218, 0.7333152294158936], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1364", "similarity": [0.6510841548442841, 0.6478641033172607, 0.6439041793346405], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1366", "similarity": [0.7224477827548981, 0.6963613331317902, 0.694293886423111], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1367", "similarity": [0.7751749455928802, 0.7610020041465759, 0.758496418595314], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1371", "similarity": [0.7912206947803497, 0.7911145687103271, 0.7819504141807556], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1372", "similarity": [0.699622631072998, 0.6874270141124725, 0.6745124459266663], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1373", "similarity": [0.6991589069366455, 0.689741849899292, 0.6877170205116272], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1374", "similarity": [0.6497040688991547, 0.6419709324836731, 0.6358833909034729], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1375", "similarity": [0.6507920622825623, 0.6493946313858032, 0.6390056908130646], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1376", "similarity": [0.8089888095855713, 0.8028970062732697, 0.7480239868164062], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1378", "similarity": [0.7386638820171356, 0.7299855053424835, 0.7162514328956604], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1380", "similarity": [0.823582798242569, 0.7887218296527863, 0.7868562489748001], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1381", "similarity": [0.7885770350694656, 0.7860381305217743, 0.7614270895719528], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1383", "similarity": [0.7504098117351532, 0.7479522824287415, 0.7457929253578186], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1386", "similarity": [0.7057912945747375, 0.6996557712554932, 0.6944002509117126], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1387", "similarity": [0.7675543129444122, 0.7537312507629395, 0.7477833330631256], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1390", "similarity": [0.7684580236673355, 0.7382077574729919, 0.737171471118927], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1391", "similarity": [0.712019145488739, 0.7030500769615173, 0.7006235122680664], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1392", "similarity": [0.7319477498531342, 0.7257728576660156, 0.7255098223686218], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1393", "similarity": [0.7605706453323364, 0.7597777247428894, 0.7471197843551636], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1394", "similarity": [0.8155699819326401, 0.7866444885730743, 0.7806155681610107], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1397", "similarity": [0.7752045542001724, 0.735956072807312, 0.7325091063976288], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1398", "similarity": [0.7656487226486206, 0.7372013330459595, 0.7126749157905579], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1400", "similarity": [0.7747121751308441, 0.7394778728485107, 0.7249526977539062], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1401", "similarity": [0.7735694646835327, 0.7248751521110535, 0.6707638204097748], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1402", "similarity": [0.7279670834541321, 0.7278309762477875, 0.7188257575035095], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1403", "similarity": [0.7486324310302734, 0.7296221852302551, 0.7276785373687744], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1404", "similarity": [0.7514962553977966, 0.7371000051498413, 0.7314556837081909], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1405", "similarity": [0.7468580007553101, 0.7451568245887756, 0.7377894818782806], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1406", "similarity": [0.6781324744224548, 0.6767546832561493, 0.6766438484191895], "relevant": [false, false, false]} +{"split": "train", "index": "unanswerable", "query_id": "1407", "similarity": [0.7519550621509552, 0.7481494545936584, 0.7462581396102905], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1", "similarity": [0.7029363811016083, 0.6937490105628967, 0.6833390593528748], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "3", "similarity": [0.76772341132164, 0.7580264359712601, 0.7542661130428314], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "5", "similarity": [0.7120313048362732, 0.70958012342453, 0.7040311992168427], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "13", "similarity": [0.764228567481041, 0.7621487379074097, 0.7551555037498474], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "36", "similarity": [0.7959075570106506, 0.7902679443359375, 0.7892882823944092], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "42", "similarity": [0.8439354300498962, 0.8001197725534439, 0.7765368223190308], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "48", "similarity": [0.7501026839017868, 0.749695360660553, 0.740513265132904], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "49", "similarity": [0.8759020417928696, 0.8029913157224655, 0.7788126170635223], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "50", "similarity": [0.81588514149189, 0.7706095576286316, 0.7578956484794617], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "51", "similarity": [0.8653968572616577, 0.7443061470985413, 0.7435750961303711], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "53", "similarity": [0.8576283156871796, 0.729396641254425, 0.7287785708904266], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "54", "similarity": [0.8804101347923279, 0.833916962146759, 0.821098119020462], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "56", "similarity": [0.8668489158153534, 0.7964489161968231, 0.7843835204839706], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "57", "similarity": [0.8624793440103531, 0.7958991676568985, 0.7861583828926086], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "70", "similarity": [0.7941447496414185, 0.7849941700696945, 0.7811834365129471], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "72", "similarity": [0.7156018614768982, 0.6958191692829132, 0.6870241761207581], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "75", "similarity": [0.7950565665960312, 0.7922124266624451, 0.7892077714204788], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "94", "similarity": [0.6568001210689545, 0.6409316062927246, 0.6396643817424774], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "99", "similarity": [0.6767255961894989, 0.675061047077179, 0.6724228858947754], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "100", "similarity": [0.7667108774185181, 0.76009301841259, 0.7414956092834473], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "113", "similarity": [0.8477151393890381, 0.7934524416923523, 0.7783675938844681], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "115", "similarity": [0.7760854065418243, 0.6117458939552307, 0.6075420379638672], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "118", "similarity": [0.9155451506376266, 0.8884675353765488, 0.7657711207866669], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "124", "similarity": [0.8785125091671944, 0.8515368700027466, 0.7774012386798859], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "127", "similarity": [0.7090146839618683, 0.704912543296814, 0.7025558352470398], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "128", "similarity": [0.657137393951416, 0.6466840207576752, 0.6339055001735687], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "129", "similarity": [0.7870830595493317, 0.6614673137664795, 0.6427783071994781], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "130", "similarity": [0.8036397844552994, 0.6525686681270599, 0.6431843042373657], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "132", "similarity": [0.7658988237380981, 0.76087786257267, 0.7546935826539993], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "133", "similarity": [0.8236173689365387, 0.7896661162376404, 0.7825583219528198], "relevant": [true, false, true]} +{"split": "test", "index": "answerable", "query_id": "137", "similarity": [0.8487936854362488, 0.7125073075294495, 0.7043752670288086], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "141", "similarity": [0.7657681107521057, 0.7417525053024292, 0.7297107875347137], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "142", "similarity": [0.8193921744823456, 0.7854724228382111, 0.7615996152162552], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "143", "similarity": [0.8276940286159515, 0.7929473966360092, 0.7671186923980713], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "146", "similarity": [0.8363286554813385, 0.8097755908966064, 0.7622812539339066], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "148", "similarity": [0.8090996742248535, 0.784025639295578, 0.7690538763999939], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "163", "similarity": [0.862811803817749, 0.8494992852210999, 0.8231061697006226], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "171", "similarity": [0.8350398540496826, 0.7640970647335052, 0.7431093454360962], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "179", "similarity": [0.8813551291823387, 0.8596816658973694, 0.858530804514885], "relevant": [true, true, true]} +{"split": "test", "index": "answerable", "query_id": "180", "similarity": [0.8139235079288483, 0.7700926959514618, 0.7428233921527863], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "183", "similarity": [0.7893843054771423, 0.7871740460395813, 0.7858666926622391], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "185", "similarity": [0.725955069065094, 0.7217798233032227, 0.7143768966197968], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "198", "similarity": [0.6554760932922363, 0.6531824469566345, 0.6525209546089172], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "208", "similarity": [0.8299831449985504, 0.7247023284435272, 0.7175058722496033], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "212", "similarity": [0.7664633393287659, 0.757150799036026, 0.7543959021568298], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "213", "similarity": [0.8306756466627121, 0.8299618810415268, 0.7776113748550415], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "216", "similarity": [0.8535353988409042, 0.7703449726104736, 0.7573475688695908], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "217", "similarity": [0.8568076640367508, 0.7729044705629349, 0.7605955749750137], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "218", "similarity": [0.8987051174044609, 0.7921966761350632, 0.7698689699172974], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "219", "similarity": [0.8878541886806488, 0.7949444949626923, 0.7712008059024811], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "230", "similarity": [0.7082990407943726, 0.6677863895893097, 0.6674472093582153], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "232", "similarity": [0.8400094211101532, 0.8346850574016571, 0.8062928766012192], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "233", "similarity": [0.8425444662570953, 0.758554220199585, 0.7544156908988953], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "236", "similarity": [0.8369491994380951, 0.7467031180858612, 0.7408315539360046], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "237", "similarity": [0.7784469723701477, 0.7682019770145416, 0.7442789077758789], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "238", "similarity": [0.7411556243896484, 0.741139829158783, 0.7405553758144379], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "239", "similarity": [0.7455036640167236, 0.7385490536689758, 0.7294548153877258], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "248", "similarity": [0.7869554162025452, 0.746204674243927, 0.7300858795642853], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "249", "similarity": [0.7635575234889984, 0.7246505618095398, 0.7204106450080872], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "261", "similarity": [0.857544332742691, 0.8441934883594513, 0.83208367228508], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "268", "similarity": [0.7248853445053101, 0.7021384537220001, 0.6887215971946716], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "269", "similarity": [0.7220316529273987, 0.6994612514972687, 0.6986546516418457], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "274", "similarity": [0.8964915871620178, 0.8413791358470917, 0.766912192106247], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "275", "similarity": [0.8363587260246277, 0.8254386484622955, 0.7870145440101624], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "279", "similarity": [0.8420855551958084, 0.7822575271129608, 0.7547663897275925], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "294", "similarity": [0.7720412015914917, 0.7576690316200256, 0.744100034236908], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "295", "similarity": [0.8135707974433899, 0.8134715259075165, 0.8076557815074921], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "298", "similarity": [0.7603715062141418, 0.7596250176429749, 0.7362123727798462], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "300", "similarity": [0.7628100961446762, 0.7295755445957184, 0.7242439389228821], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "303", "similarity": [0.7833350598812103, 0.7530089318752289, 0.7407953143119812], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "312", "similarity": [0.75364089012146, 0.746374249458313, 0.722494900226593], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "314", "similarity": [0.7565313875675201, 0.7405337989330292, 0.7186566889286041], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "324", "similarity": [0.6955782175064087, 0.6949192881584167, 0.6913672685623169], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "327", "similarity": [0.7274942100048065, 0.7185897827148438, 0.7078738808631897], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "338", "similarity": [0.8244267553091049, 0.7975046634674072, 0.748763382434845], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "343", "similarity": [0.8320137560367584, 0.7857046574354172, 0.7746187150478363], "relevant": [true, true, false]} +{"split": "test", "index": "answerable", "query_id": "350", "similarity": [0.8250823318958282, 0.7291047275066376, 0.7197912037372589], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "354", "similarity": [0.8697828054428101, 0.7774917334318161, 0.7737409174442291], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "362", "similarity": [0.7597450166940689, 0.759043738245964, 0.755070298910141], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "380", "similarity": [0.7680002301931381, 0.7615105509757996, 0.7562246918678284], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "384", "similarity": [0.7528343796730042, 0.7438760101795197, 0.7263925671577454], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "385", "similarity": [0.8376368284225464, 0.7731684446334839, 0.7307290434837341], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "386", "similarity": [0.7858518958091736, 0.7052141427993774, 0.7026928961277008], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "388", "similarity": [0.747253954410553, 0.7326339483261108, 0.7105787694454193], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "399", "similarity": [0.8879928290843964, 0.7791419923305511, 0.7485264837741852], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "410", "similarity": [0.8052110970020294, 0.7049797177314758, 0.683527410030365], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "411", "similarity": [0.7912569046020508, 0.7061430811882019, 0.7047926187515259], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "415", "similarity": [0.8360879570245743, 0.8331219553947449, 0.8114665597677231], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "421", "similarity": [0.7417535781860352, 0.7304936051368713, 0.709075003862381], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "431", "similarity": [0.761710911989212, 0.7525125592947006, 0.74610835313797], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "436", "similarity": [0.8197819143533707, 0.7594025582075119, 0.7561256289482117], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "437", "similarity": [0.7988032102584839, 0.790080800652504, 0.7874240428209305], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "439", "similarity": [0.7849586606025696, 0.7692846059799194, 0.7674060463905334], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "440", "similarity": [0.7472258508205414, 0.7448928952217102, 0.7420444190502167], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "443", "similarity": [0.8870245292782784, 0.7759211659431458, 0.761216402053833], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "452", "similarity": [0.7664143741130829, 0.7644934058189392, 0.7525120079517365], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "475", "similarity": [0.7881592512130737, 0.7634416818618774, 0.7615413665771484], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "478", "similarity": [0.8307082206010818, 0.7440131604671478, 0.7400738000869751], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "491", "similarity": [0.7934356480836868, 0.7343559265136719, 0.7331064343452454], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "501", "similarity": [0.771100789308548, 0.7289299964904785, 0.7232643961906433], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "502", "similarity": [0.665950357913971, 0.6619696319103241, 0.6585739254951477], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "507", "similarity": [0.8801887482404709, 0.809212788939476, 0.8091888725757599], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "508", "similarity": [0.7495159208774567, 0.7063907980918884, 0.6944817304611206], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "513", "similarity": [0.7682482153177261, 0.7501061260700226, 0.7376741468906403], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "514", "similarity": [0.8675504624843597, 0.8314205408096313, 0.8266224563121796], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "516", "similarity": [0.8525822758674622, 0.8068366646766663, 0.7743600010871887], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "517", "similarity": [0.7903604805469513, 0.7824482172727585, 0.7742674350738525], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "521", "similarity": [0.7834121882915497, 0.7198183536529541, 0.7115947604179382], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "525", "similarity": [0.8758071064949036, 0.7693173587322235, 0.7675777077674866], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "527", "similarity": [0.773534744977951, 0.7613107115030289, 0.7545612752437592], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "528", "similarity": [0.8129747658967972, 0.7274459600448608, 0.7258059680461884], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "532", "similarity": [0.7980749756097794, 0.7325800061225891, 0.7266129851341248], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "533", "similarity": [0.7881137877702713, 0.7263603806495667, 0.7213621437549591], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "535", "similarity": [0.7655723989009857, 0.7615721225738525, 0.7525089085102081], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "536", "similarity": [0.7917886674404144, 0.7321686148643494, 0.7107288241386414], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "539", "similarity": [0.8691039532423019, 0.7835469543933868, 0.7752222567796707], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "540", "similarity": [0.8001385778188705, 0.7939575612545013, 0.7869390547275543], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "544", "similarity": [0.7655007243156433, 0.7589653730392456, 0.7459238171577454], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "549", "similarity": [0.7583030462265015, 0.7097672522068024, 0.706944614648819], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "551", "similarity": [0.8008323460817337, 0.7742499113082886, 0.7685754150152206], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "552", "similarity": [0.8790151029825211, 0.8068269491195679, 0.7660653442144394], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "554", "similarity": [0.760211169719696, 0.7512370049953461, 0.750274121761322], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "560", "similarity": [0.7784685343503952, 0.7746509611606598, 0.7674212455749512], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "569", "similarity": [0.8442529737949371, 0.8044003546237946, 0.7944575250148773], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "575", "similarity": [0.7852035313844681, 0.7579565942287445, 0.7323789596557617], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "577", "similarity": [0.7506194561719894, 0.7319298386573792, 0.7166398465633392], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "578", "similarity": [0.7811170816421509, 0.7568121552467346, 0.7494983077049255], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "587", "similarity": [0.794885128736496, 0.7440000772476196, 0.7329695522785187], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "589", "similarity": [0.911148339509964, 0.7435694932937622, 0.7312192916870117], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "593", "similarity": [0.840502142906189, 0.7589736133813858, 0.7536534070968628], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "597", "similarity": [0.7965594530105591, 0.7933168560266495, 0.7904607057571411], "relevant": [true, true, true]} +{"split": "test", "index": "answerable", "query_id": "598", "similarity": [0.7979578375816345, 0.7946074903011322, 0.7915992140769958], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "613", "similarity": [0.881292387843132, 0.7771831452846527, 0.7517833113670349], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "619", "similarity": [0.7215523421764374, 0.7183295488357544, 0.7164791524410248], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "623", "similarity": [0.8261176347732544, 0.8091426342725754, 0.8007083535194397], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "628", "similarity": [0.7850408256053925, 0.7752825319766998, 0.7493198215961456], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "636", "similarity": [0.8293750435113907, 0.7764180302619934, 0.751686692237854], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "637", "similarity": [0.7247857451438904, 0.7008589506149292, 0.6810854077339172], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "641", "similarity": [0.8042619526386261, 0.7205309271812439, 0.7135331332683563], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "644", "similarity": [0.7702750265598297, 0.7539838999509811, 0.7498028874397278], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "649", "similarity": [0.672608494758606, 0.6499870419502258, 0.6371825933456421], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "659", "similarity": [0.650026947259903, 0.6417555510997772, 0.6382255554199219], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "660", "similarity": [0.6613682508468628, 0.6588095724582672, 0.6433593332767487], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "674", "similarity": [0.7743014395236969, 0.7672757208347321, 0.7631520628929138], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "684", "similarity": [0.7838437557220459, 0.7775910496711731, 0.7302187979221344], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "690", "similarity": [0.7026431262493134, 0.6996148824691772, 0.6911275386810303], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "691", "similarity": [0.7825896143913269, 0.7568004131317139, 0.750620037317276], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "692", "similarity": [0.798763632774353, 0.7262070775032043, 0.7226062417030334], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "693", "similarity": [0.8312391936779022, 0.7065634727478027, 0.7054753303527832], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "700", "similarity": [0.8472352027893066, 0.8298687636852264, 0.7837457060813904], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "702", "similarity": [0.8395679593086243, 0.8323096334934235, 0.7784163355827332], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "715", "similarity": [0.8208154737949371, 0.8094618618488312, 0.7860832810401917], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "716", "similarity": [0.8103711903095245, 0.7648439258337021, 0.7513183653354645], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "718", "similarity": [0.777699664235115, 0.741447240114212, 0.7347411513328552], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "721", "similarity": [0.8130327612161636, 0.7524165511131287, 0.7274701595306396], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "723", "similarity": [0.8839032426476479, 0.7665507197380066, 0.7483245134353638], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "727", "similarity": [0.828539252281189, 0.7392005026340485, 0.7380672097206116], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "728", "similarity": [0.8287231922149658, 0.7347556948661804, 0.7332133054733276], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "729", "similarity": [0.7671551555395126, 0.7457635402679443, 0.7336677610874176], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "742", "similarity": [0.7816434502601624, 0.7033832967281342, 0.7022213935852051], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "743", "similarity": [0.7920422255992889, 0.7356070280075073, 0.7221891283988953], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "744", "similarity": [0.8451002836227417, 0.8413776755332947, 0.8104537427425385], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "756", "similarity": [0.8299048095941544, 0.8122470676898956, 0.8013694584369659], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "759", "similarity": [0.8421260714530945, 0.8344922959804535, 0.8211539834737778], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "768", "similarity": [0.7411505579948425, 0.7150000035762787, 0.6888008713722229], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "770", "similarity": [0.8368267118930817, 0.7788041532039642, 0.7478071451187134], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "775", "similarity": [0.8026803731918335, 0.7813616245985031, 0.7532719075679779], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "781", "similarity": [0.8716769516468048, 0.8591830432415009, 0.8482467085123062], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "783", "similarity": [0.7948863655328751, 0.784265398979187, 0.7762290835380554], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "784", "similarity": [0.8244532495737076, 0.8100689947605133, 0.8084197044372559], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "785", "similarity": [0.7889743745326996, 0.7334842681884766, 0.7304971218109131], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "793", "similarity": [0.7724388241767883, 0.7587174028158188, 0.7268569767475128], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "800", "similarity": [0.8289507627487183, 0.8234221041202545, 0.816665455698967], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "805", "similarity": [0.8869497776031494, 0.7611818611621857, 0.7568326890468597], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "808", "similarity": [0.7167440056800842, 0.7000648677349091, 0.6852685511112213], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "811", "similarity": [0.7795698791742325, 0.7115486264228821, 0.7094067931175232], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "814", "similarity": [0.8954996466636658, 0.7359445095062256, 0.7319276928901672], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "820", "similarity": [0.7403759360313416, 0.7353923320770264, 0.7330943048000336], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "821", "similarity": [0.7407223582267761, 0.7379066646099091, 0.735599160194397], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "823", "similarity": [0.7974947392940521, 0.7779718041419983, 0.7545285075902939], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "830", "similarity": [0.7423371970653534, 0.7399648427963257, 0.7319430112838745], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "831", "similarity": [0.7466650605201721, 0.7402245998382568, 0.7280697226524353], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "832", "similarity": [0.83174167573452, 0.7507630735635757, 0.742770254611969], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "834", "similarity": [0.7914370000362396, 0.7329479455947876, 0.7189933359622955], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "837", "similarity": [0.7649929821491241, 0.7576069384813309, 0.7470101118087769], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "839", "similarity": [0.7620907127857208, 0.7419877648353577, 0.7289429306983948], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "845", "similarity": [0.8685175776481628, 0.848732203245163, 0.8460465371608734], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "847", "similarity": [0.7838225811719894, 0.7473157346248627, 0.7126085758209229], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "852", "similarity": [0.7522986680269241, 0.7267070412635803, 0.6999932825565338], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "859", "similarity": [0.7629775404930115, 0.7575729787349701, 0.7459883987903595], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "870", "similarity": [0.7546938359737396, 0.7390194535255432, 0.7333924770355225], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "873", "similarity": [0.71230748295784, 0.705065906047821, 0.700303703546524], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "879", "similarity": [0.7975278198719025, 0.7047696709632874, 0.7044182419776917], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "880", "similarity": [0.8111192882061005, 0.7742561846971512, 0.7648105919361115], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "882", "similarity": [0.7969813644886017, 0.7425941824913025, 0.7287073135375977], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "887", "similarity": [0.7091203927993774, 0.7080304622650146, 0.6801060736179352], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "903", "similarity": [0.810256615281105, 0.7642746865749359, 0.7522061467170715], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "904", "similarity": [0.8804208859801292, 0.7566303163766861, 0.7525677382946014], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "907", "similarity": [0.8844327181577682, 0.8224397599697113, 0.7520658522844315], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "911", "similarity": [0.7723871171474457, 0.7446613013744354, 0.7318137586116791], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "913", "similarity": [0.7429296374320984, 0.7418681979179382, 0.7273560166358948], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "914", "similarity": [0.7595296502113342, 0.7366532683372498, 0.7259669899940491], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "921", "similarity": [0.8315577805042267, 0.7625961750745773, 0.7433757483959198], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "922", "similarity": [0.8233658075332642, 0.734434574842453, 0.7184143662452698], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "936", "similarity": [0.7855901420116425, 0.742929995059967, 0.738252729177475], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "956", "similarity": [0.8176441937685013, 0.7734988331794739, 0.7648598253726959], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "957", "similarity": [0.7256578207015991, 0.7171511650085449, 0.7008177638053894], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "960", "similarity": [0.847189337015152, 0.7333106696605682, 0.729676365852356], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "967", "similarity": [0.7660316079854965, 0.7092241048812866, 0.6955503523349762], "relevant": [true, true, false]} +{"split": "test", "index": "answerable", "query_id": "971", "similarity": [0.8615225255489349, 0.8588536381721497, 0.8562845587730408], "relevant": [true, true, false]} +{"split": "test", "index": "answerable", "query_id": "975", "similarity": [0.7453554272651672, 0.7286932468414307, 0.7245600521564484], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "982", "similarity": [0.7940777093172073, 0.7264578938484192, 0.7214467525482178], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "985", "similarity": [0.8450209200382233, 0.762986570596695, 0.7479864358901978], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "993", "similarity": [0.8506144732236862, 0.8394210487604141, 0.7863108366727829], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1012", "similarity": [0.9152038022875786, 0.8500750660896301, 0.8079778254032135], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1014", "similarity": [0.7866782546043396, 0.759058490395546, 0.7369645237922668], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1019", "similarity": [0.7656744420528412, 0.7567866742610931, 0.7482101321220398], "relevant": [false, false, true]} +{"split": "test", "index": "answerable", "query_id": "1020", "similarity": [0.8098451495170593, 0.7987809628248215, 0.7634838819503784], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1021", "similarity": [0.8183752596378326, 0.8013875037431717, 0.7752389013767242], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1024", "similarity": [0.7773071229457855, 0.7487127482891083, 0.746694028377533], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1029", "similarity": [0.8628725409507751, 0.8283500373363495, 0.803783044219017], "relevant": [true, true, false]} +{"split": "test", "index": "answerable", "query_id": "1041", "similarity": [0.8660165518522263, 0.8547260463237762, 0.8391537219285965], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1049", "similarity": [0.7519171237945557, 0.7514179944992065, 0.6915662586688995], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1062", "similarity": [0.8178398460149765, 0.7836065888404846, 0.7592867314815521], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1086", "similarity": [0.8909801542758942, 0.705142080783844, 0.673597514629364], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1088", "similarity": [0.7874546051025391, 0.7560275048017502, 0.7528949677944183], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1089", "similarity": [0.8557438105344772, 0.7938258051872253, 0.7660177052021027], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1099", "similarity": [0.8029463887214661, 0.799681767821312, 0.7868390381336212], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1100", "similarity": [0.7971938103437424, 0.7814894616603851, 0.7686410546302795], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1104", "similarity": [0.8384292423725128, 0.7604284286499023, 0.748061329126358], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1107", "similarity": [0.7743974924087524, 0.7666935324668884, 0.7633379101753235], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1110", "similarity": [0.7517037242650986, 0.7323667407035828, 0.7284021973609924], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1121", "similarity": [0.7775002717971802, 0.7693438977003098, 0.7520267516374588], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1130", "similarity": [0.8326743245124817, 0.7961269319057465, 0.7908194065093994], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1132", "similarity": [0.8183781802654266, 0.7927461862564087, 0.7811104357242584], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1137", "similarity": [0.8251103907823563, 0.7391822934150696, 0.7320400476455688], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1140", "similarity": [0.767652153968811, 0.7511867582798004, 0.7321124076843262], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1144", "similarity": [0.8376381248235703, 0.7674277722835541, 0.7555492967367172], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1146", "similarity": [0.7683923691511154, 0.6384587287902832, 0.6333682537078857], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1150", "similarity": [0.8416435122489929, 0.7744574546813965, 0.750085860490799], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1163", "similarity": [0.8339056968688965, 0.7352535724639893, 0.7229893207550049], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1175", "similarity": [0.6788469552993774, 0.6702433824539185, 0.6665136814117432], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1179", "similarity": [0.740827888250351, 0.7390636801719666, 0.7295312881469727], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1180", "similarity": [0.7579579651355743, 0.738152027130127, 0.7312131524085999], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1185", "similarity": [0.8049778640270233, 0.8024595081806183, 0.777779832482338], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1187", "similarity": [0.7890508323907852, 0.7812856435775757, 0.7607564032077789], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1191", "similarity": [0.6705031991004944, 0.6635471284389496, 0.6513506472110748], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1194", "similarity": [0.7607233822345734, 0.7227251529693604, 0.7223069667816162], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1196", "similarity": [0.673975944519043, 0.6688997149467468, 0.6549594700336456], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1197", "similarity": [0.6924685835838318, 0.6746512055397034, 0.6519043445587158], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1199", "similarity": [0.748276948928833, 0.7410315275192261, 0.7174493968486786], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1200", "similarity": [0.738024890422821, 0.7112044095993042, 0.7088710069656372], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1202", "similarity": [0.8070042431354523, 0.7800656855106354, 0.7379585802555084], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1204", "similarity": [0.7793199270963669, 0.7366514205932617, 0.7281931936740875], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1207", "similarity": [0.7991433292627335, 0.7273439168930054, 0.7225707769393921], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1213", "similarity": [0.7317626178264618, 0.7293062806129456, 0.7264629006385803], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1216", "similarity": [0.8669848293066025, 0.7468224167823792, 0.7454152405261993], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1221", "similarity": [0.7241076231002808, 0.7224636673927307, 0.7216868996620178], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1225", "similarity": [0.7908404767513275, 0.7756432741880417, 0.7566201388835907], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1226", "similarity": [0.7823041677474976, 0.7346232533454895, 0.7333442568778992], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1232", "similarity": [0.8173831701278687, 0.736847460269928, 0.7227635085582733], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1241", "similarity": [0.8032843768596649, 0.7675262987613678, 0.755263552069664], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1245", "similarity": [0.6093050241470337, 0.6070595383644104, 0.6012250781059265], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1259", "similarity": [0.8447959125041962, 0.8295777142047882, 0.774319127202034], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1262", "similarity": [0.8133216053247452, 0.7958488017320633, 0.7714706659317017], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1266", "similarity": [0.8628692030906677, 0.8419828712940216, 0.8264861553907394], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1270", "similarity": [0.8005024492740631, 0.701309859752655, 0.6902666091918945], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1271", "similarity": [0.8261932879686356, 0.7205764651298523, 0.7002262771129608], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1272", "similarity": [0.7212342023849487, 0.7106541395187378, 0.7070806622505188], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1273", "similarity": [0.8897695094347, 0.8085397481918335, 0.7750474661588669], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1274", "similarity": [0.8450797647237778, 0.8418130129575729, 0.8056619614362717], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1278", "similarity": [0.7758548855781555, 0.7484519779682159, 0.7361946105957031], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1279", "similarity": [0.7712298035621643, 0.7555335164070129, 0.7459572851657867], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1280", "similarity": [0.827012836933136, 0.8071885257959366, 0.7858406156301498], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1281", "similarity": [0.7784529477357864, 0.7386350035667419, 0.7245036363601685], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1282", "similarity": [0.7234574258327484, 0.6802079677581787, 0.6774957180023193], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1290", "similarity": [0.8792228847742081, 0.7739570587873459, 0.7439085245132446], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1292", "similarity": [0.7539853304624557, 0.7216142117977142, 0.7145121097564697], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1298", "similarity": [0.9148239195346832, 0.7015049159526825, 0.683366060256958], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1303", "similarity": [0.7046567797660828, 0.6493729948997498, 0.6451824605464935], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1316", "similarity": [0.77081698179245, 0.7639243900775909, 0.7638518512248993], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1319", "similarity": [0.7860507071018219, 0.7849607467651367, 0.7342851758003235], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1320", "similarity": [0.8360067158937454, 0.792366161942482, 0.769587829709053], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1332", "similarity": [0.7793323397636414, 0.753076046705246, 0.7459698021411896], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1335", "similarity": [0.8225918412208557, 0.8173884153366089, 0.7706536948680878], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1336", "similarity": [0.8236823976039886, 0.8147284835577011, 0.7702942490577698], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1337", "similarity": [0.7898253500461578, 0.7462815046310425, 0.7457302212715149], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1339", "similarity": [0.7511343359947205, 0.6407075822353363, 0.6393257975578308], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1344", "similarity": [0.8526198863983154, 0.8315247595310211, 0.8005094230175018], "relevant": [false, true, false]} +{"split": "test", "index": "answerable", "query_id": "1352", "similarity": [0.8466326892375946, 0.7250054478645325, 0.7024728059768677], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1359", "similarity": [0.8665230721235275, 0.7966189086437225, 0.744536817073822], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1362", "similarity": [0.657408595085144, 0.6573453545570374, 0.6346705853939056], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1363", "similarity": [0.6818379163742065, 0.6558543741703033, 0.6550711393356323], "relevant": [false, false, false]} +{"split": "test", "index": "answerable", "query_id": "1368", "similarity": [0.7772565484046936, 0.7646338492631912, 0.7484412789344788], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1370", "similarity": [0.8010129332542419, 0.763783186674118, 0.7526881694793701], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1379", "similarity": [0.8650477081537247, 0.8504252433776855, 0.8446377962827682], "relevant": [true, true, true]} +{"split": "test", "index": "answerable", "query_id": "1382", "similarity": [0.7828246504068375, 0.7563636898994446, 0.7549264878034592], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1385", "similarity": [0.764193058013916, 0.706710159778595, 0.6923578381538391], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1389", "similarity": [0.8384910225868225, 0.783025711774826, 0.7598305344581604], "relevant": [true, false, false]} +{"split": "test", "index": "answerable", "query_id": "1395", "similarity": [0.7630792856216431, 0.7320650517940521, 0.7310499250888824], "relevant": [false, false, true]} +{"split": "test", "index": "unanswerable", "query_id": "1", "similarity": [0.7029363811016083, 0.6937490105628967, 0.6833390593528748], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "3", "similarity": [0.76772341132164, 0.7580264359712601, 0.7542661130428314], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "5", "similarity": [0.7120313048362732, 0.7040311992168427, 0.6838917434215546], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "13", "similarity": [0.7621487379074097, 0.7551555037498474, 0.7480845749378204], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "36", "similarity": [0.7959075570106506, 0.7902679443359375, 0.7892882823944092], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "42", "similarity": [0.8001197725534439, 0.7765368223190308, 0.7435933947563171], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "48", "similarity": [0.7501026839017868, 0.749695360660553, 0.740513265132904], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "49", "similarity": [0.8029913157224655, 0.7788126170635223, 0.764490470290184], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "50", "similarity": [0.7706095576286316, 0.7578956484794617, 0.7495352327823639], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "51", "similarity": [0.7443061470985413, 0.7435750961303711, 0.7391541600227356], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "53", "similarity": [0.729396641254425, 0.7287785708904266, 0.7284884750843048], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "54", "similarity": [0.833916962146759, 0.821098119020462, 0.8149524480104446], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "56", "similarity": [0.7964489161968231, 0.7843835204839706, 0.7770878374576569], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "57", "similarity": [0.7958991676568985, 0.7861583828926086, 0.7780857980251312], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "70", "similarity": [0.7941447496414185, 0.7849941700696945, 0.7811834365129471], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "72", "similarity": [0.6958191692829132, 0.6870241761207581, 0.6835286021232605], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "75", "similarity": [0.7922124266624451, 0.7892077714204788, 0.7730134129524231], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "94", "similarity": [0.6409316062927246, 0.6396643817424774, 0.6360529661178589], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "99", "similarity": [0.6724228858947754, 0.6720548868179321, 0.6677436828613281], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "100", "similarity": [0.7667108774185181, 0.7414956092834473, 0.7385158538818359], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "113", "similarity": [0.7934524416923523, 0.7783675938844681, 0.7776359021663666], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "115", "similarity": [0.6117458939552307, 0.6075420379638672, 0.5999412834644318], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "118", "similarity": [0.8884675353765488, 0.7657711207866669, 0.7625764012336731], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "124", "similarity": [0.8515368700027466, 0.7774012386798859, 0.7722839415073395], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "127", "similarity": [0.704912543296814, 0.7025558352470398, 0.692731499671936], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "128", "similarity": [0.657137393951416, 0.6466840207576752, 0.6339055001735687], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "129", "similarity": [0.6614673137664795, 0.6427783071994781, 0.6287756562232971], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "130", "similarity": [0.6525686681270599, 0.6431843042373657, 0.6257853209972382], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "132", "similarity": [0.7658988237380981, 0.76087786257267, 0.7546935826539993], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "133", "similarity": [0.7896661162376404, 0.742158442735672, 0.7338775992393494], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "137", "similarity": [0.7125073075294495, 0.7043752670288086, 0.699680358171463], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "141", "similarity": [0.7417525053024292, 0.7297107875347137, 0.7068961262702942], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "142", "similarity": [0.7854724228382111, 0.7615996152162552, 0.7579139620065689], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "143", "similarity": [0.7929473966360092, 0.7671186923980713, 0.754847839474678], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "146", "similarity": [0.8097755908966064, 0.7622812539339066, 0.7458203136920929], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "148", "similarity": [0.784025639295578, 0.7690538763999939, 0.7620041370391846], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "163", "similarity": [0.8494992852210999, 0.8231061697006226, 0.7785960584878922], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "171", "similarity": [0.7640970647335052, 0.7431093454360962, 0.736943781375885], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "179", "similarity": [0.8264829218387604, 0.7888946682214737, 0.7818214595317841], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "180", "similarity": [0.7700926959514618, 0.7428233921527863, 0.7421081066131592], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "183", "similarity": [0.7893843054771423, 0.7858666926622391, 0.7851394712924957], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "185", "similarity": [0.7217798233032227, 0.7070990800857544, 0.7057316899299622], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "198", "similarity": [0.6554760932922363, 0.6531824469566345, 0.6525209546089172], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "208", "similarity": [0.7247023284435272, 0.7175058722496033, 0.7158440351486206], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "212", "similarity": [0.7664633393287659, 0.757150799036026, 0.7543959021568298], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "213", "similarity": [0.8306756466627121, 0.8299618810415268, 0.7776113748550415], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "216", "similarity": [0.7703449726104736, 0.7573475688695908, 0.7521823644638062], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "217", "similarity": [0.7729044705629349, 0.7605955749750137, 0.7603683769702911], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "218", "similarity": [0.7921966761350632, 0.7698689699172974, 0.7690748274326324], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "219", "similarity": [0.7949444949626923, 0.7712008059024811, 0.7650498151779175], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "230", "similarity": [0.6677863895893097, 0.6624345779418945, 0.6576047539710999], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "232", "similarity": [0.8400094211101532, 0.8062928766012192, 0.6829822659492493], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "233", "similarity": [0.758554220199585, 0.7544156908988953, 0.7334019541740417], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "236", "similarity": [0.7467031180858612, 0.7408315539360046, 0.739338755607605], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "237", "similarity": [0.7784469723701477, 0.7682019770145416, 0.7088387906551361], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "238", "similarity": [0.741139829158783, 0.7331408858299255, 0.7305283546447754], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "239", "similarity": [0.7455036640167236, 0.7385490536689758, 0.7294548153877258], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "248", "similarity": [0.746204674243927, 0.7300858795642853, 0.7266176044940948], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "249", "similarity": [0.7246505618095398, 0.7182492017745972, 0.7058922052383423], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "261", "similarity": [0.857544332742691, 0.8441934883594513, 0.83208367228508], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "268", "similarity": [0.7021384537220001, 0.6887215971946716, 0.6812728941440582], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "269", "similarity": [0.6994612514972687, 0.6986546516418457, 0.6967234313488007], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "274", "similarity": [0.8413791358470917, 0.766912192106247, 0.7597303241491318], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "275", "similarity": [0.8254386484622955, 0.7870145440101624, 0.7650186866521835], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "279", "similarity": [0.7822575271129608, 0.7547663897275925, 0.7324564754962921], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "294", "similarity": [0.7720412015914917, 0.7576690316200256, 0.744100034236908], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "295", "similarity": [0.8135707974433899, 0.8134715259075165, 0.8076557815074921], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "298", "similarity": [0.7596250176429749, 0.7362123727798462, 0.7315994799137115], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "300", "similarity": [0.7628100961446762, 0.7242439389228821, 0.7202023565769196], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "303", "similarity": [0.7833350598812103, 0.7530089318752289, 0.7407953143119812], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "312", "similarity": [0.75364089012146, 0.746374249458313, 0.722494900226593], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "314", "similarity": [0.7565313875675201, 0.7405337989330292, 0.7186566889286041], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "324", "similarity": [0.6955782175064087, 0.6949192881584167, 0.6913672685623169], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "327", "similarity": [0.7185897827148438, 0.7072920203208923, 0.7072582542896271], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "338", "similarity": [0.7975046634674072, 0.748763382434845, 0.7442920804023743], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "343", "similarity": [0.7729334533214569, 0.766059398651123, 0.7653561532497406], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "350", "similarity": [0.7291047275066376, 0.7197912037372589, 0.7193441390991211], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "354", "similarity": [0.7774917334318161, 0.7737409174442291, 0.7571388334035873], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "362", "similarity": [0.759043738245964, 0.755070298910141, 0.7527332603931427], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "380", "similarity": [0.7615105509757996, 0.7562246918678284, 0.7476686835289001], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "384", "similarity": [0.7528343796730042, 0.7263925671577454, 0.7232518196105957], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "385", "similarity": [0.8376368284225464, 0.7307290434837341, 0.7295404672622681], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "386", "similarity": [0.7052141427993774, 0.7026928961277008, 0.6826918721199036], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "388", "similarity": [0.7326339483261108, 0.7105787694454193, 0.709447979927063], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "399", "similarity": [0.7791419923305511, 0.7485264837741852, 0.7384714484214783], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "410", "similarity": [0.7049797177314758, 0.683527410030365, 0.6722372174263], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "411", "similarity": [0.7061430811882019, 0.7047926187515259, 0.6886550188064575], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "415", "similarity": [0.8360879570245743, 0.8331219553947449, 0.8114665597677231], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "421", "similarity": [0.7417535781860352, 0.7304936051368713, 0.709075003862381], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "431", "similarity": [0.761710911989212, 0.7525125592947006, 0.74610835313797], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "436", "similarity": [0.7594025582075119, 0.7561256289482117, 0.7487668991088867], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "437", "similarity": [0.7988032102584839, 0.790080800652504, 0.7874240428209305], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "439", "similarity": [0.7692846059799194, 0.7674060463905334, 0.767299473285675], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "440", "similarity": [0.7448928952217102, 0.7420444190502167, 0.7367921471595764], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "443", "similarity": [0.7759211659431458, 0.761216402053833, 0.7594750225543976], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "452", "similarity": [0.7644934058189392, 0.7525120079517365, 0.7496446669101715], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "475", "similarity": [0.7881592512130737, 0.7634416818618774, 0.7615413665771484], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "478", "similarity": [0.7440131604671478, 0.7400738000869751, 0.7320411205291748], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "491", "similarity": [0.7343559265136719, 0.7331064343452454, 0.7212390005588531], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "501", "similarity": [0.7289299964904785, 0.7232643961906433, 0.7210421562194824], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "502", "similarity": [0.665950357913971, 0.6619696319103241, 0.6585739254951477], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "507", "similarity": [0.8801887482404709, 0.809212788939476, 0.8091888725757599], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "508", "similarity": [0.7495159208774567, 0.7063907980918884, 0.6944817304611206], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "513", "similarity": [0.7682482153177261, 0.7376741468906403, 0.7250625193119049], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "514", "similarity": [0.8314205408096313, 0.8266224563121796, 0.814179390668869], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "516", "similarity": [0.8068366646766663, 0.7743600010871887, 0.7720349282026291], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "517", "similarity": [0.7903604805469513, 0.7824482172727585, 0.7742674350738525], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "521", "similarity": [0.7198183536529541, 0.7115947604179382, 0.7052816450595856], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "525", "similarity": [0.7693173587322235, 0.7675777077674866, 0.7585359215736389], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "527", "similarity": [0.773534744977951, 0.7613107115030289, 0.7545612752437592], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "528", "similarity": [0.7258059680461884, 0.7213800549507141, 0.7153647541999817], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "532", "similarity": [0.7266129851341248, 0.7256234884262085, 0.7249929904937744], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "533", "similarity": [0.7263603806495667, 0.7213621437549591, 0.7140765488147736], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "535", "similarity": [0.7655723989009857, 0.7525089085102081, 0.7520381659269333], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "536", "similarity": [0.7321686148643494, 0.7107288241386414, 0.7062258124351501], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "539", "similarity": [0.7835469543933868, 0.7752222567796707, 0.7714233696460724], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "540", "similarity": [0.7939575612545013, 0.7869390547275543, 0.7732616066932678], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "544", "similarity": [0.7655007243156433, 0.7589653730392456, 0.7459238171577454], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "549", "similarity": [0.7097672522068024, 0.706944614648819, 0.7042387127876282], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "551", "similarity": [0.8008323460817337, 0.7685754150152206, 0.763638436794281], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "552", "similarity": [0.8068269491195679, 0.7660653442144394, 0.7645672559738159], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "554", "similarity": [0.760211169719696, 0.7512370049953461, 0.750274121761322], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "560", "similarity": [0.7784685343503952, 0.7746509611606598, 0.7674212455749512], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "569", "similarity": [0.8442529737949371, 0.7944575250148773, 0.7853033095598221], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "575", "similarity": [0.7579565942287445, 0.7323789596557617, 0.7284194231033325], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "577", "similarity": [0.7506194561719894, 0.7319298386573792, 0.7166398465633392], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "578", "similarity": [0.7568121552467346, 0.7494983077049255, 0.7427420020103455], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "587", "similarity": [0.794885128736496, 0.7440000772476196, 0.7304946184158325], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "589", "similarity": [0.7435694932937622, 0.7312192916870117, 0.7274719476699829], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "593", "similarity": [0.7589736133813858, 0.7536534070968628, 0.7362264394760132], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "597", "similarity": [0.7695851922035217, 0.762396901845932, 0.7536479532718658], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "598", "similarity": [0.7866725623607635, 0.7781177014112473, 0.7709923088550568], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "613", "similarity": [0.7771831452846527, 0.7517833113670349, 0.7386051416397095], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "619", "similarity": [0.7215523421764374, 0.7183295488357544, 0.7164791524410248], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "623", "similarity": [0.8261176347732544, 0.8091426342725754, 0.8007083535194397], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "628", "similarity": [0.7752825319766998, 0.7493198215961456, 0.7387538552284241], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "636", "similarity": [0.7764180302619934, 0.751686692237854, 0.7387111186981201], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "637", "similarity": [0.7247857451438904, 0.6810854077339172, 0.6774901151657104], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "641", "similarity": [0.7205309271812439, 0.7135331332683563, 0.7003925144672394], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "644", "similarity": [0.7539838999509811, 0.7498028874397278, 0.7399303317070007], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "649", "similarity": [0.6371825933456421, 0.6320307850837708, 0.6316461563110352], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "659", "similarity": [0.650026947259903, 0.6417555510997772, 0.6382255554199219], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "660", "similarity": [0.6613682508468628, 0.6588095724582672, 0.6433593332767487], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "674", "similarity": [0.7743014395236969, 0.7631520628929138, 0.7367751598358154], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "684", "similarity": [0.7838437557220459, 0.7775910496711731, 0.7047359049320221], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "690", "similarity": [0.7026431262493134, 0.6996148824691772, 0.6911275386810303], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "691", "similarity": [0.7825896143913269, 0.750620037317276, 0.7452578544616699], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "692", "similarity": [0.7262070775032043, 0.7226062417030334, 0.7201774716377258], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "693", "similarity": [0.7065634727478027, 0.7054753303527832, 0.703249454498291], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "700", "similarity": [0.8298687636852264, 0.7837457060813904, 0.7629614472389221], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "702", "similarity": [0.8323096334934235, 0.7784163355827332, 0.7700539380311966], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "715", "similarity": [0.8208154737949371, 0.8094618618488312, 0.7860832810401917], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "716", "similarity": [0.8103711903095245, 0.7648439258337021, 0.7513183653354645], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "718", "similarity": [0.7347411513328552, 0.7336439490318298, 0.7321698069572449], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "721", "similarity": [0.7274701595306396, 0.7252560257911682, 0.7042293846607208], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "723", "similarity": [0.7665507197380066, 0.7483245134353638, 0.7367261052131653], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "727", "similarity": [0.7392005026340485, 0.7380672097206116, 0.7377597689628601], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "728", "similarity": [0.7347556948661804, 0.7332133054733276, 0.7322185039520264], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "729", "similarity": [0.7457635402679443, 0.7336677610874176, 0.7195152044296265], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "742", "similarity": [0.7033832967281342, 0.7022213935852051, 0.7000100910663605], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "743", "similarity": [0.7356070280075073, 0.7221891283988953, 0.7188420593738556], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "744", "similarity": [0.8451002836227417, 0.8413776755332947, 0.7386681139469147], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "756", "similarity": [0.8299048095941544, 0.8122470676898956, 0.7548938393592834], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "759", "similarity": [0.8344922959804535, 0.8211539834737778, 0.7990319430828094], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "768", "similarity": [0.7411505579948425, 0.7150000035762787, 0.6888008713722229], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "770", "similarity": [0.7788041532039642, 0.7478071451187134, 0.740228146314621], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "775", "similarity": [0.8026803731918335, 0.7813616245985031, 0.7532719075679779], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "781", "similarity": [0.8591830432415009, 0.8250378966331482, 0.7860068082809448], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "783", "similarity": [0.7948863655328751, 0.7762290835380554, 0.7657938301563263], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "784", "similarity": [0.8100689947605133, 0.8084197044372559, 0.805113285779953], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "785", "similarity": [0.7889743745326996, 0.7334842681884766, 0.7304971218109131], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "793", "similarity": [0.7724388241767883, 0.7268569767475128, 0.7196374535560608], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "800", "similarity": [0.8289507627487183, 0.8234221041202545, 0.8005592525005341], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "805", "similarity": [0.7611818611621857, 0.7568326890468597, 0.7566337883472443], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "808", "similarity": [0.7000648677349091, 0.6852685511112213, 0.678263247013092], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "811", "similarity": [0.7115486264228821, 0.7094067931175232, 0.7067081928253174], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "814", "similarity": [0.7359445095062256, 0.7351368963718414, 0.7319276928901672], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "820", "similarity": [0.7403759360313416, 0.7353923320770264, 0.7324269711971283], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "821", "similarity": [0.7407223582267761, 0.7379066646099091, 0.735599160194397], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "823", "similarity": [0.7779718041419983, 0.7545285075902939, 0.7471126317977905], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "830", "similarity": [0.7423371970653534, 0.7399648427963257, 0.7319430112838745], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "831", "similarity": [0.7466650605201721, 0.7402245998382568, 0.7280697226524353], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "832", "similarity": [0.7507630735635757, 0.742770254611969, 0.7380167245864868], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "834", "similarity": [0.7914370000362396, 0.7329479455947876, 0.7189933359622955], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "837", "similarity": [0.7649929821491241, 0.7470101118087769, 0.741224616765976], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "839", "similarity": [0.7419877648353577, 0.7144302427768707, 0.7116941809654236], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "845", "similarity": [0.8685175776481628, 0.848732203245163, 0.8460465371608734], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "847", "similarity": [0.7473157346248627, 0.7126085758209229, 0.7017037868499756], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "852", "similarity": [0.7522986680269241, 0.6999932825565338, 0.6980593800544739], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "859", "similarity": [0.7629775404930115, 0.7459883987903595, 0.7325558066368103], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "870", "similarity": [0.7546938359737396, 0.7390194535255432, 0.7298319935798645], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "873", "similarity": [0.71230748295784, 0.700303703546524, 0.693898618221283], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "879", "similarity": [0.7047696709632874, 0.7044182419776917, 0.7009522616863251], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "880", "similarity": [0.7742561846971512, 0.7648105919361115, 0.754446804523468], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "882", "similarity": [0.7425941824913025, 0.7287073135375977, 0.717049777507782], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "887", "similarity": [0.7091203927993774, 0.7080304622650146, 0.6801060736179352], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "903", "similarity": [0.7642746865749359, 0.7522061467170715, 0.7439824342727661], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "904", "similarity": [0.7525677382946014, 0.751628041267395, 0.748858630657196], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "907", "similarity": [0.8224397599697113, 0.7520658522844315, 0.7462199926376343], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "911", "similarity": [0.7446613013744354, 0.7318137586116791, 0.7263806462287903], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "913", "similarity": [0.7429296374320984, 0.7418681979179382, 0.7273560166358948], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "914", "similarity": [0.7595296502113342, 0.7366532683372498, 0.7259669899940491], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "921", "similarity": [0.7625961750745773, 0.7433757483959198, 0.7410108149051666], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "922", "similarity": [0.734434574842453, 0.7184143662452698, 0.7149204909801483], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "936", "similarity": [0.742929995059967, 0.738252729177475, 0.724001944065094], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "956", "similarity": [0.7734988331794739, 0.7648598253726959, 0.7617708742618561], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "957", "similarity": [0.7171511650085449, 0.7008177638053894, 0.6889441013336182], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "960", "similarity": [0.7333106696605682, 0.729676365852356, 0.7286441326141357], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "967", "similarity": [0.6955503523349762, 0.6868831515312195, 0.6866341233253479], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "971", "similarity": [0.8562845587730408, 0.8534707725048065, 0.8523468375205994], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "975", "similarity": [0.7453554272651672, 0.7286932468414307, 0.7245600521564484], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "982", "similarity": [0.7264578938484192, 0.7214467525482178, 0.7206145524978638], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "985", "similarity": [0.762986570596695, 0.7479864358901978, 0.7471787929534912], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "993", "similarity": [0.8394210487604141, 0.7863108366727829, 0.769094854593277], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1012", "similarity": [0.8500750660896301, 0.8079778254032135, 0.7742600739002228], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1014", "similarity": [0.759058490395546, 0.7262263894081116, 0.7257663011550903], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1019", "similarity": [0.7656744420528412, 0.7567866742610931, 0.7465376555919647], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1020", "similarity": [0.8098451495170593, 0.7634838819503784, 0.7612511813640594], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1021", "similarity": [0.8183752596378326, 0.7752389013767242, 0.7656659632921219], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1024", "similarity": [0.7773071229457855, 0.7487127482891083, 0.746694028377533], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1029", "similarity": [0.803783044219017, 0.7880780696868896, 0.7803459167480469], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1041", "similarity": [0.8660165518522263, 0.8547260463237762, 0.8391537219285965], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1049", "similarity": [0.7519171237945557, 0.7514179944992065, 0.6915662586688995], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1062", "similarity": [0.7836065888404846, 0.7592867314815521, 0.7436666190624237], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1086", "similarity": [0.705142080783844, 0.673597514629364, 0.6733212471008301], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1088", "similarity": [0.7560275048017502, 0.7528949677944183, 0.7336575984954834], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1089", "similarity": [0.7938258051872253, 0.7660177052021027, 0.7539996355772018], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1099", "similarity": [0.8029463887214661, 0.799681767821312, 0.7868390381336212], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1100", "similarity": [0.7971938103437424, 0.7814894616603851, 0.7686410546302795], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1104", "similarity": [0.7604284286499023, 0.748061329126358, 0.7453789114952087], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1107", "similarity": [0.7666935324668884, 0.7633379101753235, 0.6976696252822876], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1110", "similarity": [0.7517037242650986, 0.7323667407035828, 0.7284021973609924], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1121", "similarity": [0.7775002717971802, 0.7520267516374588, 0.7434476017951965], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1130", "similarity": [0.7961269319057465, 0.7908194065093994, 0.7784222364425659], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1132", "similarity": [0.8183781802654266, 0.7927461862564087, 0.7811104357242584], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1137", "similarity": [0.7391822934150696, 0.7320400476455688, 0.7310544550418854], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1140", "similarity": [0.7511867582798004, 0.7321124076843262, 0.7246317863464355], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1144", "similarity": [0.7674277722835541, 0.7555492967367172, 0.7330150604248047], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1146", "similarity": [0.6384587287902832, 0.6333682537078857, 0.6183847784996033], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1150", "similarity": [0.7744574546813965, 0.750085860490799, 0.7370292842388153], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1163", "similarity": [0.7352535724639893, 0.7229893207550049, 0.7082043886184692], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1175", "similarity": [0.6788469552993774, 0.6665136814117432, 0.6643029451370239], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1179", "similarity": [0.740827888250351, 0.7390636801719666, 0.7295312881469727], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1180", "similarity": [0.7579579651355743, 0.7312131524085999, 0.7236147522926331], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1185", "similarity": [0.8024595081806183, 0.777779832482338, 0.7658783942461014], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1187", "similarity": [0.7890508323907852, 0.7607564032077789, 0.7379032969474792], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1191", "similarity": [0.6705031991004944, 0.6635471284389496, 0.6513506472110748], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1194", "similarity": [0.7227251529693604, 0.7223069667816162, 0.7015037834644318], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1196", "similarity": [0.673975944519043, 0.6688997149467468, 0.6549594700336456], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1197", "similarity": [0.6924685835838318, 0.6746512055397034, 0.6519043445587158], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1199", "similarity": [0.748276948928833, 0.7410315275192261, 0.7174493968486786], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1200", "similarity": [0.738024890422821, 0.7112044095993042, 0.7088710069656372], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1202", "similarity": [0.7800656855106354, 0.7379585802555084, 0.7338744699954987], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1204", "similarity": [0.7366514205932617, 0.7281931936740875, 0.7202237248420715], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1207", "similarity": [0.7273439168930054, 0.7225707769393921, 0.7176903486251831], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1213", "similarity": [0.7317626178264618, 0.7293062806129456, 0.7264629006385803], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1216", "similarity": [0.7468224167823792, 0.7454152405261993, 0.7425310015678406], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1221", "similarity": [0.7241076231002808, 0.7224636673927307, 0.7216868996620178], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1225", "similarity": [0.7756432741880417, 0.7566201388835907, 0.7558136284351349], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1226", "similarity": [0.7823041677474976, 0.7346232533454895, 0.7333442568778992], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1232", "similarity": [0.736847460269928, 0.7227635085582733, 0.7192588448524475], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1241", "similarity": [0.7675262987613678, 0.755263552069664, 0.7540930360555649], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1245", "similarity": [0.6093050241470337, 0.6012250781059265, 0.5966856479644775], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1259", "similarity": [0.8295777142047882, 0.774319127202034, 0.7557564675807953], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1262", "similarity": [0.7958488017320633, 0.7714706659317017, 0.7663780450820923], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1266", "similarity": [0.8075172156095505, 0.8051984012126923, 0.8018884807825089], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1270", "similarity": [0.701309859752655, 0.6902666091918945, 0.678877592086792], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1271", "similarity": [0.7205764651298523, 0.7002262771129608, 0.6934643983840942], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1272", "similarity": [0.7212342023849487, 0.7106541395187378, 0.7070806622505188], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1273", "similarity": [0.8085397481918335, 0.7750474661588669, 0.7648250013589859], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1274", "similarity": [0.8418130129575729, 0.8056619614362717, 0.7879739552736282], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1278", "similarity": [0.7484519779682159, 0.7361946105957031, 0.7302991449832916], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1279", "similarity": [0.7555335164070129, 0.7459572851657867, 0.7263928949832916], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1280", "similarity": [0.827012836933136, 0.8071885257959366, 0.7858406156301498], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1281", "similarity": [0.7784529477357864, 0.7386350035667419, 0.7245036363601685], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1282", "similarity": [0.6802079677581787, 0.6774957180023193, 0.6600225567817688], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1290", "similarity": [0.7739570587873459, 0.7439085245132446, 0.7414477467536926], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1292", "similarity": [0.7216142117977142, 0.7145121097564697, 0.7138743102550507], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1298", "similarity": [0.7015049159526825, 0.683366060256958, 0.6824226975440979], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1303", "similarity": [0.6493729948997498, 0.6451824605464935, 0.6417464911937714], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1316", "similarity": [0.77081698179245, 0.7639243900775909, 0.7638518512248993], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1319", "similarity": [0.7849607467651367, 0.7342851758003235, 0.7273035943508148], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1320", "similarity": [0.792366161942482, 0.769587829709053, 0.7678298056125641], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1332", "similarity": [0.7793323397636414, 0.753076046705246, 0.7459698021411896], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1335", "similarity": [0.8173884153366089, 0.7706536948680878, 0.7629219591617584], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1336", "similarity": [0.8236823976039886, 0.7702942490577698, 0.7674665451049805], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1337", "similarity": [0.7462815046310425, 0.7457302212715149, 0.7428813576698303], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1339", "similarity": [0.6407075822353363, 0.6393257975578308, 0.6347869336605072], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1344", "similarity": [0.8526198863983154, 0.8005094230175018, 0.7991660833358765], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1352", "similarity": [0.7250054478645325, 0.6947504281997681, 0.6944907605648041], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1359", "similarity": [0.7966189086437225, 0.744536817073822, 0.7423273324966431], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1362", "similarity": [0.657408595085144, 0.6573453545570374, 0.6346705853939056], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1363", "similarity": [0.6818379163742065, 0.6558543741703033, 0.6550711393356323], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1368", "similarity": [0.7646338492631912, 0.7484412789344788, 0.741333544254303], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1370", "similarity": [0.763783186674118, 0.7526881694793701, 0.7482672929763794], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1379", "similarity": [0.830168604850769, 0.7939124554395676, 0.7860750108957291], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1382", "similarity": [0.7563636898994446, 0.7549264878034592, 0.7454427480697632], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1385", "similarity": [0.6923578381538391, 0.6850172579288483, 0.6835412383079529], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1389", "similarity": [0.7598305344581604, 0.756094679236412, 0.7381424903869629], "relevant": [false, false, false]} +{"split": "test", "index": "unanswerable", "query_id": "1395", "similarity": [0.7630792856216431, 0.7320650517940521, 0.7291585206985474], "relevant": [false, false, false]} diff --git a/evaluation/embedder_choice.py b/evaluation/embedder_choice.py new file mode 100644 index 0000000..96ffa50 --- /dev/null +++ b/evaluation/embedder_choice.py @@ -0,0 +1,124 @@ +"""How the embedding model was chosen, and a check on a second dataset. Vector search only. + +Rules, fixed before running: +- SciFact (`--dataset scifact`): three small models. The one with the best nDCG@10 on the 809 + training claims becomes the app's default. The 300 test claims are scored in the same run for + the record; they play no part in the choice. +- FiQA (`--dataset fiqa`): the chosen model against the old one (all-MiniLM-L6-v2) on the 648 + test questions, a dataset that played no part in the choice. If the chosen model is clearly + worse there (paired 95% interval for nDCG@10 entirely below 0), the app goes back to the old one. + +Search here is exact: every passage's vector is compared with the question's (cosine similarity, +top 10). The app uses ChromaDB, whose search is approximate, so its numbers can differ slightly +(`evaluation.retrieval_eval` gives the app's). + + python -m evaluation.embedder_choice --dataset scifact + python -m evaluation.embedder_choice --dataset fiqa +""" + +import argparse +import json +import sys +import time +from pathlib import Path + +import numpy as np +import pandas as pd +from sentence_transformers import SentenceTransformer + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "src")) + +from rag_engine import EMBEDDERS # noqa: E402 + +from evaluation.compare import paired_bootstrap # noqa: E402 +from evaluation.metrics import hit_at_k, ndcg_at_k # noqa: E402 +from evaluation.retrieval_eval import load # noqa: E402 + +CANDIDATES = { + "all-MiniLM-L6-v2": EMBEDDERS["all-MiniLM-L6-v2"], + "bge-small-en-v1.5": EMBEDDERS["bge-small-en-v1.5"], + "e5-small-v2": {"repo": "intfloat/e5-small-v2", "revision": "ffb93f3bd4047442299a41ebb6fa998a38507c52", + "query_prefix": "query: ", "passage_prefix": "passage: "}, +} +OLD, NEW = "all-MiniLM-L6-v2", "bge-small-en-v1.5" +PLAN = {"scifact": {"models": list(CANDIDATES), "splits": ["train", "test"]}, + "fiqa": {"models": [OLD, NEW], "splits": ["test"]}} +K = 10 +OUT_DIR = Path(__file__).resolve().parent + + +def top_k(q: np.ndarray, docs: np.ndarray, k: int = K) -> np.ndarray: + """Indexes of the k most similar passages for each question, best first.""" + sim = q @ docs.T + part = np.argpartition(-sim, k, axis=1)[:, :k] + order = np.argsort(-np.take_along_axis(sim, part, axis=1), axis=1) + return np.take_along_axis(part, order, axis=1) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("--dataset", choices=sorted(PLAN), default="scifact") + args = parser.parse_args() + plan = PLAN[args.dataset] + + splits = {} + for split in plan["splits"]: + corpus, questions, relevant = load(args.dataset, split) + splits[split] = (questions, relevant) + texts = [(t + "\n" + b).strip() for t, b in zip(corpus["title"].fillna("").astype(str), + corpus["text"].fillna("").astype(str))] + ids = np.array(corpus["_id"].astype(str).tolist()) + print(f"{args.dataset}: {len(texts)} passages, " + + ", ".join(f"{s} {len(q)} questions" for s, (q, _) in splits.items()), flush=True) + + result = {"dataset": args.dataset, "passages": len(texts), "candidates": {}} + rows = [] + for name in plan["models"]: + spec = CANDIDATES[name] + model = SentenceTransformer(spec["repo"], revision=spec["revision"], device="cpu") + t0 = time.time() + docs = model.encode([spec["passage_prefix"] + t for t in texts], batch_size=64, + normalize_embeddings=True) + entry = {"repo": spec["repo"], "revision": spec["revision"], + "minutes_to_embed_passages": round((time.time() - t0) / 60, 1)} + for split, (questions, relevant) in splits.items(): + qids = list(questions) + q = model.encode([spec["query_prefix"] + questions[i] for i in qids], batch_size=64, + normalize_embeddings=True) + for qid, best in zip(qids, top_k(q, docs)): + ranked = ids[best].tolist() + rows.append({"model": name, "split": split, "query_id": qid, + "ndcg10": ndcg_at_k(ranked, relevant[qid], K), + "hit3": hit_at_k(ranked, relevant[qid], 3)}) + mine = [r for r in rows if r["model"] == name and r["split"] == split] + entry[split] = {"ndcg@10": round(float(np.mean([r["ndcg10"] for r in mine])), 4), + "hit@3": round(float(np.mean([r["hit3"] for r in mine])), 4), + "questions": len(mine)} + result["candidates"][name] = entry + print(name, entry, flush=True) + + per_q = pd.DataFrame(rows) + result["difference_new_minus_old"] = {} + for split in plan["splits"]: + a = per_q[(per_q.model == NEW) & (per_q.split == split)].set_index("query_id") + b = per_q[(per_q.model == OLD) & (per_q.split == split)].set_index("query_id").loc[a.index] + result["difference_new_minus_old"][split] = {} + for metric in ("ndcg10", "hit3"): + mean, low, high = paired_bootstrap((a[metric] - b[metric]).to_numpy()) + result["difference_new_minus_old"][split][metric] = { + "difference": round(mean, 4), "interval_95": [round(low, 4), round(high, 4)]} + + if args.dataset == "scifact": + result["chosen_on_train"] = max(plan["models"], + key=lambda n: result["candidates"][n]["train"]["ndcg@10"]) + else: + upper = result["difference_new_minus_old"]["test"]["ndcg10"]["interval_95"][1] + result["keep_new_model"] = bool(upper >= 0) + (OUT_DIR / f"embedder_choice_{args.dataset}.json").write_text(json.dumps(result, indent=2) + "\n") + per_q.round(5).to_csv(OUT_DIR / f"embedder_choice_per_question_{args.dataset}.csv", index=False) + print(json.dumps({k: v for k, v in result.items() if k != "candidates"}, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/evaluation/embedder_choice_fiqa.json b/evaluation/embedder_choice_fiqa.json new file mode 100644 index 0000000..ea85cc3 --- /dev/null +++ b/evaluation/embedder_choice_fiqa.json @@ -0,0 +1,45 @@ +{ + "dataset": "fiqa", + "passages": 57638, + "candidates": { + "all-MiniLM-L6-v2": { + "repo": "sentence-transformers/all-MiniLM-L6-v2", + "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", + "minutes_to_embed_passages": 11.0, + "test": { + "ndcg@10": 0.3687, + "hit@3": 0.5123, + "questions": 648 + } + }, + "bge-small-en-v1.5": { + "repo": "BAAI/bge-small-en-v1.5", + "revision": "5c38ec7c405ec4b44b94cc5a9bb96e735b38267a", + "minutes_to_embed_passages": 53.3, + "test": { + "ndcg@10": 0.4035, + "hit@3": 0.5432, + "questions": 648 + } + } + }, + "difference_new_minus_old": { + "test": { + "ndcg10": { + "difference": 0.0348, + "interval_95": [ + 0.0167, + 0.053 + ] + }, + "hit3": { + "difference": 0.0309, + "interval_95": [ + 0.0015, + 0.0617 + ] + } + } + }, + "keep_new_model": true +} diff --git a/evaluation/embedder_choice_per_question_fiqa.csv b/evaluation/embedder_choice_per_question_fiqa.csv new file mode 100644 index 0000000..2a30733 --- /dev/null +++ b/evaluation/embedder_choice_per_question_fiqa.csv @@ -0,0 +1,1297 @@ +model,split,query_id,ndcg10,hit3 +all-MiniLM-L6-v2,test,8,0.61315,1.0 +all-MiniLM-L6-v2,test,15,1.0,1.0 +all-MiniLM-L6-v2,test,18,1.0,1.0 +all-MiniLM-L6-v2,test,26,0.0,0.0 +all-MiniLM-L6-v2,test,34,0.0,0.0 +all-MiniLM-L6-v2,test,42,0.0,0.0 +all-MiniLM-L6-v2,test,56,1.0,1.0 +all-MiniLM-L6-v2,test,68,0.0,0.0 +all-MiniLM-L6-v2,test,89,0.24739,0.0 +all-MiniLM-L6-v2,test,90,0.63093,1.0 +all-MiniLM-L6-v2,test,94,0.43068,0.0 +all-MiniLM-L6-v2,test,98,0.61315,1.0 +all-MiniLM-L6-v2,test,104,0.30657,1.0 +all-MiniLM-L6-v2,test,106,0.43068,0.0 +all-MiniLM-L6-v2,test,109,1.0,1.0 +all-MiniLM-L6-v2,test,475,0.5,1.0 +all-MiniLM-L6-v2,test,503,0.0,0.0 +all-MiniLM-L6-v2,test,504,0.0,0.0 +all-MiniLM-L6-v2,test,515,0.31546,0.0 +all-MiniLM-L6-v2,test,529,0.0,0.0 +all-MiniLM-L6-v2,test,547,0.0,0.0 +all-MiniLM-L6-v2,test,549,0.35621,0.0 +all-MiniLM-L6-v2,test,559,0.0,0.0 +all-MiniLM-L6-v2,test,570,0.0,0.0 +all-MiniLM-L6-v2,test,585,0.0,0.0 +all-MiniLM-L6-v2,test,588,0.30657,1.0 +all-MiniLM-L6-v2,test,594,0.61315,1.0 +all-MiniLM-L6-v2,test,603,1.0,1.0 +all-MiniLM-L6-v2,test,604,0.64478,1.0 +all-MiniLM-L6-v2,test,620,0.0,0.0 +all-MiniLM-L6-v2,test,622,0.20438,0.0 +all-MiniLM-L6-v2,test,659,0.61147,1.0 +all-MiniLM-L6-v2,test,672,0.0,0.0 +all-MiniLM-L6-v2,test,684,0.0,0.0 +all-MiniLM-L6-v2,test,687,0.0,0.0 +all-MiniLM-L6-v2,test,689,0.0,0.0 +all-MiniLM-L6-v2,test,691,0.30103,0.0 +all-MiniLM-L6-v2,test,699,0.63093,1.0 +all-MiniLM-L6-v2,test,701,0.0,0.0 +all-MiniLM-L6-v2,test,715,0.0,0.0 +all-MiniLM-L6-v2,test,721,1.0,1.0 +all-MiniLM-L6-v2,test,744,0.0,0.0 +all-MiniLM-L6-v2,test,750,0.81753,1.0 +all-MiniLM-L6-v2,test,753,0.0,0.0 +all-MiniLM-L6-v2,test,766,0.29608,1.0 +all-MiniLM-L6-v2,test,776,0.29346,1.0 +all-MiniLM-L6-v2,test,810,0.28906,0.0 +all-MiniLM-L6-v2,test,813,0.30657,1.0 +all-MiniLM-L6-v2,test,849,0.63093,1.0 +all-MiniLM-L6-v2,test,852,0.70392,1.0 +all-MiniLM-L6-v2,test,853,0.0,0.0 +all-MiniLM-L6-v2,test,858,0.11285,0.0 +all-MiniLM-L6-v2,test,859,0.0,0.0 +all-MiniLM-L6-v2,test,864,0.7323,1.0 +all-MiniLM-L6-v2,test,879,1.0,1.0 +all-MiniLM-L6-v2,test,885,0.0,0.0 +all-MiniLM-L6-v2,test,895,0.63093,1.0 +all-MiniLM-L6-v2,test,904,1.0,1.0 +all-MiniLM-L6-v2,test,928,0.61315,1.0 +all-MiniLM-L6-v2,test,929,1.0,1.0 +all-MiniLM-L6-v2,test,932,0.0,0.0 +all-MiniLM-L6-v2,test,939,1.0,1.0 +all-MiniLM-L6-v2,test,945,0.63093,1.0 +all-MiniLM-L6-v2,test,957,0.67139,1.0 +all-MiniLM-L6-v2,test,988,0.0,0.0 +all-MiniLM-L6-v2,test,1074,1.0,1.0 +all-MiniLM-L6-v2,test,1085,0.17724,0.0 +all-MiniLM-L6-v2,test,1090,0.21841,0.0 +all-MiniLM-L6-v2,test,1150,0.0,0.0 +all-MiniLM-L6-v2,test,1157,0.38685,1.0 +all-MiniLM-L6-v2,test,1159,0.0,0.0 +all-MiniLM-L6-v2,test,1198,1.0,1.0 +all-MiniLM-L6-v2,test,1230,0.0,0.0 +all-MiniLM-L6-v2,test,1281,0.63093,1.0 +all-MiniLM-L6-v2,test,1284,0.63093,1.0 +all-MiniLM-L6-v2,test,1297,1.0,1.0 +all-MiniLM-L6-v2,test,1306,0.28564,0.0 +all-MiniLM-L6-v2,test,1309,0.39038,1.0 +all-MiniLM-L6-v2,test,1310,0.0,0.0 +all-MiniLM-L6-v2,test,1321,0.0,0.0 +all-MiniLM-L6-v2,test,1322,0.13013,0.0 +all-MiniLM-L6-v2,test,1391,0.29608,1.0 +all-MiniLM-L6-v2,test,1393,0.5204,1.0 +all-MiniLM-L6-v2,test,1415,1.0,1.0 +all-MiniLM-L6-v2,test,1416,0.0,0.0 +all-MiniLM-L6-v2,test,1441,1.0,1.0 +all-MiniLM-L6-v2,test,1451,0.63093,1.0 +all-MiniLM-L6-v2,test,1469,0.0,0.0 +all-MiniLM-L6-v2,test,1530,0.57801,1.0 +all-MiniLM-L6-v2,test,1670,0.76536,1.0 +all-MiniLM-L6-v2,test,1676,0.55851,1.0 +all-MiniLM-L6-v2,test,1736,0.50874,1.0 +all-MiniLM-L6-v2,test,1748,0.76536,1.0 +all-MiniLM-L6-v2,test,1783,0.0,0.0 +all-MiniLM-L6-v2,test,1812,0.0,0.0 +all-MiniLM-L6-v2,test,1815,1.0,1.0 +all-MiniLM-L6-v2,test,1819,0.1513,1.0 +all-MiniLM-L6-v2,test,1824,0.33333,0.0 +all-MiniLM-L6-v2,test,1826,1.0,1.0 +all-MiniLM-L6-v2,test,1832,0.0,0.0 +all-MiniLM-L6-v2,test,1871,0.5414,1.0 +all-MiniLM-L6-v2,test,1877,1.0,1.0 +all-MiniLM-L6-v2,test,1889,0.0,0.0 +all-MiniLM-L6-v2,test,1915,0.61315,1.0 +all-MiniLM-L6-v2,test,1920,0.38685,0.0 +all-MiniLM-L6-v2,test,1933,1.0,1.0 +all-MiniLM-L6-v2,test,1948,1.0,1.0 +all-MiniLM-L6-v2,test,1994,0.5684,1.0 +all-MiniLM-L6-v2,test,2010,0.83855,1.0 +all-MiniLM-L6-v2,test,2051,0.0,0.0 +all-MiniLM-L6-v2,test,2070,0.0,0.0 +all-MiniLM-L6-v2,test,2075,0.61956,1.0 +all-MiniLM-L6-v2,test,2076,1.0,1.0 +all-MiniLM-L6-v2,test,2088,0.0,0.0 +all-MiniLM-L6-v2,test,2108,0.29608,1.0 +all-MiniLM-L6-v2,test,2118,0.0,0.0 +all-MiniLM-L6-v2,test,2154,0.0,0.0 +all-MiniLM-L6-v2,test,2181,0.0,0.0 +all-MiniLM-L6-v2,test,2183,0.28162,1.0 +all-MiniLM-L6-v2,test,2204,0.92021,1.0 +all-MiniLM-L6-v2,test,2264,0.0,0.0 +all-MiniLM-L6-v2,test,2296,0.23505,1.0 +all-MiniLM-L6-v2,test,2306,0.61315,1.0 +all-MiniLM-L6-v2,test,2316,0.81753,1.0 +all-MiniLM-L6-v2,test,2318,0.63093,1.0 +all-MiniLM-L6-v2,test,2330,0.61315,1.0 +all-MiniLM-L6-v2,test,2334,0.80704,1.0 +all-MiniLM-L6-v2,test,2348,0.22009,1.0 +all-MiniLM-L6-v2,test,2376,0.90901,1.0 +all-MiniLM-L6-v2,test,2383,0.30657,1.0 +all-MiniLM-L6-v2,test,2384,0.38685,0.0 +all-MiniLM-L6-v2,test,2385,0.61315,1.0 +all-MiniLM-L6-v2,test,2388,1.0,1.0 +all-MiniLM-L6-v2,test,2395,0.0,0.0 +all-MiniLM-L6-v2,test,2398,0.10699,0.0 +all-MiniLM-L6-v2,test,2399,1.0,1.0 +all-MiniLM-L6-v2,test,2400,0.46928,1.0 +all-MiniLM-L6-v2,test,2407,0.13013,0.0 +all-MiniLM-L6-v2,test,2416,0.83187,1.0 +all-MiniLM-L6-v2,test,2423,0.49177,1.0 +all-MiniLM-L6-v2,test,2443,0.0,0.0 +all-MiniLM-L6-v2,test,2445,1.0,1.0 +all-MiniLM-L6-v2,test,2460,1.0,1.0 +all-MiniLM-L6-v2,test,2465,0.0,0.0 +all-MiniLM-L6-v2,test,2472,0.0,0.0 +all-MiniLM-L6-v2,test,2486,1.0,1.0 +all-MiniLM-L6-v2,test,2498,0.0,0.0 +all-MiniLM-L6-v2,test,2513,0.43068,0.0 +all-MiniLM-L6-v2,test,2516,0.63668,1.0 +all-MiniLM-L6-v2,test,2549,0.61315,1.0 +all-MiniLM-L6-v2,test,2551,0.53072,1.0 +all-MiniLM-L6-v2,test,2568,0.64135,1.0 +all-MiniLM-L6-v2,test,2579,0.19092,1.0 +all-MiniLM-L6-v2,test,2580,0.46928,1.0 +all-MiniLM-L6-v2,test,2587,0.63093,1.0 +all-MiniLM-L6-v2,test,2589,0.38365,0.0 +all-MiniLM-L6-v2,test,2590,0.6257,1.0 +all-MiniLM-L6-v2,test,2593,0.20211,0.0 +all-MiniLM-L6-v2,test,2598,0.2372,0.0 +all-MiniLM-L6-v2,test,2648,0.63093,1.0 +all-MiniLM-L6-v2,test,2676,0.43068,0.0 +all-MiniLM-L6-v2,test,2685,0.07614,0.0 +all-MiniLM-L6-v2,test,2695,1.0,1.0 +all-MiniLM-L6-v2,test,2713,0.0,0.0 +all-MiniLM-L6-v2,test,2724,0.20438,0.0 +all-MiniLM-L6-v2,test,2737,1.0,1.0 +all-MiniLM-L6-v2,test,2747,1.0,1.0 +all-MiniLM-L6-v2,test,2749,0.85034,1.0 +all-MiniLM-L6-v2,test,2790,0.27488,1.0 +all-MiniLM-L6-v2,test,2801,0.61315,1.0 +all-MiniLM-L6-v2,test,2856,0.73659,1.0 +all-MiniLM-L6-v2,test,2857,0.58557,1.0 +all-MiniLM-L6-v2,test,2880,1.0,1.0 +all-MiniLM-L6-v2,test,2885,0.0,0.0 +all-MiniLM-L6-v2,test,2891,1.0,1.0 +all-MiniLM-L6-v2,test,2895,0.0,0.0 +all-MiniLM-L6-v2,test,2903,0.0,0.0 +all-MiniLM-L6-v2,test,2923,0.50438,1.0 +all-MiniLM-L6-v2,test,2964,0.17724,0.0 +all-MiniLM-L6-v2,test,2968,0.62405,1.0 +all-MiniLM-L6-v2,test,2994,0.13565,0.0 +all-MiniLM-L6-v2,test,3006,0.19092,1.0 +all-MiniLM-L6-v2,test,3008,0.2463,1.0 +all-MiniLM-L6-v2,test,3014,0.0,0.0 +all-MiniLM-L6-v2,test,3033,0.61315,1.0 +all-MiniLM-L6-v2,test,3039,0.63093,1.0 +all-MiniLM-L6-v2,test,3049,0.39038,1.0 +all-MiniLM-L6-v2,test,3051,1.0,1.0 +all-MiniLM-L6-v2,test,3067,0.0,0.0 +all-MiniLM-L6-v2,test,3085,0.5,1.0 +all-MiniLM-L6-v2,test,3091,0.90837,1.0 +all-MiniLM-L6-v2,test,3103,0.63093,1.0 +all-MiniLM-L6-v2,test,3115,0.49352,1.0 +all-MiniLM-L6-v2,test,3125,0.5,1.0 +all-MiniLM-L6-v2,test,3148,0.50874,1.0 +all-MiniLM-L6-v2,test,3149,0.17724,0.0 +all-MiniLM-L6-v2,test,3177,0.0,0.0 +all-MiniLM-L6-v2,test,3179,0.61315,1.0 +all-MiniLM-L6-v2,test,3186,0.61315,1.0 +all-MiniLM-L6-v2,test,3189,0.0,0.0 +all-MiniLM-L6-v2,test,3254,0.46928,1.0 +all-MiniLM-L6-v2,test,3264,0.36945,1.0 +all-MiniLM-L6-v2,test,3357,0.31546,0.0 +all-MiniLM-L6-v2,test,3369,0.0,0.0 +all-MiniLM-L6-v2,test,3394,0.63668,1.0 +all-MiniLM-L6-v2,test,3404,0.18463,0.0 +all-MiniLM-L6-v2,test,3405,0.30103,0.0 +all-MiniLM-L6-v2,test,3446,0.55315,1.0 +all-MiniLM-L6-v2,test,3451,0.27488,1.0 +all-MiniLM-L6-v2,test,3453,0.87722,1.0 +all-MiniLM-L6-v2,test,3480,1.0,1.0 +all-MiniLM-L6-v2,test,3490,0.0,0.0 +all-MiniLM-L6-v2,test,3500,0.3026,1.0 +all-MiniLM-L6-v2,test,3503,0.62721,1.0 +all-MiniLM-L6-v2,test,3512,0.0,0.0 +all-MiniLM-L6-v2,test,3528,1.0,1.0 +all-MiniLM-L6-v2,test,3530,0.29608,1.0 +all-MiniLM-L6-v2,test,3534,0.0,0.0 +all-MiniLM-L6-v2,test,3566,0.49819,1.0 +all-MiniLM-L6-v2,test,3569,0.0,0.0 +all-MiniLM-L6-v2,test,3594,0.55851,1.0 +all-MiniLM-L6-v2,test,3612,0.0,0.0 +all-MiniLM-L6-v2,test,3615,0.0,0.0 +all-MiniLM-L6-v2,test,3625,0.46928,1.0 +all-MiniLM-L6-v2,test,3682,0.0,0.0 +all-MiniLM-L6-v2,test,3683,0.57097,1.0 +all-MiniLM-L6-v2,test,3694,0.0,0.0 +all-MiniLM-L6-v2,test,3724,0.07336,0.0 +all-MiniLM-L6-v2,test,3735,0.0,0.0 +all-MiniLM-L6-v2,test,3759,0.0,0.0 +all-MiniLM-L6-v2,test,3767,0.0,0.0 +all-MiniLM-L6-v2,test,3771,0.49352,1.0 +all-MiniLM-L6-v2,test,3781,0.49115,1.0 +all-MiniLM-L6-v2,test,3789,0.63668,1.0 +all-MiniLM-L6-v2,test,3791,0.6257,1.0 +all-MiniLM-L6-v2,test,3801,0.0,0.0 +all-MiniLM-L6-v2,test,3822,0.15102,0.0 +all-MiniLM-L6-v2,test,3829,0.18458,0.0 +all-MiniLM-L6-v2,test,3830,1.0,1.0 +all-MiniLM-L6-v2,test,3837,0.63093,1.0 +all-MiniLM-L6-v2,test,3859,0.0,0.0 +all-MiniLM-L6-v2,test,3875,0.91972,1.0 +all-MiniLM-L6-v2,test,3888,0.0,0.0 +all-MiniLM-L6-v2,test,3909,0.33014,1.0 +all-MiniLM-L6-v2,test,3932,0.0,0.0 +all-MiniLM-L6-v2,test,3934,0.34621,1.0 +all-MiniLM-L6-v2,test,3995,0.0,0.0 +all-MiniLM-L6-v2,test,4007,0.46928,1.0 +all-MiniLM-L6-v2,test,4011,0.9134,1.0 +all-MiniLM-L6-v2,test,4019,0.0,0.0 +all-MiniLM-L6-v2,test,4031,0.0,0.0 +all-MiniLM-L6-v2,test,4037,0.61315,1.0 +all-MiniLM-L6-v2,test,4047,0.7537,1.0 +all-MiniLM-L6-v2,test,4071,0.0,0.0 +all-MiniLM-L6-v2,test,4084,0.87722,1.0 +all-MiniLM-L6-v2,test,4102,0.13906,0.0 +all-MiniLM-L6-v2,test,4103,0.0,0.0 +all-MiniLM-L6-v2,test,4105,0.20438,0.0 +all-MiniLM-L6-v2,test,4116,0.61315,1.0 +all-MiniLM-L6-v2,test,4125,0.0,0.0 +all-MiniLM-L6-v2,test,4142,0.52728,1.0 +all-MiniLM-L6-v2,test,4153,0.0,0.0 +all-MiniLM-L6-v2,test,4179,0.63093,1.0 +all-MiniLM-L6-v2,test,4188,0.0,0.0 +all-MiniLM-L6-v2,test,4205,0.13565,0.0 +all-MiniLM-L6-v2,test,4233,0.80621,1.0 +all-MiniLM-L6-v2,test,4265,0.3026,1.0 +all-MiniLM-L6-v2,test,4286,0.61315,1.0 +all-MiniLM-L6-v2,test,4289,0.79772,1.0 +all-MiniLM-L6-v2,test,4306,0.51353,1.0 +all-MiniLM-L6-v2,test,4312,0.44596,1.0 +all-MiniLM-L6-v2,test,4335,0.63668,1.0 +all-MiniLM-L6-v2,test,4339,1.0,1.0 +all-MiniLM-L6-v2,test,4394,0.0,0.0 +all-MiniLM-L6-v2,test,4409,0.52711,1.0 +all-MiniLM-L6-v2,test,4411,0.0,0.0 +all-MiniLM-L6-v2,test,4414,0.0,0.0 +all-MiniLM-L6-v2,test,4415,0.19519,1.0 +all-MiniLM-L6-v2,test,4433,0.0,0.0 +all-MiniLM-L6-v2,test,4447,0.0,0.0 +all-MiniLM-L6-v2,test,4464,0.63093,1.0 +all-MiniLM-L6-v2,test,4465,0.0,0.0 +all-MiniLM-L6-v2,test,4484,1.0,1.0 +all-MiniLM-L6-v2,test,4499,0.0,0.0 +all-MiniLM-L6-v2,test,4500,0.0,0.0 +all-MiniLM-L6-v2,test,4504,0.38685,1.0 +all-MiniLM-L6-v2,test,4514,0.39369,1.0 +all-MiniLM-L6-v2,test,4523,0.33916,1.0 +all-MiniLM-L6-v2,test,4539,0.21841,0.0 +all-MiniLM-L6-v2,test,4571,0.76536,1.0 +all-MiniLM-L6-v2,test,4600,0.43068,0.0 +all-MiniLM-L6-v2,test,4605,0.37852,1.0 +all-MiniLM-L6-v2,test,4615,0.80947,1.0 +all-MiniLM-L6-v2,test,4640,0.15643,0.0 +all-MiniLM-L6-v2,test,4641,0.61084,1.0 +all-MiniLM-L6-v2,test,4678,0.0,0.0 +all-MiniLM-L6-v2,test,4681,1.0,1.0 +all-MiniLM-L6-v2,test,4700,1.0,1.0 +all-MiniLM-L6-v2,test,4714,0.83187,1.0 +all-MiniLM-L6-v2,test,4756,0.35621,0.0 +all-MiniLM-L6-v2,test,4767,0.0,0.0 +all-MiniLM-L6-v2,test,4775,0.0,0.0 +all-MiniLM-L6-v2,test,4777,0.0,0.0 +all-MiniLM-L6-v2,test,4785,1.0,1.0 +all-MiniLM-L6-v2,test,4804,0.50127,0.0 +all-MiniLM-L6-v2,test,4813,1.0,1.0 +all-MiniLM-L6-v2,test,4823,0.46928,1.0 +all-MiniLM-L6-v2,test,4827,0.63093,1.0 +all-MiniLM-L6-v2,test,4837,0.0,0.0 +all-MiniLM-L6-v2,test,4844,0.61315,1.0 +all-MiniLM-L6-v2,test,4845,0.76536,1.0 +all-MiniLM-L6-v2,test,4846,0.0,0.0 +all-MiniLM-L6-v2,test,4863,0.29608,1.0 +all-MiniLM-L6-v2,test,4865,0.0,0.0 +all-MiniLM-L6-v2,test,4920,0.96747,1.0 +all-MiniLM-L6-v2,test,4942,0.0,0.0 +all-MiniLM-L6-v2,test,4946,0.5,1.0 +all-MiniLM-L6-v2,test,4955,0.0,0.0 +all-MiniLM-L6-v2,test,4962,0.0,0.0 +all-MiniLM-L6-v2,test,4968,0.35621,0.0 +all-MiniLM-L6-v2,test,4981,0.67139,1.0 +all-MiniLM-L6-v2,test,4999,0.39038,1.0 +all-MiniLM-L6-v2,test,5021,0.0,0.0 +all-MiniLM-L6-v2,test,5030,0.0,0.0 +all-MiniLM-L6-v2,test,5045,0.38685,0.0 +all-MiniLM-L6-v2,test,5054,0.0,0.0 +all-MiniLM-L6-v2,test,5061,0.0,0.0 +all-MiniLM-L6-v2,test,5064,0.61315,1.0 +all-MiniLM-L6-v2,test,5067,0.0,0.0 +all-MiniLM-L6-v2,test,5080,0.0,0.0 +all-MiniLM-L6-v2,test,5083,0.18458,0.0 +all-MiniLM-L6-v2,test,5085,0.31546,0.0 +all-MiniLM-L6-v2,test,5086,1.0,1.0 +all-MiniLM-L6-v2,test,5090,0.61315,1.0 +all-MiniLM-L6-v2,test,5125,1.0,1.0 +all-MiniLM-L6-v2,test,5134,0.2372,0.0 +all-MiniLM-L6-v2,test,5150,0.0,0.0 +all-MiniLM-L6-v2,test,5155,0.0,0.0 +all-MiniLM-L6-v2,test,5172,0.0,0.0 +all-MiniLM-L6-v2,test,5178,0.50874,1.0 +all-MiniLM-L6-v2,test,5185,0.0,0.0 +all-MiniLM-L6-v2,test,5196,0.0,0.0 +all-MiniLM-L6-v2,test,5206,0.0,0.0 +all-MiniLM-L6-v2,test,5228,0.0,0.0 +all-MiniLM-L6-v2,test,5231,0.0,0.0 +all-MiniLM-L6-v2,test,5241,0.14607,0.0 +all-MiniLM-L6-v2,test,5254,0.18458,0.0 +all-MiniLM-L6-v2,test,5255,1.0,1.0 +all-MiniLM-L6-v2,test,5264,0.61573,1.0 +all-MiniLM-L6-v2,test,5271,0.47762,1.0 +all-MiniLM-L6-v2,test,5331,1.0,1.0 +all-MiniLM-L6-v2,test,5343,0.38685,0.0 +all-MiniLM-L6-v2,test,5347,1.0,1.0 +all-MiniLM-L6-v2,test,5356,0.16958,1.0 +all-MiniLM-L6-v2,test,5369,0.0,0.0 +all-MiniLM-L6-v2,test,5374,0.0,0.0 +all-MiniLM-L6-v2,test,5380,1.0,1.0 +all-MiniLM-L6-v2,test,5402,0.0,0.0 +all-MiniLM-L6-v2,test,5410,0.29608,1.0 +all-MiniLM-L6-v2,test,5422,0.16813,0.0 +all-MiniLM-L6-v2,test,5427,0.0,0.0 +all-MiniLM-L6-v2,test,5460,0.0,0.0 +all-MiniLM-L6-v2,test,5464,0.15643,0.0 +all-MiniLM-L6-v2,test,5503,0.61315,1.0 +all-MiniLM-L6-v2,test,5505,0.63093,1.0 +all-MiniLM-L6-v2,test,5511,0.22009,1.0 +all-MiniLM-L6-v2,test,5534,0.61315,1.0 +all-MiniLM-L6-v2,test,5549,0.0,0.0 +all-MiniLM-L6-v2,test,5585,0.0,0.0 +all-MiniLM-L6-v2,test,5592,1.0,1.0 +all-MiniLM-L6-v2,test,5616,0.5,1.0 +all-MiniLM-L6-v2,test,5620,0.0,0.0 +all-MiniLM-L6-v2,test,5646,1.0,1.0 +all-MiniLM-L6-v2,test,5653,1.0,1.0 +all-MiniLM-L6-v2,test,5683,1.0,1.0 +all-MiniLM-L6-v2,test,5710,0.0,0.0 +all-MiniLM-L6-v2,test,5741,0.30657,1.0 +all-MiniLM-L6-v2,test,5763,0.34337,0.0 +all-MiniLM-L6-v2,test,5782,0.32098,1.0 +all-MiniLM-L6-v2,test,5790,0.0,0.0 +all-MiniLM-L6-v2,test,5808,0.0,0.0 +all-MiniLM-L6-v2,test,5853,0.21079,1.0 +all-MiniLM-L6-v2,test,5862,0.19519,1.0 +all-MiniLM-L6-v2,test,5888,0.19343,0.0 +all-MiniLM-L6-v2,test,5903,0.0,0.0 +all-MiniLM-L6-v2,test,5906,0.26407,0.0 +all-MiniLM-L6-v2,test,5940,0.0,0.0 +all-MiniLM-L6-v2,test,5951,0.33627,1.0 +all-MiniLM-L6-v2,test,5970,0.23464,1.0 +all-MiniLM-L6-v2,test,5981,0.98961,1.0 +all-MiniLM-L6-v2,test,5993,0.23297,0.0 +all-MiniLM-L6-v2,test,6002,0.79358,1.0 +all-MiniLM-L6-v2,test,6004,0.0,0.0 +all-MiniLM-L6-v2,test,6005,0.28952,1.0 +all-MiniLM-L6-v2,test,6009,0.35621,0.0 +all-MiniLM-L6-v2,test,6041,0.63668,1.0 +all-MiniLM-L6-v2,test,6080,0.61315,1.0 +all-MiniLM-L6-v2,test,6110,0.2439,1.0 +all-MiniLM-L6-v2,test,6121,0.18458,0.0 +all-MiniLM-L6-v2,test,6122,0.46928,1.0 +all-MiniLM-L6-v2,test,6131,0.29062,1.0 +all-MiniLM-L6-v2,test,6133,0.38685,1.0 +all-MiniLM-L6-v2,test,6142,0.61315,1.0 +all-MiniLM-L6-v2,test,6146,1.0,1.0 +all-MiniLM-L6-v2,test,6199,0.39038,1.0 +all-MiniLM-L6-v2,test,6219,1.0,1.0 +all-MiniLM-L6-v2,test,6221,0.0,0.0 +all-MiniLM-L6-v2,test,6252,0.0,0.0 +all-MiniLM-L6-v2,test,6262,0.47762,1.0 +all-MiniLM-L6-v2,test,6278,0.30657,1.0 +all-MiniLM-L6-v2,test,6395,0.0,0.0 +all-MiniLM-L6-v2,test,6410,0.0,0.0 +all-MiniLM-L6-v2,test,6420,1.0,1.0 +all-MiniLM-L6-v2,test,6441,0.31546,0.0 +all-MiniLM-L6-v2,test,6467,0.0,0.0 +all-MiniLM-L6-v2,test,6468,0.61315,1.0 +all-MiniLM-L6-v2,test,6479,1.0,1.0 +all-MiniLM-L6-v2,test,6525,0.46928,1.0 +all-MiniLM-L6-v2,test,6554,0.57143,1.0 +all-MiniLM-L6-v2,test,6562,0.0,0.0 +all-MiniLM-L6-v2,test,6611,1.0,1.0 +all-MiniLM-L6-v2,test,6612,0.0,0.0 +all-MiniLM-L6-v2,test,6625,0.5,1.0 +all-MiniLM-L6-v2,test,6629,0.0,0.0 +all-MiniLM-L6-v2,test,6635,0.39732,1.0 +all-MiniLM-L6-v2,test,6644,0.0,0.0 +all-MiniLM-L6-v2,test,6647,0.76536,1.0 +all-MiniLM-L6-v2,test,6668,0.38685,1.0 +all-MiniLM-L6-v2,test,6679,0.47762,1.0 +all-MiniLM-L6-v2,test,6683,0.0,0.0 +all-MiniLM-L6-v2,test,6713,0.61315,1.0 +all-MiniLM-L6-v2,test,6715,0.2372,0.0 +all-MiniLM-L6-v2,test,6746,0.61315,1.0 +all-MiniLM-L6-v2,test,6787,0.0,0.0 +all-MiniLM-L6-v2,test,6792,0.0,0.0 +all-MiniLM-L6-v2,test,6800,1.0,1.0 +all-MiniLM-L6-v2,test,6803,0.58557,1.0 +all-MiniLM-L6-v2,test,6807,1.0,1.0 +all-MiniLM-L6-v2,test,6814,0.0,0.0 +all-MiniLM-L6-v2,test,6832,0.0,0.0 +all-MiniLM-L6-v2,test,6835,0.61315,1.0 +all-MiniLM-L6-v2,test,6849,0.31546,0.0 +all-MiniLM-L6-v2,test,6862,1.0,1.0 +all-MiniLM-L6-v2,test,6867,0.22415,1.0 +all-MiniLM-L6-v2,test,6875,0.0,0.0 +all-MiniLM-L6-v2,test,6890,0.61315,1.0 +all-MiniLM-L6-v2,test,6891,0.14804,0.0 +all-MiniLM-L6-v2,test,6896,0.0,0.0 +all-MiniLM-L6-v2,test,6901,0.18458,0.0 +all-MiniLM-L6-v2,test,6907,0.0,0.0 +all-MiniLM-L6-v2,test,6909,0.0,0.0 +all-MiniLM-L6-v2,test,6959,0.61315,1.0 +all-MiniLM-L6-v2,test,6985,1.0,1.0 +all-MiniLM-L6-v2,test,7017,0.0,0.0 +all-MiniLM-L6-v2,test,7068,0.18458,0.0 +all-MiniLM-L6-v2,test,7071,0.0,0.0 +all-MiniLM-L6-v2,test,7080,1.0,1.0 +all-MiniLM-L6-v2,test,7096,0.0,0.0 +all-MiniLM-L6-v2,test,7098,0.83155,1.0 +all-MiniLM-L6-v2,test,7105,0.63093,1.0 +all-MiniLM-L6-v2,test,7109,0.38365,0.0 +all-MiniLM-L6-v2,test,7124,0.5414,1.0 +all-MiniLM-L6-v2,test,7141,0.0,0.0 +all-MiniLM-L6-v2,test,7145,0.15643,0.0 +all-MiniLM-L6-v2,test,7178,0.5,1.0 +all-MiniLM-L6-v2,test,7188,0.0,0.0 +all-MiniLM-L6-v2,test,7205,1.0,1.0 +all-MiniLM-L6-v2,test,7206,0.0,0.0 +all-MiniLM-L6-v2,test,7218,0.38685,0.0 +all-MiniLM-L6-v2,test,7221,0.0,0.0 +all-MiniLM-L6-v2,test,7269,0.35621,0.0 +all-MiniLM-L6-v2,test,7279,0.52944,1.0 +all-MiniLM-L6-v2,test,7295,0.0,0.0 +all-MiniLM-L6-v2,test,7311,0.0,0.0 +all-MiniLM-L6-v2,test,7326,0.0,0.0 +all-MiniLM-L6-v2,test,7329,0.61315,1.0 +all-MiniLM-L6-v2,test,7344,0.38685,1.0 +all-MiniLM-L6-v2,test,7345,0.23464,1.0 +all-MiniLM-L6-v2,test,7377,0.43068,0.0 +all-MiniLM-L6-v2,test,7431,0.0,0.0 +all-MiniLM-L6-v2,test,7441,0.29608,1.0 +all-MiniLM-L6-v2,test,7445,0.0,0.0 +all-MiniLM-L6-v2,test,7448,0.63093,1.0 +all-MiniLM-L6-v2,test,7456,0.0,0.0 +all-MiniLM-L6-v2,test,7463,0.33916,1.0 +all-MiniLM-L6-v2,test,7467,0.63093,1.0 +all-MiniLM-L6-v2,test,7484,1.0,1.0 +all-MiniLM-L6-v2,test,7509,0.0,0.0 +all-MiniLM-L6-v2,test,7512,0.0,0.0 +all-MiniLM-L6-v2,test,7513,0.0,0.0 +all-MiniLM-L6-v2,test,7529,0.17724,0.0 +all-MiniLM-L6-v2,test,7533,0.0,0.0 +all-MiniLM-L6-v2,test,7534,0.46928,1.0 +all-MiniLM-L6-v2,test,7590,0.65092,1.0 +all-MiniLM-L6-v2,test,7592,0.0,0.0 +all-MiniLM-L6-v2,test,7594,0.0,0.0 +all-MiniLM-L6-v2,test,7622,0.0,0.0 +all-MiniLM-L6-v2,test,7633,0.39038,1.0 +all-MiniLM-L6-v2,test,7674,0.14127,0.0 +all-MiniLM-L6-v2,test,7700,0.0,0.0 +all-MiniLM-L6-v2,test,7702,0.57064,1.0 +all-MiniLM-L6-v2,test,7705,0.46928,1.0 +all-MiniLM-L6-v2,test,7734,0.38685,1.0 +all-MiniLM-L6-v2,test,7747,0.0,0.0 +all-MiniLM-L6-v2,test,7754,0.38685,0.0 +all-MiniLM-L6-v2,test,7758,1.0,1.0 +all-MiniLM-L6-v2,test,7801,0.2372,0.0 +all-MiniLM-L6-v2,test,7803,0.0,0.0 +all-MiniLM-L6-v2,test,7823,0.23464,1.0 +all-MiniLM-L6-v2,test,7876,0.5,1.0 +all-MiniLM-L6-v2,test,7879,0.14127,0.0 +all-MiniLM-L6-v2,test,7880,0.0,0.0 +all-MiniLM-L6-v2,test,7911,1.0,1.0 +all-MiniLM-L6-v2,test,7925,0.10779,0.0 +all-MiniLM-L6-v2,test,7928,0.19519,1.0 +all-MiniLM-L6-v2,test,7936,0.38268,1.0 +all-MiniLM-L6-v2,test,7992,0.0,0.0 +all-MiniLM-L6-v2,test,8002,0.44412,1.0 +all-MiniLM-L6-v2,test,8005,0.0,0.0 +all-MiniLM-L6-v2,test,8013,0.0,0.0 +all-MiniLM-L6-v2,test,8017,0.43068,0.0 +all-MiniLM-L6-v2,test,8034,0.0,0.0 +all-MiniLM-L6-v2,test,8040,0.63093,1.0 +all-MiniLM-L6-v2,test,8072,0.29608,1.0 +all-MiniLM-L6-v2,test,8079,0.70392,1.0 +all-MiniLM-L6-v2,test,8102,0.16813,0.0 +all-MiniLM-L6-v2,test,8116,0.38685,1.0 +all-MiniLM-L6-v2,test,8121,1.0,1.0 +all-MiniLM-L6-v2,test,8202,0.0,0.0 +all-MiniLM-L6-v2,test,8230,0.0,0.0 +all-MiniLM-L6-v2,test,8247,0.16813,0.0 +all-MiniLM-L6-v2,test,8271,0.61315,1.0 +all-MiniLM-L6-v2,test,8275,0.0,0.0 +all-MiniLM-L6-v2,test,8296,1.0,1.0 +all-MiniLM-L6-v2,test,8332,0.80725,1.0 +all-MiniLM-L6-v2,test,8351,0.0,0.0 +all-MiniLM-L6-v2,test,8378,0.46928,1.0 +all-MiniLM-L6-v2,test,8456,0.0,0.0 +all-MiniLM-L6-v2,test,8475,0.78687,1.0 +all-MiniLM-L6-v2,test,8507,0.15643,0.0 +all-MiniLM-L6-v2,test,8512,1.0,1.0 +all-MiniLM-L6-v2,test,8513,0.0,0.0 +all-MiniLM-L6-v2,test,8532,0.14127,0.0 +all-MiniLM-L6-v2,test,8537,0.0,0.0 +all-MiniLM-L6-v2,test,8539,0.2463,1.0 +all-MiniLM-L6-v2,test,8544,1.0,1.0 +all-MiniLM-L6-v2,test,8592,0.34337,0.0 +all-MiniLM-L6-v2,test,8632,0.30657,1.0 +all-MiniLM-L6-v2,test,8635,0.0,0.0 +all-MiniLM-L6-v2,test,8702,0.38268,1.0 +all-MiniLM-L6-v2,test,8779,0.26407,0.0 +all-MiniLM-L6-v2,test,8789,0.76536,1.0 +all-MiniLM-L6-v2,test,8795,1.0,1.0 +all-MiniLM-L6-v2,test,8832,1.0,1.0 +all-MiniLM-L6-v2,test,8834,0.58423,1.0 +all-MiniLM-L6-v2,test,8855,0.0,0.0 +all-MiniLM-L6-v2,test,8874,0.0,0.0 +all-MiniLM-L6-v2,test,8934,0.0,0.0 +all-MiniLM-L6-v2,test,8937,0.0,0.0 +all-MiniLM-L6-v2,test,8947,0.61315,1.0 +all-MiniLM-L6-v2,test,8959,0.0,0.0 +all-MiniLM-L6-v2,test,8970,0.91972,1.0 +all-MiniLM-L6-v2,test,8974,0.50263,1.0 +all-MiniLM-L6-v2,test,8982,0.0,0.0 +all-MiniLM-L6-v2,test,9060,0.61315,1.0 +all-MiniLM-L6-v2,test,9088,0.0,0.0 +all-MiniLM-L6-v2,test,9108,0.0,0.0 +all-MiniLM-L6-v2,test,9115,0.36927,0.0 +all-MiniLM-L6-v2,test,9126,0.0,0.0 +all-MiniLM-L6-v2,test,9164,0.20211,0.0 +all-MiniLM-L6-v2,test,9174,0.16958,1.0 +all-MiniLM-L6-v2,test,9188,0.87722,1.0 +all-MiniLM-L6-v2,test,9245,0.0,0.0 +all-MiniLM-L6-v2,test,9275,0.17724,0.0 +all-MiniLM-L6-v2,test,9291,0.0,0.0 +all-MiniLM-L6-v2,test,9296,0.61315,1.0 +all-MiniLM-L6-v2,test,9329,0.0,0.0 +all-MiniLM-L6-v2,test,9332,0.0,0.0 +all-MiniLM-L6-v2,test,9381,0.0,0.0 +all-MiniLM-L6-v2,test,9385,0.0,0.0 +all-MiniLM-L6-v2,test,9391,0.44149,1.0 +all-MiniLM-L6-v2,test,9403,0.0,0.0 +all-MiniLM-L6-v2,test,9481,0.54377,1.0 +all-MiniLM-L6-v2,test,9487,0.33333,0.0 +all-MiniLM-L6-v2,test,9548,0.0,0.0 +all-MiniLM-L6-v2,test,9556,0.0,0.0 +all-MiniLM-L6-v2,test,9565,0.80657,1.0 +all-MiniLM-L6-v2,test,9598,0.0,0.0 +all-MiniLM-L6-v2,test,9617,0.46928,1.0 +all-MiniLM-L6-v2,test,9633,0.26407,0.0 +all-MiniLM-L6-v2,test,9643,0.0,0.0 +all-MiniLM-L6-v2,test,9644,1.0,1.0 +all-MiniLM-L6-v2,test,9646,0.69343,1.0 +all-MiniLM-L6-v2,test,9668,0.0,0.0 +all-MiniLM-L6-v2,test,9701,0.0,0.0 +all-MiniLM-L6-v2,test,9733,0.0,0.0 +all-MiniLM-L6-v2,test,9735,0.0,0.0 +all-MiniLM-L6-v2,test,9737,0.43675,1.0 +all-MiniLM-L6-v2,test,9771,0.0,0.0 +all-MiniLM-L6-v2,test,9808,0.0,0.0 +all-MiniLM-L6-v2,test,9824,1.0,1.0 +all-MiniLM-L6-v2,test,9871,0.0,0.0 +all-MiniLM-L6-v2,test,9882,0.0,0.0 +all-MiniLM-L6-v2,test,9925,0.38685,1.0 +all-MiniLM-L6-v2,test,9929,0.69343,1.0 +all-MiniLM-L6-v2,test,9961,0.0,0.0 +all-MiniLM-L6-v2,test,9979,0.52051,1.0 +all-MiniLM-L6-v2,test,10034,0.17724,0.0 +all-MiniLM-L6-v2,test,10039,0.63093,1.0 +all-MiniLM-L6-v2,test,10109,0.51865,1.0 +all-MiniLM-L6-v2,test,10122,0.12315,0.0 +all-MiniLM-L6-v2,test,10136,0.23464,1.0 +all-MiniLM-L6-v2,test,10137,0.5,1.0 +all-MiniLM-L6-v2,test,10152,1.0,1.0 +all-MiniLM-L6-v2,test,10183,0.0,0.0 +all-MiniLM-L6-v2,test,10213,0.0,0.0 +all-MiniLM-L6-v2,test,10246,0.0,0.0 +all-MiniLM-L6-v2,test,10267,0.14607,0.0 +all-MiniLM-L6-v2,test,10414,1.0,1.0 +all-MiniLM-L6-v2,test,10447,0.23464,1.0 +all-MiniLM-L6-v2,test,10462,0.0,0.0 +all-MiniLM-L6-v2,test,10482,0.0,0.0 +all-MiniLM-L6-v2,test,10497,0.77524,1.0 +all-MiniLM-L6-v2,test,10526,0.21841,0.0 +all-MiniLM-L6-v2,test,10547,0.0,0.0 +all-MiniLM-L6-v2,test,10558,0.0,0.0 +all-MiniLM-L6-v2,test,10596,0.68435,1.0 +all-MiniLM-L6-v2,test,10601,0.0,0.0 +all-MiniLM-L6-v2,test,10628,0.38685,0.0 +all-MiniLM-L6-v2,test,10639,0.0,0.0 +all-MiniLM-L6-v2,test,10645,0.41183,0.0 +all-MiniLM-L6-v2,test,10674,0.7157,1.0 +all-MiniLM-L6-v2,test,10710,0.63093,1.0 +all-MiniLM-L6-v2,test,10734,0.57097,1.0 +all-MiniLM-L6-v2,test,10792,1.0,1.0 +all-MiniLM-L6-v2,test,10808,0.67876,1.0 +all-MiniLM-L6-v2,test,10809,0.35015,0.0 +all-MiniLM-L6-v2,test,10812,0.58557,1.0 +all-MiniLM-L6-v2,test,10827,0.0,0.0 +all-MiniLM-L6-v2,test,10845,1.0,1.0 +all-MiniLM-L6-v2,test,10912,0.0,0.0 +all-MiniLM-L6-v2,test,10932,1.0,1.0 +all-MiniLM-L6-v2,test,10975,0.0,0.0 +all-MiniLM-L6-v2,test,10979,0.55851,1.0 +all-MiniLM-L6-v2,test,10994,0.0,0.0 +all-MiniLM-L6-v2,test,11039,0.395,1.0 +all-MiniLM-L6-v2,test,11054,1.0,1.0 +all-MiniLM-L6-v2,test,11088,0.63093,1.0 +bge-small-en-v1.5,test,8,0.61315,1.0 +bge-small-en-v1.5,test,15,1.0,1.0 +bge-small-en-v1.5,test,18,1.0,1.0 +bge-small-en-v1.5,test,26,0.21841,0.0 +bge-small-en-v1.5,test,34,0.0,0.0 +bge-small-en-v1.5,test,42,0.0,0.0 +bge-small-en-v1.5,test,56,1.0,1.0 +bge-small-en-v1.5,test,68,0.0,0.0 +bge-small-en-v1.5,test,89,0.11706,0.0 +bge-small-en-v1.5,test,90,0.0,0.0 +bge-small-en-v1.5,test,94,0.63093,1.0 +bge-small-en-v1.5,test,98,0.61315,1.0 +bge-small-en-v1.5,test,104,0.61315,1.0 +bge-small-en-v1.5,test,106,0.63093,1.0 +bge-small-en-v1.5,test,109,1.0,1.0 +bge-small-en-v1.5,test,475,1.0,1.0 +bge-small-en-v1.5,test,503,0.0,0.0 +bge-small-en-v1.5,test,504,0.0,0.0 +bge-small-en-v1.5,test,515,1.0,1.0 +bge-small-en-v1.5,test,529,0.0,0.0 +bge-small-en-v1.5,test,547,0.0,0.0 +bge-small-en-v1.5,test,549,1.0,1.0 +bge-small-en-v1.5,test,559,0.0,0.0 +bge-small-en-v1.5,test,570,0.0,0.0 +bge-small-en-v1.5,test,585,0.38685,1.0 +bge-small-en-v1.5,test,588,0.30657,1.0 +bge-small-en-v1.5,test,594,0.61315,1.0 +bge-small-en-v1.5,test,603,1.0,1.0 +bge-small-en-v1.5,test,604,0.63405,1.0 +bge-small-en-v1.5,test,620,0.10699,0.0 +bge-small-en-v1.5,test,622,0.0,0.0 +bge-small-en-v1.5,test,659,0.57369,1.0 +bge-small-en-v1.5,test,672,0.0,0.0 +bge-small-en-v1.5,test,684,0.0,0.0 +bge-small-en-v1.5,test,687,0.0,0.0 +bge-small-en-v1.5,test,689,0.0,0.0 +bge-small-en-v1.5,test,691,0.0,0.0 +bge-small-en-v1.5,test,699,0.63093,1.0 +bge-small-en-v1.5,test,701,0.0,0.0 +bge-small-en-v1.5,test,715,0.0,0.0 +bge-small-en-v1.5,test,721,1.0,1.0 +bge-small-en-v1.5,test,744,0.0,0.0 +bge-small-en-v1.5,test,750,0.81753,1.0 +bge-small-en-v1.5,test,753,0.17724,0.0 +bge-small-en-v1.5,test,766,0.14804,0.0 +bge-small-en-v1.5,test,776,0.38113,1.0 +bge-small-en-v1.5,test,810,0.0,0.0 +bge-small-en-v1.5,test,813,0.0,0.0 +bge-small-en-v1.5,test,849,0.0,0.0 +bge-small-en-v1.5,test,852,0.90603,1.0 +bge-small-en-v1.5,test,853,0.0,0.0 +bge-small-en-v1.5,test,858,0.0,0.0 +bge-small-en-v1.5,test,859,0.0,0.0 +bge-small-en-v1.5,test,864,0.58961,1.0 +bge-small-en-v1.5,test,879,1.0,1.0 +bge-small-en-v1.5,test,885,0.0,0.0 +bge-small-en-v1.5,test,895,1.0,1.0 +bge-small-en-v1.5,test,904,1.0,1.0 +bge-small-en-v1.5,test,928,0.61315,1.0 +bge-small-en-v1.5,test,929,1.0,1.0 +bge-small-en-v1.5,test,932,0.26407,0.0 +bge-small-en-v1.5,test,939,0.63093,1.0 +bge-small-en-v1.5,test,945,0.35621,0.0 +bge-small-en-v1.5,test,957,0.63644,1.0 +bge-small-en-v1.5,test,988,0.0,0.0 +bge-small-en-v1.5,test,1074,1.0,1.0 +bge-small-en-v1.5,test,1085,0.0,0.0 +bge-small-en-v1.5,test,1090,0.0,0.0 +bge-small-en-v1.5,test,1150,0.0,0.0 +bge-small-en-v1.5,test,1157,0.38685,1.0 +bge-small-en-v1.5,test,1159,0.0,0.0 +bge-small-en-v1.5,test,1198,1.0,1.0 +bge-small-en-v1.5,test,1230,0.5,1.0 +bge-small-en-v1.5,test,1281,1.0,1.0 +bge-small-en-v1.5,test,1284,0.63093,1.0 +bge-small-en-v1.5,test,1297,0.63093,1.0 +bge-small-en-v1.5,test,1306,0.0,0.0 +bge-small-en-v1.5,test,1309,0.7877,1.0 +bge-small-en-v1.5,test,1310,0.31546,0.0 +bge-small-en-v1.5,test,1321,0.0,0.0 +bge-small-en-v1.5,test,1322,0.39038,1.0 +bge-small-en-v1.5,test,1391,0.14127,0.0 +bge-small-en-v1.5,test,1393,0.49352,1.0 +bge-small-en-v1.5,test,1415,0.97093,1.0 +bge-small-en-v1.5,test,1416,0.0,0.0 +bge-small-en-v1.5,test,1441,1.0,1.0 +bge-small-en-v1.5,test,1451,1.0,1.0 +bge-small-en-v1.5,test,1469,0.63093,1.0 +bge-small-en-v1.5,test,1530,0.67199,1.0 +bge-small-en-v1.5,test,1670,0.76536,1.0 +bge-small-en-v1.5,test,1676,0.65424,1.0 +bge-small-en-v1.5,test,1736,0.21399,1.0 +bge-small-en-v1.5,test,1748,0.67139,1.0 +bge-small-en-v1.5,test,1783,0.0,0.0 +bge-small-en-v1.5,test,1812,0.19343,0.0 +bge-small-en-v1.5,test,1815,1.0,1.0 +bge-small-en-v1.5,test,1819,0.24239,1.0 +bge-small-en-v1.5,test,1824,0.63093,1.0 +bge-small-en-v1.5,test,1826,0.94939,1.0 +bge-small-en-v1.5,test,1832,1.0,1.0 +bge-small-en-v1.5,test,1871,0.69756,1.0 +bge-small-en-v1.5,test,1877,1.0,1.0 +bge-small-en-v1.5,test,1889,0.0,0.0 +bge-small-en-v1.5,test,1915,0.69343,1.0 +bge-small-en-v1.5,test,1920,0.43068,0.0 +bge-small-en-v1.5,test,1933,1.0,1.0 +bge-small-en-v1.5,test,1948,0.5,1.0 +bge-small-en-v1.5,test,1994,0.21885,0.0 +bge-small-en-v1.5,test,2010,0.81265,1.0 +bge-small-en-v1.5,test,2051,0.0,0.0 +bge-small-en-v1.5,test,2070,0.0,0.0 +bge-small-en-v1.5,test,2075,0.52204,1.0 +bge-small-en-v1.5,test,2076,1.0,1.0 +bge-small-en-v1.5,test,2088,0.0,0.0 +bge-small-en-v1.5,test,2108,0.13565,0.0 +bge-small-en-v1.5,test,2118,0.0,0.0 +bge-small-en-v1.5,test,2154,0.0,0.0 +bge-small-en-v1.5,test,2181,0.0,0.0 +bge-small-en-v1.5,test,2183,0.73592,1.0 +bge-small-en-v1.5,test,2204,0.9099,1.0 +bge-small-en-v1.5,test,2264,0.61315,1.0 +bge-small-en-v1.5,test,2296,0.31339,1.0 +bge-small-en-v1.5,test,2306,0.61315,1.0 +bge-small-en-v1.5,test,2316,1.0,1.0 +bge-small-en-v1.5,test,2318,0.0,0.0 +bge-small-en-v1.5,test,2330,0.91972,1.0 +bge-small-en-v1.5,test,2334,0.70392,1.0 +bge-small-en-v1.5,test,2348,0.38431,1.0 +bge-small-en-v1.5,test,2376,0.81783,1.0 +bge-small-en-v1.5,test,2383,0.61315,1.0 +bge-small-en-v1.5,test,2384,0.5,1.0 +bge-small-en-v1.5,test,2385,0.61315,1.0 +bge-small-en-v1.5,test,2388,1.0,1.0 +bge-small-en-v1.5,test,2395,0.30103,0.0 +bge-small-en-v1.5,test,2398,0.0,0.0 +bge-small-en-v1.5,test,2399,1.0,1.0 +bge-small-en-v1.5,test,2400,0.29608,1.0 +bge-small-en-v1.5,test,2407,0.0,0.0 +bge-small-en-v1.5,test,2416,0.70953,1.0 +bge-small-en-v1.5,test,2423,0.27976,1.0 +bge-small-en-v1.5,test,2443,0.0,0.0 +bge-small-en-v1.5,test,2445,1.0,1.0 +bge-small-en-v1.5,test,2460,1.0,1.0 +bge-small-en-v1.5,test,2465,0.0,0.0 +bge-small-en-v1.5,test,2472,0.0,0.0 +bge-small-en-v1.5,test,2486,1.0,1.0 +bge-small-en-v1.5,test,2498,0.0,0.0 +bge-small-en-v1.5,test,2513,0.0,0.0 +bge-small-en-v1.5,test,2516,0.63668,1.0 +bge-small-en-v1.5,test,2549,0.0,0.0 +bge-small-en-v1.5,test,2551,1.0,1.0 +bge-small-en-v1.5,test,2568,0.51865,1.0 +bge-small-en-v1.5,test,2579,0.29871,1.0 +bge-small-en-v1.5,test,2580,0.46928,1.0 +bge-small-en-v1.5,test,2587,1.0,1.0 +bge-small-en-v1.5,test,2589,0.9469,1.0 +bge-small-en-v1.5,test,2590,0.46928,1.0 +bge-small-en-v1.5,test,2593,0.29608,1.0 +bge-small-en-v1.5,test,2598,0.38685,1.0 +bge-small-en-v1.5,test,2648,1.0,1.0 +bge-small-en-v1.5,test,2676,0.63093,1.0 +bge-small-en-v1.5,test,2685,0.25294,1.0 +bge-small-en-v1.5,test,2695,1.0,1.0 +bge-small-en-v1.5,test,2713,0.0,0.0 +bge-small-en-v1.5,test,2724,0.20438,0.0 +bge-small-en-v1.5,test,2737,1.0,1.0 +bge-small-en-v1.5,test,2747,0.5,1.0 +bge-small-en-v1.5,test,2749,1.0,1.0 +bge-small-en-v1.5,test,2790,0.35762,1.0 +bge-small-en-v1.5,test,2801,0.61315,1.0 +bge-small-en-v1.5,test,2856,0.54456,1.0 +bge-small-en-v1.5,test,2857,0.58557,1.0 +bge-small-en-v1.5,test,2880,1.0,1.0 +bge-small-en-v1.5,test,2885,0.0,0.0 +bge-small-en-v1.5,test,2891,1.0,1.0 +bge-small-en-v1.5,test,2895,0.0,0.0 +bge-small-en-v1.5,test,2903,0.0,0.0 +bge-small-en-v1.5,test,2923,0.85393,1.0 +bge-small-en-v1.5,test,2964,0.0,0.0 +bge-small-en-v1.5,test,2968,1.0,1.0 +bge-small-en-v1.5,test,2994,0.15643,0.0 +bge-small-en-v1.5,test,3006,0.3026,1.0 +bge-small-en-v1.5,test,3008,0.51353,1.0 +bge-small-en-v1.5,test,3014,0.61315,1.0 +bge-small-en-v1.5,test,3033,0.61315,1.0 +bge-small-en-v1.5,test,3039,0.35621,0.0 +bge-small-en-v1.5,test,3049,0.63668,1.0 +bge-small-en-v1.5,test,3051,1.0,1.0 +bge-small-en-v1.5,test,3067,0.0,0.0 +bge-small-en-v1.5,test,3085,0.60526,1.0 +bge-small-en-v1.5,test,3091,0.98038,1.0 +bge-small-en-v1.5,test,3103,1.0,1.0 +bge-small-en-v1.5,test,3115,0.4539,1.0 +bge-small-en-v1.5,test,3125,1.0,1.0 +bge-small-en-v1.5,test,3148,0.55315,1.0 +bge-small-en-v1.5,test,3149,0.19343,0.0 +bge-small-en-v1.5,test,3177,0.0,0.0 +bge-small-en-v1.5,test,3179,0.61315,1.0 +bge-small-en-v1.5,test,3186,0.91972,1.0 +bge-small-en-v1.5,test,3189,0.0,0.0 +bge-small-en-v1.5,test,3254,0.46928,1.0 +bge-small-en-v1.5,test,3264,0.63668,1.0 +bge-small-en-v1.5,test,3357,0.5,1.0 +bge-small-en-v1.5,test,3369,0.0,0.0 +bge-small-en-v1.5,test,3394,0.63668,1.0 +bge-small-en-v1.5,test,3404,0.13744,1.0 +bge-small-en-v1.5,test,3405,0.30103,0.0 +bge-small-en-v1.5,test,3446,0.55315,1.0 +bge-small-en-v1.5,test,3451,0.27488,1.0 +bge-small-en-v1.5,test,3453,0.42279,0.0 +bge-small-en-v1.5,test,3480,1.0,1.0 +bge-small-en-v1.5,test,3490,0.0,0.0 +bge-small-en-v1.5,test,3500,0.28638,1.0 +bge-small-en-v1.5,test,3503,0.6223,1.0 +bge-small-en-v1.5,test,3512,1.0,1.0 +bge-small-en-v1.5,test,3528,0.30103,0.0 +bge-small-en-v1.5,test,3530,0.46928,1.0 +bge-small-en-v1.5,test,3534,0.0,0.0 +bge-small-en-v1.5,test,3566,0.20211,0.0 +bge-small-en-v1.5,test,3569,0.0,0.0 +bge-small-en-v1.5,test,3594,0.7877,1.0 +bge-small-en-v1.5,test,3612,0.0,0.0 +bge-small-en-v1.5,test,3615,0.69343,1.0 +bge-small-en-v1.5,test,3625,0.29608,1.0 +bge-small-en-v1.5,test,3682,0.0,0.0 +bge-small-en-v1.5,test,3683,0.52449,1.0 +bge-small-en-v1.5,test,3694,0.61315,1.0 +bge-small-en-v1.5,test,3724,0.13962,0.0 +bge-small-en-v1.5,test,3735,0.29608,1.0 +bge-small-en-v1.5,test,3759,0.0,0.0 +bge-small-en-v1.5,test,3767,0.11305,0.0 +bge-small-en-v1.5,test,3771,0.49352,1.0 +bge-small-en-v1.5,test,3781,0.91972,1.0 +bge-small-en-v1.5,test,3789,0.76681,1.0 +bge-small-en-v1.5,test,3791,0.53072,1.0 +bge-small-en-v1.5,test,3801,0.61315,1.0 +bge-small-en-v1.5,test,3822,0.2463,1.0 +bge-small-en-v1.5,test,3829,0.26407,0.0 +bge-small-en-v1.5,test,3830,0.28906,0.0 +bge-small-en-v1.5,test,3837,1.0,1.0 +bge-small-en-v1.5,test,3859,0.0,0.0 +bge-small-en-v1.5,test,3875,1.0,1.0 +bge-small-en-v1.5,test,3888,0.14127,0.0 +bge-small-en-v1.5,test,3909,0.13886,1.0 +bge-small-en-v1.5,test,3932,0.65092,1.0 +bge-small-en-v1.5,test,3934,0.0,0.0 +bge-small-en-v1.5,test,3995,0.12081,0.0 +bge-small-en-v1.5,test,4007,0.67139,1.0 +bge-small-en-v1.5,test,4011,1.0,1.0 +bge-small-en-v1.5,test,4019,0.16958,1.0 +bge-small-en-v1.5,test,4031,0.0,0.0 +bge-small-en-v1.5,test,4037,0.0,0.0 +bge-small-en-v1.5,test,4047,0.90055,1.0 +bge-small-en-v1.5,test,4071,0.0,0.0 +bge-small-en-v1.5,test,4084,0.43062,0.0 +bge-small-en-v1.5,test,4102,0.13906,0.0 +bge-small-en-v1.5,test,4103,0.0,0.0 +bge-small-en-v1.5,test,4105,0.0,0.0 +bge-small-en-v1.5,test,4116,0.91972,1.0 +bge-small-en-v1.5,test,4125,0.0,0.0 +bge-small-en-v1.5,test,4142,0.30718,0.0 +bge-small-en-v1.5,test,4153,0.0,0.0 +bge-small-en-v1.5,test,4179,1.0,1.0 +bge-small-en-v1.5,test,4188,0.28906,0.0 +bge-small-en-v1.5,test,4205,0.53072,1.0 +bge-small-en-v1.5,test,4233,0.69921,1.0 +bge-small-en-v1.5,test,4265,0.59439,1.0 +bge-small-en-v1.5,test,4286,0.61315,1.0 +bge-small-en-v1.5,test,4289,0.38685,1.0 +bge-small-en-v1.5,test,4306,0.52051,1.0 +bge-small-en-v1.5,test,4312,0.5307,1.0 +bge-small-en-v1.5,test,4335,0.7542,1.0 +bge-small-en-v1.5,test,4339,1.0,1.0 +bge-small-en-v1.5,test,4394,0.0,0.0 +bge-small-en-v1.5,test,4409,0.52,1.0 +bge-small-en-v1.5,test,4411,0.13565,0.0 +bge-small-en-v1.5,test,4414,0.38685,1.0 +bge-small-en-v1.5,test,4415,0.0,0.0 +bge-small-en-v1.5,test,4433,0.0,0.0 +bge-small-en-v1.5,test,4447,0.43068,0.0 +bge-small-en-v1.5,test,4464,0.28906,0.0 +bge-small-en-v1.5,test,4465,0.0,0.0 +bge-small-en-v1.5,test,4484,1.0,1.0 +bge-small-en-v1.5,test,4499,0.0,0.0 +bge-small-en-v1.5,test,4500,0.0,0.0 +bge-small-en-v1.5,test,4504,1.0,1.0 +bge-small-en-v1.5,test,4514,0.41039,1.0 +bge-small-en-v1.5,test,4523,0.32704,1.0 +bge-small-en-v1.5,test,4539,0.18458,0.0 +bge-small-en-v1.5,test,4571,0.76536,1.0 +bge-small-en-v1.5,test,4600,1.0,1.0 +bge-small-en-v1.5,test,4605,0.61656,1.0 +bge-small-en-v1.5,test,4615,0.74028,1.0 +bge-small-en-v1.5,test,4640,0.0,0.0 +bge-small-en-v1.5,test,4641,0.62955,1.0 +bge-small-en-v1.5,test,4678,0.0,0.0 +bge-small-en-v1.5,test,4681,1.0,1.0 +bge-small-en-v1.5,test,4700,1.0,1.0 +bge-small-en-v1.5,test,4714,0.63668,1.0 +bge-small-en-v1.5,test,4756,0.0,0.0 +bge-small-en-v1.5,test,4767,0.0,0.0 +bge-small-en-v1.5,test,4775,0.0,0.0 +bge-small-en-v1.5,test,4777,0.0,0.0 +bge-small-en-v1.5,test,4785,1.0,1.0 +bge-small-en-v1.5,test,4804,0.5,1.0 +bge-small-en-v1.5,test,4813,0.63093,1.0 +bge-small-en-v1.5,test,4823,0.46928,1.0 +bge-small-en-v1.5,test,4827,0.63093,1.0 +bge-small-en-v1.5,test,4837,0.0,0.0 +bge-small-en-v1.5,test,4844,0.61315,1.0 +bge-small-en-v1.5,test,4845,0.76536,1.0 +bge-small-en-v1.5,test,4846,0.0,0.0 +bge-small-en-v1.5,test,4863,0.63644,1.0 +bge-small-en-v1.5,test,4865,0.0,0.0 +bge-small-en-v1.5,test,4920,1.0,1.0 +bge-small-en-v1.5,test,4942,0.0,0.0 +bge-small-en-v1.5,test,4946,1.0,1.0 +bge-small-en-v1.5,test,4955,0.0,0.0 +bge-small-en-v1.5,test,4962,0.0,0.0 +bge-small-en-v1.5,test,4968,0.0,0.0 +bge-small-en-v1.5,test,4981,0.76536,1.0 +bge-small-en-v1.5,test,4999,0.69756,1.0 +bge-small-en-v1.5,test,5021,0.0,0.0 +bge-small-en-v1.5,test,5030,0.0,0.0 +bge-small-en-v1.5,test,5045,0.31546,0.0 +bge-small-en-v1.5,test,5054,0.0,0.0 +bge-small-en-v1.5,test,5061,0.0,0.0 +bge-small-en-v1.5,test,5064,0.87722,1.0 +bge-small-en-v1.5,test,5067,0.0,0.0 +bge-small-en-v1.5,test,5080,0.0,0.0 +bge-small-en-v1.5,test,5083,0.38685,1.0 +bge-small-en-v1.5,test,5085,1.0,1.0 +bge-small-en-v1.5,test,5086,1.0,1.0 +bge-small-en-v1.5,test,5090,0.61315,1.0 +bge-small-en-v1.5,test,5125,0.0,0.0 +bge-small-en-v1.5,test,5134,0.2372,0.0 +bge-small-en-v1.5,test,5150,0.0,0.0 +bge-small-en-v1.5,test,5155,0.0,0.0 +bge-small-en-v1.5,test,5172,0.0,0.0 +bge-small-en-v1.5,test,5178,0.72273,1.0 +bge-small-en-v1.5,test,5185,0.0,0.0 +bge-small-en-v1.5,test,5196,0.0,0.0 +bge-small-en-v1.5,test,5206,0.11706,0.0 +bge-small-en-v1.5,test,5228,0.0,0.0 +bge-small-en-v1.5,test,5231,0.0,0.0 +bge-small-en-v1.5,test,5241,0.45221,1.0 +bge-small-en-v1.5,test,5254,0.2372,0.0 +bge-small-en-v1.5,test,5255,1.0,1.0 +bge-small-en-v1.5,test,5264,0.69921,1.0 +bge-small-en-v1.5,test,5271,0.20211,0.0 +bge-small-en-v1.5,test,5331,0.63093,1.0 +bge-small-en-v1.5,test,5343,0.38685,0.0 +bge-small-en-v1.5,test,5347,1.0,1.0 +bge-small-en-v1.5,test,5356,0.31203,1.0 +bge-small-en-v1.5,test,5369,0.0,0.0 +bge-small-en-v1.5,test,5374,0.0,0.0 +bge-small-en-v1.5,test,5380,1.0,1.0 +bge-small-en-v1.5,test,5402,0.0,0.0 +bge-small-en-v1.5,test,5410,0.46928,1.0 +bge-small-en-v1.5,test,5422,0.19519,1.0 +bge-small-en-v1.5,test,5427,0.38685,1.0 +bge-small-en-v1.5,test,5460,0.0,0.0 +bge-small-en-v1.5,test,5464,0.0,0.0 +bge-small-en-v1.5,test,5503,0.61315,1.0 +bge-small-en-v1.5,test,5505,1.0,1.0 +bge-small-en-v1.5,test,5511,0.11005,1.0 +bge-small-en-v1.5,test,5534,0.38685,1.0 +bge-small-en-v1.5,test,5549,0.0,0.0 +bge-small-en-v1.5,test,5585,0.30103,0.0 +bge-small-en-v1.5,test,5592,1.0,1.0 +bge-small-en-v1.5,test,5616,0.30103,0.0 +bge-small-en-v1.5,test,5620,0.0,0.0 +bge-small-en-v1.5,test,5646,1.0,1.0 +bge-small-en-v1.5,test,5653,1.0,1.0 +bge-small-en-v1.5,test,5683,1.0,1.0 +bge-small-en-v1.5,test,5710,0.0,0.0 +bge-small-en-v1.5,test,5741,0.61315,1.0 +bge-small-en-v1.5,test,5763,0.29608,1.0 +bge-small-en-v1.5,test,5782,0.33916,1.0 +bge-small-en-v1.5,test,5790,0.0,0.0 +bge-small-en-v1.5,test,5808,0.38685,0.0 +bge-small-en-v1.5,test,5853,0.23271,1.0 +bge-small-en-v1.5,test,5862,0.16813,0.0 +bge-small-en-v1.5,test,5888,0.17724,0.0 +bge-small-en-v1.5,test,5903,0.0,0.0 +bge-small-en-v1.5,test,5906,0.0,0.0 +bge-small-en-v1.5,test,5940,0.0,0.0 +bge-small-en-v1.5,test,5951,0.35257,1.0 +bge-small-en-v1.5,test,5970,0.43173,1.0 +bge-small-en-v1.5,test,5981,1.0,1.0 +bge-small-en-v1.5,test,5993,0.4035,1.0 +bge-small-en-v1.5,test,6002,0.81659,1.0 +bge-small-en-v1.5,test,6004,0.0,0.0 +bge-small-en-v1.5,test,6005,0.45374,1.0 +bge-small-en-v1.5,test,6009,0.0,0.0 +bge-small-en-v1.5,test,6041,0.74953,1.0 +bge-small-en-v1.5,test,6080,0.61315,1.0 +bge-small-en-v1.5,test,6110,0.36188,1.0 +bge-small-en-v1.5,test,6121,0.81753,1.0 +bge-small-en-v1.5,test,6122,0.46928,1.0 +bge-small-en-v1.5,test,6131,0.41528,1.0 +bge-small-en-v1.5,test,6133,0.2372,0.0 +bge-small-en-v1.5,test,6142,0.61315,1.0 +bge-small-en-v1.5,test,6146,1.0,1.0 +bge-small-en-v1.5,test,6199,0.63668,1.0 +bge-small-en-v1.5,test,6219,0.87722,1.0 +bge-small-en-v1.5,test,6221,0.0,0.0 +bge-small-en-v1.5,test,6252,0.0,0.0 +bge-small-en-v1.5,test,6262,0.76536,1.0 +bge-small-en-v1.5,test,6278,0.69343,1.0 +bge-small-en-v1.5,test,6395,0.0,0.0 +bge-small-en-v1.5,test,6410,0.0,0.0 +bge-small-en-v1.5,test,6420,0.91972,1.0 +bge-small-en-v1.5,test,6441,0.28906,0.0 +bge-small-en-v1.5,test,6467,0.0,0.0 +bge-small-en-v1.5,test,6468,0.61315,1.0 +bge-small-en-v1.5,test,6479,1.0,1.0 +bge-small-en-v1.5,test,6525,0.29608,1.0 +bge-small-en-v1.5,test,6554,0.61315,1.0 +bge-small-en-v1.5,test,6562,0.39038,1.0 +bge-small-en-v1.5,test,6611,1.0,1.0 +bge-small-en-v1.5,test,6612,0.16716,0.0 +bge-small-en-v1.5,test,6625,1.0,1.0 +bge-small-en-v1.5,test,6629,0.0,0.0 +bge-small-en-v1.5,test,6635,0.52944,1.0 +bge-small-en-v1.5,test,6644,0.0,0.0 +bge-small-en-v1.5,test,6647,0.76536,1.0 +bge-small-en-v1.5,test,6668,0.20438,0.0 +bge-small-en-v1.5,test,6679,0.53072,1.0 +bge-small-en-v1.5,test,6683,0.0,0.0 +bge-small-en-v1.5,test,6713,0.61315,1.0 +bge-small-en-v1.5,test,6715,0.41444,0.0 +bge-small-en-v1.5,test,6746,0.38685,1.0 +bge-small-en-v1.5,test,6787,0.0,0.0 +bge-small-en-v1.5,test,6792,0.0,0.0 +bge-small-en-v1.5,test,6800,1.0,1.0 +bge-small-en-v1.5,test,6803,0.67602,1.0 +bge-small-en-v1.5,test,6807,1.0,1.0 +bge-small-en-v1.5,test,6814,0.0,0.0 +bge-small-en-v1.5,test,6832,0.19343,0.0 +bge-small-en-v1.5,test,6835,0.38685,1.0 +bge-small-en-v1.5,test,6849,0.0,0.0 +bge-small-en-v1.5,test,6862,1.0,1.0 +bge-small-en-v1.5,test,6867,0.55464,1.0 +bge-small-en-v1.5,test,6875,0.0,0.0 +bge-small-en-v1.5,test,6890,0.61315,1.0 +bge-small-en-v1.5,test,6891,0.46928,1.0 +bge-small-en-v1.5,test,6896,0.0,0.0 +bge-small-en-v1.5,test,6901,0.0,0.0 +bge-small-en-v1.5,test,6907,0.0,0.0 +bge-small-en-v1.5,test,6909,0.0,0.0 +bge-small-en-v1.5,test,6959,0.61315,1.0 +bge-small-en-v1.5,test,6985,1.0,1.0 +bge-small-en-v1.5,test,7017,0.0,0.0 +bge-small-en-v1.5,test,7068,0.20438,0.0 +bge-small-en-v1.5,test,7071,0.0,0.0 +bge-small-en-v1.5,test,7080,0.61315,1.0 +bge-small-en-v1.5,test,7096,0.5,1.0 +bge-small-en-v1.5,test,7098,0.79772,1.0 +bge-small-en-v1.5,test,7105,0.5,1.0 +bge-small-en-v1.5,test,7109,0.20211,0.0 +bge-small-en-v1.5,test,7124,0.39038,1.0 +bge-small-en-v1.5,test,7141,0.0,0.0 +bge-small-en-v1.5,test,7145,0.43173,1.0 +bge-small-en-v1.5,test,7178,1.0,1.0 +bge-small-en-v1.5,test,7188,1.0,1.0 +bge-small-en-v1.5,test,7205,1.0,1.0 +bge-small-en-v1.5,test,7206,0.0,0.0 +bge-small-en-v1.5,test,7218,0.0,0.0 +bge-small-en-v1.5,test,7221,0.0,0.0 +bge-small-en-v1.5,test,7269,0.35621,0.0 +bge-small-en-v1.5,test,7279,0.74953,1.0 +bge-small-en-v1.5,test,7295,0.0,0.0 +bge-small-en-v1.5,test,7311,0.0,0.0 +bge-small-en-v1.5,test,7326,0.0,0.0 +bge-small-en-v1.5,test,7329,0.38685,1.0 +bge-small-en-v1.5,test,7344,0.61315,1.0 +bge-small-en-v1.5,test,7345,0.33776,0.0 +bge-small-en-v1.5,test,7377,0.0,0.0 +bge-small-en-v1.5,test,7431,0.0,0.0 +bge-small-en-v1.5,test,7441,0.23464,1.0 +bge-small-en-v1.5,test,7445,0.0,0.0 +bge-small-en-v1.5,test,7448,0.35621,0.0 +bge-small-en-v1.5,test,7456,0.0,0.0 +bge-small-en-v1.5,test,7463,0.0,0.0 +bge-small-en-v1.5,test,7467,1.0,1.0 +bge-small-en-v1.5,test,7484,1.0,1.0 +bge-small-en-v1.5,test,7509,0.0,0.0 +bge-small-en-v1.5,test,7512,0.0,0.0 +bge-small-en-v1.5,test,7513,0.0,0.0 +bge-small-en-v1.5,test,7529,0.20438,0.0 +bge-small-en-v1.5,test,7533,0.0,0.0 +bge-small-en-v1.5,test,7534,0.46928,1.0 +bge-small-en-v1.5,test,7590,0.69343,1.0 +bge-small-en-v1.5,test,7592,0.0,0.0 +bge-small-en-v1.5,test,7594,0.0,0.0 +bge-small-en-v1.5,test,7622,0.0,0.0 +bge-small-en-v1.5,test,7633,0.39038,1.0 +bge-small-en-v1.5,test,7674,0.0,0.0 +bge-small-en-v1.5,test,7700,0.16813,0.0 +bge-small-en-v1.5,test,7702,0.21841,0.0 +bge-small-en-v1.5,test,7705,0.43173,1.0 +bge-small-en-v1.5,test,7734,0.61315,1.0 +bge-small-en-v1.5,test,7747,0.0,0.0 +bge-small-en-v1.5,test,7754,1.0,1.0 +bge-small-en-v1.5,test,7758,1.0,1.0 +bge-small-en-v1.5,test,7801,0.81753,1.0 +bge-small-en-v1.5,test,7803,0.0,0.0 +bge-small-en-v1.5,test,7823,0.46928,1.0 +bge-small-en-v1.5,test,7876,0.0,0.0 +bge-small-en-v1.5,test,7879,0.0,0.0 +bge-small-en-v1.5,test,7880,0.0,0.0 +bge-small-en-v1.5,test,7911,1.0,1.0 +bge-small-en-v1.5,test,7925,0.0,0.0 +bge-small-en-v1.5,test,7928,0.0,0.0 +bge-small-en-v1.5,test,7936,0.53072,1.0 +bge-small-en-v1.5,test,7992,0.17724,0.0 +bge-small-en-v1.5,test,8002,0.41618,1.0 +bge-small-en-v1.5,test,8005,0.0,0.0 +bge-small-en-v1.5,test,8013,0.0,0.0 +bge-small-en-v1.5,test,8017,0.43068,0.0 +bge-small-en-v1.5,test,8034,0.0,0.0 +bge-small-en-v1.5,test,8040,0.63093,1.0 +bge-small-en-v1.5,test,8072,0.44412,1.0 +bge-small-en-v1.5,test,8079,1.0,1.0 +bge-small-en-v1.5,test,8102,0.2463,1.0 +bge-small-en-v1.5,test,8116,0.57064,1.0 +bge-small-en-v1.5,test,8121,0.31546,0.0 +bge-small-en-v1.5,test,8202,0.0,0.0 +bge-small-en-v1.5,test,8230,0.17724,0.0 +bge-small-en-v1.5,test,8247,0.0,0.0 +bge-small-en-v1.5,test,8271,0.61315,1.0 +bge-small-en-v1.5,test,8275,0.0,0.0 +bge-small-en-v1.5,test,8296,1.0,1.0 +bge-small-en-v1.5,test,8332,0.67139,1.0 +bge-small-en-v1.5,test,8351,0.0,0.0 +bge-small-en-v1.5,test,8378,0.16716,0.0 +bge-small-en-v1.5,test,8456,0.46928,1.0 +bge-small-en-v1.5,test,8475,0.90209,1.0 +bge-small-en-v1.5,test,8507,0.0,0.0 +bge-small-en-v1.5,test,8512,0.0,0.0 +bge-small-en-v1.5,test,8513,0.0,0.0 +bge-small-en-v1.5,test,8532,0.29608,1.0 +bge-small-en-v1.5,test,8537,0.35621,0.0 +bge-small-en-v1.5,test,8539,0.11752,0.0 +bge-small-en-v1.5,test,8544,1.0,1.0 +bge-small-en-v1.5,test,8592,0.3152,0.0 +bge-small-en-v1.5,test,8632,0.61315,1.0 +bge-small-en-v1.5,test,8635,0.0,0.0 +bge-small-en-v1.5,test,8702,0.67139,1.0 +bge-small-en-v1.5,test,8779,0.0,0.0 +bge-small-en-v1.5,test,8789,0.76536,1.0 +bge-small-en-v1.5,test,8795,1.0,1.0 +bge-small-en-v1.5,test,8832,1.0,1.0 +bge-small-en-v1.5,test,8834,0.11706,0.0 +bge-small-en-v1.5,test,8855,0.0,0.0 +bge-small-en-v1.5,test,8874,0.0,0.0 +bge-small-en-v1.5,test,8934,0.0,0.0 +bge-small-en-v1.5,test,8937,0.35621,0.0 +bge-small-en-v1.5,test,8947,0.30657,1.0 +bge-small-en-v1.5,test,8959,0.0,0.0 +bge-small-en-v1.5,test,8970,0.91972,1.0 +bge-small-en-v1.5,test,8974,0.36188,1.0 +bge-small-en-v1.5,test,8982,0.0,0.0 +bge-small-en-v1.5,test,9060,0.61315,1.0 +bge-small-en-v1.5,test,9088,0.0,0.0 +bge-small-en-v1.5,test,9108,0.0,0.0 +bge-small-en-v1.5,test,9115,0.39107,1.0 +bge-small-en-v1.5,test,9126,0.0,0.0 +bge-small-en-v1.5,test,9164,0.23464,1.0 +bge-small-en-v1.5,test,9174,0.21399,1.0 +bge-small-en-v1.5,test,9188,0.61315,1.0 +bge-small-en-v1.5,test,9245,0.0,0.0 +bge-small-en-v1.5,test,9275,0.0,0.0 +bge-small-en-v1.5,test,9291,0.0,0.0 +bge-small-en-v1.5,test,9296,0.38685,1.0 +bge-small-en-v1.5,test,9329,0.61315,1.0 +bge-small-en-v1.5,test,9332,0.0,0.0 +bge-small-en-v1.5,test,9381,0.0,0.0 +bge-small-en-v1.5,test,9385,0.35621,0.0 +bge-small-en-v1.5,test,9391,0.55349,1.0 +bge-small-en-v1.5,test,9403,0.0,0.0 +bge-small-en-v1.5,test,9481,0.62405,1.0 +bge-small-en-v1.5,test,9487,0.0,0.0 +bge-small-en-v1.5,test,9548,0.0,0.0 +bge-small-en-v1.5,test,9556,1.0,1.0 +bge-small-en-v1.5,test,9565,1.0,1.0 +bge-small-en-v1.5,test,9598,0.0,0.0 +bge-small-en-v1.5,test,9617,0.46928,1.0 +bge-small-en-v1.5,test,9633,0.30657,1.0 +bge-small-en-v1.5,test,9643,0.38685,0.0 +bge-small-en-v1.5,test,9644,0.63093,1.0 +bge-small-en-v1.5,test,9646,0.61315,1.0 +bge-small-en-v1.5,test,9668,0.29608,1.0 +bge-small-en-v1.5,test,9701,0.0,0.0 +bge-small-en-v1.5,test,9733,0.20211,0.0 +bge-small-en-v1.5,test,9735,0.0,0.0 +bge-small-en-v1.5,test,9737,0.87108,1.0 +bge-small-en-v1.5,test,9771,0.0,0.0 +bge-small-en-v1.5,test,9808,0.46928,1.0 +bge-small-en-v1.5,test,9824,1.0,1.0 +bge-small-en-v1.5,test,9871,0.0,0.0 +bge-small-en-v1.5,test,9882,0.0,0.0 +bge-small-en-v1.5,test,9925,0.61315,1.0 +bge-small-en-v1.5,test,9929,0.61315,1.0 +bge-small-en-v1.5,test,9961,0.0,0.0 +bge-small-en-v1.5,test,9979,0.39038,1.0 +bge-small-en-v1.5,test,10034,0.20438,0.0 +bge-small-en-v1.5,test,10039,1.0,1.0 +bge-small-en-v1.5,test,10109,0.41231,1.0 +bge-small-en-v1.5,test,10122,0.19519,1.0 +bge-small-en-v1.5,test,10136,0.14127,0.0 +bge-small-en-v1.5,test,10137,0.43068,0.0 +bge-small-en-v1.5,test,10152,1.0,1.0 +bge-small-en-v1.5,test,10183,0.18458,0.0 +bge-small-en-v1.5,test,10213,0.0,0.0 +bge-small-en-v1.5,test,10246,0.0,0.0 +bge-small-en-v1.5,test,10267,0.14607,0.0 +bge-small-en-v1.5,test,10414,0.83155,1.0 +bge-small-en-v1.5,test,10447,0.23464,1.0 +bge-small-en-v1.5,test,10462,0.0,0.0 +bge-small-en-v1.5,test,10482,0.0,0.0 +bge-small-en-v1.5,test,10497,0.49436,1.0 +bge-small-en-v1.5,test,10526,0.26407,0.0 +bge-small-en-v1.5,test,10547,0.0,0.0 +bge-small-en-v1.5,test,10558,1.0,1.0 +bge-small-en-v1.5,test,10596,0.81227,1.0 +bge-small-en-v1.5,test,10601,1.0,1.0 +bge-small-en-v1.5,test,10628,0.33333,0.0 +bge-small-en-v1.5,test,10639,0.11305,0.0 +bge-small-en-v1.5,test,10645,0.2372,0.0 +bge-small-en-v1.5,test,10674,0.73659,1.0 +bge-small-en-v1.5,test,10710,0.5,1.0 +bge-small-en-v1.5,test,10734,0.96349,1.0 +bge-small-en-v1.5,test,10792,0.69343,1.0 +bge-small-en-v1.5,test,10808,0.20211,0.0 +bge-small-en-v1.5,test,10809,0.46928,1.0 +bge-small-en-v1.5,test,10812,0.76681,1.0 +bge-small-en-v1.5,test,10827,0.0,0.0 +bge-small-en-v1.5,test,10845,1.0,1.0 +bge-small-en-v1.5,test,10912,0.0,0.0 +bge-small-en-v1.5,test,10932,1.0,1.0 +bge-small-en-v1.5,test,10975,0.0,0.0 +bge-small-en-v1.5,test,10979,0.58557,1.0 +bge-small-en-v1.5,test,10994,0.21841,0.0 +bge-small-en-v1.5,test,11039,0.4134,1.0 +bge-small-en-v1.5,test,11054,1.0,1.0 +bge-small-en-v1.5,test,11088,1.0,1.0 diff --git a/evaluation/embedder_choice_per_question_scifact.csv b/evaluation/embedder_choice_per_question_scifact.csv new file mode 100644 index 0000000..b74fe6b --- /dev/null +++ b/evaluation/embedder_choice_per_question_scifact.csv @@ -0,0 +1,3328 @@ +model,split,query_id,ndcg10,hit3 +all-MiniLM-L6-v2,train,0,0.43068,0.0 +all-MiniLM-L6-v2,train,2,0.63093,1.0 +all-MiniLM-L6-v2,train,4,0.0,0.0 +all-MiniLM-L6-v2,train,6,0.43068,0.0 +all-MiniLM-L6-v2,train,9,1.0,1.0 +all-MiniLM-L6-v2,train,10,0.0,0.0 +all-MiniLM-L6-v2,train,11,0.0,0.0 +all-MiniLM-L6-v2,train,12,1.0,1.0 +all-MiniLM-L6-v2,train,14,0.0,0.0 +all-MiniLM-L6-v2,train,15,0.0,0.0 +all-MiniLM-L6-v2,train,17,0.0,0.0 +all-MiniLM-L6-v2,train,18,0.0,0.0 +all-MiniLM-L6-v2,train,19,1.0,1.0 +all-MiniLM-L6-v2,train,20,1.0,1.0 +all-MiniLM-L6-v2,train,21,1.0,1.0 +all-MiniLM-L6-v2,train,22,1.0,1.0 +all-MiniLM-L6-v2,train,24,0.63093,1.0 +all-MiniLM-L6-v2,train,25,0.5,1.0 +all-MiniLM-L6-v2,train,26,0.31546,0.0 +all-MiniLM-L6-v2,train,27,0.30103,0.0 +all-MiniLM-L6-v2,train,28,1.0,1.0 +all-MiniLM-L6-v2,train,30,1.0,1.0 +all-MiniLM-L6-v2,train,32,1.0,1.0 +all-MiniLM-L6-v2,train,34,1.0,1.0 +all-MiniLM-L6-v2,train,35,0.58028,1.0 +all-MiniLM-L6-v2,train,37,0.43062,0.0 +all-MiniLM-L6-v2,train,39,1.0,1.0 +all-MiniLM-L6-v2,train,40,1.0,1.0 +all-MiniLM-L6-v2,train,41,1.0,1.0 +all-MiniLM-L6-v2,train,43,1.0,1.0 +all-MiniLM-L6-v2,train,44,1.0,1.0 +all-MiniLM-L6-v2,train,45,1.0,1.0 +all-MiniLM-L6-v2,train,46,1.0,1.0 +all-MiniLM-L6-v2,train,47,1.0,1.0 +all-MiniLM-L6-v2,train,52,1.0,1.0 +all-MiniLM-L6-v2,train,55,1.0,1.0 +all-MiniLM-L6-v2,train,58,1.0,1.0 +all-MiniLM-L6-v2,train,60,0.48248,0.0 +all-MiniLM-L6-v2,train,61,0.30657,1.0 +all-MiniLM-L6-v2,train,62,0.0,0.0 +all-MiniLM-L6-v2,train,63,0.0,0.0 +all-MiniLM-L6-v2,train,64,0.0,0.0 +all-MiniLM-L6-v2,train,66,0.0,0.0 +all-MiniLM-L6-v2,train,67,1.0,1.0 +all-MiniLM-L6-v2,train,68,1.0,1.0 +all-MiniLM-L6-v2,train,69,0.0,0.0 +all-MiniLM-L6-v2,train,71,1.0,1.0 +all-MiniLM-L6-v2,train,73,1.0,1.0 +all-MiniLM-L6-v2,train,74,1.0,1.0 +all-MiniLM-L6-v2,train,76,1.0,1.0 +all-MiniLM-L6-v2,train,77,1.0,1.0 +all-MiniLM-L6-v2,train,78,1.0,1.0 +all-MiniLM-L6-v2,train,79,0.38685,0.0 +all-MiniLM-L6-v2,train,80,0.31546,0.0 +all-MiniLM-L6-v2,train,81,0.28906,0.0 +all-MiniLM-L6-v2,train,82,1.0,1.0 +all-MiniLM-L6-v2,train,85,0.85034,1.0 +all-MiniLM-L6-v2,train,86,0.85034,1.0 +all-MiniLM-L6-v2,train,88,0.62405,1.0 +all-MiniLM-L6-v2,train,89,0.65092,1.0 +all-MiniLM-L6-v2,train,90,0.63093,1.0 +all-MiniLM-L6-v2,train,91,0.0,0.0 +all-MiniLM-L6-v2,train,92,0.0,0.0 +all-MiniLM-L6-v2,train,93,0.28906,0.0 +all-MiniLM-L6-v2,train,95,0.33333,0.0 +all-MiniLM-L6-v2,train,96,1.0,1.0 +all-MiniLM-L6-v2,train,98,1.0,1.0 +all-MiniLM-L6-v2,train,104,1.0,1.0 +all-MiniLM-L6-v2,train,105,1.0,1.0 +all-MiniLM-L6-v2,train,106,1.0,1.0 +all-MiniLM-L6-v2,train,108,0.90603,1.0 +all-MiniLM-L6-v2,train,109,1.0,1.0 +all-MiniLM-L6-v2,train,111,1.0,1.0 +all-MiniLM-L6-v2,train,112,1.0,1.0 +all-MiniLM-L6-v2,train,114,1.0,1.0 +all-MiniLM-L6-v2,train,116,1.0,1.0 +all-MiniLM-L6-v2,train,119,1.0,1.0 +all-MiniLM-L6-v2,train,120,1.0,1.0 +all-MiniLM-L6-v2,train,121,0.43068,0.0 +all-MiniLM-L6-v2,train,122,0.33333,0.0 +all-MiniLM-L6-v2,train,123,1.0,1.0 +all-MiniLM-L6-v2,train,126,1.0,1.0 +all-MiniLM-L6-v2,train,134,0.0,0.0 +all-MiniLM-L6-v2,train,138,1.0,1.0 +all-MiniLM-L6-v2,train,139,0.0,0.0 +all-MiniLM-L6-v2,train,144,1.0,1.0 +all-MiniLM-L6-v2,train,149,0.63093,1.0 +all-MiniLM-L6-v2,train,152,1.0,1.0 +all-MiniLM-L6-v2,train,153,1.0,1.0 +all-MiniLM-L6-v2,train,154,1.0,1.0 +all-MiniLM-L6-v2,train,155,1.0,1.0 +all-MiniLM-L6-v2,train,156,1.0,1.0 +all-MiniLM-L6-v2,train,157,0.43068,0.0 +all-MiniLM-L6-v2,train,159,1.0,1.0 +all-MiniLM-L6-v2,train,160,1.0,1.0 +all-MiniLM-L6-v2,train,161,0.63093,1.0 +all-MiniLM-L6-v2,train,164,0.63093,1.0 +all-MiniLM-L6-v2,train,165,0.5,1.0 +all-MiniLM-L6-v2,train,166,1.0,1.0 +all-MiniLM-L6-v2,train,167,1.0,1.0 +all-MiniLM-L6-v2,train,168,1.0,1.0 +all-MiniLM-L6-v2,train,169,0.63093,1.0 +all-MiniLM-L6-v2,train,172,1.0,1.0 +all-MiniLM-L6-v2,train,173,0.0,0.0 +all-MiniLM-L6-v2,train,174,1.0,1.0 +all-MiniLM-L6-v2,train,175,1.0,1.0 +all-MiniLM-L6-v2,train,176,0.0,0.0 +all-MiniLM-L6-v2,train,177,1.0,1.0 +all-MiniLM-L6-v2,train,178,1.0,1.0 +all-MiniLM-L6-v2,train,181,1.0,1.0 +all-MiniLM-L6-v2,train,182,0.0,0.0 +all-MiniLM-L6-v2,train,184,1.0,1.0 +all-MiniLM-L6-v2,train,186,1.0,1.0 +all-MiniLM-L6-v2,train,187,1.0,1.0 +all-MiniLM-L6-v2,train,189,1.0,1.0 +all-MiniLM-L6-v2,train,196,1.0,1.0 +all-MiniLM-L6-v2,train,197,0.33333,0.0 +all-MiniLM-L6-v2,train,199,0.0,0.0 +all-MiniLM-L6-v2,train,200,0.31546,0.0 +all-MiniLM-L6-v2,train,201,0.63093,1.0 +all-MiniLM-L6-v2,train,203,1.0,1.0 +all-MiniLM-L6-v2,train,204,0.63093,1.0 +all-MiniLM-L6-v2,train,205,1.0,1.0 +all-MiniLM-L6-v2,train,209,0.0,0.0 +all-MiniLM-L6-v2,train,210,1.0,1.0 +all-MiniLM-L6-v2,train,211,0.0,0.0 +all-MiniLM-L6-v2,train,214,0.35621,0.0 +all-MiniLM-L6-v2,train,220,1.0,1.0 +all-MiniLM-L6-v2,train,221,1.0,1.0 +all-MiniLM-L6-v2,train,222,1.0,1.0 +all-MiniLM-L6-v2,train,223,0.43068,0.0 +all-MiniLM-L6-v2,train,224,0.0,0.0 +all-MiniLM-L6-v2,train,225,0.0,0.0 +all-MiniLM-L6-v2,train,226,0.0,0.0 +all-MiniLM-L6-v2,train,227,1.0,1.0 +all-MiniLM-L6-v2,train,228,1.0,1.0 +all-MiniLM-L6-v2,train,229,1.0,1.0 +all-MiniLM-L6-v2,train,235,1.0,1.0 +all-MiniLM-L6-v2,train,241,0.61315,1.0 +all-MiniLM-L6-v2,train,242,0.61315,1.0 +all-MiniLM-L6-v2,train,243,1.0,1.0 +all-MiniLM-L6-v2,train,244,1.0,1.0 +all-MiniLM-L6-v2,train,245,0.38685,1.0 +all-MiniLM-L6-v2,train,246,0.38685,1.0 +all-MiniLM-L6-v2,train,247,1.0,1.0 +all-MiniLM-L6-v2,train,250,0.63093,1.0 +all-MiniLM-L6-v2,train,251,0.63093,1.0 +all-MiniLM-L6-v2,train,253,0.5,1.0 +all-MiniLM-L6-v2,train,254,0.5,1.0 +all-MiniLM-L6-v2,train,255,0.63093,1.0 +all-MiniLM-L6-v2,train,256,0.63093,1.0 +all-MiniLM-L6-v2,train,258,0.0,0.0 +all-MiniLM-L6-v2,train,259,0.0,0.0 +all-MiniLM-L6-v2,train,262,0.0,0.0 +all-MiniLM-L6-v2,train,263,0.38365,0.0 +all-MiniLM-L6-v2,train,264,1.0,1.0 +all-MiniLM-L6-v2,train,265,1.0,1.0 +all-MiniLM-L6-v2,train,266,1.0,1.0 +all-MiniLM-L6-v2,train,267,0.83155,1.0 +all-MiniLM-L6-v2,train,272,1.0,1.0 +all-MiniLM-L6-v2,train,277,1.0,1.0 +all-MiniLM-L6-v2,train,278,1.0,1.0 +all-MiniLM-L6-v2,train,280,1.0,1.0 +all-MiniLM-L6-v2,train,281,0.63093,1.0 +all-MiniLM-L6-v2,train,283,1.0,1.0 +all-MiniLM-L6-v2,train,285,0.43068,0.0 +all-MiniLM-L6-v2,train,286,1.0,1.0 +all-MiniLM-L6-v2,train,287,1.0,1.0 +all-MiniLM-L6-v2,train,290,0.63093,1.0 +all-MiniLM-L6-v2,train,292,0.5,1.0 +all-MiniLM-L6-v2,train,293,0.0,0.0 +all-MiniLM-L6-v2,train,296,0.63093,1.0 +all-MiniLM-L6-v2,train,299,1.0,1.0 +all-MiniLM-L6-v2,train,301,0.43068,0.0 +all-MiniLM-L6-v2,train,304,0.0,0.0 +all-MiniLM-L6-v2,train,305,1.0,1.0 +all-MiniLM-L6-v2,train,306,1.0,1.0 +all-MiniLM-L6-v2,train,308,1.0,1.0 +all-MiniLM-L6-v2,train,309,1.0,1.0 +all-MiniLM-L6-v2,train,310,0.0,0.0 +all-MiniLM-L6-v2,train,313,0.0,0.0 +all-MiniLM-L6-v2,train,315,0.0,0.0 +all-MiniLM-L6-v2,train,316,0.35621,0.0 +all-MiniLM-L6-v2,train,317,0.0,0.0 +all-MiniLM-L6-v2,train,323,0.43068,0.0 +all-MiniLM-L6-v2,train,325,0.0,0.0 +all-MiniLM-L6-v2,train,326,0.0,0.0 +all-MiniLM-L6-v2,train,330,1.0,1.0 +all-MiniLM-L6-v2,train,331,1.0,1.0 +all-MiniLM-L6-v2,train,332,1.0,1.0 +all-MiniLM-L6-v2,train,333,1.0,1.0 +all-MiniLM-L6-v2,train,334,1.0,1.0 +all-MiniLM-L6-v2,train,335,1.0,1.0 +all-MiniLM-L6-v2,train,336,0.63093,1.0 +all-MiniLM-L6-v2,train,337,0.63093,1.0 +all-MiniLM-L6-v2,train,339,0.63093,1.0 +all-MiniLM-L6-v2,train,340,1.0,1.0 +all-MiniLM-L6-v2,train,341,1.0,1.0 +all-MiniLM-L6-v2,train,342,0.81753,1.0 +all-MiniLM-L6-v2,train,345,0.43068,0.0 +all-MiniLM-L6-v2,train,346,0.33333,0.0 +all-MiniLM-L6-v2,train,347,0.30103,0.0 +all-MiniLM-L6-v2,train,349,1.0,1.0 +all-MiniLM-L6-v2,train,351,0.0,0.0 +all-MiniLM-L6-v2,train,352,0.0,0.0 +all-MiniLM-L6-v2,train,355,0.87722,1.0 +all-MiniLM-L6-v2,train,356,0.63093,1.0 +all-MiniLM-L6-v2,train,357,1.0,1.0 +all-MiniLM-L6-v2,train,358,1.0,1.0 +all-MiniLM-L6-v2,train,361,1.0,1.0 +all-MiniLM-L6-v2,train,363,0.0,0.0 +all-MiniLM-L6-v2,train,364,0.0,0.0 +all-MiniLM-L6-v2,train,365,1.0,1.0 +all-MiniLM-L6-v2,train,366,1.0,1.0 +all-MiniLM-L6-v2,train,367,0.63093,1.0 +all-MiniLM-L6-v2,train,368,0.63093,1.0 +all-MiniLM-L6-v2,train,369,1.0,1.0 +all-MiniLM-L6-v2,train,370,1.0,1.0 +all-MiniLM-L6-v2,train,371,1.0,1.0 +all-MiniLM-L6-v2,train,372,1.0,1.0 +all-MiniLM-L6-v2,train,375,1.0,1.0 +all-MiniLM-L6-v2,train,376,1.0,1.0 +all-MiniLM-L6-v2,train,377,0.5,1.0 +all-MiniLM-L6-v2,train,378,0.4216,1.0 +all-MiniLM-L6-v2,train,379,1.0,1.0 +all-MiniLM-L6-v2,train,381,1.0,1.0 +all-MiniLM-L6-v2,train,382,0.0,0.0 +all-MiniLM-L6-v2,train,383,0.0,0.0 +all-MiniLM-L6-v2,train,389,1.0,1.0 +all-MiniLM-L6-v2,train,390,1.0,1.0 +all-MiniLM-L6-v2,train,391,0.63093,1.0 +all-MiniLM-L6-v2,train,392,1.0,1.0 +all-MiniLM-L6-v2,train,393,0.63093,1.0 +all-MiniLM-L6-v2,train,394,0.43068,0.0 +all-MiniLM-L6-v2,train,396,0.63093,1.0 +all-MiniLM-L6-v2,train,397,0.63093,1.0 +all-MiniLM-L6-v2,train,398,0.0,0.0 +all-MiniLM-L6-v2,train,400,1.0,1.0 +all-MiniLM-L6-v2,train,401,0.38685,0.0 +all-MiniLM-L6-v2,train,403,0.0,0.0 +all-MiniLM-L6-v2,train,404,0.0,0.0 +all-MiniLM-L6-v2,train,406,0.0,0.0 +all-MiniLM-L6-v2,train,407,0.28906,0.0 +all-MiniLM-L6-v2,train,413,0.0,0.0 +all-MiniLM-L6-v2,train,414,0.0,0.0 +all-MiniLM-L6-v2,train,416,0.0,0.0 +all-MiniLM-L6-v2,train,417,0.0,0.0 +all-MiniLM-L6-v2,train,418,0.0,0.0 +all-MiniLM-L6-v2,train,420,1.0,1.0 +all-MiniLM-L6-v2,train,422,0.0,0.0 +all-MiniLM-L6-v2,train,423,0.0,0.0 +all-MiniLM-L6-v2,train,425,1.0,1.0 +all-MiniLM-L6-v2,train,426,0.63093,1.0 +all-MiniLM-L6-v2,train,428,0.35621,0.0 +all-MiniLM-L6-v2,train,429,0.63093,1.0 +all-MiniLM-L6-v2,train,430,0.0,0.0 +all-MiniLM-L6-v2,train,432,1.0,1.0 +all-MiniLM-L6-v2,train,434,0.63093,1.0 +all-MiniLM-L6-v2,train,435,0.63093,1.0 +all-MiniLM-L6-v2,train,441,1.0,1.0 +all-MiniLM-L6-v2,train,444,1.0,1.0 +all-MiniLM-L6-v2,train,445,1.0,1.0 +all-MiniLM-L6-v2,train,447,1.0,1.0 +all-MiniLM-L6-v2,train,448,1.0,1.0 +all-MiniLM-L6-v2,train,449,0.0,0.0 +all-MiniLM-L6-v2,train,453,1.0,1.0 +all-MiniLM-L6-v2,train,454,1.0,1.0 +all-MiniLM-L6-v2,train,455,0.0,0.0 +all-MiniLM-L6-v2,train,456,0.0,0.0 +all-MiniLM-L6-v2,train,458,1.0,1.0 +all-MiniLM-L6-v2,train,461,0.63093,1.0 +all-MiniLM-L6-v2,train,463,0.63093,1.0 +all-MiniLM-L6-v2,train,466,0.63093,1.0 +all-MiniLM-L6-v2,train,469,0.63093,1.0 +all-MiniLM-L6-v2,train,470,0.63093,1.0 +all-MiniLM-L6-v2,train,472,0.90603,1.0 +all-MiniLM-L6-v2,train,473,0.0,0.0 +all-MiniLM-L6-v2,train,474,0.0,0.0 +all-MiniLM-L6-v2,train,479,1.0,1.0 +all-MiniLM-L6-v2,train,480,1.0,1.0 +all-MiniLM-L6-v2,train,481,0.35621,0.0 +all-MiniLM-L6-v2,train,482,1.0,1.0 +all-MiniLM-L6-v2,train,483,0.5,1.0 +all-MiniLM-L6-v2,train,484,0.0,0.0 +all-MiniLM-L6-v2,train,485,1.0,1.0 +all-MiniLM-L6-v2,train,486,0.0,0.0 +all-MiniLM-L6-v2,train,487,1.0,1.0 +all-MiniLM-L6-v2,train,488,1.0,1.0 +all-MiniLM-L6-v2,train,489,0.0,0.0 +all-MiniLM-L6-v2,train,490,1.0,1.0 +all-MiniLM-L6-v2,train,492,0.0,0.0 +all-MiniLM-L6-v2,train,493,0.0,0.0 +all-MiniLM-L6-v2,train,494,0.0,0.0 +all-MiniLM-L6-v2,train,495,1.0,1.0 +all-MiniLM-L6-v2,train,498,1.0,1.0 +all-MiniLM-L6-v2,train,499,1.0,1.0 +all-MiniLM-L6-v2,train,500,1.0,1.0 +all-MiniLM-L6-v2,train,504,0.63093,1.0 +all-MiniLM-L6-v2,train,505,1.0,1.0 +all-MiniLM-L6-v2,train,506,0.38685,0.0 +all-MiniLM-L6-v2,train,509,0.30103,0.0 +all-MiniLM-L6-v2,train,515,1.0,1.0 +all-MiniLM-L6-v2,train,523,1.0,1.0 +all-MiniLM-L6-v2,train,524,1.0,1.0 +all-MiniLM-L6-v2,train,526,0.30103,0.0 +all-MiniLM-L6-v2,train,529,0.50657,0.0 +all-MiniLM-L6-v2,train,530,0.41323,0.0 +all-MiniLM-L6-v2,train,531,0.54307,1.0 +all-MiniLM-L6-v2,train,537,1.0,1.0 +all-MiniLM-L6-v2,train,541,0.3487,0.0 +all-MiniLM-L6-v2,train,542,1.0,1.0 +all-MiniLM-L6-v2,train,545,0.0,0.0 +all-MiniLM-L6-v2,train,547,0.63093,1.0 +all-MiniLM-L6-v2,train,548,0.63093,1.0 +all-MiniLM-L6-v2,train,550,0.63093,1.0 +all-MiniLM-L6-v2,train,553,1.0,1.0 +all-MiniLM-L6-v2,train,555,0.0,0.0 +all-MiniLM-L6-v2,train,557,0.63093,1.0 +all-MiniLM-L6-v2,train,559,0.0,0.0 +all-MiniLM-L6-v2,train,562,1.0,1.0 +all-MiniLM-L6-v2,train,563,1.0,1.0 +all-MiniLM-L6-v2,train,564,1.0,1.0 +all-MiniLM-L6-v2,train,565,1.0,1.0 +all-MiniLM-L6-v2,train,566,1.0,1.0 +all-MiniLM-L6-v2,train,568,1.0,1.0 +all-MiniLM-L6-v2,train,570,1.0,1.0 +all-MiniLM-L6-v2,train,571,1.0,1.0 +all-MiniLM-L6-v2,train,572,0.0,0.0 +all-MiniLM-L6-v2,train,573,1.0,1.0 +all-MiniLM-L6-v2,train,574,1.0,1.0 +all-MiniLM-L6-v2,train,576,1.0,1.0 +all-MiniLM-L6-v2,train,579,1.0,1.0 +all-MiniLM-L6-v2,train,580,1.0,1.0 +all-MiniLM-L6-v2,train,582,1.0,1.0 +all-MiniLM-L6-v2,train,584,1.0,1.0 +all-MiniLM-L6-v2,train,585,1.0,1.0 +all-MiniLM-L6-v2,train,588,0.0,0.0 +all-MiniLM-L6-v2,train,590,1.0,1.0 +all-MiniLM-L6-v2,train,591,1.0,1.0 +all-MiniLM-L6-v2,train,592,1.0,1.0 +all-MiniLM-L6-v2,train,594,1.0,1.0 +all-MiniLM-L6-v2,train,595,0.63093,1.0 +all-MiniLM-L6-v2,train,600,0.63093,1.0 +all-MiniLM-L6-v2,train,601,1.0,1.0 +all-MiniLM-L6-v2,train,602,0.0,0.0 +all-MiniLM-L6-v2,train,603,1.0,1.0 +all-MiniLM-L6-v2,train,606,0.35621,0.0 +all-MiniLM-L6-v2,train,607,0.0,0.0 +all-MiniLM-L6-v2,train,609,0.0,0.0 +all-MiniLM-L6-v2,train,610,0.0,0.0 +all-MiniLM-L6-v2,train,611,0.5,1.0 +all-MiniLM-L6-v2,train,612,1.0,1.0 +all-MiniLM-L6-v2,train,614,1.0,1.0 +all-MiniLM-L6-v2,train,615,1.0,1.0 +all-MiniLM-L6-v2,train,616,0.0,0.0 +all-MiniLM-L6-v2,train,617,0.0,0.0 +all-MiniLM-L6-v2,train,618,0.5,1.0 +all-MiniLM-L6-v2,train,620,0.0,0.0 +all-MiniLM-L6-v2,train,621,1.0,1.0 +all-MiniLM-L6-v2,train,622,0.0,0.0 +all-MiniLM-L6-v2,train,624,1.0,1.0 +all-MiniLM-L6-v2,train,625,1.0,1.0 +all-MiniLM-L6-v2,train,626,1.0,1.0 +all-MiniLM-L6-v2,train,631,1.0,1.0 +all-MiniLM-L6-v2,train,632,1.0,1.0 +all-MiniLM-L6-v2,train,633,1.0,1.0 +all-MiniLM-L6-v2,train,635,0.63093,1.0 +all-MiniLM-L6-v2,train,638,0.43068,0.0 +all-MiniLM-L6-v2,train,640,0.43068,0.0 +all-MiniLM-L6-v2,train,642,0.0,0.0 +all-MiniLM-L6-v2,train,643,1.0,1.0 +all-MiniLM-L6-v2,train,645,0.0,0.0 +all-MiniLM-L6-v2,train,646,0.0,0.0 +all-MiniLM-L6-v2,train,647,0.63093,1.0 +all-MiniLM-L6-v2,train,648,0.5,1.0 +all-MiniLM-L6-v2,train,650,1.0,1.0 +all-MiniLM-L6-v2,train,651,0.63093,1.0 +all-MiniLM-L6-v2,train,652,0.63093,1.0 +all-MiniLM-L6-v2,train,653,0.63093,1.0 +all-MiniLM-L6-v2,train,654,1.0,1.0 +all-MiniLM-L6-v2,train,655,1.0,1.0 +all-MiniLM-L6-v2,train,657,0.5,1.0 +all-MiniLM-L6-v2,train,658,1.0,1.0 +all-MiniLM-L6-v2,train,661,1.0,1.0 +all-MiniLM-L6-v2,train,662,1.0,1.0 +all-MiniLM-L6-v2,train,663,1.0,1.0 +all-MiniLM-L6-v2,train,665,1.0,1.0 +all-MiniLM-L6-v2,train,666,1.0,1.0 +all-MiniLM-L6-v2,train,667,1.0,1.0 +all-MiniLM-L6-v2,train,668,1.0,1.0 +all-MiniLM-L6-v2,train,669,0.85034,1.0 +all-MiniLM-L6-v2,train,670,1.0,1.0 +all-MiniLM-L6-v2,train,671,1.0,1.0 +all-MiniLM-L6-v2,train,672,1.0,1.0 +all-MiniLM-L6-v2,train,673,0.38685,0.0 +all-MiniLM-L6-v2,train,676,1.0,1.0 +all-MiniLM-L6-v2,train,677,1.0,1.0 +all-MiniLM-L6-v2,train,679,0.0,0.0 +all-MiniLM-L6-v2,train,680,0.63093,1.0 +all-MiniLM-L6-v2,train,681,1.0,1.0 +all-MiniLM-L6-v2,train,682,0.63093,1.0 +all-MiniLM-L6-v2,train,683,1.0,1.0 +all-MiniLM-L6-v2,train,685,0.5,1.0 +all-MiniLM-L6-v2,train,686,0.35621,0.0 +all-MiniLM-L6-v2,train,687,0.43068,0.0 +all-MiniLM-L6-v2,train,688,0.5,1.0 +all-MiniLM-L6-v2,train,689,0.0,0.0 +all-MiniLM-L6-v2,train,694,1.0,1.0 +all-MiniLM-L6-v2,train,696,0.31546,0.0 +all-MiniLM-L6-v2,train,698,0.5,1.0 +all-MiniLM-L6-v2,train,703,0.63093,1.0 +all-MiniLM-L6-v2,train,704,0.0,0.0 +all-MiniLM-L6-v2,train,705,0.63093,1.0 +all-MiniLM-L6-v2,train,709,0.63093,1.0 +all-MiniLM-L6-v2,train,710,0.5,1.0 +all-MiniLM-L6-v2,train,713,0.0,0.0 +all-MiniLM-L6-v2,train,714,0.0,0.0 +all-MiniLM-L6-v2,train,717,1.0,1.0 +all-MiniLM-L6-v2,train,724,1.0,1.0 +all-MiniLM-L6-v2,train,726,0.65092,1.0 +all-MiniLM-L6-v2,train,730,1.0,1.0 +all-MiniLM-L6-v2,train,732,0.5,1.0 +all-MiniLM-L6-v2,train,733,0.35621,0.0 +all-MiniLM-L6-v2,train,734,0.43068,0.0 +all-MiniLM-L6-v2,train,736,1.0,1.0 +all-MiniLM-L6-v2,train,737,0.61315,1.0 +all-MiniLM-L6-v2,train,738,0.46928,1.0 +all-MiniLM-L6-v2,train,739,0.0,0.0 +all-MiniLM-L6-v2,train,740,1.0,1.0 +all-MiniLM-L6-v2,train,745,1.0,1.0 +all-MiniLM-L6-v2,train,746,1.0,1.0 +all-MiniLM-L6-v2,train,747,1.0,1.0 +all-MiniLM-L6-v2,train,748,1.0,1.0 +all-MiniLM-L6-v2,train,749,0.0,0.0 +all-MiniLM-L6-v2,train,751,1.0,1.0 +all-MiniLM-L6-v2,train,752,1.0,1.0 +all-MiniLM-L6-v2,train,753,0.0,0.0 +all-MiniLM-L6-v2,train,755,0.0,0.0 +all-MiniLM-L6-v2,train,757,1.0,1.0 +all-MiniLM-L6-v2,train,758,0.0,0.0 +all-MiniLM-L6-v2,train,760,1.0,1.0 +all-MiniLM-L6-v2,train,761,0.0,0.0 +all-MiniLM-L6-v2,train,762,0.0,0.0 +all-MiniLM-L6-v2,train,764,1.0,1.0 +all-MiniLM-L6-v2,train,765,1.0,1.0 +all-MiniLM-L6-v2,train,766,1.0,1.0 +all-MiniLM-L6-v2,train,767,1.0,1.0 +all-MiniLM-L6-v2,train,771,1.0,1.0 +all-MiniLM-L6-v2,train,772,1.0,1.0 +all-MiniLM-L6-v2,train,774,0.0,0.0 +all-MiniLM-L6-v2,train,776,0.0,0.0 +all-MiniLM-L6-v2,train,777,0.0,0.0 +all-MiniLM-L6-v2,train,778,1.0,1.0 +all-MiniLM-L6-v2,train,779,1.0,1.0 +all-MiniLM-L6-v2,train,780,1.0,1.0 +all-MiniLM-L6-v2,train,782,0.5,1.0 +all-MiniLM-L6-v2,train,787,1.0,1.0 +all-MiniLM-L6-v2,train,788,1.0,1.0 +all-MiniLM-L6-v2,train,789,1.0,1.0 +all-MiniLM-L6-v2,train,790,1.0,1.0 +all-MiniLM-L6-v2,train,791,0.63093,1.0 +all-MiniLM-L6-v2,train,792,1.0,1.0 +all-MiniLM-L6-v2,train,795,1.0,1.0 +all-MiniLM-L6-v2,train,797,1.0,1.0 +all-MiniLM-L6-v2,train,798,0.63093,1.0 +all-MiniLM-L6-v2,train,799,1.0,1.0 +all-MiniLM-L6-v2,train,801,1.0,1.0 +all-MiniLM-L6-v2,train,802,1.0,1.0 +all-MiniLM-L6-v2,train,803,1.0,1.0 +all-MiniLM-L6-v2,train,804,0.63093,1.0 +all-MiniLM-L6-v2,train,807,1.0,1.0 +all-MiniLM-L6-v2,train,810,0.63093,1.0 +all-MiniLM-L6-v2,train,812,1.0,1.0 +all-MiniLM-L6-v2,train,813,1.0,1.0 +all-MiniLM-L6-v2,train,815,0.63093,1.0 +all-MiniLM-L6-v2,train,816,0.5,1.0 +all-MiniLM-L6-v2,train,817,0.5,1.0 +all-MiniLM-L6-v2,train,818,0.0,0.0 +all-MiniLM-L6-v2,train,822,0.63093,1.0 +all-MiniLM-L6-v2,train,825,0.63093,1.0 +all-MiniLM-L6-v2,train,826,0.38685,0.0 +all-MiniLM-L6-v2,train,828,0.0,0.0 +all-MiniLM-L6-v2,train,835,0.63093,1.0 +all-MiniLM-L6-v2,train,838,0.0,0.0 +all-MiniLM-L6-v2,train,840,1.0,1.0 +all-MiniLM-L6-v2,train,841,0.43068,0.0 +all-MiniLM-L6-v2,train,844,1.0,1.0 +all-MiniLM-L6-v2,train,846,0.63093,1.0 +all-MiniLM-L6-v2,train,848,1.0,1.0 +all-MiniLM-L6-v2,train,853,1.0,1.0 +all-MiniLM-L6-v2,train,854,1.0,1.0 +all-MiniLM-L6-v2,train,855,1.0,1.0 +all-MiniLM-L6-v2,train,856,0.0,0.0 +all-MiniLM-L6-v2,train,857,0.0,0.0 +all-MiniLM-L6-v2,train,858,0.5,1.0 +all-MiniLM-L6-v2,train,860,1.0,1.0 +all-MiniLM-L6-v2,train,861,1.0,1.0 +all-MiniLM-L6-v2,train,863,0.52498,1.0 +all-MiniLM-L6-v2,train,866,0.63093,1.0 +all-MiniLM-L6-v2,train,867,1.0,1.0 +all-MiniLM-L6-v2,train,871,0.0,0.0 +all-MiniLM-L6-v2,train,876,0.0,0.0 +all-MiniLM-L6-v2,train,877,1.0,1.0 +all-MiniLM-L6-v2,train,881,1.0,1.0 +all-MiniLM-L6-v2,train,883,1.0,1.0 +all-MiniLM-L6-v2,train,884,1.0,1.0 +all-MiniLM-L6-v2,train,885,1.0,1.0 +all-MiniLM-L6-v2,train,886,1.0,1.0 +all-MiniLM-L6-v2,train,890,0.43068,0.0 +all-MiniLM-L6-v2,train,891,0.43068,0.0 +all-MiniLM-L6-v2,train,893,1.0,1.0 +all-MiniLM-L6-v2,train,894,0.0,0.0 +all-MiniLM-L6-v2,train,895,0.0,0.0 +all-MiniLM-L6-v2,train,896,1.0,1.0 +all-MiniLM-L6-v2,train,897,1.0,1.0 +all-MiniLM-L6-v2,train,898,0.79772,1.0 +all-MiniLM-L6-v2,train,899,0.83155,1.0 +all-MiniLM-L6-v2,train,900,0.0,0.0 +all-MiniLM-L6-v2,train,901,1.0,1.0 +all-MiniLM-L6-v2,train,902,1.0,1.0 +all-MiniLM-L6-v2,train,908,1.0,1.0 +all-MiniLM-L6-v2,train,909,1.0,1.0 +all-MiniLM-L6-v2,train,910,1.0,1.0 +all-MiniLM-L6-v2,train,912,1.0,1.0 +all-MiniLM-L6-v2,train,916,0.0,0.0 +all-MiniLM-L6-v2,train,917,1.0,1.0 +all-MiniLM-L6-v2,train,919,1.0,1.0 +all-MiniLM-L6-v2,train,923,1.0,1.0 +all-MiniLM-L6-v2,train,925,1.0,1.0 +all-MiniLM-L6-v2,train,926,1.0,1.0 +all-MiniLM-L6-v2,train,927,1.0,1.0 +all-MiniLM-L6-v2,train,928,0.63093,1.0 +all-MiniLM-L6-v2,train,929,0.5,1.0 +all-MiniLM-L6-v2,train,930,0.63093,1.0 +all-MiniLM-L6-v2,train,933,1.0,1.0 +all-MiniLM-L6-v2,train,934,0.38685,0.0 +all-MiniLM-L6-v2,train,935,0.0,0.0 +all-MiniLM-L6-v2,train,938,0.38685,0.0 +all-MiniLM-L6-v2,train,939,0.31546,0.0 +all-MiniLM-L6-v2,train,940,1.0,1.0 +all-MiniLM-L6-v2,train,941,1.0,1.0 +all-MiniLM-L6-v2,train,942,0.35621,0.0 +all-MiniLM-L6-v2,train,944,1.0,1.0 +all-MiniLM-L6-v2,train,945,0.6506,1.0 +all-MiniLM-L6-v2,train,946,0.6665,1.0 +all-MiniLM-L6-v2,train,949,0.0,0.0 +all-MiniLM-L6-v2,train,951,0.5,1.0 +all-MiniLM-L6-v2,train,952,1.0,1.0 +all-MiniLM-L6-v2,train,953,1.0,1.0 +all-MiniLM-L6-v2,train,954,1.0,1.0 +all-MiniLM-L6-v2,train,955,0.83155,1.0 +all-MiniLM-L6-v2,train,959,1.0,1.0 +all-MiniLM-L6-v2,train,962,0.9469,1.0 +all-MiniLM-L6-v2,train,963,0.83155,1.0 +all-MiniLM-L6-v2,train,964,0.83155,1.0 +all-MiniLM-L6-v2,train,965,1.0,1.0 +all-MiniLM-L6-v2,train,969,0.62405,1.0 +all-MiniLM-L6-v2,train,970,0.62405,1.0 +all-MiniLM-L6-v2,train,972,0.70451,1.0 +all-MiniLM-L6-v2,train,973,0.76064,1.0 +all-MiniLM-L6-v2,train,976,0.0,0.0 +all-MiniLM-L6-v2,train,977,0.0,0.0 +all-MiniLM-L6-v2,train,978,0.0,0.0 +all-MiniLM-L6-v2,train,979,1.0,1.0 +all-MiniLM-L6-v2,train,980,1.0,1.0 +all-MiniLM-L6-v2,train,984,1.0,1.0 +all-MiniLM-L6-v2,train,988,0.0,0.0 +all-MiniLM-L6-v2,train,989,0.31546,0.0 +all-MiniLM-L6-v2,train,990,1.0,1.0 +all-MiniLM-L6-v2,train,992,1.0,1.0 +all-MiniLM-L6-v2,train,994,1.0,1.0 +all-MiniLM-L6-v2,train,996,0.31546,0.0 +all-MiniLM-L6-v2,train,997,1.0,1.0 +all-MiniLM-L6-v2,train,998,1.0,1.0 +all-MiniLM-L6-v2,train,999,1.0,1.0 +all-MiniLM-L6-v2,train,1000,0.63093,1.0 +all-MiniLM-L6-v2,train,1001,0.63093,1.0 +all-MiniLM-L6-v2,train,1002,0.0,0.0 +all-MiniLM-L6-v2,train,1003,0.29608,1.0 +all-MiniLM-L6-v2,train,1004,0.69788,1.0 +all-MiniLM-L6-v2,train,1005,0.69788,1.0 +all-MiniLM-L6-v2,train,1006,1.0,1.0 +all-MiniLM-L6-v2,train,1008,0.0,0.0 +all-MiniLM-L6-v2,train,1009,1.0,1.0 +all-MiniLM-L6-v2,train,1011,1.0,1.0 +all-MiniLM-L6-v2,train,1015,1.0,1.0 +all-MiniLM-L6-v2,train,1016,1.0,1.0 +all-MiniLM-L6-v2,train,1018,1.0,1.0 +all-MiniLM-L6-v2,train,1023,1.0,1.0 +all-MiniLM-L6-v2,train,1025,0.63093,1.0 +all-MiniLM-L6-v2,train,1026,0.63093,1.0 +all-MiniLM-L6-v2,train,1027,0.38685,0.0 +all-MiniLM-L6-v2,train,1028,0.85034,1.0 +all-MiniLM-L6-v2,train,1030,1.0,1.0 +all-MiniLM-L6-v2,train,1031,1.0,1.0 +all-MiniLM-L6-v2,train,1032,0.63093,1.0 +all-MiniLM-L6-v2,train,1033,0.5,1.0 +all-MiniLM-L6-v2,train,1034,0.63093,1.0 +all-MiniLM-L6-v2,train,1035,0.63093,1.0 +all-MiniLM-L6-v2,train,1036,0.30103,0.0 +all-MiniLM-L6-v2,train,1037,1.0,1.0 +all-MiniLM-L6-v2,train,1038,1.0,1.0 +all-MiniLM-L6-v2,train,1040,0.0,0.0 +all-MiniLM-L6-v2,train,1042,1.0,1.0 +all-MiniLM-L6-v2,train,1043,1.0,1.0 +all-MiniLM-L6-v2,train,1044,1.0,1.0 +all-MiniLM-L6-v2,train,1045,1.0,1.0 +all-MiniLM-L6-v2,train,1046,0.0,0.0 +all-MiniLM-L6-v2,train,1047,0.0,0.0 +all-MiniLM-L6-v2,train,1048,0.0,0.0 +all-MiniLM-L6-v2,train,1050,1.0,1.0 +all-MiniLM-L6-v2,train,1052,0.43068,0.0 +all-MiniLM-L6-v2,train,1053,0.5,1.0 +all-MiniLM-L6-v2,train,1054,1.0,1.0 +all-MiniLM-L6-v2,train,1055,1.0,1.0 +all-MiniLM-L6-v2,train,1056,0.5,1.0 +all-MiniLM-L6-v2,train,1058,1.0,1.0 +all-MiniLM-L6-v2,train,1065,1.0,1.0 +all-MiniLM-L6-v2,train,1067,0.0,0.0 +all-MiniLM-L6-v2,train,1068,0.0,0.0 +all-MiniLM-L6-v2,train,1069,1.0,1.0 +all-MiniLM-L6-v2,train,1070,0.38685,0.0 +all-MiniLM-L6-v2,train,1072,0.63093,1.0 +all-MiniLM-L6-v2,train,1073,0.63093,1.0 +all-MiniLM-L6-v2,train,1074,0.0,0.0 +all-MiniLM-L6-v2,train,1075,0.0,0.0 +all-MiniLM-L6-v2,train,1081,1.0,1.0 +all-MiniLM-L6-v2,train,1084,1.0,1.0 +all-MiniLM-L6-v2,train,1085,1.0,1.0 +all-MiniLM-L6-v2,train,1087,1.0,1.0 +all-MiniLM-L6-v2,train,1090,1.0,1.0 +all-MiniLM-L6-v2,train,1091,0.0,0.0 +all-MiniLM-L6-v2,train,1096,0.0,0.0 +all-MiniLM-L6-v2,train,1097,0.33333,0.0 +all-MiniLM-L6-v2,train,1098,0.0,0.0 +all-MiniLM-L6-v2,train,1101,1.0,1.0 +all-MiniLM-L6-v2,train,1102,1.0,1.0 +all-MiniLM-L6-v2,train,1103,1.0,1.0 +all-MiniLM-L6-v2,train,1105,1.0,1.0 +all-MiniLM-L6-v2,train,1106,1.0,1.0 +all-MiniLM-L6-v2,train,1109,0.0,0.0 +all-MiniLM-L6-v2,train,1111,0.0,0.0 +all-MiniLM-L6-v2,train,1112,0.0,0.0 +all-MiniLM-L6-v2,train,1114,1.0,1.0 +all-MiniLM-L6-v2,train,1115,1.0,1.0 +all-MiniLM-L6-v2,train,1118,1.0,1.0 +all-MiniLM-L6-v2,train,1119,0.73283,1.0 +all-MiniLM-L6-v2,train,1120,0.73283,1.0 +all-MiniLM-L6-v2,train,1125,1.0,1.0 +all-MiniLM-L6-v2,train,1126,1.0,1.0 +all-MiniLM-L6-v2,train,1127,0.0,0.0 +all-MiniLM-L6-v2,train,1128,0.18458,0.0 +all-MiniLM-L6-v2,train,1133,1.0,1.0 +all-MiniLM-L6-v2,train,1134,1.0,1.0 +all-MiniLM-L6-v2,train,1135,1.0,1.0 +all-MiniLM-L6-v2,train,1136,1.0,1.0 +all-MiniLM-L6-v2,train,1138,0.0,0.0 +all-MiniLM-L6-v2,train,1139,0.63093,1.0 +all-MiniLM-L6-v2,train,1141,1.0,1.0 +all-MiniLM-L6-v2,train,1142,1.0,1.0 +all-MiniLM-L6-v2,train,1145,1.0,1.0 +all-MiniLM-L6-v2,train,1148,1.0,1.0 +all-MiniLM-L6-v2,train,1153,1.0,1.0 +all-MiniLM-L6-v2,train,1156,0.0,0.0 +all-MiniLM-L6-v2,train,1157,0.0,0.0 +all-MiniLM-L6-v2,train,1158,0.0,0.0 +all-MiniLM-L6-v2,train,1159,0.0,0.0 +all-MiniLM-L6-v2,train,1161,0.38685,0.0 +all-MiniLM-L6-v2,train,1162,1.0,1.0 +all-MiniLM-L6-v2,train,1164,1.0,1.0 +all-MiniLM-L6-v2,train,1165,1.0,1.0 +all-MiniLM-L6-v2,train,1166,0.0,0.0 +all-MiniLM-L6-v2,train,1168,0.63093,1.0 +all-MiniLM-L6-v2,train,1169,0.43068,0.0 +all-MiniLM-L6-v2,train,1170,1.0,1.0 +all-MiniLM-L6-v2,train,1171,1.0,1.0 +all-MiniLM-L6-v2,train,1173,1.0,1.0 +all-MiniLM-L6-v2,train,1174,0.0,0.0 +all-MiniLM-L6-v2,train,1176,1.0,1.0 +all-MiniLM-L6-v2,train,1177,1.0,1.0 +all-MiniLM-L6-v2,train,1178,0.0,0.0 +all-MiniLM-L6-v2,train,1181,0.98289,1.0 +all-MiniLM-L6-v2,train,1182,0.63093,1.0 +all-MiniLM-L6-v2,train,1183,1.0,1.0 +all-MiniLM-L6-v2,train,1184,0.5,1.0 +all-MiniLM-L6-v2,train,1186,1.0,1.0 +all-MiniLM-L6-v2,train,1188,1.0,1.0 +all-MiniLM-L6-v2,train,1190,1.0,1.0 +all-MiniLM-L6-v2,train,1193,0.63093,1.0 +all-MiniLM-L6-v2,train,1195,1.0,1.0 +all-MiniLM-L6-v2,train,1205,1.0,1.0 +all-MiniLM-L6-v2,train,1206,1.0,1.0 +all-MiniLM-L6-v2,train,1208,1.0,1.0 +all-MiniLM-L6-v2,train,1209,0.0,0.0 +all-MiniLM-L6-v2,train,1210,1.0,1.0 +all-MiniLM-L6-v2,train,1211,1.0,1.0 +all-MiniLM-L6-v2,train,1212,0.0,0.0 +all-MiniLM-L6-v2,train,1214,0.0,0.0 +all-MiniLM-L6-v2,train,1215,0.28906,0.0 +all-MiniLM-L6-v2,train,1218,0.0,0.0 +all-MiniLM-L6-v2,train,1219,0.81753,1.0 +all-MiniLM-L6-v2,train,1220,0.63093,1.0 +all-MiniLM-L6-v2,train,1223,0.5,1.0 +all-MiniLM-L6-v2,train,1224,0.80657,1.0 +all-MiniLM-L6-v2,train,1227,1.0,1.0 +all-MiniLM-L6-v2,train,1228,1.0,1.0 +all-MiniLM-L6-v2,train,1229,1.0,1.0 +all-MiniLM-L6-v2,train,1230,1.0,1.0 +all-MiniLM-L6-v2,train,1231,1.0,1.0 +all-MiniLM-L6-v2,train,1234,1.0,1.0 +all-MiniLM-L6-v2,train,1235,0.43068,0.0 +all-MiniLM-L6-v2,train,1236,0.43068,0.0 +all-MiniLM-L6-v2,train,1237,1.0,1.0 +all-MiniLM-L6-v2,train,1238,1.0,1.0 +all-MiniLM-L6-v2,train,1239,0.26407,0.0 +all-MiniLM-L6-v2,train,1244,0.43068,0.0 +all-MiniLM-L6-v2,train,1246,0.0,0.0 +all-MiniLM-L6-v2,train,1247,0.0,0.0 +all-MiniLM-L6-v2,train,1248,1.0,1.0 +all-MiniLM-L6-v2,train,1249,1.0,1.0 +all-MiniLM-L6-v2,train,1253,0.43068,0.0 +all-MiniLM-L6-v2,train,1254,1.0,1.0 +all-MiniLM-L6-v2,train,1255,0.0,0.0 +all-MiniLM-L6-v2,train,1257,0.0,0.0 +all-MiniLM-L6-v2,train,1258,0.0,0.0 +all-MiniLM-L6-v2,train,1260,0.63093,1.0 +all-MiniLM-L6-v2,train,1261,0.63093,1.0 +all-MiniLM-L6-v2,train,1263,1.0,1.0 +all-MiniLM-L6-v2,train,1265,0.38685,0.0 +all-MiniLM-L6-v2,train,1268,0.63093,1.0 +all-MiniLM-L6-v2,train,1269,1.0,1.0 +all-MiniLM-L6-v2,train,1275,1.0,1.0 +all-MiniLM-L6-v2,train,1276,0.63093,1.0 +all-MiniLM-L6-v2,train,1284,1.0,1.0 +all-MiniLM-L6-v2,train,1288,1.0,1.0 +all-MiniLM-L6-v2,train,1289,1.0,1.0 +all-MiniLM-L6-v2,train,1291,1.0,1.0 +all-MiniLM-L6-v2,train,1293,1.0,1.0 +all-MiniLM-L6-v2,train,1294,0.62405,1.0 +all-MiniLM-L6-v2,train,1295,1.0,1.0 +all-MiniLM-L6-v2,train,1297,1.0,1.0 +all-MiniLM-L6-v2,train,1300,0.0,0.0 +all-MiniLM-L6-v2,train,1302,0.0,0.0 +all-MiniLM-L6-v2,train,1304,0.0,0.0 +all-MiniLM-L6-v2,train,1305,1.0,1.0 +all-MiniLM-L6-v2,train,1306,0.0,0.0 +all-MiniLM-L6-v2,train,1307,0.0,0.0 +all-MiniLM-L6-v2,train,1308,0.0,0.0 +all-MiniLM-L6-v2,train,1309,0.0,0.0 +all-MiniLM-L6-v2,train,1310,0.5,1.0 +all-MiniLM-L6-v2,train,1311,1.0,1.0 +all-MiniLM-L6-v2,train,1312,1.0,1.0 +all-MiniLM-L6-v2,train,1314,0.38685,1.0 +all-MiniLM-L6-v2,train,1315,0.38685,1.0 +all-MiniLM-L6-v2,train,1322,1.0,1.0 +all-MiniLM-L6-v2,train,1323,1.0,1.0 +all-MiniLM-L6-v2,train,1324,1.0,1.0 +all-MiniLM-L6-v2,train,1325,0.0,0.0 +all-MiniLM-L6-v2,train,1327,1.0,1.0 +all-MiniLM-L6-v2,train,1328,0.63093,1.0 +all-MiniLM-L6-v2,train,1330,0.0,0.0 +all-MiniLM-L6-v2,train,1331,0.0,0.0 +all-MiniLM-L6-v2,train,1333,1.0,1.0 +all-MiniLM-L6-v2,train,1334,0.43735,1.0 +all-MiniLM-L6-v2,train,1340,1.0,1.0 +all-MiniLM-L6-v2,train,1341,1.0,1.0 +all-MiniLM-L6-v2,train,1342,0.63093,1.0 +all-MiniLM-L6-v2,train,1345,0.0,0.0 +all-MiniLM-L6-v2,train,1346,0.0,0.0 +all-MiniLM-L6-v2,train,1347,1.0,1.0 +all-MiniLM-L6-v2,train,1348,1.0,1.0 +all-MiniLM-L6-v2,train,1349,1.0,1.0 +all-MiniLM-L6-v2,train,1350,1.0,1.0 +all-MiniLM-L6-v2,train,1351,1.0,1.0 +all-MiniLM-L6-v2,train,1353,0.63093,1.0 +all-MiniLM-L6-v2,train,1355,1.0,1.0 +all-MiniLM-L6-v2,train,1356,1.0,1.0 +all-MiniLM-L6-v2,train,1360,1.0,1.0 +all-MiniLM-L6-v2,train,1361,0.91972,1.0 +all-MiniLM-L6-v2,train,1364,0.0,0.0 +all-MiniLM-L6-v2,train,1366,0.63093,1.0 +all-MiniLM-L6-v2,train,1367,0.43068,0.0 +all-MiniLM-L6-v2,train,1371,1.0,1.0 +all-MiniLM-L6-v2,train,1372,1.0,1.0 +all-MiniLM-L6-v2,train,1373,1.0,1.0 +all-MiniLM-L6-v2,train,1374,0.63093,1.0 +all-MiniLM-L6-v2,train,1375,0.63093,1.0 +all-MiniLM-L6-v2,train,1376,0.63093,1.0 +all-MiniLM-L6-v2,train,1378,1.0,1.0 +all-MiniLM-L6-v2,train,1380,0.71226,1.0 +all-MiniLM-L6-v2,train,1381,1.0,1.0 +all-MiniLM-L6-v2,train,1383,1.0,1.0 +all-MiniLM-L6-v2,train,1386,0.63093,1.0 +all-MiniLM-L6-v2,train,1387,0.0,0.0 +all-MiniLM-L6-v2,train,1390,1.0,1.0 +all-MiniLM-L6-v2,train,1391,1.0,1.0 +all-MiniLM-L6-v2,train,1392,1.0,1.0 +all-MiniLM-L6-v2,train,1393,0.87722,1.0 +all-MiniLM-L6-v2,train,1394,0.43068,0.0 +all-MiniLM-L6-v2,train,1397,0.63093,1.0 +all-MiniLM-L6-v2,train,1398,0.0,0.0 +all-MiniLM-L6-v2,train,1400,0.0,0.0 +all-MiniLM-L6-v2,train,1401,1.0,1.0 +all-MiniLM-L6-v2,train,1402,0.0,0.0 +all-MiniLM-L6-v2,train,1403,0.81753,1.0 +all-MiniLM-L6-v2,train,1404,0.61315,1.0 +all-MiniLM-L6-v2,train,1405,0.5,1.0 +all-MiniLM-L6-v2,train,1406,1.0,1.0 +all-MiniLM-L6-v2,train,1407,0.0,0.0 +all-MiniLM-L6-v2,test,1,0.38685,0.0 +all-MiniLM-L6-v2,test,3,0.5,1.0 +all-MiniLM-L6-v2,test,5,0.63093,1.0 +all-MiniLM-L6-v2,test,13,0.0,0.0 +all-MiniLM-L6-v2,test,36,0.49115,1.0 +all-MiniLM-L6-v2,test,42,0.63093,1.0 +all-MiniLM-L6-v2,test,48,0.0,0.0 +all-MiniLM-L6-v2,test,49,1.0,1.0 +all-MiniLM-L6-v2,test,50,1.0,1.0 +all-MiniLM-L6-v2,test,51,1.0,1.0 +all-MiniLM-L6-v2,test,53,1.0,1.0 +all-MiniLM-L6-v2,test,54,1.0,1.0 +all-MiniLM-L6-v2,test,56,1.0,1.0 +all-MiniLM-L6-v2,test,57,1.0,1.0 +all-MiniLM-L6-v2,test,70,0.0,0.0 +all-MiniLM-L6-v2,test,72,1.0,1.0 +all-MiniLM-L6-v2,test,75,0.63093,1.0 +all-MiniLM-L6-v2,test,94,0.5,1.0 +all-MiniLM-L6-v2,test,99,0.63093,1.0 +all-MiniLM-L6-v2,test,100,0.5,1.0 +all-MiniLM-L6-v2,test,113,1.0,1.0 +all-MiniLM-L6-v2,test,115,1.0,1.0 +all-MiniLM-L6-v2,test,118,0.63093,1.0 +all-MiniLM-L6-v2,test,124,1.0,1.0 +all-MiniLM-L6-v2,test,127,1.0,1.0 +all-MiniLM-L6-v2,test,128,0.0,0.0 +all-MiniLM-L6-v2,test,129,1.0,1.0 +all-MiniLM-L6-v2,test,130,1.0,1.0 +all-MiniLM-L6-v2,test,132,0.0,0.0 +all-MiniLM-L6-v2,test,133,0.63995,1.0 +all-MiniLM-L6-v2,test,137,1.0,1.0 +all-MiniLM-L6-v2,test,141,0.91972,1.0 +all-MiniLM-L6-v2,test,142,1.0,1.0 +all-MiniLM-L6-v2,test,143,1.0,1.0 +all-MiniLM-L6-v2,test,146,1.0,1.0 +all-MiniLM-L6-v2,test,148,0.63093,1.0 +all-MiniLM-L6-v2,test,163,1.0,1.0 +all-MiniLM-L6-v2,test,171,1.0,1.0 +all-MiniLM-L6-v2,test,179,0.95583,1.0 +all-MiniLM-L6-v2,test,180,1.0,1.0 +all-MiniLM-L6-v2,test,183,1.0,1.0 +all-MiniLM-L6-v2,test,185,1.0,1.0 +all-MiniLM-L6-v2,test,198,0.0,0.0 +all-MiniLM-L6-v2,test,208,0.5,1.0 +all-MiniLM-L6-v2,test,212,0.35621,0.0 +all-MiniLM-L6-v2,test,213,0.38685,0.0 +all-MiniLM-L6-v2,test,216,1.0,1.0 +all-MiniLM-L6-v2,test,217,1.0,1.0 +all-MiniLM-L6-v2,test,218,1.0,1.0 +all-MiniLM-L6-v2,test,219,1.0,1.0 +all-MiniLM-L6-v2,test,230,1.0,1.0 +all-MiniLM-L6-v2,test,232,1.0,1.0 +all-MiniLM-L6-v2,test,233,1.0,1.0 +all-MiniLM-L6-v2,test,236,1.0,1.0 +all-MiniLM-L6-v2,test,237,1.0,1.0 +all-MiniLM-L6-v2,test,238,0.63093,1.0 +all-MiniLM-L6-v2,test,239,0.0,0.0 +all-MiniLM-L6-v2,test,248,0.5,1.0 +all-MiniLM-L6-v2,test,249,0.5,1.0 +all-MiniLM-L6-v2,test,261,0.20438,0.0 +all-MiniLM-L6-v2,test,268,1.0,1.0 +all-MiniLM-L6-v2,test,269,1.0,1.0 +all-MiniLM-L6-v2,test,274,1.0,1.0 +all-MiniLM-L6-v2,test,275,0.67973,1.0 +all-MiniLM-L6-v2,test,279,1.0,1.0 +all-MiniLM-L6-v2,test,294,0.0,0.0 +all-MiniLM-L6-v2,test,295,0.5,1.0 +all-MiniLM-L6-v2,test,298,1.0,1.0 +all-MiniLM-L6-v2,test,300,0.43068,0.0 +all-MiniLM-L6-v2,test,303,0.0,0.0 +all-MiniLM-L6-v2,test,312,0.0,0.0 +all-MiniLM-L6-v2,test,314,0.0,0.0 +all-MiniLM-L6-v2,test,324,0.63093,1.0 +all-MiniLM-L6-v2,test,327,1.0,1.0 +all-MiniLM-L6-v2,test,338,0.63093,1.0 +all-MiniLM-L6-v2,test,343,0.83155,1.0 +all-MiniLM-L6-v2,test,350,1.0,1.0 +all-MiniLM-L6-v2,test,354,1.0,1.0 +all-MiniLM-L6-v2,test,362,1.0,1.0 +all-MiniLM-L6-v2,test,380,1.0,1.0 +all-MiniLM-L6-v2,test,384,0.0,0.0 +all-MiniLM-L6-v2,test,385,0.69343,1.0 +all-MiniLM-L6-v2,test,386,1.0,1.0 +all-MiniLM-L6-v2,test,388,0.63093,1.0 +all-MiniLM-L6-v2,test,399,1.0,1.0 +all-MiniLM-L6-v2,test,410,1.0,1.0 +all-MiniLM-L6-v2,test,411,1.0,1.0 +all-MiniLM-L6-v2,test,415,0.0,0.0 +all-MiniLM-L6-v2,test,421,0.0,0.0 +all-MiniLM-L6-v2,test,431,0.0,0.0 +all-MiniLM-L6-v2,test,436,1.0,1.0 +all-MiniLM-L6-v2,test,437,0.0,0.0 +all-MiniLM-L6-v2,test,439,1.0,1.0 +all-MiniLM-L6-v2,test,440,1.0,1.0 +all-MiniLM-L6-v2,test,443,1.0,1.0 +all-MiniLM-L6-v2,test,452,0.17724,0.0 +all-MiniLM-L6-v2,test,475,0.0,0.0 +all-MiniLM-L6-v2,test,478,1.0,1.0 +all-MiniLM-L6-v2,test,491,1.0,1.0 +all-MiniLM-L6-v2,test,501,1.0,1.0 +all-MiniLM-L6-v2,test,502,0.0,0.0 +all-MiniLM-L6-v2,test,507,0.63093,1.0 +all-MiniLM-L6-v2,test,508,0.0,0.0 +all-MiniLM-L6-v2,test,513,0.63093,1.0 +all-MiniLM-L6-v2,test,514,1.0,1.0 +all-MiniLM-L6-v2,test,516,1.0,1.0 +all-MiniLM-L6-v2,test,517,0.0,0.0 +all-MiniLM-L6-v2,test,521,1.0,1.0 +all-MiniLM-L6-v2,test,525,1.0,1.0 +all-MiniLM-L6-v2,test,527,0.0,0.0 +all-MiniLM-L6-v2,test,528,1.0,1.0 +all-MiniLM-L6-v2,test,532,1.0,1.0 +all-MiniLM-L6-v2,test,533,1.0,1.0 +all-MiniLM-L6-v2,test,535,0.43068,0.0 +all-MiniLM-L6-v2,test,536,1.0,1.0 +all-MiniLM-L6-v2,test,539,1.0,1.0 +all-MiniLM-L6-v2,test,540,0.26407,0.0 +all-MiniLM-L6-v2,test,544,0.0,0.0 +all-MiniLM-L6-v2,test,549,1.0,1.0 +all-MiniLM-L6-v2,test,551,0.63093,1.0 +all-MiniLM-L6-v2,test,552,1.0,1.0 +all-MiniLM-L6-v2,test,554,0.0,0.0 +all-MiniLM-L6-v2,test,560,0.0,0.0 +all-MiniLM-L6-v2,test,569,0.43068,0.0 +all-MiniLM-L6-v2,test,575,0.5,1.0 +all-MiniLM-L6-v2,test,577,0.31546,0.0 +all-MiniLM-L6-v2,test,578,1.0,1.0 +all-MiniLM-L6-v2,test,587,0.0,0.0 +all-MiniLM-L6-v2,test,589,1.0,1.0 +all-MiniLM-L6-v2,test,593,1.0,1.0 +all-MiniLM-L6-v2,test,597,0.79287,1.0 +all-MiniLM-L6-v2,test,598,0.33333,0.0 +all-MiniLM-L6-v2,test,613,1.0,1.0 +all-MiniLM-L6-v2,test,619,0.0,0.0 +all-MiniLM-L6-v2,test,623,0.0,0.0 +all-MiniLM-L6-v2,test,628,0.43068,0.0 +all-MiniLM-L6-v2,test,636,1.0,1.0 +all-MiniLM-L6-v2,test,637,0.43068,0.0 +all-MiniLM-L6-v2,test,641,0.83155,1.0 +all-MiniLM-L6-v2,test,644,0.0,0.0 +all-MiniLM-L6-v2,test,649,1.0,1.0 +all-MiniLM-L6-v2,test,659,0.43068,0.0 +all-MiniLM-L6-v2,test,660,0.0,0.0 +all-MiniLM-L6-v2,test,674,0.33333,0.0 +all-MiniLM-L6-v2,test,684,1.0,1.0 +all-MiniLM-L6-v2,test,690,0.0,0.0 +all-MiniLM-L6-v2,test,691,0.43068,0.0 +all-MiniLM-L6-v2,test,692,1.0,1.0 +all-MiniLM-L6-v2,test,693,1.0,1.0 +all-MiniLM-L6-v2,test,700,0.63093,1.0 +all-MiniLM-L6-v2,test,702,0.63093,1.0 +all-MiniLM-L6-v2,test,715,0.0,0.0 +all-MiniLM-L6-v2,test,716,0.0,0.0 +all-MiniLM-L6-v2,test,718,1.0,1.0 +all-MiniLM-L6-v2,test,721,1.0,1.0 +all-MiniLM-L6-v2,test,723,1.0,1.0 +all-MiniLM-L6-v2,test,727,0.43068,0.0 +all-MiniLM-L6-v2,test,728,0.65092,1.0 +all-MiniLM-L6-v2,test,729,1.0,1.0 +all-MiniLM-L6-v2,test,742,0.30103,0.0 +all-MiniLM-L6-v2,test,743,0.0,0.0 +all-MiniLM-L6-v2,test,744,0.5,1.0 +all-MiniLM-L6-v2,test,756,0.63093,1.0 +all-MiniLM-L6-v2,test,759,1.0,1.0 +all-MiniLM-L6-v2,test,768,0.0,0.0 +all-MiniLM-L6-v2,test,770,1.0,1.0 +all-MiniLM-L6-v2,test,775,0.0,0.0 +all-MiniLM-L6-v2,test,781,0.63093,1.0 +all-MiniLM-L6-v2,test,783,0.38685,0.0 +all-MiniLM-L6-v2,test,784,0.63093,1.0 +all-MiniLM-L6-v2,test,785,0.0,0.0 +all-MiniLM-L6-v2,test,793,1.0,1.0 +all-MiniLM-L6-v2,test,800,0.63093,1.0 +all-MiniLM-L6-v2,test,805,1.0,1.0 +all-MiniLM-L6-v2,test,808,1.0,1.0 +all-MiniLM-L6-v2,test,811,1.0,1.0 +all-MiniLM-L6-v2,test,814,1.0,1.0 +all-MiniLM-L6-v2,test,820,0.0,0.0 +all-MiniLM-L6-v2,test,821,0.0,0.0 +all-MiniLM-L6-v2,test,823,1.0,1.0 +all-MiniLM-L6-v2,test,830,0.0,0.0 +all-MiniLM-L6-v2,test,831,0.0,0.0 +all-MiniLM-L6-v2,test,832,1.0,1.0 +all-MiniLM-L6-v2,test,834,0.0,0.0 +all-MiniLM-L6-v2,test,837,0.63093,1.0 +all-MiniLM-L6-v2,test,839,0.5,1.0 +all-MiniLM-L6-v2,test,845,0.43068,0.0 +all-MiniLM-L6-v2,test,847,1.0,1.0 +all-MiniLM-L6-v2,test,852,0.63093,1.0 +all-MiniLM-L6-v2,test,859,0.43068,0.0 +all-MiniLM-L6-v2,test,870,0.0,0.0 +all-MiniLM-L6-v2,test,873,0.26762,1.0 +all-MiniLM-L6-v2,test,879,1.0,1.0 +all-MiniLM-L6-v2,test,880,0.63093,1.0 +all-MiniLM-L6-v2,test,882,1.0,1.0 +all-MiniLM-L6-v2,test,887,0.0,0.0 +all-MiniLM-L6-v2,test,903,1.0,1.0 +all-MiniLM-L6-v2,test,904,1.0,1.0 +all-MiniLM-L6-v2,test,907,1.0,1.0 +all-MiniLM-L6-v2,test,911,1.0,1.0 +all-MiniLM-L6-v2,test,913,0.31546,0.0 +all-MiniLM-L6-v2,test,914,0.43068,0.0 +all-MiniLM-L6-v2,test,921,1.0,1.0 +all-MiniLM-L6-v2,test,922,1.0,1.0 +all-MiniLM-L6-v2,test,936,1.0,1.0 +all-MiniLM-L6-v2,test,956,1.0,1.0 +all-MiniLM-L6-v2,test,957,0.5,1.0 +all-MiniLM-L6-v2,test,960,1.0,1.0 +all-MiniLM-L6-v2,test,967,1.0,1.0 +all-MiniLM-L6-v2,test,971,0.73157,1.0 +all-MiniLM-L6-v2,test,975,0.0,0.0 +all-MiniLM-L6-v2,test,982,1.0,1.0 +all-MiniLM-L6-v2,test,985,1.0,1.0 +all-MiniLM-L6-v2,test,993,0.63093,1.0 +all-MiniLM-L6-v2,test,1012,1.0,1.0 +all-MiniLM-L6-v2,test,1014,1.0,1.0 +all-MiniLM-L6-v2,test,1019,0.43068,0.0 +all-MiniLM-L6-v2,test,1020,0.63093,1.0 +all-MiniLM-L6-v2,test,1021,0.63093,1.0 +all-MiniLM-L6-v2,test,1024,0.0,0.0 +all-MiniLM-L6-v2,test,1029,0.96747,1.0 +all-MiniLM-L6-v2,test,1041,0.0,0.0 +all-MiniLM-L6-v2,test,1049,0.0,0.0 +all-MiniLM-L6-v2,test,1062,0.63093,1.0 +all-MiniLM-L6-v2,test,1086,1.0,1.0 +all-MiniLM-L6-v2,test,1088,1.0,1.0 +all-MiniLM-L6-v2,test,1089,1.0,1.0 +all-MiniLM-L6-v2,test,1099,0.5,1.0 +all-MiniLM-L6-v2,test,1100,0.43068,0.0 +all-MiniLM-L6-v2,test,1104,1.0,1.0 +all-MiniLM-L6-v2,test,1107,0.5,1.0 +all-MiniLM-L6-v2,test,1110,0.0,0.0 +all-MiniLM-L6-v2,test,1121,1.0,1.0 +all-MiniLM-L6-v2,test,1130,1.0,1.0 +all-MiniLM-L6-v2,test,1132,0.2372,0.0 +all-MiniLM-L6-v2,test,1137,1.0,1.0 +all-MiniLM-L6-v2,test,1140,1.0,1.0 +all-MiniLM-L6-v2,test,1144,1.0,1.0 +all-MiniLM-L6-v2,test,1146,1.0,1.0 +all-MiniLM-L6-v2,test,1150,1.0,1.0 +all-MiniLM-L6-v2,test,1163,1.0,1.0 +all-MiniLM-L6-v2,test,1175,0.0,0.0 +all-MiniLM-L6-v2,test,1179,0.0,0.0 +all-MiniLM-L6-v2,test,1180,0.0,0.0 +all-MiniLM-L6-v2,test,1185,0.5,1.0 +all-MiniLM-L6-v2,test,1187,0.63093,1.0 +all-MiniLM-L6-v2,test,1191,0.63093,1.0 +all-MiniLM-L6-v2,test,1194,1.0,1.0 +all-MiniLM-L6-v2,test,1196,0.38685,0.0 +all-MiniLM-L6-v2,test,1197,0.38685,0.0 +all-MiniLM-L6-v2,test,1199,0.0,0.0 +all-MiniLM-L6-v2,test,1200,0.0,0.0 +all-MiniLM-L6-v2,test,1202,0.63093,1.0 +all-MiniLM-L6-v2,test,1204,1.0,1.0 +all-MiniLM-L6-v2,test,1207,1.0,1.0 +all-MiniLM-L6-v2,test,1213,0.0,0.0 +all-MiniLM-L6-v2,test,1216,1.0,1.0 +all-MiniLM-L6-v2,test,1221,0.31546,0.0 +all-MiniLM-L6-v2,test,1225,1.0,1.0 +all-MiniLM-L6-v2,test,1226,0.0,0.0 +all-MiniLM-L6-v2,test,1232,1.0,1.0 +all-MiniLM-L6-v2,test,1241,1.0,1.0 +all-MiniLM-L6-v2,test,1245,0.30103,0.0 +all-MiniLM-L6-v2,test,1259,0.63093,1.0 +all-MiniLM-L6-v2,test,1262,1.0,1.0 +all-MiniLM-L6-v2,test,1266,0.38685,0.0 +all-MiniLM-L6-v2,test,1270,1.0,1.0 +all-MiniLM-L6-v2,test,1271,1.0,1.0 +all-MiniLM-L6-v2,test,1272,0.0,0.0 +all-MiniLM-L6-v2,test,1273,1.0,1.0 +all-MiniLM-L6-v2,test,1274,0.9469,1.0 +all-MiniLM-L6-v2,test,1278,1.0,1.0 +all-MiniLM-L6-v2,test,1279,1.0,1.0 +all-MiniLM-L6-v2,test,1280,0.0,0.0 +all-MiniLM-L6-v2,test,1281,0.31546,0.0 +all-MiniLM-L6-v2,test,1282,1.0,1.0 +all-MiniLM-L6-v2,test,1290,1.0,1.0 +all-MiniLM-L6-v2,test,1292,1.0,1.0 +all-MiniLM-L6-v2,test,1298,1.0,1.0 +all-MiniLM-L6-v2,test,1303,1.0,1.0 +all-MiniLM-L6-v2,test,1316,0.0,0.0 +all-MiniLM-L6-v2,test,1319,1.0,1.0 +all-MiniLM-L6-v2,test,1320,1.0,1.0 +all-MiniLM-L6-v2,test,1332,0.0,0.0 +all-MiniLM-L6-v2,test,1335,1.0,1.0 +all-MiniLM-L6-v2,test,1336,1.0,1.0 +all-MiniLM-L6-v2,test,1337,0.28906,0.0 +all-MiniLM-L6-v2,test,1339,1.0,1.0 +all-MiniLM-L6-v2,test,1344,0.63093,1.0 +all-MiniLM-L6-v2,test,1352,1.0,1.0 +all-MiniLM-L6-v2,test,1359,1.0,1.0 +all-MiniLM-L6-v2,test,1362,0.0,0.0 +all-MiniLM-L6-v2,test,1363,0.0,0.0 +all-MiniLM-L6-v2,test,1368,0.0,0.0 +all-MiniLM-L6-v2,test,1370,0.5,1.0 +all-MiniLM-L6-v2,test,1379,0.95583,1.0 +all-MiniLM-L6-v2,test,1382,1.0,1.0 +all-MiniLM-L6-v2,test,1385,0.63093,1.0 +all-MiniLM-L6-v2,test,1389,1.0,1.0 +all-MiniLM-L6-v2,test,1395,0.5,1.0 +bge-small-en-v1.5,train,0,0.0,0.0 +bge-small-en-v1.5,train,2,0.5,1.0 +bge-small-en-v1.5,train,4,0.43068,0.0 +bge-small-en-v1.5,train,6,1.0,1.0 +bge-small-en-v1.5,train,9,1.0,1.0 +bge-small-en-v1.5,train,10,0.35621,0.0 +bge-small-en-v1.5,train,11,0.43068,0.0 +bge-small-en-v1.5,train,12,1.0,1.0 +bge-small-en-v1.5,train,14,0.0,0.0 +bge-small-en-v1.5,train,15,0.0,0.0 +bge-small-en-v1.5,train,17,0.0,0.0 +bge-small-en-v1.5,train,18,0.33333,0.0 +bge-small-en-v1.5,train,19,0.63093,1.0 +bge-small-en-v1.5,train,20,1.0,1.0 +bge-small-en-v1.5,train,21,1.0,1.0 +bge-small-en-v1.5,train,22,1.0,1.0 +bge-small-en-v1.5,train,24,1.0,1.0 +bge-small-en-v1.5,train,25,1.0,1.0 +bge-small-en-v1.5,train,26,0.35621,0.0 +bge-small-en-v1.5,train,27,0.35621,0.0 +bge-small-en-v1.5,train,28,1.0,1.0 +bge-small-en-v1.5,train,30,1.0,1.0 +bge-small-en-v1.5,train,32,1.0,1.0 +bge-small-en-v1.5,train,34,0.63093,1.0 +bge-small-en-v1.5,train,35,0.83155,1.0 +bge-small-en-v1.5,train,37,0.81753,1.0 +bge-small-en-v1.5,train,39,1.0,1.0 +bge-small-en-v1.5,train,40,1.0,1.0 +bge-small-en-v1.5,train,41,1.0,1.0 +bge-small-en-v1.5,train,43,1.0,1.0 +bge-small-en-v1.5,train,44,1.0,1.0 +bge-small-en-v1.5,train,45,1.0,1.0 +bge-small-en-v1.5,train,46,1.0,1.0 +bge-small-en-v1.5,train,47,1.0,1.0 +bge-small-en-v1.5,train,52,1.0,1.0 +bge-small-en-v1.5,train,55,1.0,1.0 +bge-small-en-v1.5,train,58,1.0,1.0 +bge-small-en-v1.5,train,60,0.79772,1.0 +bge-small-en-v1.5,train,61,0.83155,1.0 +bge-small-en-v1.5,train,62,0.38685,0.0 +bge-small-en-v1.5,train,63,0.0,0.0 +bge-small-en-v1.5,train,64,0.0,0.0 +bge-small-en-v1.5,train,66,0.5,1.0 +bge-small-en-v1.5,train,67,1.0,1.0 +bge-small-en-v1.5,train,68,1.0,1.0 +bge-small-en-v1.5,train,69,0.49115,1.0 +bge-small-en-v1.5,train,71,1.0,1.0 +bge-small-en-v1.5,train,73,0.5,1.0 +bge-small-en-v1.5,train,74,1.0,1.0 +bge-small-en-v1.5,train,76,1.0,1.0 +bge-small-en-v1.5,train,77,1.0,1.0 +bge-small-en-v1.5,train,78,1.0,1.0 +bge-small-en-v1.5,train,79,0.63093,1.0 +bge-small-en-v1.5,train,80,0.38685,0.0 +bge-small-en-v1.5,train,81,0.0,0.0 +bge-small-en-v1.5,train,82,1.0,1.0 +bge-small-en-v1.5,train,85,0.69343,1.0 +bge-small-en-v1.5,train,86,0.69343,1.0 +bge-small-en-v1.5,train,88,0.69343,1.0 +bge-small-en-v1.5,train,89,0.65092,1.0 +bge-small-en-v1.5,train,90,0.63093,1.0 +bge-small-en-v1.5,train,91,0.0,0.0 +bge-small-en-v1.5,train,92,0.0,0.0 +bge-small-en-v1.5,train,93,0.0,0.0 +bge-small-en-v1.5,train,95,0.38685,0.0 +bge-small-en-v1.5,train,96,1.0,1.0 +bge-small-en-v1.5,train,98,1.0,1.0 +bge-small-en-v1.5,train,104,1.0,1.0 +bge-small-en-v1.5,train,105,1.0,1.0 +bge-small-en-v1.5,train,106,1.0,1.0 +bge-small-en-v1.5,train,108,0.90603,1.0 +bge-small-en-v1.5,train,109,1.0,1.0 +bge-small-en-v1.5,train,111,1.0,1.0 +bge-small-en-v1.5,train,112,1.0,1.0 +bge-small-en-v1.5,train,114,1.0,1.0 +bge-small-en-v1.5,train,116,1.0,1.0 +bge-small-en-v1.5,train,119,1.0,1.0 +bge-small-en-v1.5,train,120,1.0,1.0 +bge-small-en-v1.5,train,121,1.0,1.0 +bge-small-en-v1.5,train,122,1.0,1.0 +bge-small-en-v1.5,train,123,1.0,1.0 +bge-small-en-v1.5,train,126,1.0,1.0 +bge-small-en-v1.5,train,134,0.63093,1.0 +bge-small-en-v1.5,train,138,1.0,1.0 +bge-small-en-v1.5,train,139,0.0,0.0 +bge-small-en-v1.5,train,144,1.0,1.0 +bge-small-en-v1.5,train,149,1.0,1.0 +bge-small-en-v1.5,train,152,1.0,1.0 +bge-small-en-v1.5,train,153,1.0,1.0 +bge-small-en-v1.5,train,154,1.0,1.0 +bge-small-en-v1.5,train,155,0.63093,1.0 +bge-small-en-v1.5,train,156,0.63093,1.0 +bge-small-en-v1.5,train,157,1.0,1.0 +bge-small-en-v1.5,train,159,1.0,1.0 +bge-small-en-v1.5,train,160,1.0,1.0 +bge-small-en-v1.5,train,161,0.43068,0.0 +bge-small-en-v1.5,train,164,1.0,1.0 +bge-small-en-v1.5,train,165,0.5,1.0 +bge-small-en-v1.5,train,166,1.0,1.0 +bge-small-en-v1.5,train,167,1.0,1.0 +bge-small-en-v1.5,train,168,1.0,1.0 +bge-small-en-v1.5,train,169,0.63093,1.0 +bge-small-en-v1.5,train,172,1.0,1.0 +bge-small-en-v1.5,train,173,0.0,0.0 +bge-small-en-v1.5,train,174,0.38685,0.0 +bge-small-en-v1.5,train,175,0.5,1.0 +bge-small-en-v1.5,train,176,0.63093,1.0 +bge-small-en-v1.5,train,177,1.0,1.0 +bge-small-en-v1.5,train,178,1.0,1.0 +bge-small-en-v1.5,train,181,1.0,1.0 +bge-small-en-v1.5,train,182,0.0,0.0 +bge-small-en-v1.5,train,184,1.0,1.0 +bge-small-en-v1.5,train,186,1.0,1.0 +bge-small-en-v1.5,train,187,1.0,1.0 +bge-small-en-v1.5,train,189,1.0,1.0 +bge-small-en-v1.5,train,196,1.0,1.0 +bge-small-en-v1.5,train,197,0.5,1.0 +bge-small-en-v1.5,train,199,0.30103,0.0 +bge-small-en-v1.5,train,200,0.63093,1.0 +bge-small-en-v1.5,train,201,1.0,1.0 +bge-small-en-v1.5,train,203,1.0,1.0 +bge-small-en-v1.5,train,204,0.63093,1.0 +bge-small-en-v1.5,train,205,1.0,1.0 +bge-small-en-v1.5,train,209,0.33333,0.0 +bge-small-en-v1.5,train,210,1.0,1.0 +bge-small-en-v1.5,train,211,1.0,1.0 +bge-small-en-v1.5,train,214,0.30103,0.0 +bge-small-en-v1.5,train,220,1.0,1.0 +bge-small-en-v1.5,train,221,1.0,1.0 +bge-small-en-v1.5,train,222,1.0,1.0 +bge-small-en-v1.5,train,223,1.0,1.0 +bge-small-en-v1.5,train,224,0.0,0.0 +bge-small-en-v1.5,train,225,0.0,0.0 +bge-small-en-v1.5,train,226,0.0,0.0 +bge-small-en-v1.5,train,227,1.0,1.0 +bge-small-en-v1.5,train,228,1.0,1.0 +bge-small-en-v1.5,train,229,1.0,1.0 +bge-small-en-v1.5,train,235,1.0,1.0 +bge-small-en-v1.5,train,241,0.69343,1.0 +bge-small-en-v1.5,train,242,0.65092,1.0 +bge-small-en-v1.5,train,243,1.0,1.0 +bge-small-en-v1.5,train,244,1.0,1.0 +bge-small-en-v1.5,train,245,0.61315,1.0 +bge-small-en-v1.5,train,246,0.61315,1.0 +bge-small-en-v1.5,train,247,0.43068,0.0 +bge-small-en-v1.5,train,250,1.0,1.0 +bge-small-en-v1.5,train,251,1.0,1.0 +bge-small-en-v1.5,train,253,0.5,1.0 +bge-small-en-v1.5,train,254,0.5,1.0 +bge-small-en-v1.5,train,255,1.0,1.0 +bge-small-en-v1.5,train,256,1.0,1.0 +bge-small-en-v1.5,train,258,0.5,1.0 +bge-small-en-v1.5,train,259,0.0,0.0 +bge-small-en-v1.5,train,262,0.0,0.0 +bge-small-en-v1.5,train,263,0.71226,1.0 +bge-small-en-v1.5,train,264,1.0,1.0 +bge-small-en-v1.5,train,265,1.0,1.0 +bge-small-en-v1.5,train,266,0.63093,1.0 +bge-small-en-v1.5,train,267,0.80657,1.0 +bge-small-en-v1.5,train,272,1.0,1.0 +bge-small-en-v1.5,train,277,1.0,1.0 +bge-small-en-v1.5,train,278,1.0,1.0 +bge-small-en-v1.5,train,280,1.0,1.0 +bge-small-en-v1.5,train,281,1.0,1.0 +bge-small-en-v1.5,train,283,1.0,1.0 +bge-small-en-v1.5,train,285,0.63093,1.0 +bge-small-en-v1.5,train,286,1.0,1.0 +bge-small-en-v1.5,train,287,1.0,1.0 +bge-small-en-v1.5,train,290,1.0,1.0 +bge-small-en-v1.5,train,292,1.0,1.0 +bge-small-en-v1.5,train,293,0.5,1.0 +bge-small-en-v1.5,train,296,1.0,1.0 +bge-small-en-v1.5,train,299,1.0,1.0 +bge-small-en-v1.5,train,301,0.5,1.0 +bge-small-en-v1.5,train,304,1.0,1.0 +bge-small-en-v1.5,train,305,1.0,1.0 +bge-small-en-v1.5,train,306,1.0,1.0 +bge-small-en-v1.5,train,308,1.0,1.0 +bge-small-en-v1.5,train,309,1.0,1.0 +bge-small-en-v1.5,train,310,0.0,0.0 +bge-small-en-v1.5,train,313,0.0,0.0 +bge-small-en-v1.5,train,315,0.0,0.0 +bge-small-en-v1.5,train,316,0.63093,1.0 +bge-small-en-v1.5,train,317,1.0,1.0 +bge-small-en-v1.5,train,323,0.0,0.0 +bge-small-en-v1.5,train,325,0.5,1.0 +bge-small-en-v1.5,train,326,0.43068,0.0 +bge-small-en-v1.5,train,330,1.0,1.0 +bge-small-en-v1.5,train,331,1.0,1.0 +bge-small-en-v1.5,train,332,1.0,1.0 +bge-small-en-v1.5,train,333,1.0,1.0 +bge-small-en-v1.5,train,334,1.0,1.0 +bge-small-en-v1.5,train,335,1.0,1.0 +bge-small-en-v1.5,train,336,0.63093,1.0 +bge-small-en-v1.5,train,337,0.63093,1.0 +bge-small-en-v1.5,train,339,1.0,1.0 +bge-small-en-v1.5,train,340,1.0,1.0 +bge-small-en-v1.5,train,341,1.0,1.0 +bge-small-en-v1.5,train,342,1.0,1.0 +bge-small-en-v1.5,train,345,0.0,0.0 +bge-small-en-v1.5,train,346,0.63093,1.0 +bge-small-en-v1.5,train,347,0.5,1.0 +bge-small-en-v1.5,train,349,1.0,1.0 +bge-small-en-v1.5,train,351,0.0,0.0 +bge-small-en-v1.5,train,352,0.0,0.0 +bge-small-en-v1.5,train,355,0.87722,1.0 +bge-small-en-v1.5,train,356,1.0,1.0 +bge-small-en-v1.5,train,357,1.0,1.0 +bge-small-en-v1.5,train,358,1.0,1.0 +bge-small-en-v1.5,train,361,0.5,1.0 +bge-small-en-v1.5,train,363,0.0,0.0 +bge-small-en-v1.5,train,364,1.0,1.0 +bge-small-en-v1.5,train,365,1.0,1.0 +bge-small-en-v1.5,train,366,1.0,1.0 +bge-small-en-v1.5,train,367,1.0,1.0 +bge-small-en-v1.5,train,368,1.0,1.0 +bge-small-en-v1.5,train,369,1.0,1.0 +bge-small-en-v1.5,train,370,1.0,1.0 +bge-small-en-v1.5,train,371,1.0,1.0 +bge-small-en-v1.5,train,372,1.0,1.0 +bge-small-en-v1.5,train,375,1.0,1.0 +bge-small-en-v1.5,train,376,1.0,1.0 +bge-small-en-v1.5,train,377,1.0,1.0 +bge-small-en-v1.5,train,378,0.82077,1.0 +bge-small-en-v1.5,train,379,1.0,1.0 +bge-small-en-v1.5,train,381,1.0,1.0 +bge-small-en-v1.5,train,382,0.31546,0.0 +bge-small-en-v1.5,train,383,0.63093,1.0 +bge-small-en-v1.5,train,389,1.0,1.0 +bge-small-en-v1.5,train,390,1.0,1.0 +bge-small-en-v1.5,train,391,1.0,1.0 +bge-small-en-v1.5,train,392,1.0,1.0 +bge-small-en-v1.5,train,393,1.0,1.0 +bge-small-en-v1.5,train,394,0.5,1.0 +bge-small-en-v1.5,train,396,1.0,1.0 +bge-small-en-v1.5,train,397,1.0,1.0 +bge-small-en-v1.5,train,398,0.0,0.0 +bge-small-en-v1.5,train,400,1.0,1.0 +bge-small-en-v1.5,train,401,1.0,1.0 +bge-small-en-v1.5,train,403,1.0,1.0 +bge-small-en-v1.5,train,404,0.5,1.0 +bge-small-en-v1.5,train,406,0.0,0.0 +bge-small-en-v1.5,train,407,1.0,1.0 +bge-small-en-v1.5,train,413,0.43068,0.0 +bge-small-en-v1.5,train,414,0.33333,0.0 +bge-small-en-v1.5,train,416,0.63093,1.0 +bge-small-en-v1.5,train,417,0.63093,1.0 +bge-small-en-v1.5,train,418,0.0,0.0 +bge-small-en-v1.5,train,420,1.0,1.0 +bge-small-en-v1.5,train,422,0.0,0.0 +bge-small-en-v1.5,train,423,0.35621,0.0 +bge-small-en-v1.5,train,425,1.0,1.0 +bge-small-en-v1.5,train,426,0.5,1.0 +bge-small-en-v1.5,train,428,1.0,1.0 +bge-small-en-v1.5,train,429,1.0,1.0 +bge-small-en-v1.5,train,430,0.0,0.0 +bge-small-en-v1.5,train,432,1.0,1.0 +bge-small-en-v1.5,train,434,0.43068,0.0 +bge-small-en-v1.5,train,435,0.38685,0.0 +bge-small-en-v1.5,train,441,0.63093,1.0 +bge-small-en-v1.5,train,444,1.0,1.0 +bge-small-en-v1.5,train,445,1.0,1.0 +bge-small-en-v1.5,train,447,1.0,1.0 +bge-small-en-v1.5,train,448,1.0,1.0 +bge-small-en-v1.5,train,449,0.20438,0.0 +bge-small-en-v1.5,train,453,1.0,1.0 +bge-small-en-v1.5,train,454,1.0,1.0 +bge-small-en-v1.5,train,455,0.33333,0.0 +bge-small-en-v1.5,train,456,0.0,0.0 +bge-small-en-v1.5,train,458,1.0,1.0 +bge-small-en-v1.5,train,461,1.0,1.0 +bge-small-en-v1.5,train,463,0.43068,0.0 +bge-small-en-v1.5,train,466,0.5,1.0 +bge-small-en-v1.5,train,469,1.0,1.0 +bge-small-en-v1.5,train,470,0.5,1.0 +bge-small-en-v1.5,train,472,0.88546,1.0 +bge-small-en-v1.5,train,473,0.0,0.0 +bge-small-en-v1.5,train,474,0.0,0.0 +bge-small-en-v1.5,train,479,1.0,1.0 +bge-small-en-v1.5,train,480,1.0,1.0 +bge-small-en-v1.5,train,481,1.0,1.0 +bge-small-en-v1.5,train,482,1.0,1.0 +bge-small-en-v1.5,train,483,0.35621,0.0 +bge-small-en-v1.5,train,484,0.5,1.0 +bge-small-en-v1.5,train,485,1.0,1.0 +bge-small-en-v1.5,train,486,0.63093,1.0 +bge-small-en-v1.5,train,487,1.0,1.0 +bge-small-en-v1.5,train,488,1.0,1.0 +bge-small-en-v1.5,train,489,0.30103,0.0 +bge-small-en-v1.5,train,490,1.0,1.0 +bge-small-en-v1.5,train,492,1.0,1.0 +bge-small-en-v1.5,train,493,0.0,0.0 +bge-small-en-v1.5,train,494,0.33333,0.0 +bge-small-en-v1.5,train,495,1.0,1.0 +bge-small-en-v1.5,train,498,1.0,1.0 +bge-small-en-v1.5,train,499,1.0,1.0 +bge-small-en-v1.5,train,500,1.0,1.0 +bge-small-en-v1.5,train,504,1.0,1.0 +bge-small-en-v1.5,train,505,1.0,1.0 +bge-small-en-v1.5,train,506,1.0,1.0 +bge-small-en-v1.5,train,509,0.0,0.0 +bge-small-en-v1.5,train,515,1.0,1.0 +bge-small-en-v1.5,train,523,0.63093,1.0 +bge-small-en-v1.5,train,524,0.63093,1.0 +bge-small-en-v1.5,train,526,0.5,1.0 +bge-small-en-v1.5,train,529,0.57801,1.0 +bge-small-en-v1.5,train,530,0.47616,1.0 +bge-small-en-v1.5,train,531,0.43675,1.0 +bge-small-en-v1.5,train,537,1.0,1.0 +bge-small-en-v1.5,train,541,0.70392,1.0 +bge-small-en-v1.5,train,542,1.0,1.0 +bge-small-en-v1.5,train,545,0.30103,0.0 +bge-small-en-v1.5,train,547,1.0,1.0 +bge-small-en-v1.5,train,548,1.0,1.0 +bge-small-en-v1.5,train,550,0.0,0.0 +bge-small-en-v1.5,train,553,1.0,1.0 +bge-small-en-v1.5,train,555,0.5,1.0 +bge-small-en-v1.5,train,557,1.0,1.0 +bge-small-en-v1.5,train,559,0.0,0.0 +bge-small-en-v1.5,train,562,1.0,1.0 +bge-small-en-v1.5,train,563,1.0,1.0 +bge-small-en-v1.5,train,564,1.0,1.0 +bge-small-en-v1.5,train,565,1.0,1.0 +bge-small-en-v1.5,train,566,1.0,1.0 +bge-small-en-v1.5,train,568,1.0,1.0 +bge-small-en-v1.5,train,570,1.0,1.0 +bge-small-en-v1.5,train,571,1.0,1.0 +bge-small-en-v1.5,train,572,0.43068,0.0 +bge-small-en-v1.5,train,573,1.0,1.0 +bge-small-en-v1.5,train,574,1.0,1.0 +bge-small-en-v1.5,train,576,1.0,1.0 +bge-small-en-v1.5,train,579,1.0,1.0 +bge-small-en-v1.5,train,580,1.0,1.0 +bge-small-en-v1.5,train,582,1.0,1.0 +bge-small-en-v1.5,train,584,1.0,1.0 +bge-small-en-v1.5,train,585,1.0,1.0 +bge-small-en-v1.5,train,588,0.5,1.0 +bge-small-en-v1.5,train,590,1.0,1.0 +bge-small-en-v1.5,train,591,1.0,1.0 +bge-small-en-v1.5,train,592,1.0,1.0 +bge-small-en-v1.5,train,594,1.0,1.0 +bge-small-en-v1.5,train,595,1.0,1.0 +bge-small-en-v1.5,train,600,1.0,1.0 +bge-small-en-v1.5,train,601,1.0,1.0 +bge-small-en-v1.5,train,602,0.0,0.0 +bge-small-en-v1.5,train,603,0.43068,0.0 +bge-small-en-v1.5,train,606,0.33333,0.0 +bge-small-en-v1.5,train,607,1.0,1.0 +bge-small-en-v1.5,train,609,0.0,0.0 +bge-small-en-v1.5,train,610,0.0,0.0 +bge-small-en-v1.5,train,611,1.0,1.0 +bge-small-en-v1.5,train,612,1.0,1.0 +bge-small-en-v1.5,train,614,1.0,1.0 +bge-small-en-v1.5,train,615,1.0,1.0 +bge-small-en-v1.5,train,616,0.0,0.0 +bge-small-en-v1.5,train,617,0.0,0.0 +bge-small-en-v1.5,train,618,1.0,1.0 +bge-small-en-v1.5,train,620,0.0,0.0 +bge-small-en-v1.5,train,621,1.0,1.0 +bge-small-en-v1.5,train,622,0.63093,1.0 +bge-small-en-v1.5,train,624,1.0,1.0 +bge-small-en-v1.5,train,625,1.0,1.0 +bge-small-en-v1.5,train,626,1.0,1.0 +bge-small-en-v1.5,train,631,1.0,1.0 +bge-small-en-v1.5,train,632,1.0,1.0 +bge-small-en-v1.5,train,633,1.0,1.0 +bge-small-en-v1.5,train,635,1.0,1.0 +bge-small-en-v1.5,train,638,0.63093,1.0 +bge-small-en-v1.5,train,640,0.63093,1.0 +bge-small-en-v1.5,train,642,0.63093,1.0 +bge-small-en-v1.5,train,643,1.0,1.0 +bge-small-en-v1.5,train,645,0.5,1.0 +bge-small-en-v1.5,train,646,0.33333,0.0 +bge-small-en-v1.5,train,647,1.0,1.0 +bge-small-en-v1.5,train,648,1.0,1.0 +bge-small-en-v1.5,train,650,1.0,1.0 +bge-small-en-v1.5,train,651,1.0,1.0 +bge-small-en-v1.5,train,652,1.0,1.0 +bge-small-en-v1.5,train,653,0.63093,1.0 +bge-small-en-v1.5,train,654,1.0,1.0 +bge-small-en-v1.5,train,655,1.0,1.0 +bge-small-en-v1.5,train,657,1.0,1.0 +bge-small-en-v1.5,train,658,1.0,1.0 +bge-small-en-v1.5,train,661,1.0,1.0 +bge-small-en-v1.5,train,662,1.0,1.0 +bge-small-en-v1.5,train,663,1.0,1.0 +bge-small-en-v1.5,train,665,1.0,1.0 +bge-small-en-v1.5,train,666,1.0,1.0 +bge-small-en-v1.5,train,667,1.0,1.0 +bge-small-en-v1.5,train,668,1.0,1.0 +bge-small-en-v1.5,train,669,1.0,1.0 +bge-small-en-v1.5,train,670,0.63093,1.0 +bge-small-en-v1.5,train,671,1.0,1.0 +bge-small-en-v1.5,train,672,1.0,1.0 +bge-small-en-v1.5,train,673,0.63093,1.0 +bge-small-en-v1.5,train,676,0.5,1.0 +bge-small-en-v1.5,train,677,0.38685,0.0 +bge-small-en-v1.5,train,679,0.0,0.0 +bge-small-en-v1.5,train,680,1.0,1.0 +bge-small-en-v1.5,train,681,1.0,1.0 +bge-small-en-v1.5,train,682,1.0,1.0 +bge-small-en-v1.5,train,683,1.0,1.0 +bge-small-en-v1.5,train,685,1.0,1.0 +bge-small-en-v1.5,train,686,0.0,0.0 +bge-small-en-v1.5,train,687,0.5,1.0 +bge-small-en-v1.5,train,688,0.63093,1.0 +bge-small-en-v1.5,train,689,0.0,0.0 +bge-small-en-v1.5,train,694,1.0,1.0 +bge-small-en-v1.5,train,696,0.63093,1.0 +bge-small-en-v1.5,train,698,0.5,1.0 +bge-small-en-v1.5,train,703,1.0,1.0 +bge-small-en-v1.5,train,704,0.0,0.0 +bge-small-en-v1.5,train,705,1.0,1.0 +bge-small-en-v1.5,train,709,1.0,1.0 +bge-small-en-v1.5,train,710,0.63093,1.0 +bge-small-en-v1.5,train,713,0.0,0.0 +bge-small-en-v1.5,train,714,0.0,0.0 +bge-small-en-v1.5,train,717,1.0,1.0 +bge-small-en-v1.5,train,724,1.0,1.0 +bge-small-en-v1.5,train,726,0.83155,1.0 +bge-small-en-v1.5,train,730,1.0,1.0 +bge-small-en-v1.5,train,732,0.0,0.0 +bge-small-en-v1.5,train,733,0.33333,0.0 +bge-small-en-v1.5,train,734,1.0,1.0 +bge-small-en-v1.5,train,736,1.0,1.0 +bge-small-en-v1.5,train,737,0.91972,1.0 +bge-small-en-v1.5,train,738,0.88546,1.0 +bge-small-en-v1.5,train,739,0.0,0.0 +bge-small-en-v1.5,train,740,1.0,1.0 +bge-small-en-v1.5,train,745,1.0,1.0 +bge-small-en-v1.5,train,746,1.0,1.0 +bge-small-en-v1.5,train,747,1.0,1.0 +bge-small-en-v1.5,train,748,1.0,1.0 +bge-small-en-v1.5,train,749,0.33333,0.0 +bge-small-en-v1.5,train,751,1.0,1.0 +bge-small-en-v1.5,train,752,1.0,1.0 +bge-small-en-v1.5,train,753,0.0,0.0 +bge-small-en-v1.5,train,755,0.0,0.0 +bge-small-en-v1.5,train,757,1.0,1.0 +bge-small-en-v1.5,train,758,0.38685,0.0 +bge-small-en-v1.5,train,760,1.0,1.0 +bge-small-en-v1.5,train,761,0.28906,0.0 +bge-small-en-v1.5,train,762,1.0,1.0 +bge-small-en-v1.5,train,764,1.0,1.0 +bge-small-en-v1.5,train,765,1.0,1.0 +bge-small-en-v1.5,train,766,0.63093,1.0 +bge-small-en-v1.5,train,767,1.0,1.0 +bge-small-en-v1.5,train,771,1.0,1.0 +bge-small-en-v1.5,train,772,1.0,1.0 +bge-small-en-v1.5,train,774,0.38685,0.0 +bge-small-en-v1.5,train,776,0.35621,0.0 +bge-small-en-v1.5,train,777,0.35621,0.0 +bge-small-en-v1.5,train,778,1.0,1.0 +bge-small-en-v1.5,train,779,1.0,1.0 +bge-small-en-v1.5,train,780,1.0,1.0 +bge-small-en-v1.5,train,782,0.5,1.0 +bge-small-en-v1.5,train,787,1.0,1.0 +bge-small-en-v1.5,train,788,1.0,1.0 +bge-small-en-v1.5,train,789,1.0,1.0 +bge-small-en-v1.5,train,790,1.0,1.0 +bge-small-en-v1.5,train,791,1.0,1.0 +bge-small-en-v1.5,train,792,1.0,1.0 +bge-small-en-v1.5,train,795,1.0,1.0 +bge-small-en-v1.5,train,797,1.0,1.0 +bge-small-en-v1.5,train,798,0.63093,1.0 +bge-small-en-v1.5,train,799,1.0,1.0 +bge-small-en-v1.5,train,801,1.0,1.0 +bge-small-en-v1.5,train,802,1.0,1.0 +bge-small-en-v1.5,train,803,1.0,1.0 +bge-small-en-v1.5,train,804,1.0,1.0 +bge-small-en-v1.5,train,807,1.0,1.0 +bge-small-en-v1.5,train,810,1.0,1.0 +bge-small-en-v1.5,train,812,1.0,1.0 +bge-small-en-v1.5,train,813,1.0,1.0 +bge-small-en-v1.5,train,815,1.0,1.0 +bge-small-en-v1.5,train,816,1.0,1.0 +bge-small-en-v1.5,train,817,0.63093,1.0 +bge-small-en-v1.5,train,818,0.63093,1.0 +bge-small-en-v1.5,train,822,0.63093,1.0 +bge-small-en-v1.5,train,825,0.63093,1.0 +bge-small-en-v1.5,train,826,0.5,1.0 +bge-small-en-v1.5,train,828,0.30103,0.0 +bge-small-en-v1.5,train,835,0.43068,0.0 +bge-small-en-v1.5,train,838,0.0,0.0 +bge-small-en-v1.5,train,840,1.0,1.0 +bge-small-en-v1.5,train,841,1.0,1.0 +bge-small-en-v1.5,train,844,1.0,1.0 +bge-small-en-v1.5,train,846,1.0,1.0 +bge-small-en-v1.5,train,848,1.0,1.0 +bge-small-en-v1.5,train,853,1.0,1.0 +bge-small-en-v1.5,train,854,1.0,1.0 +bge-small-en-v1.5,train,855,1.0,1.0 +bge-small-en-v1.5,train,856,0.0,0.0 +bge-small-en-v1.5,train,857,0.0,0.0 +bge-small-en-v1.5,train,858,0.5,1.0 +bge-small-en-v1.5,train,860,1.0,1.0 +bge-small-en-v1.5,train,861,1.0,1.0 +bge-small-en-v1.5,train,863,0.65092,1.0 +bge-small-en-v1.5,train,866,0.63093,1.0 +bge-small-en-v1.5,train,867,1.0,1.0 +bge-small-en-v1.5,train,871,0.5,1.0 +bge-small-en-v1.5,train,876,1.0,1.0 +bge-small-en-v1.5,train,877,1.0,1.0 +bge-small-en-v1.5,train,881,1.0,1.0 +bge-small-en-v1.5,train,883,1.0,1.0 +bge-small-en-v1.5,train,884,1.0,1.0 +bge-small-en-v1.5,train,885,1.0,1.0 +bge-small-en-v1.5,train,886,1.0,1.0 +bge-small-en-v1.5,train,890,0.63093,1.0 +bge-small-en-v1.5,train,891,0.63093,1.0 +bge-small-en-v1.5,train,893,1.0,1.0 +bge-small-en-v1.5,train,894,0.0,0.0 +bge-small-en-v1.5,train,895,0.0,0.0 +bge-small-en-v1.5,train,896,0.0,0.0 +bge-small-en-v1.5,train,897,0.0,0.0 +bge-small-en-v1.5,train,898,0.61315,1.0 +bge-small-en-v1.5,train,899,0.61315,1.0 +bge-small-en-v1.5,train,900,0.0,0.0 +bge-small-en-v1.5,train,901,1.0,1.0 +bge-small-en-v1.5,train,902,1.0,1.0 +bge-small-en-v1.5,train,908,1.0,1.0 +bge-small-en-v1.5,train,909,1.0,1.0 +bge-small-en-v1.5,train,910,1.0,1.0 +bge-small-en-v1.5,train,912,1.0,1.0 +bge-small-en-v1.5,train,916,0.0,0.0 +bge-small-en-v1.5,train,917,1.0,1.0 +bge-small-en-v1.5,train,919,1.0,1.0 +bge-small-en-v1.5,train,923,1.0,1.0 +bge-small-en-v1.5,train,925,1.0,1.0 +bge-small-en-v1.5,train,926,1.0,1.0 +bge-small-en-v1.5,train,927,1.0,1.0 +bge-small-en-v1.5,train,928,1.0,1.0 +bge-small-en-v1.5,train,929,1.0,1.0 +bge-small-en-v1.5,train,930,1.0,1.0 +bge-small-en-v1.5,train,933,1.0,1.0 +bge-small-en-v1.5,train,934,0.5,1.0 +bge-small-en-v1.5,train,935,0.38685,0.0 +bge-small-en-v1.5,train,938,0.35621,0.0 +bge-small-en-v1.5,train,939,0.43068,0.0 +bge-small-en-v1.5,train,940,1.0,1.0 +bge-small-en-v1.5,train,941,1.0,1.0 +bge-small-en-v1.5,train,942,0.0,0.0 +bge-small-en-v1.5,train,944,1.0,1.0 +bge-small-en-v1.5,train,945,0.671,1.0 +bge-small-en-v1.5,train,946,0.69339,1.0 +bge-small-en-v1.5,train,949,0.0,0.0 +bge-small-en-v1.5,train,951,1.0,1.0 +bge-small-en-v1.5,train,952,1.0,1.0 +bge-small-en-v1.5,train,953,1.0,1.0 +bge-small-en-v1.5,train,954,1.0,1.0 +bge-small-en-v1.5,train,955,0.57064,1.0 +bge-small-en-v1.5,train,959,1.0,1.0 +bge-small-en-v1.5,train,962,0.71226,1.0 +bge-small-en-v1.5,train,963,0.59124,1.0 +bge-small-en-v1.5,train,964,0.59124,1.0 +bge-small-en-v1.5,train,965,1.0,1.0 +bge-small-en-v1.5,train,969,0.85034,1.0 +bge-small-en-v1.5,train,970,0.83155,1.0 +bge-small-en-v1.5,train,972,0.89889,1.0 +bge-small-en-v1.5,train,973,0.62552,1.0 +bge-small-en-v1.5,train,976,0.0,0.0 +bge-small-en-v1.5,train,977,0.0,0.0 +bge-small-en-v1.5,train,978,0.0,0.0 +bge-small-en-v1.5,train,979,1.0,1.0 +bge-small-en-v1.5,train,980,1.0,1.0 +bge-small-en-v1.5,train,984,1.0,1.0 +bge-small-en-v1.5,train,988,0.0,0.0 +bge-small-en-v1.5,train,989,0.63093,1.0 +bge-small-en-v1.5,train,990,1.0,1.0 +bge-small-en-v1.5,train,992,1.0,1.0 +bge-small-en-v1.5,train,994,1.0,1.0 +bge-small-en-v1.5,train,996,1.0,1.0 +bge-small-en-v1.5,train,997,1.0,1.0 +bge-small-en-v1.5,train,998,1.0,1.0 +bge-small-en-v1.5,train,999,1.0,1.0 +bge-small-en-v1.5,train,1000,1.0,1.0 +bge-small-en-v1.5,train,1001,1.0,1.0 +bge-small-en-v1.5,train,1002,0.0,0.0 +bge-small-en-v1.5,train,1003,0.46928,1.0 +bge-small-en-v1.5,train,1004,0.96747,1.0 +bge-small-en-v1.5,train,1005,0.96747,1.0 +bge-small-en-v1.5,train,1006,1.0,1.0 +bge-small-en-v1.5,train,1008,0.0,0.0 +bge-small-en-v1.5,train,1009,1.0,1.0 +bge-small-en-v1.5,train,1011,1.0,1.0 +bge-small-en-v1.5,train,1015,1.0,1.0 +bge-small-en-v1.5,train,1016,1.0,1.0 +bge-small-en-v1.5,train,1018,0.63093,1.0 +bge-small-en-v1.5,train,1023,1.0,1.0 +bge-small-en-v1.5,train,1025,1.0,1.0 +bge-small-en-v1.5,train,1026,0.63093,1.0 +bge-small-en-v1.5,train,1027,0.63093,1.0 +bge-small-en-v1.5,train,1028,1.0,1.0 +bge-small-en-v1.5,train,1030,1.0,1.0 +bge-small-en-v1.5,train,1031,1.0,1.0 +bge-small-en-v1.5,train,1032,0.63093,1.0 +bge-small-en-v1.5,train,1033,1.0,1.0 +bge-small-en-v1.5,train,1034,0.63093,1.0 +bge-small-en-v1.5,train,1035,0.63093,1.0 +bge-small-en-v1.5,train,1036,0.0,0.0 +bge-small-en-v1.5,train,1037,1.0,1.0 +bge-small-en-v1.5,train,1038,1.0,1.0 +bge-small-en-v1.5,train,1040,0.378,0.0 +bge-small-en-v1.5,train,1042,1.0,1.0 +bge-small-en-v1.5,train,1043,1.0,1.0 +bge-small-en-v1.5,train,1044,1.0,1.0 +bge-small-en-v1.5,train,1045,1.0,1.0 +bge-small-en-v1.5,train,1046,0.0,0.0 +bge-small-en-v1.5,train,1047,0.63093,1.0 +bge-small-en-v1.5,train,1048,0.38685,0.0 +bge-small-en-v1.5,train,1050,1.0,1.0 +bge-small-en-v1.5,train,1052,0.43068,0.0 +bge-small-en-v1.5,train,1053,0.43068,0.0 +bge-small-en-v1.5,train,1054,1.0,1.0 +bge-small-en-v1.5,train,1055,1.0,1.0 +bge-small-en-v1.5,train,1056,1.0,1.0 +bge-small-en-v1.5,train,1058,1.0,1.0 +bge-small-en-v1.5,train,1065,1.0,1.0 +bge-small-en-v1.5,train,1067,0.0,0.0 +bge-small-en-v1.5,train,1068,0.0,0.0 +bge-small-en-v1.5,train,1069,1.0,1.0 +bge-small-en-v1.5,train,1070,0.33333,0.0 +bge-small-en-v1.5,train,1072,1.0,1.0 +bge-small-en-v1.5,train,1073,1.0,1.0 +bge-small-en-v1.5,train,1074,0.43068,0.0 +bge-small-en-v1.5,train,1075,0.63093,1.0 +bge-small-en-v1.5,train,1081,1.0,1.0 +bge-small-en-v1.5,train,1084,1.0,1.0 +bge-small-en-v1.5,train,1085,1.0,1.0 +bge-small-en-v1.5,train,1087,1.0,1.0 +bge-small-en-v1.5,train,1090,1.0,1.0 +bge-small-en-v1.5,train,1091,0.0,0.0 +bge-small-en-v1.5,train,1096,0.0,0.0 +bge-small-en-v1.5,train,1097,1.0,1.0 +bge-small-en-v1.5,train,1098,0.0,0.0 +bge-small-en-v1.5,train,1101,1.0,1.0 +bge-small-en-v1.5,train,1102,1.0,1.0 +bge-small-en-v1.5,train,1103,1.0,1.0 +bge-small-en-v1.5,train,1105,1.0,1.0 +bge-small-en-v1.5,train,1106,1.0,1.0 +bge-small-en-v1.5,train,1109,0.0,0.0 +bge-small-en-v1.5,train,1111,0.0,0.0 +bge-small-en-v1.5,train,1112,0.30103,0.0 +bge-small-en-v1.5,train,1114,1.0,1.0 +bge-small-en-v1.5,train,1115,1.0,1.0 +bge-small-en-v1.5,train,1118,1.0,1.0 +bge-small-en-v1.5,train,1119,0.73283,1.0 +bge-small-en-v1.5,train,1120,1.0,1.0 +bge-small-en-v1.5,train,1125,1.0,1.0 +bge-small-en-v1.5,train,1126,1.0,1.0 +bge-small-en-v1.5,train,1127,0.0,0.0 +bge-small-en-v1.5,train,1128,0.18458,0.0 +bge-small-en-v1.5,train,1133,1.0,1.0 +bge-small-en-v1.5,train,1134,1.0,1.0 +bge-small-en-v1.5,train,1135,1.0,1.0 +bge-small-en-v1.5,train,1136,1.0,1.0 +bge-small-en-v1.5,train,1138,0.28906,0.0 +bge-small-en-v1.5,train,1139,1.0,1.0 +bge-small-en-v1.5,train,1141,1.0,1.0 +bge-small-en-v1.5,train,1142,1.0,1.0 +bge-small-en-v1.5,train,1145,1.0,1.0 +bge-small-en-v1.5,train,1148,1.0,1.0 +bge-small-en-v1.5,train,1153,1.0,1.0 +bge-small-en-v1.5,train,1156,1.0,1.0 +bge-small-en-v1.5,train,1157,1.0,1.0 +bge-small-en-v1.5,train,1158,1.0,1.0 +bge-small-en-v1.5,train,1159,1.0,1.0 +bge-small-en-v1.5,train,1161,0.35621,0.0 +bge-small-en-v1.5,train,1162,1.0,1.0 +bge-small-en-v1.5,train,1164,0.5,1.0 +bge-small-en-v1.5,train,1165,0.35621,0.0 +bge-small-en-v1.5,train,1166,0.5,1.0 +bge-small-en-v1.5,train,1168,0.43068,0.0 +bge-small-en-v1.5,train,1169,1.0,1.0 +bge-small-en-v1.5,train,1170,1.0,1.0 +bge-small-en-v1.5,train,1171,1.0,1.0 +bge-small-en-v1.5,train,1173,1.0,1.0 +bge-small-en-v1.5,train,1174,0.0,0.0 +bge-small-en-v1.5,train,1176,1.0,1.0 +bge-small-en-v1.5,train,1177,1.0,1.0 +bge-small-en-v1.5,train,1178,0.0,0.0 +bge-small-en-v1.5,train,1181,1.0,1.0 +bge-small-en-v1.5,train,1182,1.0,1.0 +bge-small-en-v1.5,train,1183,1.0,1.0 +bge-small-en-v1.5,train,1184,1.0,1.0 +bge-small-en-v1.5,train,1186,0.31546,0.0 +bge-small-en-v1.5,train,1188,1.0,1.0 +bge-small-en-v1.5,train,1190,0.0,0.0 +bge-small-en-v1.5,train,1193,0.5,1.0 +bge-small-en-v1.5,train,1195,0.63093,1.0 +bge-small-en-v1.5,train,1205,1.0,1.0 +bge-small-en-v1.5,train,1206,1.0,1.0 +bge-small-en-v1.5,train,1208,1.0,1.0 +bge-small-en-v1.5,train,1209,0.35621,0.0 +bge-small-en-v1.5,train,1210,1.0,1.0 +bge-small-en-v1.5,train,1211,1.0,1.0 +bge-small-en-v1.5,train,1212,0.0,0.0 +bge-small-en-v1.5,train,1214,0.0,0.0 +bge-small-en-v1.5,train,1215,0.43068,0.0 +bge-small-en-v1.5,train,1218,0.0,0.0 +bge-small-en-v1.5,train,1219,0.85034,1.0 +bge-small-en-v1.5,train,1220,1.0,1.0 +bge-small-en-v1.5,train,1223,1.0,1.0 +bge-small-en-v1.5,train,1224,0.59124,1.0 +bge-small-en-v1.5,train,1227,0.63093,1.0 +bge-small-en-v1.5,train,1228,0.63093,1.0 +bge-small-en-v1.5,train,1229,0.63093,1.0 +bge-small-en-v1.5,train,1230,1.0,1.0 +bge-small-en-v1.5,train,1231,1.0,1.0 +bge-small-en-v1.5,train,1234,1.0,1.0 +bge-small-en-v1.5,train,1235,0.5,1.0 +bge-small-en-v1.5,train,1236,0.5,1.0 +bge-small-en-v1.5,train,1237,1.0,1.0 +bge-small-en-v1.5,train,1238,1.0,1.0 +bge-small-en-v1.5,train,1239,0.81753,1.0 +bge-small-en-v1.5,train,1244,0.35621,0.0 +bge-small-en-v1.5,train,1246,0.38685,0.0 +bge-small-en-v1.5,train,1247,0.35621,0.0 +bge-small-en-v1.5,train,1248,1.0,1.0 +bge-small-en-v1.5,train,1249,1.0,1.0 +bge-small-en-v1.5,train,1253,0.0,0.0 +bge-small-en-v1.5,train,1254,1.0,1.0 +bge-small-en-v1.5,train,1255,0.5,1.0 +bge-small-en-v1.5,train,1257,0.0,0.0 +bge-small-en-v1.5,train,1258,0.0,0.0 +bge-small-en-v1.5,train,1260,1.0,1.0 +bge-small-en-v1.5,train,1261,1.0,1.0 +bge-small-en-v1.5,train,1263,1.0,1.0 +bge-small-en-v1.5,train,1265,1.0,1.0 +bge-small-en-v1.5,train,1268,0.5,1.0 +bge-small-en-v1.5,train,1269,1.0,1.0 +bge-small-en-v1.5,train,1275,0.63093,1.0 +bge-small-en-v1.5,train,1276,0.63093,1.0 +bge-small-en-v1.5,train,1284,1.0,1.0 +bge-small-en-v1.5,train,1288,1.0,1.0 +bge-small-en-v1.5,train,1289,1.0,1.0 +bge-small-en-v1.5,train,1291,1.0,1.0 +bge-small-en-v1.5,train,1293,1.0,1.0 +bge-small-en-v1.5,train,1294,0.65092,1.0 +bge-small-en-v1.5,train,1295,1.0,1.0 +bge-small-en-v1.5,train,1297,1.0,1.0 +bge-small-en-v1.5,train,1300,0.0,0.0 +bge-small-en-v1.5,train,1302,0.63093,1.0 +bge-small-en-v1.5,train,1304,0.63093,1.0 +bge-small-en-v1.5,train,1305,1.0,1.0 +bge-small-en-v1.5,train,1306,0.42177,0.0 +bge-small-en-v1.5,train,1307,0.0,0.0 +bge-small-en-v1.5,train,1308,0.0,0.0 +bge-small-en-v1.5,train,1309,0.0,0.0 +bge-small-en-v1.5,train,1310,1.0,1.0 +bge-small-en-v1.5,train,1311,0.30103,0.0 +bge-small-en-v1.5,train,1312,0.5,1.0 +bge-small-en-v1.5,train,1314,0.56409,1.0 +bge-small-en-v1.5,train,1315,0.56409,1.0 +bge-small-en-v1.5,train,1322,1.0,1.0 +bge-small-en-v1.5,train,1323,1.0,1.0 +bge-small-en-v1.5,train,1324,1.0,1.0 +bge-small-en-v1.5,train,1325,0.0,0.0 +bge-small-en-v1.5,train,1327,1.0,1.0 +bge-small-en-v1.5,train,1328,1.0,1.0 +bge-small-en-v1.5,train,1330,0.0,0.0 +bge-small-en-v1.5,train,1331,0.5,1.0 +bge-small-en-v1.5,train,1333,1.0,1.0 +bge-small-en-v1.5,train,1334,0.67139,1.0 +bge-small-en-v1.5,train,1340,1.0,1.0 +bge-small-en-v1.5,train,1341,1.0,1.0 +bge-small-en-v1.5,train,1342,0.0,0.0 +bge-small-en-v1.5,train,1345,0.0,0.0 +bge-small-en-v1.5,train,1346,0.38685,0.0 +bge-small-en-v1.5,train,1347,1.0,1.0 +bge-small-en-v1.5,train,1348,1.0,1.0 +bge-small-en-v1.5,train,1349,1.0,1.0 +bge-small-en-v1.5,train,1350,1.0,1.0 +bge-small-en-v1.5,train,1351,1.0,1.0 +bge-small-en-v1.5,train,1353,0.63093,1.0 +bge-small-en-v1.5,train,1355,0.63093,1.0 +bge-small-en-v1.5,train,1356,1.0,1.0 +bge-small-en-v1.5,train,1360,1.0,1.0 +bge-small-en-v1.5,train,1361,0.85034,1.0 +bge-small-en-v1.5,train,1364,0.0,0.0 +bge-small-en-v1.5,train,1366,1.0,1.0 +bge-small-en-v1.5,train,1367,1.0,1.0 +bge-small-en-v1.5,train,1371,1.0,1.0 +bge-small-en-v1.5,train,1372,1.0,1.0 +bge-small-en-v1.5,train,1373,1.0,1.0 +bge-small-en-v1.5,train,1374,1.0,1.0 +bge-small-en-v1.5,train,1375,1.0,1.0 +bge-small-en-v1.5,train,1376,0.5,1.0 +bge-small-en-v1.5,train,1378,1.0,1.0 +bge-small-en-v1.5,train,1380,0.67973,1.0 +bge-small-en-v1.5,train,1381,1.0,1.0 +bge-small-en-v1.5,train,1383,1.0,1.0 +bge-small-en-v1.5,train,1386,1.0,1.0 +bge-small-en-v1.5,train,1387,1.0,1.0 +bge-small-en-v1.5,train,1390,0.63093,1.0 +bge-small-en-v1.5,train,1391,1.0,1.0 +bge-small-en-v1.5,train,1392,1.0,1.0 +bge-small-en-v1.5,train,1393,0.87722,1.0 +bge-small-en-v1.5,train,1394,0.31546,0.0 +bge-small-en-v1.5,train,1397,0.63093,1.0 +bge-small-en-v1.5,train,1398,0.63093,1.0 +bge-small-en-v1.5,train,1400,0.63093,1.0 +bge-small-en-v1.5,train,1401,0.63093,1.0 +bge-small-en-v1.5,train,1402,0.0,0.0 +bge-small-en-v1.5,train,1403,1.0,1.0 +bge-small-en-v1.5,train,1404,1.0,1.0 +bge-small-en-v1.5,train,1405,1.0,1.0 +bge-small-en-v1.5,train,1406,1.0,1.0 +bge-small-en-v1.5,train,1407,0.17724,0.0 +bge-small-en-v1.5,test,1,0.0,0.0 +bge-small-en-v1.5,test,3,0.38685,0.0 +bge-small-en-v1.5,test,5,0.63093,1.0 +bge-small-en-v1.5,test,13,0.0,0.0 +bge-small-en-v1.5,test,36,0.41444,0.0 +bge-small-en-v1.5,test,42,1.0,1.0 +bge-small-en-v1.5,test,48,0.43068,0.0 +bge-small-en-v1.5,test,49,1.0,1.0 +bge-small-en-v1.5,test,50,1.0,1.0 +bge-small-en-v1.5,test,51,1.0,1.0 +bge-small-en-v1.5,test,53,1.0,1.0 +bge-small-en-v1.5,test,54,1.0,1.0 +bge-small-en-v1.5,test,56,1.0,1.0 +bge-small-en-v1.5,test,57,1.0,1.0 +bge-small-en-v1.5,test,70,0.42279,0.0 +bge-small-en-v1.5,test,72,1.0,1.0 +bge-small-en-v1.5,test,75,1.0,1.0 +bge-small-en-v1.5,test,94,1.0,1.0 +bge-small-en-v1.5,test,99,0.63093,1.0 +bge-small-en-v1.5,test,100,0.63093,1.0 +bge-small-en-v1.5,test,113,1.0,1.0 +bge-small-en-v1.5,test,115,1.0,1.0 +bge-small-en-v1.5,test,118,1.0,1.0 +bge-small-en-v1.5,test,124,1.0,1.0 +bge-small-en-v1.5,test,127,1.0,1.0 +bge-small-en-v1.5,test,128,0.0,0.0 +bge-small-en-v1.5,test,129,1.0,1.0 +bge-small-en-v1.5,test,130,1.0,1.0 +bge-small-en-v1.5,test,132,0.0,0.0 +bge-small-en-v1.5,test,133,0.78601,1.0 +bge-small-en-v1.5,test,137,1.0,1.0 +bge-small-en-v1.5,test,141,0.87722,1.0 +bge-small-en-v1.5,test,142,1.0,1.0 +bge-small-en-v1.5,test,143,1.0,1.0 +bge-small-en-v1.5,test,146,1.0,1.0 +bge-small-en-v1.5,test,148,1.0,1.0 +bge-small-en-v1.5,test,163,1.0,1.0 +bge-small-en-v1.5,test,171,1.0,1.0 +bge-small-en-v1.5,test,179,0.97093,1.0 +bge-small-en-v1.5,test,180,1.0,1.0 +bge-small-en-v1.5,test,183,0.63093,1.0 +bge-small-en-v1.5,test,185,0.5,1.0 +bge-small-en-v1.5,test,198,0.30103,0.0 +bge-small-en-v1.5,test,208,1.0,1.0 +bge-small-en-v1.5,test,212,0.35621,0.0 +bge-small-en-v1.5,test,213,0.31546,0.0 +bge-small-en-v1.5,test,216,1.0,1.0 +bge-small-en-v1.5,test,217,1.0,1.0 +bge-small-en-v1.5,test,218,1.0,1.0 +bge-small-en-v1.5,test,219,1.0,1.0 +bge-small-en-v1.5,test,230,1.0,1.0 +bge-small-en-v1.5,test,232,0.63093,1.0 +bge-small-en-v1.5,test,233,1.0,1.0 +bge-small-en-v1.5,test,236,1.0,1.0 +bge-small-en-v1.5,test,237,0.5,1.0 +bge-small-en-v1.5,test,238,0.5,1.0 +bge-small-en-v1.5,test,239,0.0,0.0 +bge-small-en-v1.5,test,248,1.0,1.0 +bge-small-en-v1.5,test,249,1.0,1.0 +bge-small-en-v1.5,test,261,0.21841,0.0 +bge-small-en-v1.5,test,268,1.0,1.0 +bge-small-en-v1.5,test,269,1.0,1.0 +bge-small-en-v1.5,test,274,1.0,1.0 +bge-small-en-v1.5,test,275,0.63644,1.0 +bge-small-en-v1.5,test,279,1.0,1.0 +bge-small-en-v1.5,test,294,0.30103,0.0 +bge-small-en-v1.5,test,295,0.31546,0.0 +bge-small-en-v1.5,test,298,1.0,1.0 +bge-small-en-v1.5,test,300,0.63093,1.0 +bge-small-en-v1.5,test,303,0.0,0.0 +bge-small-en-v1.5,test,312,0.0,0.0 +bge-small-en-v1.5,test,314,0.0,0.0 +bge-small-en-v1.5,test,324,0.0,0.0 +bge-small-en-v1.5,test,327,1.0,1.0 +bge-small-en-v1.5,test,338,1.0,1.0 +bge-small-en-v1.5,test,343,1.0,1.0 +bge-small-en-v1.5,test,350,1.0,1.0 +bge-small-en-v1.5,test,354,1.0,1.0 +bge-small-en-v1.5,test,362,1.0,1.0 +bge-small-en-v1.5,test,380,1.0,1.0 +bge-small-en-v1.5,test,384,0.63093,1.0 +bge-small-en-v1.5,test,385,0.56409,1.0 +bge-small-en-v1.5,test,386,1.0,1.0 +bge-small-en-v1.5,test,388,1.0,1.0 +bge-small-en-v1.5,test,399,1.0,1.0 +bge-small-en-v1.5,test,410,1.0,1.0 +bge-small-en-v1.5,test,411,1.0,1.0 +bge-small-en-v1.5,test,415,0.38685,0.0 +bge-small-en-v1.5,test,421,0.0,0.0 +bge-small-en-v1.5,test,431,0.0,0.0 +bge-small-en-v1.5,test,436,1.0,1.0 +bge-small-en-v1.5,test,437,0.0,0.0 +bge-small-en-v1.5,test,439,1.0,1.0 +bge-small-en-v1.5,test,440,1.0,1.0 +bge-small-en-v1.5,test,443,1.0,1.0 +bge-small-en-v1.5,test,452,0.61315,1.0 +bge-small-en-v1.5,test,475,0.0,0.0 +bge-small-en-v1.5,test,478,1.0,1.0 +bge-small-en-v1.5,test,491,1.0,1.0 +bge-small-en-v1.5,test,501,1.0,1.0 +bge-small-en-v1.5,test,502,0.0,0.0 +bge-small-en-v1.5,test,507,0.38685,0.0 +bge-small-en-v1.5,test,508,0.0,0.0 +bge-small-en-v1.5,test,513,0.63093,1.0 +bge-small-en-v1.5,test,514,1.0,1.0 +bge-small-en-v1.5,test,516,1.0,1.0 +bge-small-en-v1.5,test,517,0.0,0.0 +bge-small-en-v1.5,test,521,1.0,1.0 +bge-small-en-v1.5,test,525,1.0,1.0 +bge-small-en-v1.5,test,527,0.43068,0.0 +bge-small-en-v1.5,test,528,1.0,1.0 +bge-small-en-v1.5,test,532,1.0,1.0 +bge-small-en-v1.5,test,533,1.0,1.0 +bge-small-en-v1.5,test,535,0.63093,1.0 +bge-small-en-v1.5,test,536,1.0,1.0 +bge-small-en-v1.5,test,539,1.0,1.0 +bge-small-en-v1.5,test,540,0.61315,1.0 +bge-small-en-v1.5,test,544,0.35621,0.0 +bge-small-en-v1.5,test,549,1.0,1.0 +bge-small-en-v1.5,test,551,0.0,0.0 +bge-small-en-v1.5,test,552,1.0,1.0 +bge-small-en-v1.5,test,554,0.38685,0.0 +bge-small-en-v1.5,test,560,0.0,0.0 +bge-small-en-v1.5,test,569,0.63093,1.0 +bge-small-en-v1.5,test,575,1.0,1.0 +bge-small-en-v1.5,test,577,0.38685,0.0 +bge-small-en-v1.5,test,578,1.0,1.0 +bge-small-en-v1.5,test,587,0.5,1.0 +bge-small-en-v1.5,test,589,1.0,1.0 +bge-small-en-v1.5,test,593,1.0,1.0 +bge-small-en-v1.5,test,597,1.0,1.0 +bge-small-en-v1.5,test,598,1.0,1.0 +bge-small-en-v1.5,test,613,1.0,1.0 +bge-small-en-v1.5,test,619,0.0,0.0 +bge-small-en-v1.5,test,623,0.35621,0.0 +bge-small-en-v1.5,test,628,1.0,1.0 +bge-small-en-v1.5,test,636,1.0,1.0 +bge-small-en-v1.5,test,637,0.63093,1.0 +bge-small-en-v1.5,test,641,0.79772,1.0 +bge-small-en-v1.5,test,644,1.0,1.0 +bge-small-en-v1.5,test,649,1.0,1.0 +bge-small-en-v1.5,test,659,0.43068,0.0 +bge-small-en-v1.5,test,660,0.0,0.0 +bge-small-en-v1.5,test,674,0.28906,0.0 +bge-small-en-v1.5,test,684,0.5,1.0 +bge-small-en-v1.5,test,690,0.0,0.0 +bge-small-en-v1.5,test,691,0.63093,1.0 +bge-small-en-v1.5,test,692,1.0,1.0 +bge-small-en-v1.5,test,693,1.0,1.0 +bge-small-en-v1.5,test,700,1.0,1.0 +bge-small-en-v1.5,test,702,1.0,1.0 +bge-small-en-v1.5,test,715,0.0,0.0 +bge-small-en-v1.5,test,716,0.0,0.0 +bge-small-en-v1.5,test,718,1.0,1.0 +bge-small-en-v1.5,test,721,1.0,1.0 +bge-small-en-v1.5,test,723,1.0,1.0 +bge-small-en-v1.5,test,727,0.35621,0.0 +bge-small-en-v1.5,test,728,0.83155,1.0 +bge-small-en-v1.5,test,729,1.0,1.0 +bge-small-en-v1.5,test,742,1.0,1.0 +bge-small-en-v1.5,test,743,1.0,1.0 +bge-small-en-v1.5,test,744,0.5,1.0 +bge-small-en-v1.5,test,756,0.5,1.0 +bge-small-en-v1.5,test,759,1.0,1.0 +bge-small-en-v1.5,test,768,0.0,0.0 +bge-small-en-v1.5,test,770,1.0,1.0 +bge-small-en-v1.5,test,775,0.38685,0.0 +bge-small-en-v1.5,test,781,1.0,1.0 +bge-small-en-v1.5,test,783,0.31546,0.0 +bge-small-en-v1.5,test,784,1.0,1.0 +bge-small-en-v1.5,test,785,0.43068,0.0 +bge-small-en-v1.5,test,793,0.63093,1.0 +bge-small-en-v1.5,test,800,0.5,1.0 +bge-small-en-v1.5,test,805,1.0,1.0 +bge-small-en-v1.5,test,808,1.0,1.0 +bge-small-en-v1.5,test,811,1.0,1.0 +bge-small-en-v1.5,test,814,1.0,1.0 +bge-small-en-v1.5,test,820,0.0,0.0 +bge-small-en-v1.5,test,821,0.0,0.0 +bge-small-en-v1.5,test,823,1.0,1.0 +bge-small-en-v1.5,test,830,0.0,0.0 +bge-small-en-v1.5,test,831,0.0,0.0 +bge-small-en-v1.5,test,832,1.0,1.0 +bge-small-en-v1.5,test,834,0.0,0.0 +bge-small-en-v1.5,test,837,0.63093,1.0 +bge-small-en-v1.5,test,839,1.0,1.0 +bge-small-en-v1.5,test,845,0.38685,0.0 +bge-small-en-v1.5,test,847,1.0,1.0 +bge-small-en-v1.5,test,852,0.63093,1.0 +bge-small-en-v1.5,test,859,0.63093,1.0 +bge-small-en-v1.5,test,870,0.5,1.0 +bge-small-en-v1.5,test,873,0.36005,1.0 +bge-small-en-v1.5,test,879,1.0,1.0 +bge-small-en-v1.5,test,880,1.0,1.0 +bge-small-en-v1.5,test,882,1.0,1.0 +bge-small-en-v1.5,test,887,0.0,0.0 +bge-small-en-v1.5,test,903,1.0,1.0 +bge-small-en-v1.5,test,904,1.0,1.0 +bge-small-en-v1.5,test,907,1.0,1.0 +bge-small-en-v1.5,test,911,1.0,1.0 +bge-small-en-v1.5,test,913,0.0,0.0 +bge-small-en-v1.5,test,914,0.0,0.0 +bge-small-en-v1.5,test,921,1.0,1.0 +bge-small-en-v1.5,test,922,1.0,1.0 +bge-small-en-v1.5,test,936,1.0,1.0 +bge-small-en-v1.5,test,956,1.0,1.0 +bge-small-en-v1.5,test,957,1.0,1.0 +bge-small-en-v1.5,test,960,1.0,1.0 +bge-small-en-v1.5,test,967,1.0,1.0 +bge-small-en-v1.5,test,971,0.90587,1.0 +bge-small-en-v1.5,test,975,0.0,0.0 +bge-small-en-v1.5,test,982,1.0,1.0 +bge-small-en-v1.5,test,985,1.0,1.0 +bge-small-en-v1.5,test,993,1.0,1.0 +bge-small-en-v1.5,test,1012,1.0,1.0 +bge-small-en-v1.5,test,1014,1.0,1.0 +bge-small-en-v1.5,test,1019,0.5,1.0 +bge-small-en-v1.5,test,1020,0.63093,1.0 +bge-small-en-v1.5,test,1021,0.63093,1.0 +bge-small-en-v1.5,test,1024,0.33333,0.0 +bge-small-en-v1.5,test,1029,0.96747,1.0 +bge-small-en-v1.5,test,1041,0.42177,0.0 +bge-small-en-v1.5,test,1049,0.35621,0.0 +bge-small-en-v1.5,test,1062,1.0,1.0 +bge-small-en-v1.5,test,1086,1.0,1.0 +bge-small-en-v1.5,test,1088,1.0,1.0 +bge-small-en-v1.5,test,1089,1.0,1.0 +bge-small-en-v1.5,test,1099,0.31546,0.0 +bge-small-en-v1.5,test,1100,0.0,0.0 +bge-small-en-v1.5,test,1104,1.0,1.0 +bge-small-en-v1.5,test,1107,1.0,1.0 +bge-small-en-v1.5,test,1110,0.0,0.0 +bge-small-en-v1.5,test,1121,0.63093,1.0 +bge-small-en-v1.5,test,1130,1.0,1.0 +bge-small-en-v1.5,test,1132,0.19343,0.0 +bge-small-en-v1.5,test,1137,1.0,1.0 +bge-small-en-v1.5,test,1140,1.0,1.0 +bge-small-en-v1.5,test,1144,1.0,1.0 +bge-small-en-v1.5,test,1146,1.0,1.0 +bge-small-en-v1.5,test,1150,1.0,1.0 +bge-small-en-v1.5,test,1163,1.0,1.0 +bge-small-en-v1.5,test,1175,0.0,0.0 +bge-small-en-v1.5,test,1179,0.0,0.0 +bge-small-en-v1.5,test,1180,0.63093,1.0 +bge-small-en-v1.5,test,1185,1.0,1.0 +bge-small-en-v1.5,test,1187,0.63093,1.0 +bge-small-en-v1.5,test,1191,0.0,0.0 +bge-small-en-v1.5,test,1194,1.0,1.0 +bge-small-en-v1.5,test,1196,0.38685,0.0 +bge-small-en-v1.5,test,1197,0.38685,0.0 +bge-small-en-v1.5,test,1199,0.0,0.0 +bge-small-en-v1.5,test,1200,0.28906,0.0 +bge-small-en-v1.5,test,1202,1.0,1.0 +bge-small-en-v1.5,test,1204,1.0,1.0 +bge-small-en-v1.5,test,1207,1.0,1.0 +bge-small-en-v1.5,test,1213,0.0,0.0 +bge-small-en-v1.5,test,1216,1.0,1.0 +bge-small-en-v1.5,test,1221,0.31546,0.0 +bge-small-en-v1.5,test,1225,1.0,1.0 +bge-small-en-v1.5,test,1226,0.0,0.0 +bge-small-en-v1.5,test,1232,1.0,1.0 +bge-small-en-v1.5,test,1241,1.0,1.0 +bge-small-en-v1.5,test,1245,0.63093,1.0 +bge-small-en-v1.5,test,1259,1.0,1.0 +bge-small-en-v1.5,test,1262,1.0,1.0 +bge-small-en-v1.5,test,1266,1.0,1.0 +bge-small-en-v1.5,test,1270,1.0,1.0 +bge-small-en-v1.5,test,1271,1.0,1.0 +bge-small-en-v1.5,test,1272,0.0,0.0 +bge-small-en-v1.5,test,1273,1.0,1.0 +bge-small-en-v1.5,test,1274,0.83855,1.0 +bge-small-en-v1.5,test,1278,1.0,1.0 +bge-small-en-v1.5,test,1279,1.0,1.0 +bge-small-en-v1.5,test,1280,0.35621,0.0 +bge-small-en-v1.5,test,1281,0.33333,0.0 +bge-small-en-v1.5,test,1282,1.0,1.0 +bge-small-en-v1.5,test,1290,1.0,1.0 +bge-small-en-v1.5,test,1292,1.0,1.0 +bge-small-en-v1.5,test,1298,1.0,1.0 +bge-small-en-v1.5,test,1303,1.0,1.0 +bge-small-en-v1.5,test,1316,0.0,0.0 +bge-small-en-v1.5,test,1319,1.0,1.0 +bge-small-en-v1.5,test,1320,1.0,1.0 +bge-small-en-v1.5,test,1332,0.0,0.0 +bge-small-en-v1.5,test,1335,1.0,1.0 +bge-small-en-v1.5,test,1336,0.63093,1.0 +bge-small-en-v1.5,test,1337,1.0,1.0 +bge-small-en-v1.5,test,1339,1.0,1.0 +bge-small-en-v1.5,test,1344,0.63093,1.0 +bge-small-en-v1.5,test,1352,1.0,1.0 +bge-small-en-v1.5,test,1359,1.0,1.0 +bge-small-en-v1.5,test,1362,0.0,0.0 +bge-small-en-v1.5,test,1363,0.0,0.0 +bge-small-en-v1.5,test,1368,1.0,1.0 +bge-small-en-v1.5,test,1370,1.0,1.0 +bge-small-en-v1.5,test,1379,0.97093,1.0 +bge-small-en-v1.5,test,1382,1.0,1.0 +bge-small-en-v1.5,test,1385,1.0,1.0 +bge-small-en-v1.5,test,1389,1.0,1.0 +bge-small-en-v1.5,test,1395,0.5,1.0 +e5-small-v2,train,0,0.38685,0.0 +e5-small-v2,train,2,0.63093,1.0 +e5-small-v2,train,4,0.35621,0.0 +e5-small-v2,train,6,1.0,1.0 +e5-small-v2,train,9,1.0,1.0 +e5-small-v2,train,10,0.0,0.0 +e5-small-v2,train,11,0.30103,0.0 +e5-small-v2,train,12,1.0,1.0 +e5-small-v2,train,14,0.0,0.0 +e5-small-v2,train,15,0.0,0.0 +e5-small-v2,train,17,0.0,0.0 +e5-small-v2,train,18,0.35621,0.0 +e5-small-v2,train,19,1.0,1.0 +e5-small-v2,train,20,1.0,1.0 +e5-small-v2,train,21,1.0,1.0 +e5-small-v2,train,22,1.0,1.0 +e5-small-v2,train,24,1.0,1.0 +e5-small-v2,train,25,1.0,1.0 +e5-small-v2,train,26,0.0,0.0 +e5-small-v2,train,27,0.0,0.0 +e5-small-v2,train,28,1.0,1.0 +e5-small-v2,train,30,1.0,1.0 +e5-small-v2,train,32,1.0,1.0 +e5-small-v2,train,34,1.0,1.0 +e5-small-v2,train,35,0.38685,1.0 +e5-small-v2,train,37,0.61315,1.0 +e5-small-v2,train,39,1.0,1.0 +e5-small-v2,train,40,1.0,1.0 +e5-small-v2,train,41,1.0,1.0 +e5-small-v2,train,43,1.0,1.0 +e5-small-v2,train,44,1.0,1.0 +e5-small-v2,train,45,1.0,1.0 +e5-small-v2,train,46,1.0,1.0 +e5-small-v2,train,47,1.0,1.0 +e5-small-v2,train,52,1.0,1.0 +e5-small-v2,train,55,1.0,1.0 +e5-small-v2,train,58,1.0,1.0 +e5-small-v2,train,60,0.48381,1.0 +e5-small-v2,train,61,0.54377,1.0 +e5-small-v2,train,62,0.63093,1.0 +e5-small-v2,train,63,0.0,0.0 +e5-small-v2,train,64,0.0,0.0 +e5-small-v2,train,66,0.0,0.0 +e5-small-v2,train,67,1.0,1.0 +e5-small-v2,train,68,1.0,1.0 +e5-small-v2,train,69,0.61315,1.0 +e5-small-v2,train,71,1.0,1.0 +e5-small-v2,train,73,0.0,0.0 +e5-small-v2,train,74,1.0,1.0 +e5-small-v2,train,76,1.0,1.0 +e5-small-v2,train,77,1.0,1.0 +e5-small-v2,train,78,1.0,1.0 +e5-small-v2,train,79,0.63093,1.0 +e5-small-v2,train,80,0.31546,0.0 +e5-small-v2,train,81,0.0,0.0 +e5-small-v2,train,82,1.0,1.0 +e5-small-v2,train,85,1.0,1.0 +e5-small-v2,train,86,1.0,1.0 +e5-small-v2,train,88,0.69343,1.0 +e5-small-v2,train,89,0.65092,1.0 +e5-small-v2,train,90,0.63093,1.0 +e5-small-v2,train,91,0.0,0.0 +e5-small-v2,train,92,0.0,0.0 +e5-small-v2,train,93,0.0,0.0 +e5-small-v2,train,95,0.0,0.0 +e5-small-v2,train,96,1.0,1.0 +e5-small-v2,train,98,1.0,1.0 +e5-small-v2,train,104,1.0,1.0 +e5-small-v2,train,105,1.0,1.0 +e5-small-v2,train,106,1.0,1.0 +e5-small-v2,train,108,0.96747,1.0 +e5-small-v2,train,109,1.0,1.0 +e5-small-v2,train,111,1.0,1.0 +e5-small-v2,train,112,1.0,1.0 +e5-small-v2,train,114,1.0,1.0 +e5-small-v2,train,116,1.0,1.0 +e5-small-v2,train,119,0.63093,1.0 +e5-small-v2,train,120,0.63093,1.0 +e5-small-v2,train,121,0.63093,1.0 +e5-small-v2,train,122,0.63093,1.0 +e5-small-v2,train,123,1.0,1.0 +e5-small-v2,train,126,1.0,1.0 +e5-small-v2,train,134,0.38685,0.0 +e5-small-v2,train,138,1.0,1.0 +e5-small-v2,train,139,0.0,0.0 +e5-small-v2,train,144,1.0,1.0 +e5-small-v2,train,149,1.0,1.0 +e5-small-v2,train,152,1.0,1.0 +e5-small-v2,train,153,1.0,1.0 +e5-small-v2,train,154,1.0,1.0 +e5-small-v2,train,155,0.63093,1.0 +e5-small-v2,train,156,1.0,1.0 +e5-small-v2,train,157,1.0,1.0 +e5-small-v2,train,159,1.0,1.0 +e5-small-v2,train,160,1.0,1.0 +e5-small-v2,train,161,1.0,1.0 +e5-small-v2,train,164,1.0,1.0 +e5-small-v2,train,165,0.63093,1.0 +e5-small-v2,train,166,1.0,1.0 +e5-small-v2,train,167,0.63093,1.0 +e5-small-v2,train,168,0.63093,1.0 +e5-small-v2,train,169,1.0,1.0 +e5-small-v2,train,172,1.0,1.0 +e5-small-v2,train,173,0.0,0.0 +e5-small-v2,train,174,0.63093,1.0 +e5-small-v2,train,175,0.5,1.0 +e5-small-v2,train,176,0.43068,0.0 +e5-small-v2,train,177,1.0,1.0 +e5-small-v2,train,178,0.97093,1.0 +e5-small-v2,train,181,1.0,1.0 +e5-small-v2,train,182,0.30103,0.0 +e5-small-v2,train,184,1.0,1.0 +e5-small-v2,train,186,1.0,1.0 +e5-small-v2,train,187,1.0,1.0 +e5-small-v2,train,189,1.0,1.0 +e5-small-v2,train,196,1.0,1.0 +e5-small-v2,train,197,0.5,1.0 +e5-small-v2,train,199,0.30103,0.0 +e5-small-v2,train,200,1.0,1.0 +e5-small-v2,train,201,1.0,1.0 +e5-small-v2,train,203,1.0,1.0 +e5-small-v2,train,204,0.63093,1.0 +e5-small-v2,train,205,1.0,1.0 +e5-small-v2,train,209,0.63093,1.0 +e5-small-v2,train,210,1.0,1.0 +e5-small-v2,train,211,1.0,1.0 +e5-small-v2,train,214,0.33333,0.0 +e5-small-v2,train,220,1.0,1.0 +e5-small-v2,train,221,1.0,1.0 +e5-small-v2,train,222,1.0,1.0 +e5-small-v2,train,223,1.0,1.0 +e5-small-v2,train,224,0.0,0.0 +e5-small-v2,train,225,0.0,0.0 +e5-small-v2,train,226,0.0,0.0 +e5-small-v2,train,227,0.5,1.0 +e5-small-v2,train,228,1.0,1.0 +e5-small-v2,train,229,1.0,1.0 +e5-small-v2,train,235,1.0,1.0 +e5-small-v2,train,241,0.87722,1.0 +e5-small-v2,train,242,0.87722,1.0 +e5-small-v2,train,243,1.0,1.0 +e5-small-v2,train,244,1.0,1.0 +e5-small-v2,train,245,0.61315,1.0 +e5-small-v2,train,246,0.61315,1.0 +e5-small-v2,train,247,0.63093,1.0 +e5-small-v2,train,250,1.0,1.0 +e5-small-v2,train,251,1.0,1.0 +e5-small-v2,train,253,0.5,1.0 +e5-small-v2,train,254,0.5,1.0 +e5-small-v2,train,255,1.0,1.0 +e5-small-v2,train,256,1.0,1.0 +e5-small-v2,train,258,0.63093,1.0 +e5-small-v2,train,259,0.0,0.0 +e5-small-v2,train,262,0.0,0.0 +e5-small-v2,train,263,0.85293,1.0 +e5-small-v2,train,264,1.0,1.0 +e5-small-v2,train,265,1.0,1.0 +e5-small-v2,train,266,0.63093,1.0 +e5-small-v2,train,267,1.0,1.0 +e5-small-v2,train,272,1.0,1.0 +e5-small-v2,train,277,1.0,1.0 +e5-small-v2,train,278,1.0,1.0 +e5-small-v2,train,280,1.0,1.0 +e5-small-v2,train,281,1.0,1.0 +e5-small-v2,train,283,1.0,1.0 +e5-small-v2,train,285,1.0,1.0 +e5-small-v2,train,286,1.0,1.0 +e5-small-v2,train,287,1.0,1.0 +e5-small-v2,train,290,1.0,1.0 +e5-small-v2,train,292,1.0,1.0 +e5-small-v2,train,293,0.0,0.0 +e5-small-v2,train,296,1.0,1.0 +e5-small-v2,train,299,1.0,1.0 +e5-small-v2,train,301,0.43068,0.0 +e5-small-v2,train,304,1.0,1.0 +e5-small-v2,train,305,1.0,1.0 +e5-small-v2,train,306,1.0,1.0 +e5-small-v2,train,308,1.0,1.0 +e5-small-v2,train,309,1.0,1.0 +e5-small-v2,train,310,0.0,0.0 +e5-small-v2,train,313,0.0,0.0 +e5-small-v2,train,315,0.0,0.0 +e5-small-v2,train,316,0.33333,0.0 +e5-small-v2,train,317,1.0,1.0 +e5-small-v2,train,323,0.0,0.0 +e5-small-v2,train,325,0.0,0.0 +e5-small-v2,train,326,0.0,0.0 +e5-small-v2,train,330,1.0,1.0 +e5-small-v2,train,331,0.5,1.0 +e5-small-v2,train,332,1.0,1.0 +e5-small-v2,train,333,1.0,1.0 +e5-small-v2,train,334,0.63093,1.0 +e5-small-v2,train,335,1.0,1.0 +e5-small-v2,train,336,0.63093,1.0 +e5-small-v2,train,337,0.63093,1.0 +e5-small-v2,train,339,1.0,1.0 +e5-small-v2,train,340,1.0,1.0 +e5-small-v2,train,341,1.0,1.0 +e5-small-v2,train,342,0.62405,1.0 +e5-small-v2,train,345,0.5,1.0 +e5-small-v2,train,346,0.38685,0.0 +e5-small-v2,train,347,0.33333,0.0 +e5-small-v2,train,349,1.0,1.0 +e5-small-v2,train,351,0.0,0.0 +e5-small-v2,train,352,0.0,0.0 +e5-small-v2,train,355,0.91972,1.0 +e5-small-v2,train,356,1.0,1.0 +e5-small-v2,train,357,1.0,1.0 +e5-small-v2,train,358,1.0,1.0 +e5-small-v2,train,361,1.0,1.0 +e5-small-v2,train,363,0.5,1.0 +e5-small-v2,train,364,1.0,1.0 +e5-small-v2,train,365,1.0,1.0 +e5-small-v2,train,366,1.0,1.0 +e5-small-v2,train,367,1.0,1.0 +e5-small-v2,train,368,1.0,1.0 +e5-small-v2,train,369,1.0,1.0 +e5-small-v2,train,370,1.0,1.0 +e5-small-v2,train,371,1.0,1.0 +e5-small-v2,train,372,1.0,1.0 +e5-small-v2,train,375,1.0,1.0 +e5-small-v2,train,376,1.0,1.0 +e5-small-v2,train,377,1.0,1.0 +e5-small-v2,train,378,0.50438,1.0 +e5-small-v2,train,379,0.33333,0.0 +e5-small-v2,train,381,0.38685,0.0 +e5-small-v2,train,382,0.28906,0.0 +e5-small-v2,train,383,0.5,1.0 +e5-small-v2,train,389,0.63093,1.0 +e5-small-v2,train,390,1.0,1.0 +e5-small-v2,train,391,1.0,1.0 +e5-small-v2,train,392,1.0,1.0 +e5-small-v2,train,393,1.0,1.0 +e5-small-v2,train,394,0.43068,0.0 +e5-small-v2,train,396,1.0,1.0 +e5-small-v2,train,397,1.0,1.0 +e5-small-v2,train,398,0.0,0.0 +e5-small-v2,train,400,1.0,1.0 +e5-small-v2,train,401,1.0,1.0 +e5-small-v2,train,403,0.35621,0.0 +e5-small-v2,train,404,0.0,0.0 +e5-small-v2,train,406,0.0,0.0 +e5-small-v2,train,407,1.0,1.0 +e5-small-v2,train,413,0.63093,1.0 +e5-small-v2,train,414,0.43068,0.0 +e5-small-v2,train,416,1.0,1.0 +e5-small-v2,train,417,1.0,1.0 +e5-small-v2,train,418,0.0,0.0 +e5-small-v2,train,420,1.0,1.0 +e5-small-v2,train,422,0.0,0.0 +e5-small-v2,train,423,0.31546,0.0 +e5-small-v2,train,425,1.0,1.0 +e5-small-v2,train,426,0.63093,1.0 +e5-small-v2,train,428,1.0,1.0 +e5-small-v2,train,429,1.0,1.0 +e5-small-v2,train,430,0.0,0.0 +e5-small-v2,train,432,1.0,1.0 +e5-small-v2,train,434,0.38685,0.0 +e5-small-v2,train,435,0.5,1.0 +e5-small-v2,train,441,0.63093,1.0 +e5-small-v2,train,444,1.0,1.0 +e5-small-v2,train,445,1.0,1.0 +e5-small-v2,train,447,1.0,1.0 +e5-small-v2,train,448,1.0,1.0 +e5-small-v2,train,449,0.26407,0.0 +e5-small-v2,train,453,1.0,1.0 +e5-small-v2,train,454,1.0,1.0 +e5-small-v2,train,455,0.35621,0.0 +e5-small-v2,train,456,0.0,0.0 +e5-small-v2,train,458,1.0,1.0 +e5-small-v2,train,461,1.0,1.0 +e5-small-v2,train,463,0.63093,1.0 +e5-small-v2,train,466,1.0,1.0 +e5-small-v2,train,469,0.5,1.0 +e5-small-v2,train,470,0.33333,0.0 +e5-small-v2,train,472,0.67973,1.0 +e5-small-v2,train,473,0.0,0.0 +e5-small-v2,train,474,0.0,0.0 +e5-small-v2,train,479,1.0,1.0 +e5-small-v2,train,480,1.0,1.0 +e5-small-v2,train,481,0.38685,0.0 +e5-small-v2,train,482,1.0,1.0 +e5-small-v2,train,483,1.0,1.0 +e5-small-v2,train,484,0.0,0.0 +e5-small-v2,train,485,1.0,1.0 +e5-small-v2,train,486,0.0,0.0 +e5-small-v2,train,487,1.0,1.0 +e5-small-v2,train,488,1.0,1.0 +e5-small-v2,train,489,0.0,0.0 +e5-small-v2,train,490,1.0,1.0 +e5-small-v2,train,492,1.0,1.0 +e5-small-v2,train,493,0.0,0.0 +e5-small-v2,train,494,0.0,0.0 +e5-small-v2,train,495,1.0,1.0 +e5-small-v2,train,498,1.0,1.0 +e5-small-v2,train,499,1.0,1.0 +e5-small-v2,train,500,1.0,1.0 +e5-small-v2,train,504,1.0,1.0 +e5-small-v2,train,505,1.0,1.0 +e5-small-v2,train,506,1.0,1.0 +e5-small-v2,train,509,0.0,0.0 +e5-small-v2,train,515,1.0,1.0 +e5-small-v2,train,523,0.63093,1.0 +e5-small-v2,train,524,0.63093,1.0 +e5-small-v2,train,526,0.43068,0.0 +e5-small-v2,train,529,0.49819,1.0 +e5-small-v2,train,530,0.41443,1.0 +e5-small-v2,train,531,0.67139,1.0 +e5-small-v2,train,537,1.0,1.0 +e5-small-v2,train,541,0.61732,1.0 +e5-small-v2,train,542,1.0,1.0 +e5-small-v2,train,545,1.0,1.0 +e5-small-v2,train,547,1.0,1.0 +e5-small-v2,train,548,1.0,1.0 +e5-small-v2,train,550,0.5,1.0 +e5-small-v2,train,553,1.0,1.0 +e5-small-v2,train,555,0.0,0.0 +e5-small-v2,train,557,0.0,0.0 +e5-small-v2,train,559,0.0,0.0 +e5-small-v2,train,562,1.0,1.0 +e5-small-v2,train,563,1.0,1.0 +e5-small-v2,train,564,1.0,1.0 +e5-small-v2,train,565,1.0,1.0 +e5-small-v2,train,566,1.0,1.0 +e5-small-v2,train,568,1.0,1.0 +e5-small-v2,train,570,1.0,1.0 +e5-small-v2,train,571,1.0,1.0 +e5-small-v2,train,572,0.0,0.0 +e5-small-v2,train,573,1.0,1.0 +e5-small-v2,train,574,1.0,1.0 +e5-small-v2,train,576,1.0,1.0 +e5-small-v2,train,579,1.0,1.0 +e5-small-v2,train,580,1.0,1.0 +e5-small-v2,train,582,1.0,1.0 +e5-small-v2,train,584,1.0,1.0 +e5-small-v2,train,585,1.0,1.0 +e5-small-v2,train,588,0.0,0.0 +e5-small-v2,train,590,1.0,1.0 +e5-small-v2,train,591,1.0,1.0 +e5-small-v2,train,592,1.0,1.0 +e5-small-v2,train,594,1.0,1.0 +e5-small-v2,train,595,1.0,1.0 +e5-small-v2,train,600,1.0,1.0 +e5-small-v2,train,601,1.0,1.0 +e5-small-v2,train,602,0.0,0.0 +e5-small-v2,train,603,0.5,1.0 +e5-small-v2,train,606,0.33333,0.0 +e5-small-v2,train,607,1.0,1.0 +e5-small-v2,train,609,0.0,0.0 +e5-small-v2,train,610,0.0,0.0 +e5-small-v2,train,611,0.31546,0.0 +e5-small-v2,train,612,1.0,1.0 +e5-small-v2,train,614,1.0,1.0 +e5-small-v2,train,615,1.0,1.0 +e5-small-v2,train,616,0.0,0.0 +e5-small-v2,train,617,0.0,0.0 +e5-small-v2,train,618,1.0,1.0 +e5-small-v2,train,620,0.0,0.0 +e5-small-v2,train,621,1.0,1.0 +e5-small-v2,train,622,0.31546,0.0 +e5-small-v2,train,624,1.0,1.0 +e5-small-v2,train,625,1.0,1.0 +e5-small-v2,train,626,1.0,1.0 +e5-small-v2,train,631,1.0,1.0 +e5-small-v2,train,632,1.0,1.0 +e5-small-v2,train,633,1.0,1.0 +e5-small-v2,train,635,1.0,1.0 +e5-small-v2,train,638,0.63093,1.0 +e5-small-v2,train,640,1.0,1.0 +e5-small-v2,train,642,1.0,1.0 +e5-small-v2,train,643,1.0,1.0 +e5-small-v2,train,645,0.63093,1.0 +e5-small-v2,train,646,0.5,1.0 +e5-small-v2,train,647,1.0,1.0 +e5-small-v2,train,648,1.0,1.0 +e5-small-v2,train,650,1.0,1.0 +e5-small-v2,train,651,1.0,1.0 +e5-small-v2,train,652,1.0,1.0 +e5-small-v2,train,653,0.63093,1.0 +e5-small-v2,train,654,1.0,1.0 +e5-small-v2,train,655,1.0,1.0 +e5-small-v2,train,657,0.43068,0.0 +e5-small-v2,train,658,1.0,1.0 +e5-small-v2,train,661,1.0,1.0 +e5-small-v2,train,662,1.0,1.0 +e5-small-v2,train,663,1.0,1.0 +e5-small-v2,train,665,1.0,1.0 +e5-small-v2,train,666,1.0,1.0 +e5-small-v2,train,667,1.0,1.0 +e5-small-v2,train,668,1.0,1.0 +e5-small-v2,train,669,1.0,1.0 +e5-small-v2,train,670,1.0,1.0 +e5-small-v2,train,671,1.0,1.0 +e5-small-v2,train,672,1.0,1.0 +e5-small-v2,train,673,0.0,0.0 +e5-small-v2,train,676,1.0,1.0 +e5-small-v2,train,677,0.63093,1.0 +e5-small-v2,train,679,0.0,0.0 +e5-small-v2,train,680,1.0,1.0 +e5-small-v2,train,681,1.0,1.0 +e5-small-v2,train,682,0.5,1.0 +e5-small-v2,train,683,1.0,1.0 +e5-small-v2,train,685,0.63093,1.0 +e5-small-v2,train,686,0.63093,1.0 +e5-small-v2,train,687,0.43068,0.0 +e5-small-v2,train,688,0.43068,0.0 +e5-small-v2,train,689,0.0,0.0 +e5-small-v2,train,694,1.0,1.0 +e5-small-v2,train,696,1.0,1.0 +e5-small-v2,train,698,1.0,1.0 +e5-small-v2,train,703,0.63093,1.0 +e5-small-v2,train,704,0.0,0.0 +e5-small-v2,train,705,1.0,1.0 +e5-small-v2,train,709,1.0,1.0 +e5-small-v2,train,710,1.0,1.0 +e5-small-v2,train,713,0.0,0.0 +e5-small-v2,train,714,0.0,0.0 +e5-small-v2,train,717,1.0,1.0 +e5-small-v2,train,724,1.0,1.0 +e5-small-v2,train,726,0.81753,1.0 +e5-small-v2,train,730,1.0,1.0 +e5-small-v2,train,732,0.0,0.0 +e5-small-v2,train,733,0.0,0.0 +e5-small-v2,train,734,1.0,1.0 +e5-small-v2,train,736,1.0,1.0 +e5-small-v2,train,737,0.58028,1.0 +e5-small-v2,train,738,0.49819,1.0 +e5-small-v2,train,739,0.0,0.0 +e5-small-v2,train,740,1.0,1.0 +e5-small-v2,train,745,1.0,1.0 +e5-small-v2,train,746,1.0,1.0 +e5-small-v2,train,747,1.0,1.0 +e5-small-v2,train,748,1.0,1.0 +e5-small-v2,train,749,0.5,1.0 +e5-small-v2,train,751,1.0,1.0 +e5-small-v2,train,752,1.0,1.0 +e5-small-v2,train,753,0.0,0.0 +e5-small-v2,train,755,0.0,0.0 +e5-small-v2,train,757,1.0,1.0 +e5-small-v2,train,758,0.0,0.0 +e5-small-v2,train,760,1.0,1.0 +e5-small-v2,train,761,0.0,0.0 +e5-small-v2,train,762,0.63093,1.0 +e5-small-v2,train,764,0.5,1.0 +e5-small-v2,train,765,0.5,1.0 +e5-small-v2,train,766,0.63093,1.0 +e5-small-v2,train,767,1.0,1.0 +e5-small-v2,train,771,1.0,1.0 +e5-small-v2,train,772,1.0,1.0 +e5-small-v2,train,774,0.0,0.0 +e5-small-v2,train,776,0.0,0.0 +e5-small-v2,train,777,0.0,0.0 +e5-small-v2,train,778,1.0,1.0 +e5-small-v2,train,779,1.0,1.0 +e5-small-v2,train,780,1.0,1.0 +e5-small-v2,train,782,0.63093,1.0 +e5-small-v2,train,787,1.0,1.0 +e5-small-v2,train,788,1.0,1.0 +e5-small-v2,train,789,1.0,1.0 +e5-small-v2,train,790,1.0,1.0 +e5-small-v2,train,791,0.5,1.0 +e5-small-v2,train,792,1.0,1.0 +e5-small-v2,train,795,0.5,1.0 +e5-small-v2,train,797,1.0,1.0 +e5-small-v2,train,798,0.63093,1.0 +e5-small-v2,train,799,0.63093,1.0 +e5-small-v2,train,801,1.0,1.0 +e5-small-v2,train,802,1.0,1.0 +e5-small-v2,train,803,1.0,1.0 +e5-small-v2,train,804,1.0,1.0 +e5-small-v2,train,807,1.0,1.0 +e5-small-v2,train,810,0.43068,0.0 +e5-small-v2,train,812,1.0,1.0 +e5-small-v2,train,813,1.0,1.0 +e5-small-v2,train,815,1.0,1.0 +e5-small-v2,train,816,1.0,1.0 +e5-small-v2,train,817,0.33333,0.0 +e5-small-v2,train,818,0.0,0.0 +e5-small-v2,train,822,1.0,1.0 +e5-small-v2,train,825,0.63093,1.0 +e5-small-v2,train,826,0.5,1.0 +e5-small-v2,train,828,0.28906,0.0 +e5-small-v2,train,835,0.38685,0.0 +e5-small-v2,train,838,0.0,0.0 +e5-small-v2,train,840,1.0,1.0 +e5-small-v2,train,841,1.0,1.0 +e5-small-v2,train,844,1.0,1.0 +e5-small-v2,train,846,0.0,0.0 +e5-small-v2,train,848,1.0,1.0 +e5-small-v2,train,853,1.0,1.0 +e5-small-v2,train,854,1.0,1.0 +e5-small-v2,train,855,1.0,1.0 +e5-small-v2,train,856,0.0,0.0 +e5-small-v2,train,857,0.0,0.0 +e5-small-v2,train,858,1.0,1.0 +e5-small-v2,train,860,1.0,1.0 +e5-small-v2,train,861,1.0,1.0 +e5-small-v2,train,863,0.69343,1.0 +e5-small-v2,train,866,0.63093,1.0 +e5-small-v2,train,867,1.0,1.0 +e5-small-v2,train,871,0.33333,0.0 +e5-small-v2,train,876,0.0,0.0 +e5-small-v2,train,877,1.0,1.0 +e5-small-v2,train,881,1.0,1.0 +e5-small-v2,train,883,1.0,1.0 +e5-small-v2,train,884,1.0,1.0 +e5-small-v2,train,885,1.0,1.0 +e5-small-v2,train,886,1.0,1.0 +e5-small-v2,train,890,0.5,1.0 +e5-small-v2,train,891,0.5,1.0 +e5-small-v2,train,893,1.0,1.0 +e5-small-v2,train,894,0.28906,0.0 +e5-small-v2,train,895,0.43068,0.0 +e5-small-v2,train,896,0.5,1.0 +e5-small-v2,train,897,0.63093,1.0 +e5-small-v2,train,898,0.61315,1.0 +e5-small-v2,train,899,0.61315,1.0 +e5-small-v2,train,900,0.0,0.0 +e5-small-v2,train,901,1.0,1.0 +e5-small-v2,train,902,1.0,1.0 +e5-small-v2,train,908,1.0,1.0 +e5-small-v2,train,909,1.0,1.0 +e5-small-v2,train,910,1.0,1.0 +e5-small-v2,train,912,1.0,1.0 +e5-small-v2,train,916,0.0,0.0 +e5-small-v2,train,917,1.0,1.0 +e5-small-v2,train,919,1.0,1.0 +e5-small-v2,train,923,1.0,1.0 +e5-small-v2,train,925,1.0,1.0 +e5-small-v2,train,926,1.0,1.0 +e5-small-v2,train,927,0.63093,1.0 +e5-small-v2,train,928,0.5,1.0 +e5-small-v2,train,929,0.43068,0.0 +e5-small-v2,train,930,1.0,1.0 +e5-small-v2,train,933,1.0,1.0 +e5-small-v2,train,934,0.0,0.0 +e5-small-v2,train,935,0.0,0.0 +e5-small-v2,train,938,0.28906,0.0 +e5-small-v2,train,939,1.0,1.0 +e5-small-v2,train,940,1.0,1.0 +e5-small-v2,train,941,1.0,1.0 +e5-small-v2,train,942,0.0,0.0 +e5-small-v2,train,944,1.0,1.0 +e5-small-v2,train,945,0.81179,1.0 +e5-small-v2,train,946,0.83268,1.0 +e5-small-v2,train,949,0.0,0.0 +e5-small-v2,train,951,1.0,1.0 +e5-small-v2,train,952,1.0,1.0 +e5-small-v2,train,953,1.0,1.0 +e5-small-v2,train,954,1.0,1.0 +e5-small-v2,train,955,1.0,1.0 +e5-small-v2,train,959,1.0,1.0 +e5-small-v2,train,962,0.69788,1.0 +e5-small-v2,train,963,0.59124,1.0 +e5-small-v2,train,964,0.83155,1.0 +e5-small-v2,train,965,0.63093,1.0 +e5-small-v2,train,969,0.56409,1.0 +e5-small-v2,train,970,0.60526,1.0 +e5-small-v2,train,972,0.98289,1.0 +e5-small-v2,train,973,0.72247,1.0 +e5-small-v2,train,976,0.0,0.0 +e5-small-v2,train,977,0.0,0.0 +e5-small-v2,train,978,0.0,0.0 +e5-small-v2,train,979,1.0,1.0 +e5-small-v2,train,980,1.0,1.0 +e5-small-v2,train,984,1.0,1.0 +e5-small-v2,train,988,0.0,0.0 +e5-small-v2,train,989,0.38685,0.0 +e5-small-v2,train,990,1.0,1.0 +e5-small-v2,train,992,0.0,0.0 +e5-small-v2,train,994,0.63093,1.0 +e5-small-v2,train,996,0.0,0.0 +e5-small-v2,train,997,0.28906,0.0 +e5-small-v2,train,998,0.5,1.0 +e5-small-v2,train,999,0.5,1.0 +e5-small-v2,train,1000,1.0,1.0 +e5-small-v2,train,1001,1.0,1.0 +e5-small-v2,train,1002,0.0,0.0 +e5-small-v2,train,1003,0.46928,1.0 +e5-small-v2,train,1004,0.96747,1.0 +e5-small-v2,train,1005,0.96747,1.0 +e5-small-v2,train,1006,1.0,1.0 +e5-small-v2,train,1008,0.0,0.0 +e5-small-v2,train,1009,1.0,1.0 +e5-small-v2,train,1011,1.0,1.0 +e5-small-v2,train,1015,1.0,1.0 +e5-small-v2,train,1016,1.0,1.0 +e5-small-v2,train,1018,1.0,1.0 +e5-small-v2,train,1023,1.0,1.0 +e5-small-v2,train,1025,0.38685,0.0 +e5-small-v2,train,1026,0.63093,1.0 +e5-small-v2,train,1027,0.5,1.0 +e5-small-v2,train,1028,0.91972,1.0 +e5-small-v2,train,1030,1.0,1.0 +e5-small-v2,train,1031,1.0,1.0 +e5-small-v2,train,1032,0.43068,0.0 +e5-small-v2,train,1033,0.63093,1.0 +e5-small-v2,train,1034,0.63093,1.0 +e5-small-v2,train,1035,0.63093,1.0 +e5-small-v2,train,1036,0.0,0.0 +e5-small-v2,train,1037,1.0,1.0 +e5-small-v2,train,1038,1.0,1.0 +e5-small-v2,train,1040,0.41183,0.0 +e5-small-v2,train,1042,1.0,1.0 +e5-small-v2,train,1043,1.0,1.0 +e5-small-v2,train,1044,1.0,1.0 +e5-small-v2,train,1045,1.0,1.0 +e5-small-v2,train,1046,0.49819,1.0 +e5-small-v2,train,1047,0.0,0.0 +e5-small-v2,train,1048,0.31546,0.0 +e5-small-v2,train,1050,1.0,1.0 +e5-small-v2,train,1052,1.0,1.0 +e5-small-v2,train,1053,1.0,1.0 +e5-small-v2,train,1054,1.0,1.0 +e5-small-v2,train,1055,1.0,1.0 +e5-small-v2,train,1056,1.0,1.0 +e5-small-v2,train,1058,1.0,1.0 +e5-small-v2,train,1065,1.0,1.0 +e5-small-v2,train,1067,0.0,0.0 +e5-small-v2,train,1068,0.0,0.0 +e5-small-v2,train,1069,1.0,1.0 +e5-small-v2,train,1070,0.31546,0.0 +e5-small-v2,train,1072,1.0,1.0 +e5-small-v2,train,1073,1.0,1.0 +e5-small-v2,train,1074,0.0,0.0 +e5-small-v2,train,1075,0.0,0.0 +e5-small-v2,train,1081,0.63093,1.0 +e5-small-v2,train,1084,0.63093,1.0 +e5-small-v2,train,1085,0.5,1.0 +e5-small-v2,train,1087,1.0,1.0 +e5-small-v2,train,1090,1.0,1.0 +e5-small-v2,train,1091,0.0,0.0 +e5-small-v2,train,1096,0.0,0.0 +e5-small-v2,train,1097,0.63093,1.0 +e5-small-v2,train,1098,0.0,0.0 +e5-small-v2,train,1101,1.0,1.0 +e5-small-v2,train,1102,0.5,1.0 +e5-small-v2,train,1103,1.0,1.0 +e5-small-v2,train,1105,1.0,1.0 +e5-small-v2,train,1106,1.0,1.0 +e5-small-v2,train,1109,0.0,0.0 +e5-small-v2,train,1111,0.0,0.0 +e5-small-v2,train,1112,0.0,0.0 +e5-small-v2,train,1114,1.0,1.0 +e5-small-v2,train,1115,1.0,1.0 +e5-small-v2,train,1118,1.0,1.0 +e5-small-v2,train,1119,0.73283,1.0 +e5-small-v2,train,1120,0.90603,1.0 +e5-small-v2,train,1125,1.0,1.0 +e5-small-v2,train,1126,1.0,1.0 +e5-small-v2,train,1127,0.43068,0.0 +e5-small-v2,train,1128,0.2372,0.0 +e5-small-v2,train,1133,1.0,1.0 +e5-small-v2,train,1134,1.0,1.0 +e5-small-v2,train,1135,1.0,1.0 +e5-small-v2,train,1136,1.0,1.0 +e5-small-v2,train,1138,0.30103,0.0 +e5-small-v2,train,1139,1.0,1.0 +e5-small-v2,train,1141,1.0,1.0 +e5-small-v2,train,1142,1.0,1.0 +e5-small-v2,train,1145,1.0,1.0 +e5-small-v2,train,1148,1.0,1.0 +e5-small-v2,train,1153,1.0,1.0 +e5-small-v2,train,1156,1.0,1.0 +e5-small-v2,train,1157,1.0,1.0 +e5-small-v2,train,1158,1.0,1.0 +e5-small-v2,train,1159,1.0,1.0 +e5-small-v2,train,1161,0.0,0.0 +e5-small-v2,train,1162,1.0,1.0 +e5-small-v2,train,1164,0.0,0.0 +e5-small-v2,train,1165,0.0,0.0 +e5-small-v2,train,1166,0.38685,0.0 +e5-small-v2,train,1168,0.0,0.0 +e5-small-v2,train,1169,1.0,1.0 +e5-small-v2,train,1170,1.0,1.0 +e5-small-v2,train,1171,1.0,1.0 +e5-small-v2,train,1173,1.0,1.0 +e5-small-v2,train,1174,0.0,0.0 +e5-small-v2,train,1176,1.0,1.0 +e5-small-v2,train,1177,1.0,1.0 +e5-small-v2,train,1178,0.0,0.0 +e5-small-v2,train,1181,1.0,1.0 +e5-small-v2,train,1182,1.0,1.0 +e5-small-v2,train,1183,1.0,1.0 +e5-small-v2,train,1184,1.0,1.0 +e5-small-v2,train,1186,0.43068,0.0 +e5-small-v2,train,1188,1.0,1.0 +e5-small-v2,train,1190,0.0,0.0 +e5-small-v2,train,1193,0.0,0.0 +e5-small-v2,train,1195,1.0,1.0 +e5-small-v2,train,1205,1.0,1.0 +e5-small-v2,train,1206,1.0,1.0 +e5-small-v2,train,1208,0.35621,0.0 +e5-small-v2,train,1209,0.0,0.0 +e5-small-v2,train,1210,1.0,1.0 +e5-small-v2,train,1211,1.0,1.0 +e5-small-v2,train,1212,0.0,0.0 +e5-small-v2,train,1214,0.0,0.0 +e5-small-v2,train,1215,1.0,1.0 +e5-small-v2,train,1218,0.0,0.0 +e5-small-v2,train,1219,0.61315,1.0 +e5-small-v2,train,1220,1.0,1.0 +e5-small-v2,train,1223,1.0,1.0 +e5-small-v2,train,1224,0.26407,0.0 +e5-small-v2,train,1227,0.63093,1.0 +e5-small-v2,train,1228,0.63093,1.0 +e5-small-v2,train,1229,1.0,1.0 +e5-small-v2,train,1230,1.0,1.0 +e5-small-v2,train,1231,1.0,1.0 +e5-small-v2,train,1234,1.0,1.0 +e5-small-v2,train,1235,0.30103,0.0 +e5-small-v2,train,1236,0.35621,0.0 +e5-small-v2,train,1237,1.0,1.0 +e5-small-v2,train,1238,1.0,1.0 +e5-small-v2,train,1239,0.80657,1.0 +e5-small-v2,train,1244,0.30103,0.0 +e5-small-v2,train,1246,0.0,0.0 +e5-small-v2,train,1247,0.0,0.0 +e5-small-v2,train,1248,1.0,1.0 +e5-small-v2,train,1249,1.0,1.0 +e5-small-v2,train,1253,0.0,0.0 +e5-small-v2,train,1254,1.0,1.0 +e5-small-v2,train,1255,0.0,0.0 +e5-small-v2,train,1257,0.0,0.0 +e5-small-v2,train,1258,0.0,0.0 +e5-small-v2,train,1260,1.0,1.0 +e5-small-v2,train,1261,1.0,1.0 +e5-small-v2,train,1263,1.0,1.0 +e5-small-v2,train,1265,1.0,1.0 +e5-small-v2,train,1268,0.43068,0.0 +e5-small-v2,train,1269,1.0,1.0 +e5-small-v2,train,1275,0.63093,1.0 +e5-small-v2,train,1276,1.0,1.0 +e5-small-v2,train,1284,1.0,1.0 +e5-small-v2,train,1288,1.0,1.0 +e5-small-v2,train,1289,1.0,1.0 +e5-small-v2,train,1291,0.63093,1.0 +e5-small-v2,train,1293,1.0,1.0 +e5-small-v2,train,1294,0.87722,1.0 +e5-small-v2,train,1295,1.0,1.0 +e5-small-v2,train,1297,1.0,1.0 +e5-small-v2,train,1300,0.0,0.0 +e5-small-v2,train,1302,0.0,0.0 +e5-small-v2,train,1304,0.0,0.0 +e5-small-v2,train,1305,1.0,1.0 +e5-small-v2,train,1306,0.19343,0.0 +e5-small-v2,train,1307,0.0,0.0 +e5-small-v2,train,1308,0.0,0.0 +e5-small-v2,train,1309,0.0,0.0 +e5-small-v2,train,1310,1.0,1.0 +e5-small-v2,train,1311,0.30103,0.0 +e5-small-v2,train,1312,1.0,1.0 +e5-small-v2,train,1314,0.30657,1.0 +e5-small-v2,train,1315,0.30657,1.0 +e5-small-v2,train,1322,1.0,1.0 +e5-small-v2,train,1323,1.0,1.0 +e5-small-v2,train,1324,1.0,1.0 +e5-small-v2,train,1325,0.0,0.0 +e5-small-v2,train,1327,0.91972,1.0 +e5-small-v2,train,1328,0.63093,1.0 +e5-small-v2,train,1330,0.0,0.0 +e5-small-v2,train,1331,0.0,0.0 +e5-small-v2,train,1333,1.0,1.0 +e5-small-v2,train,1334,0.53072,1.0 +e5-small-v2,train,1340,1.0,1.0 +e5-small-v2,train,1341,1.0,1.0 +e5-small-v2,train,1342,0.0,0.0 +e5-small-v2,train,1345,0.0,0.0 +e5-small-v2,train,1346,0.38685,0.0 +e5-small-v2,train,1347,1.0,1.0 +e5-small-v2,train,1348,1.0,1.0 +e5-small-v2,train,1349,1.0,1.0 +e5-small-v2,train,1350,1.0,1.0 +e5-small-v2,train,1351,1.0,1.0 +e5-small-v2,train,1353,0.63093,1.0 +e5-small-v2,train,1355,1.0,1.0 +e5-small-v2,train,1356,1.0,1.0 +e5-small-v2,train,1360,1.0,1.0 +e5-small-v2,train,1361,0.61315,1.0 +e5-small-v2,train,1364,0.0,0.0 +e5-small-v2,train,1366,1.0,1.0 +e5-small-v2,train,1367,1.0,1.0 +e5-small-v2,train,1371,1.0,1.0 +e5-small-v2,train,1372,1.0,1.0 +e5-small-v2,train,1373,1.0,1.0 +e5-small-v2,train,1374,0.63093,1.0 +e5-small-v2,train,1375,0.31546,0.0 +e5-small-v2,train,1376,0.63093,1.0 +e5-small-v2,train,1378,1.0,1.0 +e5-small-v2,train,1380,0.83855,1.0 +e5-small-v2,train,1381,1.0,1.0 +e5-small-v2,train,1383,1.0,1.0 +e5-small-v2,train,1386,0.63093,1.0 +e5-small-v2,train,1387,1.0,1.0 +e5-small-v2,train,1390,0.63093,1.0 +e5-small-v2,train,1391,1.0,1.0 +e5-small-v2,train,1392,1.0,1.0 +e5-small-v2,train,1393,0.91972,1.0 +e5-small-v2,train,1394,0.0,0.0 +e5-small-v2,train,1397,0.31546,0.0 +e5-small-v2,train,1398,0.33333,0.0 +e5-small-v2,train,1400,0.0,0.0 +e5-small-v2,train,1401,1.0,1.0 +e5-small-v2,train,1402,0.0,0.0 +e5-small-v2,train,1403,1.0,1.0 +e5-small-v2,train,1404,1.0,1.0 +e5-small-v2,train,1405,1.0,1.0 +e5-small-v2,train,1406,1.0,1.0 +e5-small-v2,train,1407,0.19343,0.0 +e5-small-v2,test,1,0.38685,0.0 +e5-small-v2,test,3,0.43068,0.0 +e5-small-v2,test,5,0.5,1.0 +e5-small-v2,test,13,0.0,0.0 +e5-small-v2,test,36,0.38685,1.0 +e5-small-v2,test,42,1.0,1.0 +e5-small-v2,test,48,0.5,1.0 +e5-small-v2,test,49,1.0,1.0 +e5-small-v2,test,50,1.0,1.0 +e5-small-v2,test,51,1.0,1.0 +e5-small-v2,test,53,1.0,1.0 +e5-small-v2,test,54,1.0,1.0 +e5-small-v2,test,56,1.0,1.0 +e5-small-v2,test,57,1.0,1.0 +e5-small-v2,test,70,0.38685,1.0 +e5-small-v2,test,72,0.38685,0.0 +e5-small-v2,test,75,1.0,1.0 +e5-small-v2,test,94,0.63093,1.0 +e5-small-v2,test,99,1.0,1.0 +e5-small-v2,test,100,1.0,1.0 +e5-small-v2,test,113,1.0,1.0 +e5-small-v2,test,115,1.0,1.0 +e5-small-v2,test,118,1.0,1.0 +e5-small-v2,test,124,1.0,1.0 +e5-small-v2,test,127,0.0,0.0 +e5-small-v2,test,128,0.0,0.0 +e5-small-v2,test,129,1.0,1.0 +e5-small-v2,test,130,1.0,1.0 +e5-small-v2,test,132,0.0,0.0 +e5-small-v2,test,133,0.77562,1.0 +e5-small-v2,test,137,1.0,1.0 +e5-small-v2,test,141,0.91972,1.0 +e5-small-v2,test,142,1.0,1.0 +e5-small-v2,test,143,1.0,1.0 +e5-small-v2,test,146,1.0,1.0 +e5-small-v2,test,148,1.0,1.0 +e5-small-v2,test,163,1.0,1.0 +e5-small-v2,test,171,1.0,1.0 +e5-small-v2,test,179,0.962,1.0 +e5-small-v2,test,180,1.0,1.0 +e5-small-v2,test,183,0.63093,1.0 +e5-small-v2,test,185,0.30103,0.0 +e5-small-v2,test,198,0.0,0.0 +e5-small-v2,test,208,1.0,1.0 +e5-small-v2,test,212,0.43068,0.0 +e5-small-v2,test,213,0.35621,0.0 +e5-small-v2,test,216,1.0,1.0 +e5-small-v2,test,217,1.0,1.0 +e5-small-v2,test,218,1.0,1.0 +e5-small-v2,test,219,1.0,1.0 +e5-small-v2,test,230,1.0,1.0 +e5-small-v2,test,232,0.5,1.0 +e5-small-v2,test,233,1.0,1.0 +e5-small-v2,test,236,1.0,1.0 +e5-small-v2,test,237,1.0,1.0 +e5-small-v2,test,238,0.0,0.0 +e5-small-v2,test,239,0.0,0.0 +e5-small-v2,test,248,1.0,1.0 +e5-small-v2,test,249,1.0,1.0 +e5-small-v2,test,261,0.39565,0.0 +e5-small-v2,test,268,1.0,1.0 +e5-small-v2,test,269,0.63093,1.0 +e5-small-v2,test,274,1.0,1.0 +e5-small-v2,test,275,0.81265,1.0 +e5-small-v2,test,279,1.0,1.0 +e5-small-v2,test,294,0.0,0.0 +e5-small-v2,test,295,0.30103,0.0 +e5-small-v2,test,298,1.0,1.0 +e5-small-v2,test,300,0.5,1.0 +e5-small-v2,test,303,0.0,0.0 +e5-small-v2,test,312,0.0,0.0 +e5-small-v2,test,314,0.0,0.0 +e5-small-v2,test,324,0.0,0.0 +e5-small-v2,test,327,1.0,1.0 +e5-small-v2,test,338,1.0,1.0 +e5-small-v2,test,343,0.83155,1.0 +e5-small-v2,test,350,1.0,1.0 +e5-small-v2,test,354,1.0,1.0 +e5-small-v2,test,362,1.0,1.0 +e5-small-v2,test,380,0.43068,0.0 +e5-small-v2,test,384,0.31546,0.0 +e5-small-v2,test,385,0.69343,1.0 +e5-small-v2,test,386,1.0,1.0 +e5-small-v2,test,388,1.0,1.0 +e5-small-v2,test,399,1.0,1.0 +e5-small-v2,test,410,1.0,1.0 +e5-small-v2,test,411,1.0,1.0 +e5-small-v2,test,415,0.63093,1.0 +e5-small-v2,test,421,0.0,0.0 +e5-small-v2,test,431,0.0,0.0 +e5-small-v2,test,436,1.0,1.0 +e5-small-v2,test,437,0.0,0.0 +e5-small-v2,test,439,1.0,1.0 +e5-small-v2,test,440,1.0,1.0 +e5-small-v2,test,443,1.0,1.0 +e5-small-v2,test,452,0.61315,1.0 +e5-small-v2,test,475,1.0,1.0 +e5-small-v2,test,478,0.63093,1.0 +e5-small-v2,test,491,1.0,1.0 +e5-small-v2,test,501,1.0,1.0 +e5-small-v2,test,502,0.0,0.0 +e5-small-v2,test,507,0.63093,1.0 +e5-small-v2,test,508,0.0,0.0 +e5-small-v2,test,513,0.5,1.0 +e5-small-v2,test,514,1.0,1.0 +e5-small-v2,test,516,1.0,1.0 +e5-small-v2,test,517,0.0,0.0 +e5-small-v2,test,521,0.63093,1.0 +e5-small-v2,test,525,1.0,1.0 +e5-small-v2,test,527,0.0,0.0 +e5-small-v2,test,528,1.0,1.0 +e5-small-v2,test,532,1.0,1.0 +e5-small-v2,test,533,1.0,1.0 +e5-small-v2,test,535,0.38685,0.0 +e5-small-v2,test,536,1.0,1.0 +e5-small-v2,test,539,1.0,1.0 +e5-small-v2,test,540,0.38685,1.0 +e5-small-v2,test,544,1.0,1.0 +e5-small-v2,test,549,1.0,1.0 +e5-small-v2,test,551,0.35621,0.0 +e5-small-v2,test,552,1.0,1.0 +e5-small-v2,test,554,0.0,0.0 +e5-small-v2,test,560,0.0,0.0 +e5-small-v2,test,569,0.63093,1.0 +e5-small-v2,test,575,0.63093,1.0 +e5-small-v2,test,577,0.30103,0.0 +e5-small-v2,test,578,0.5,1.0 +e5-small-v2,test,587,0.0,0.0 +e5-small-v2,test,589,1.0,1.0 +e5-small-v2,test,593,1.0,1.0 +e5-small-v2,test,597,0.90663,1.0 +e5-small-v2,test,598,0.30103,0.0 +e5-small-v2,test,613,1.0,1.0 +e5-small-v2,test,619,0.0,0.0 +e5-small-v2,test,623,0.0,0.0 +e5-small-v2,test,628,1.0,1.0 +e5-small-v2,test,636,1.0,1.0 +e5-small-v2,test,637,0.5,1.0 +e5-small-v2,test,641,1.0,1.0 +e5-small-v2,test,644,0.63093,1.0 +e5-small-v2,test,649,1.0,1.0 +e5-small-v2,test,659,0.35621,0.0 +e5-small-v2,test,660,0.0,0.0 +e5-small-v2,test,674,0.0,0.0 +e5-small-v2,test,684,1.0,1.0 +e5-small-v2,test,690,0.38685,0.0 +e5-small-v2,test,691,0.38685,0.0 +e5-small-v2,test,692,1.0,1.0 +e5-small-v2,test,693,1.0,1.0 +e5-small-v2,test,700,1.0,1.0 +e5-small-v2,test,702,1.0,1.0 +e5-small-v2,test,715,0.0,0.0 +e5-small-v2,test,716,0.0,0.0 +e5-small-v2,test,718,1.0,1.0 +e5-small-v2,test,721,1.0,1.0 +e5-small-v2,test,723,1.0,1.0 +e5-small-v2,test,727,0.33333,0.0 +e5-small-v2,test,728,0.83155,1.0 +e5-small-v2,test,729,0.63093,1.0 +e5-small-v2,test,742,0.63093,1.0 +e5-small-v2,test,743,0.5,1.0 +e5-small-v2,test,744,0.5,1.0 +e5-small-v2,test,756,0.63093,1.0 +e5-small-v2,test,759,1.0,1.0 +e5-small-v2,test,768,0.0,0.0 +e5-small-v2,test,770,1.0,1.0 +e5-small-v2,test,775,0.0,0.0 +e5-small-v2,test,781,1.0,1.0 +e5-small-v2,test,783,0.0,0.0 +e5-small-v2,test,784,0.5,1.0 +e5-small-v2,test,785,0.0,0.0 +e5-small-v2,test,793,0.5,1.0 +e5-small-v2,test,800,1.0,1.0 +e5-small-v2,test,805,1.0,1.0 +e5-small-v2,test,808,1.0,1.0 +e5-small-v2,test,811,1.0,1.0 +e5-small-v2,test,814,1.0,1.0 +e5-small-v2,test,820,0.5,1.0 +e5-small-v2,test,821,0.30103,0.0 +e5-small-v2,test,823,1.0,1.0 +e5-small-v2,test,830,0.0,0.0 +e5-small-v2,test,831,0.0,0.0 +e5-small-v2,test,832,1.0,1.0 +e5-small-v2,test,834,0.0,0.0 +e5-small-v2,test,837,0.43068,0.0 +e5-small-v2,test,839,1.0,1.0 +e5-small-v2,test,845,0.43068,0.0 +e5-small-v2,test,847,0.63093,1.0 +e5-small-v2,test,852,1.0,1.0 +e5-small-v2,test,859,1.0,1.0 +e5-small-v2,test,870,0.33333,0.0 +e5-small-v2,test,873,0.31565,1.0 +e5-small-v2,test,879,1.0,1.0 +e5-small-v2,test,880,1.0,1.0 +e5-small-v2,test,882,0.63093,1.0 +e5-small-v2,test,887,0.0,0.0 +e5-small-v2,test,903,1.0,1.0 +e5-small-v2,test,904,1.0,1.0 +e5-small-v2,test,907,1.0,1.0 +e5-small-v2,test,911,1.0,1.0 +e5-small-v2,test,913,0.0,0.0 +e5-small-v2,test,914,1.0,1.0 +e5-small-v2,test,921,1.0,1.0 +e5-small-v2,test,922,1.0,1.0 +e5-small-v2,test,936,1.0,1.0 +e5-small-v2,test,956,1.0,1.0 +e5-small-v2,test,957,1.0,1.0 +e5-small-v2,test,960,1.0,1.0 +e5-small-v2,test,967,0.85034,1.0 +e5-small-v2,test,971,0.92676,1.0 +e5-small-v2,test,975,0.0,0.0 +e5-small-v2,test,982,1.0,1.0 +e5-small-v2,test,985,1.0,1.0 +e5-small-v2,test,993,0.63093,1.0 +e5-small-v2,test,1012,1.0,1.0 +e5-small-v2,test,1014,1.0,1.0 +e5-small-v2,test,1019,1.0,1.0 +e5-small-v2,test,1020,1.0,1.0 +e5-small-v2,test,1021,1.0,1.0 +e5-small-v2,test,1024,0.63093,1.0 +e5-small-v2,test,1029,0.9469,1.0 +e5-small-v2,test,1041,0.45749,0.0 +e5-small-v2,test,1049,0.0,0.0 +e5-small-v2,test,1062,1.0,1.0 +e5-small-v2,test,1086,1.0,1.0 +e5-small-v2,test,1088,1.0,1.0 +e5-small-v2,test,1089,1.0,1.0 +e5-small-v2,test,1099,0.63093,1.0 +e5-small-v2,test,1100,0.63093,1.0 +e5-small-v2,test,1104,1.0,1.0 +e5-small-v2,test,1107,0.43068,0.0 +e5-small-v2,test,1110,0.0,0.0 +e5-small-v2,test,1121,1.0,1.0 +e5-small-v2,test,1130,1.0,1.0 +e5-small-v2,test,1132,0.30657,1.0 +e5-small-v2,test,1137,1.0,1.0 +e5-small-v2,test,1140,1.0,1.0 +e5-small-v2,test,1144,1.0,1.0 +e5-small-v2,test,1146,1.0,1.0 +e5-small-v2,test,1150,1.0,1.0 +e5-small-v2,test,1163,1.0,1.0 +e5-small-v2,test,1175,0.0,0.0 +e5-small-v2,test,1179,0.0,0.0 +e5-small-v2,test,1180,0.38685,0.0 +e5-small-v2,test,1185,1.0,1.0 +e5-small-v2,test,1187,0.63093,1.0 +e5-small-v2,test,1191,0.33333,0.0 +e5-small-v2,test,1194,1.0,1.0 +e5-small-v2,test,1196,0.43068,0.0 +e5-small-v2,test,1197,0.5,1.0 +e5-small-v2,test,1199,0.0,0.0 +e5-small-v2,test,1200,0.0,0.0 +e5-small-v2,test,1202,1.0,1.0 +e5-small-v2,test,1204,1.0,1.0 +e5-small-v2,test,1207,1.0,1.0 +e5-small-v2,test,1213,0.0,0.0 +e5-small-v2,test,1216,1.0,1.0 +e5-small-v2,test,1221,1.0,1.0 +e5-small-v2,test,1225,0.63093,1.0 +e5-small-v2,test,1226,0.0,0.0 +e5-small-v2,test,1232,1.0,1.0 +e5-small-v2,test,1241,1.0,1.0 +e5-small-v2,test,1245,0.0,0.0 +e5-small-v2,test,1259,1.0,1.0 +e5-small-v2,test,1262,1.0,1.0 +e5-small-v2,test,1266,1.0,1.0 +e5-small-v2,test,1270,1.0,1.0 +e5-small-v2,test,1271,1.0,1.0 +e5-small-v2,test,1272,0.0,0.0 +e5-small-v2,test,1273,1.0,1.0 +e5-small-v2,test,1274,0.93252,1.0 +e5-small-v2,test,1278,0.5,1.0 +e5-small-v2,test,1279,1.0,1.0 +e5-small-v2,test,1280,0.0,0.0 +e5-small-v2,test,1281,0.0,0.0 +e5-small-v2,test,1282,1.0,1.0 +e5-small-v2,test,1290,1.0,1.0 +e5-small-v2,test,1292,0.63093,1.0 +e5-small-v2,test,1298,1.0,1.0 +e5-small-v2,test,1303,1.0,1.0 +e5-small-v2,test,1316,0.0,0.0 +e5-small-v2,test,1319,0.5,1.0 +e5-small-v2,test,1320,0.63093,1.0 +e5-small-v2,test,1332,0.0,0.0 +e5-small-v2,test,1335,1.0,1.0 +e5-small-v2,test,1336,1.0,1.0 +e5-small-v2,test,1337,0.43068,0.0 +e5-small-v2,test,1339,1.0,1.0 +e5-small-v2,test,1344,1.0,1.0 +e5-small-v2,test,1352,1.0,1.0 +e5-small-v2,test,1359,1.0,1.0 +e5-small-v2,test,1362,0.0,0.0 +e5-small-v2,test,1363,0.0,0.0 +e5-small-v2,test,1368,1.0,1.0 +e5-small-v2,test,1370,1.0,1.0 +e5-small-v2,test,1379,0.94387,1.0 +e5-small-v2,test,1382,1.0,1.0 +e5-small-v2,test,1385,1.0,1.0 +e5-small-v2,test,1389,1.0,1.0 +e5-small-v2,test,1395,0.0,0.0 diff --git a/evaluation/embedder_choice_scifact.json b/evaluation/embedder_choice_scifact.json new file mode 100644 index 0000000..108d792 --- /dev/null +++ b/evaluation/embedder_choice_scifact.json @@ -0,0 +1,86 @@ +{ + "dataset": "scifact", + "passages": 5183, + "candidates": { + "all-MiniLM-L6-v2": { + "repo": "sentence-transformers/all-MiniLM-L6-v2", + "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", + "minutes_to_embed_passages": 2.5, + "train": { + "ndcg@10": 0.6602, + "hit@3": 0.7058, + "questions": 809 + }, + "test": { + "ndcg@10": 0.6451, + "hit@3": 0.6833, + "questions": 300 + } + }, + "bge-small-en-v1.5": { + "repo": "BAAI/bge-small-en-v1.5", + "revision": "5c38ec7c405ec4b44b94cc5a9bb96e735b38267a", + "minutes_to_embed_passages": 6.9, + "train": { + "ndcg@10": 0.7469, + "hit@3": 0.7911, + "questions": 809 + }, + "test": { + "ndcg@10": 0.7127, + "hit@3": 0.73, + "questions": 300 + } + }, + "e5-small-v2": { + "repo": "intfloat/e5-small-v2", + "revision": "ffb93f3bd4047442299a41ebb6fa998a38507c52", + "minutes_to_embed_passages": 5.7, + "train": { + "ndcg@10": 0.7078, + "hit@3": 0.7466, + "questions": 809 + }, + "test": { + "ndcg@10": 0.6875, + "hit@3": 0.7233, + "questions": 300 + } + } + }, + "difference_new_minus_old": { + "train": { + "ndcg10": { + "difference": 0.0867, + "interval_95": [ + 0.0683, + 0.105 + ] + }, + "hit3": { + "difference": 0.0853, + "interval_95": [ + 0.0618, + 0.11 + ] + } + }, + "test": { + "ndcg10": { + "difference": 0.0676, + "interval_95": [ + 0.0413, + 0.0941 + ] + }, + "hit3": { + "difference": 0.0467, + "interval_95": [ + 0.0133, + 0.0833 + ] + } + } + }, + "chosen_on_train": "bge-small-en-v1.5" +} diff --git a/evaluation/per_question_scifact.csv b/evaluation/per_question_scifact_all-MiniLM-L6-v2.csv similarity index 100% rename from evaluation/per_question_scifact.csv rename to evaluation/per_question_scifact_all-MiniLM-L6-v2.csv diff --git a/evaluation/per_question_scifact_bge-small-en-v1.5.csv b/evaluation/per_question_scifact_bge-small-en-v1.5.csv new file mode 100644 index 0000000..49aa6bf --- /dev/null +++ b/evaluation/per_question_scifact_bge-small-en-v1.5.csv @@ -0,0 +1,1801 @@ +setup,query_id,returned,ms,ndcg10,recall10,mrr10,hit3 +dense,1,10,21.92379,0.0,0.0,0.0,0.0 +dense,3,10,17.64912,0.38685,1.0,0.2,0.0 +dense,5,10,17.67758,0.63093,1.0,0.5,1.0 +dense,13,10,19.02904,0.0,0.0,0.0,0.0 +dense,36,10,17.36683,0.41444,1.0,0.2,0.0 +dense,42,10,19.79746,1.0,1.0,1.0,1.0 +dense,48,10,18.88342,0.43068,1.0,0.25,0.0 +dense,49,10,17.422,1.0,1.0,1.0,1.0 +dense,50,10,18.31404,1.0,1.0,1.0,1.0 +dense,51,10,18.13383,1.0,1.0,1.0,1.0 +dense,53,10,17.36267,1.0,1.0,1.0,1.0 +dense,54,10,17.81125,1.0,1.0,1.0,1.0 +dense,56,10,19.43025,1.0,1.0,1.0,1.0 +dense,57,10,19.50729,1.0,1.0,1.0,1.0 +dense,70,10,17.81438,0.42279,1.0,0.16667,0.0 +dense,72,10,16.91242,1.0,1.0,1.0,1.0 +dense,75,10,17.27446,1.0,1.0,1.0,1.0 +dense,94,10,19.45717,1.0,1.0,1.0,1.0 +dense,99,10,18.12637,0.63093,1.0,0.5,1.0 +dense,100,10,17.88117,0.63093,1.0,0.5,1.0 +dense,113,10,16.99137,1.0,1.0,1.0,1.0 +dense,115,10,18.39362,1.0,1.0,1.0,1.0 +dense,118,10,18.76962,1.0,1.0,1.0,1.0 +dense,124,10,17.54479,1.0,1.0,1.0,1.0 +dense,127,10,18.16475,1.0,1.0,1.0,1.0 +dense,128,10,18.88054,0.0,0.0,0.0,0.0 +dense,129,10,18.8725,1.0,1.0,1.0,1.0 +dense,130,10,18.12538,1.0,1.0,1.0,1.0 +dense,132,10,17.94087,0.0,0.0,0.0,0.0 +dense,133,10,17.06517,0.78601,0.8,1.0,1.0 +dense,137,10,19.32346,1.0,1.0,1.0,1.0 +dense,141,10,17.92463,0.87722,1.0,1.0,1.0 +dense,142,10,16.91692,1.0,1.0,1.0,1.0 +dense,143,10,17.02267,1.0,1.0,1.0,1.0 +dense,146,10,19.92954,1.0,1.0,1.0,1.0 +dense,148,10,18.16721,1.0,1.0,1.0,1.0 +dense,163,10,19.30558,1.0,1.0,1.0,1.0 +dense,171,10,17.79146,1.0,1.0,1.0,1.0 +dense,179,10,18.1785,0.97093,1.0,1.0,1.0 +dense,180,10,18.37483,1.0,1.0,1.0,1.0 +dense,183,10,17.88429,0.63093,1.0,0.5,1.0 +dense,185,10,19.02025,0.5,1.0,0.33333,1.0 +dense,198,10,19.04163,0.30103,1.0,0.11111,0.0 +dense,208,10,19.01158,1.0,1.0,1.0,1.0 +dense,212,10,18.78025,0.35621,1.0,0.16667,0.0 +dense,213,10,17.82879,0.31546,1.0,0.125,0.0 +dense,216,10,18.24633,1.0,1.0,1.0,1.0 +dense,217,10,19.04217,1.0,1.0,1.0,1.0 +dense,218,10,18.02729,1.0,1.0,1.0,1.0 +dense,219,10,17.42,1.0,1.0,1.0,1.0 +dense,230,10,17.90854,1.0,1.0,1.0,1.0 +dense,232,10,17.92754,0.63093,1.0,0.5,1.0 +dense,233,10,19.42696,1.0,1.0,1.0,1.0 +dense,236,10,20.092,1.0,1.0,1.0,1.0 +dense,237,10,18.90004,0.5,1.0,0.33333,1.0 +dense,238,10,19.47221,0.5,1.0,0.33333,1.0 +dense,239,10,19.33121,0.0,0.0,0.0,0.0 +dense,248,10,18.65875,1.0,1.0,1.0,1.0 +dense,249,10,18.3585,1.0,1.0,1.0,1.0 +dense,261,10,16.72688,0.21841,0.5,0.16667,0.0 +dense,268,10,18.29742,1.0,1.0,1.0,1.0 +dense,269,10,18.86446,1.0,1.0,1.0,1.0 +dense,274,10,18.69212,1.0,1.0,1.0,1.0 +dense,275,10,18.96967,0.63644,0.66667,1.0,1.0 +dense,279,10,18.85125,1.0,1.0,1.0,1.0 +dense,294,10,17.96875,0.30103,1.0,0.11111,0.0 +dense,295,10,17.33179,0.31546,1.0,0.125,0.0 +dense,298,10,15.91642,1.0,1.0,1.0,1.0 +dense,300,10,18.56292,0.63093,1.0,0.5,1.0 +dense,303,10,18.33492,0.0,0.0,0.0,0.0 +dense,312,10,18.50088,0.0,0.0,0.0,0.0 +dense,314,10,21.29704,0.0,0.0,0.0,0.0 +dense,324,10,17.78054,0.0,0.0,0.0,0.0 +dense,327,10,17.92571,1.0,1.0,1.0,1.0 +dense,338,10,17.37879,1.0,1.0,1.0,1.0 +dense,343,10,20.19533,1.0,1.0,1.0,1.0 +dense,350,10,15.41821,1.0,1.0,1.0,1.0 +dense,354,10,19.63037,1.0,1.0,1.0,1.0 +dense,362,10,16.26629,1.0,1.0,1.0,1.0 +dense,380,10,17.7035,1.0,1.0,1.0,1.0 +dense,384,10,16.61871,0.63093,1.0,0.5,1.0 +dense,385,10,17.98892,0.57143,1.0,0.5,1.0 +dense,386,10,17.16617,1.0,1.0,1.0,1.0 +dense,388,10,18.58538,1.0,1.0,1.0,1.0 +dense,399,10,17.23242,1.0,1.0,1.0,1.0 +dense,410,10,18.45925,1.0,1.0,1.0,1.0 +dense,411,10,18.22587,1.0,1.0,1.0,1.0 +dense,415,10,17.724,0.38685,1.0,0.2,0.0 +dense,421,10,17.42029,0.0,0.0,0.0,0.0 +dense,431,10,17.41046,0.0,0.0,0.0,0.0 +dense,436,10,18.1585,1.0,1.0,1.0,1.0 +dense,437,10,19.68871,0.0,0.0,0.0,0.0 +dense,439,10,17.59379,1.0,1.0,1.0,1.0 +dense,440,10,18.41304,1.0,1.0,1.0,1.0 +dense,443,10,17.60025,1.0,1.0,1.0,1.0 +dense,452,10,17.79354,0.61315,0.5,1.0,1.0 +dense,475,10,16.76967,0.0,0.0,0.0,0.0 +dense,478,10,18.31092,1.0,1.0,1.0,1.0 +dense,491,10,17.30225,1.0,1.0,1.0,1.0 +dense,501,10,18.94392,1.0,1.0,1.0,1.0 +dense,502,10,18.29421,0.0,0.0,0.0,0.0 +dense,507,10,18.88421,0.38685,1.0,0.2,0.0 +dense,508,10,17.55162,0.0,0.0,0.0,0.0 +dense,513,10,18.58558,0.63093,1.0,0.5,1.0 +dense,514,10,18.38996,1.0,1.0,1.0,1.0 +dense,516,10,19.19004,1.0,1.0,1.0,1.0 +dense,517,10,16.75554,0.0,0.0,0.0,0.0 +dense,521,10,18.41471,1.0,1.0,1.0,1.0 +dense,525,10,18.87921,1.0,1.0,1.0,1.0 +dense,527,10,17.86275,0.43068,1.0,0.25,0.0 +dense,528,10,19.76312,1.0,1.0,1.0,1.0 +dense,532,10,18.37183,1.0,1.0,1.0,1.0 +dense,533,10,18.51542,1.0,1.0,1.0,1.0 +dense,535,10,18.96287,0.63093,1.0,0.5,1.0 +dense,536,10,16.83783,1.0,1.0,1.0,1.0 +dense,539,10,18.16433,1.0,1.0,1.0,1.0 +dense,540,10,18.10354,0.61315,0.5,1.0,1.0 +dense,544,10,18.42842,0.35621,1.0,0.16667,0.0 +dense,549,10,18.56446,1.0,1.0,1.0,1.0 +dense,551,10,17.35162,0.0,0.0,0.0,0.0 +dense,552,10,15.80904,1.0,1.0,1.0,1.0 +dense,554,10,18.49442,0.38685,1.0,0.2,0.0 +dense,560,10,20.09654,0.0,0.0,0.0,0.0 +dense,569,10,18.14462,0.63093,1.0,0.5,1.0 +dense,575,10,18.7235,1.0,1.0,1.0,1.0 +dense,577,10,17.13038,0.38685,1.0,0.2,0.0 +dense,578,10,18.25108,1.0,1.0,1.0,1.0 +dense,587,10,18.08554,0.5,1.0,0.33333,1.0 +dense,589,10,19.27625,1.0,1.0,1.0,1.0 +dense,593,10,17.09029,1.0,1.0,1.0,1.0 +dense,597,10,18.49342,1.0,1.0,1.0,1.0 +dense,598,10,18.88125,1.0,1.0,1.0,1.0 +dense,613,10,19.8255,1.0,1.0,1.0,1.0 +dense,619,10,16.99867,0.0,0.0,0.0,0.0 +dense,623,10,17.83471,0.35621,1.0,0.16667,0.0 +dense,628,10,17.69479,1.0,1.0,1.0,1.0 +dense,636,10,17.18208,1.0,1.0,1.0,1.0 +dense,637,10,16.79358,0.63093,1.0,0.5,1.0 +dense,641,10,18.44575,0.79772,1.0,1.0,1.0 +dense,644,10,19.41279,1.0,1.0,1.0,1.0 +dense,649,10,18.32008,1.0,1.0,1.0,1.0 +dense,659,10,16.29967,0.43068,1.0,0.25,0.0 +dense,660,10,18.22825,0.0,0.0,0.0,0.0 +dense,674,10,19.34037,0.28906,1.0,0.1,0.0 +dense,684,10,17.573,0.5,1.0,0.33333,1.0 +dense,690,10,18.04342,0.0,0.0,0.0,0.0 +dense,691,10,17.83771,0.63093,1.0,0.5,1.0 +dense,692,10,19.37438,1.0,1.0,1.0,1.0 +dense,693,10,17.64454,1.0,1.0,1.0,1.0 +dense,700,10,17.9935,1.0,1.0,1.0,1.0 +dense,702,10,17.77121,1.0,1.0,1.0,1.0 +dense,715,10,18.69529,0.0,0.0,0.0,0.0 +dense,716,10,18.36079,0.0,0.0,0.0,0.0 +dense,718,10,18.82646,1.0,1.0,1.0,1.0 +dense,721,10,19.20563,1.0,1.0,1.0,1.0 +dense,723,10,14.46842,1.0,1.0,1.0,1.0 +dense,727,10,17.54262,0.35621,1.0,0.16667,0.0 +dense,728,10,16.96233,0.83155,1.0,1.0,1.0 +dense,729,10,16.33642,1.0,1.0,1.0,1.0 +dense,742,10,17.20317,1.0,1.0,1.0,1.0 +dense,743,10,18.12408,1.0,1.0,1.0,1.0 +dense,744,10,17.91617,0.5,1.0,0.33333,1.0 +dense,756,10,16.67967,0.5,1.0,0.33333,1.0 +dense,759,10,19.30617,1.0,1.0,1.0,1.0 +dense,768,10,17.53967,0.0,0.0,0.0,0.0 +dense,770,10,18.96608,1.0,1.0,1.0,1.0 +dense,775,10,16.78925,0.38685,1.0,0.2,0.0 +dense,781,10,19.03167,1.0,1.0,1.0,1.0 +dense,783,10,15.52267,0.31546,1.0,0.125,0.0 +dense,784,10,18.56804,1.0,1.0,1.0,1.0 +dense,785,10,19.39825,0.43068,1.0,0.25,0.0 +dense,793,10,18.30717,0.63093,1.0,0.5,1.0 +dense,800,10,17.88796,0.5,1.0,0.33333,1.0 +dense,805,10,17.99792,1.0,1.0,1.0,1.0 +dense,808,10,18.07912,1.0,1.0,1.0,1.0 +dense,811,10,17.14408,1.0,1.0,1.0,1.0 +dense,814,10,17.87075,1.0,1.0,1.0,1.0 +dense,820,10,18.62192,0.0,0.0,0.0,0.0 +dense,821,10,16.78088,0.0,0.0,0.0,0.0 +dense,823,10,18.66808,1.0,1.0,1.0,1.0 +dense,830,10,17.62546,0.0,0.0,0.0,0.0 +dense,831,10,18.25217,0.0,0.0,0.0,0.0 +dense,832,10,17.05754,1.0,1.0,1.0,1.0 +dense,834,10,19.02029,0.0,0.0,0.0,0.0 +dense,837,10,18.12383,0.63093,1.0,0.5,1.0 +dense,839,10,20.16146,1.0,1.0,1.0,1.0 +dense,845,10,16.50921,0.38685,1.0,0.2,0.0 +dense,847,10,16.52038,1.0,1.0,1.0,1.0 +dense,852,10,18.9415,0.63093,1.0,0.5,1.0 +dense,859,10,19.04833,0.63093,1.0,0.5,1.0 +dense,870,10,17.80883,0.5,1.0,0.33333,1.0 +dense,873,10,18.16163,0.36005,0.4,0.5,1.0 +dense,879,10,17.26562,1.0,1.0,1.0,1.0 +dense,880,10,18.43687,1.0,1.0,1.0,1.0 +dense,882,10,17.12587,1.0,1.0,1.0,1.0 +dense,887,10,18.76042,0.0,0.0,0.0,0.0 +dense,903,10,18.29433,1.0,1.0,1.0,1.0 +dense,904,10,18.84992,1.0,1.0,1.0,1.0 +dense,907,10,16.87975,1.0,1.0,1.0,1.0 +dense,911,10,18.89712,1.0,1.0,1.0,1.0 +dense,913,10,19.06442,0.0,0.0,0.0,0.0 +dense,914,10,18.55408,0.0,0.0,0.0,0.0 +dense,921,10,17.09112,1.0,1.0,1.0,1.0 +dense,922,10,17.04108,1.0,1.0,1.0,1.0 +dense,936,10,18.19183,1.0,1.0,1.0,1.0 +dense,956,10,19.12225,1.0,1.0,1.0,1.0 +dense,957,10,17.535,1.0,1.0,1.0,1.0 +dense,960,10,18.07129,1.0,1.0,1.0,1.0 +dense,967,10,18.27117,1.0,1.0,1.0,1.0 +dense,971,10,19.05733,0.90587,1.0,1.0,1.0 +dense,975,10,19.24421,0.0,0.0,0.0,0.0 +dense,982,10,18.56942,1.0,1.0,1.0,1.0 +dense,985,10,15.96283,1.0,1.0,1.0,1.0 +dense,993,10,19.57021,1.0,1.0,1.0,1.0 +dense,1012,10,18.00937,1.0,1.0,1.0,1.0 +dense,1014,10,19.42608,1.0,1.0,1.0,1.0 +dense,1019,10,19.07412,0.5,1.0,0.33333,1.0 +dense,1020,10,18.33171,0.63093,1.0,0.5,1.0 +dense,1021,10,18.86262,0.63093,1.0,0.5,1.0 +dense,1024,10,17.58325,0.33333,1.0,0.14286,0.0 +dense,1029,10,19.99613,0.96747,1.0,1.0,1.0 +dense,1041,10,19.67912,0.42177,1.0,0.2,0.0 +dense,1049,10,18.70463,0.35621,1.0,0.16667,0.0 +dense,1062,10,19.00292,1.0,1.0,1.0,1.0 +dense,1086,10,17.733,1.0,1.0,1.0,1.0 +dense,1088,10,18.43196,1.0,1.0,1.0,1.0 +dense,1089,10,17.44371,1.0,1.0,1.0,1.0 +dense,1099,10,18.90542,0.31546,1.0,0.125,0.0 +dense,1100,10,18.60342,0.0,0.0,0.0,0.0 +dense,1104,10,17.06663,1.0,1.0,1.0,1.0 +dense,1107,10,20.02825,1.0,1.0,1.0,1.0 +dense,1110,10,17.65958,0.0,0.0,0.0,0.0 +dense,1121,10,18.57888,0.63093,1.0,0.5,1.0 +dense,1130,10,18.07725,1.0,1.0,1.0,1.0 +dense,1132,10,19.33921,0.19343,0.5,0.125,0.0 +dense,1137,10,18.2615,1.0,1.0,1.0,1.0 +dense,1140,10,18.08925,1.0,1.0,1.0,1.0 +dense,1144,10,18.64142,1.0,1.0,1.0,1.0 +dense,1146,10,18.56467,1.0,1.0,1.0,1.0 +dense,1150,10,17.83358,1.0,1.0,1.0,1.0 +dense,1163,10,17.67633,1.0,1.0,1.0,1.0 +dense,1175,10,19.43833,0.0,0.0,0.0,0.0 +dense,1179,10,17.13537,0.0,0.0,0.0,0.0 +dense,1180,10,18.01075,0.63093,1.0,0.5,1.0 +dense,1185,10,17.04292,1.0,1.0,1.0,1.0 +dense,1187,10,16.70354,0.63093,1.0,0.5,1.0 +dense,1191,10,18.67862,0.0,0.0,0.0,0.0 +dense,1194,10,19.11404,1.0,1.0,1.0,1.0 +dense,1196,10,18.05837,0.38685,1.0,0.2,0.0 +dense,1197,10,17.66275,0.38685,1.0,0.2,0.0 +dense,1199,10,16.17721,0.0,0.0,0.0,0.0 +dense,1200,10,16.91117,0.28906,1.0,0.1,0.0 +dense,1202,10,16.91162,1.0,1.0,1.0,1.0 +dense,1204,10,23.7575,1.0,1.0,1.0,1.0 +dense,1207,10,21.84492,1.0,1.0,1.0,1.0 +dense,1213,10,25.22396,0.0,0.0,0.0,0.0 +dense,1216,10,20.53908,1.0,1.0,1.0,1.0 +dense,1221,10,17.51463,0.31546,1.0,0.125,0.0 +dense,1225,10,15.66987,1.0,1.0,1.0,1.0 +dense,1226,10,21.26383,0.0,0.0,0.0,0.0 +dense,1232,10,23.41475,1.0,1.0,1.0,1.0 +dense,1241,10,20.695,1.0,1.0,1.0,1.0 +dense,1245,10,21.21208,0.63093,1.0,0.5,1.0 +dense,1259,10,18.31554,1.0,1.0,1.0,1.0 +dense,1262,10,20.57817,1.0,1.0,1.0,1.0 +dense,1266,10,17.58063,1.0,1.0,1.0,1.0 +dense,1270,10,20.15196,1.0,1.0,1.0,1.0 +dense,1271,10,18.06692,1.0,1.0,1.0,1.0 +dense,1272,10,18.72275,0.0,0.0,0.0,0.0 +dense,1273,10,17.78258,1.0,1.0,1.0,1.0 +dense,1274,10,22.11083,0.83855,1.0,1.0,1.0 +dense,1278,10,20.843,1.0,1.0,1.0,1.0 +dense,1279,10,19.61712,1.0,1.0,1.0,1.0 +dense,1280,10,18.55429,0.35621,1.0,0.16667,0.0 +dense,1281,10,21.512,0.33333,1.0,0.14286,0.0 +dense,1282,10,19.72779,1.0,1.0,1.0,1.0 +dense,1290,10,24.09988,1.0,1.0,1.0,1.0 +dense,1292,10,27.74408,1.0,1.0,1.0,1.0 +dense,1298,10,18.54983,1.0,1.0,1.0,1.0 +dense,1303,10,21.02525,1.0,1.0,1.0,1.0 +dense,1316,10,19.74963,0.0,0.0,0.0,0.0 +dense,1319,10,18.63537,1.0,1.0,1.0,1.0 +dense,1320,10,19.04812,1.0,1.0,1.0,1.0 +dense,1332,10,25.81487,0.0,0.0,0.0,0.0 +dense,1335,10,23.21608,1.0,1.0,1.0,1.0 +dense,1336,10,20.26392,0.63093,1.0,0.5,1.0 +dense,1337,10,20.10217,1.0,1.0,1.0,1.0 +dense,1339,10,17.68583,1.0,1.0,1.0,1.0 +dense,1344,10,17.42496,0.63093,1.0,0.5,1.0 +dense,1352,10,19.37008,1.0,1.0,1.0,1.0 +dense,1359,10,16.35471,1.0,1.0,1.0,1.0 +dense,1362,10,18.74483,0.0,0.0,0.0,0.0 +dense,1363,10,20.281,0.0,0.0,0.0,0.0 +dense,1368,10,17.3875,1.0,1.0,1.0,1.0 +dense,1370,10,16.64792,1.0,1.0,1.0,1.0 +dense,1379,10,17.91012,0.97093,1.0,1.0,1.0 +dense,1382,10,20.04542,1.0,1.0,1.0,1.0 +dense,1385,10,18.05079,1.0,1.0,1.0,1.0 +dense,1389,10,20.50004,1.0,1.0,1.0,1.0 +dense,1395,10,18.17408,0.5,1.0,0.33333,1.0 +bm25,1,10,6.29533,0.0,0.0,0.0,0.0 +bm25,3,10,17.07762,1.0,1.0,1.0,1.0 +bm25,5,10,7.53017,1.0,1.0,1.0,1.0 +bm25,13,10,9.35629,0.0,0.0,0.0,0.0 +bm25,36,10,8.60604,0.21841,0.5,0.16667,0.0 +bm25,42,10,11.53825,1.0,1.0,1.0,1.0 +bm25,48,10,12.57783,0.63093,1.0,0.5,1.0 +bm25,49,10,7.59408,1.0,1.0,1.0,1.0 +bm25,50,10,7.06679,1.0,1.0,1.0,1.0 +bm25,51,10,9.15279,1.0,1.0,1.0,1.0 +bm25,53,10,9.03096,1.0,1.0,1.0,1.0 +bm25,54,10,12.38325,0.35621,1.0,0.16667,0.0 +bm25,56,10,14.15092,1.0,1.0,1.0,1.0 +bm25,57,10,13.14,1.0,1.0,1.0,1.0 +bm25,70,10,5.65029,1.0,1.0,1.0,1.0 +bm25,72,10,7.053,0.30103,1.0,0.11111,0.0 +bm25,75,10,12.56671,0.63093,1.0,0.5,1.0 +bm25,94,10,6.58108,1.0,1.0,1.0,1.0 +bm25,99,10,10.13437,1.0,1.0,1.0,1.0 +bm25,100,10,7.61533,1.0,1.0,1.0,1.0 +bm25,113,10,11.47121,1.0,1.0,1.0,1.0 +bm25,115,10,9.39625,1.0,1.0,1.0,1.0 +bm25,118,10,10.92121,1.0,1.0,1.0,1.0 +bm25,124,10,11.67704,1.0,1.0,1.0,1.0 +bm25,127,10,9.75058,0.5,1.0,0.33333,1.0 +bm25,128,10,7.58013,0.5,1.0,0.33333,1.0 +bm25,129,10,12.31604,1.0,1.0,1.0,1.0 +bm25,130,10,13.07088,1.0,1.0,1.0,1.0 +bm25,132,10,5.93233,0.0,0.0,0.0,0.0 +bm25,133,10,17.48721,0.13121,0.2,0.2,0.0 +bm25,137,10,11.31421,1.0,1.0,1.0,1.0 +bm25,141,10,10.59779,1.0,1.0,1.0,1.0 +bm25,142,10,18.22817,1.0,1.0,1.0,1.0 +bm25,143,10,15.32621,1.0,1.0,1.0,1.0 +bm25,146,10,16.63788,1.0,1.0,1.0,1.0 +bm25,148,10,4.85629,1.0,1.0,1.0,1.0 +bm25,163,10,8.27675,1.0,1.0,1.0,1.0 +bm25,171,10,9.26242,1.0,1.0,1.0,1.0 +bm25,179,10,7.60013,0.58557,0.5,1.0,1.0 +bm25,180,10,18.346,1.0,1.0,1.0,1.0 +bm25,183,10,7.21667,0.31546,1.0,0.125,0.0 +bm25,185,10,8.48408,0.63093,1.0,0.5,1.0 +bm25,198,10,4.95392,0.5,1.0,0.33333,1.0 +bm25,208,10,6.13283,1.0,1.0,1.0,1.0 +bm25,212,10,6.40204,1.0,1.0,1.0,1.0 +bm25,213,10,11.86225,0.35621,1.0,0.16667,0.0 +bm25,216,10,8.83038,1.0,1.0,1.0,1.0 +bm25,217,10,8.12929,1.0,1.0,1.0,1.0 +bm25,218,10,7.65321,1.0,1.0,1.0,1.0 +bm25,219,10,8.38029,1.0,1.0,1.0,1.0 +bm25,230,10,11.84871,1.0,1.0,1.0,1.0 +bm25,232,10,11.59871,0.63093,1.0,0.5,1.0 +bm25,233,10,10.14508,1.0,1.0,1.0,1.0 +bm25,236,10,8.16079,1.0,1.0,1.0,1.0 +bm25,237,10,9.92054,1.0,1.0,1.0,1.0 +bm25,238,10,6.359,0.0,0.0,0.0,0.0 +bm25,239,10,7.08654,0.0,0.0,0.0,0.0 +bm25,248,10,6.65221,1.0,1.0,1.0,1.0 +bm25,249,10,5.54625,0.63093,1.0,0.5,1.0 +bm25,261,10,10.11713,0.26407,0.5,0.25,0.0 +bm25,268,10,4.94325,0.63093,1.0,0.5,1.0 +bm25,269,10,3.70229,0.63093,1.0,0.5,1.0 +bm25,274,10,16.573,1.0,1.0,1.0,1.0 +bm25,275,10,13.31738,0.46928,0.33333,1.0,1.0 +bm25,279,10,7.7315,1.0,1.0,1.0,1.0 +bm25,294,10,9.67812,1.0,1.0,1.0,1.0 +bm25,295,10,14.59117,1.0,1.0,1.0,1.0 +bm25,298,10,10.99329,1.0,1.0,1.0,1.0 +bm25,300,10,22.46192,0.43068,1.0,0.25,0.0 +bm25,303,10,11.24275,0.63093,1.0,0.5,1.0 +bm25,312,10,11.34567,0.38685,1.0,0.2,0.0 +bm25,314,10,20.01087,0.5,1.0,0.33333,1.0 +bm25,324,10,5.25067,0.43068,1.0,0.25,0.0 +bm25,327,10,9.35642,1.0,1.0,1.0,1.0 +bm25,338,10,5.718,1.0,1.0,1.0,1.0 +bm25,343,10,16.16967,0.65092,1.0,0.5,1.0 +bm25,350,10,15.03671,1.0,1.0,1.0,1.0 +bm25,354,10,9.19833,1.0,1.0,1.0,1.0 +bm25,362,10,21.25671,1.0,1.0,1.0,1.0 +bm25,380,10,10.49937,1.0,1.0,1.0,1.0 +bm25,384,10,10.57408,0.0,0.0,0.0,0.0 +bm25,385,10,9.78029,0.30657,0.5,0.33333,1.0 +bm25,386,10,12.92833,1.0,1.0,1.0,1.0 +bm25,388,10,8.13271,0.5,1.0,0.33333,1.0 +bm25,399,10,8.94883,1.0,1.0,1.0,1.0 +bm25,410,10,8.9345,1.0,1.0,1.0,1.0 +bm25,411,10,8.43583,1.0,1.0,1.0,1.0 +bm25,415,10,11.08504,0.38685,1.0,0.2,0.0 +bm25,421,10,11.14942,0.0,0.0,0.0,0.0 +bm25,431,10,10.54008,0.0,0.0,0.0,0.0 +bm25,436,10,11.43858,1.0,1.0,1.0,1.0 +bm25,437,10,17.01575,0.0,0.0,0.0,0.0 +bm25,439,10,12.77575,0.63093,1.0,0.5,1.0 +bm25,440,10,12.81179,0.63093,1.0,0.5,1.0 +bm25,443,10,9.66425,1.0,1.0,1.0,1.0 +bm25,452,10,8.47329,0.2372,0.5,0.2,0.0 +bm25,475,10,9.011,0.0,0.0,0.0,0.0 +bm25,478,10,24.15158,0.63093,1.0,0.5,1.0 +bm25,491,10,12.53696,1.0,1.0,1.0,1.0 +bm25,501,10,7.14267,0.63093,1.0,0.5,1.0 +bm25,502,10,13.11929,0.0,0.0,0.0,0.0 +bm25,507,10,12.38358,1.0,1.0,1.0,1.0 +bm25,508,10,9.88996,0.0,0.0,0.0,0.0 +bm25,513,10,6.29787,1.0,1.0,1.0,1.0 +bm25,514,10,16.90458,0.0,0.0,0.0,0.0 +bm25,516,10,12.44546,1.0,1.0,1.0,1.0 +bm25,517,10,6.66379,0.0,0.0,0.0,0.0 +bm25,521,10,21.60029,1.0,1.0,1.0,1.0 +bm25,525,10,17.43475,1.0,1.0,1.0,1.0 +bm25,527,10,14.85658,0.63093,1.0,0.5,1.0 +bm25,528,10,21.84775,1.0,1.0,1.0,1.0 +bm25,532,10,5.64021,1.0,1.0,1.0,1.0 +bm25,533,10,6.09296,1.0,1.0,1.0,1.0 +bm25,535,10,8.31996,0.0,0.0,0.0,0.0 +bm25,536,10,5.95467,1.0,1.0,1.0,1.0 +bm25,539,10,5.92358,1.0,1.0,1.0,1.0 +bm25,540,10,7.0175,0.2372,0.5,0.2,0.0 +bm25,544,10,8.43867,0.0,0.0,0.0,0.0 +bm25,549,10,6.00846,1.0,1.0,1.0,1.0 +bm25,551,10,19.21742,0.43068,1.0,0.25,0.0 +bm25,552,10,19.65871,1.0,1.0,1.0,1.0 +bm25,554,10,11.36246,0.33333,1.0,0.14286,0.0 +bm25,560,10,12.57958,0.0,0.0,0.0,0.0 +bm25,569,10,8.30721,1.0,1.0,1.0,1.0 +bm25,575,10,9.92204,0.0,0.0,0.0,0.0 +bm25,577,10,20.13325,0.0,0.0,0.0,0.0 +bm25,578,10,9.95008,1.0,1.0,1.0,1.0 +bm25,587,10,22.24479,0.43068,1.0,0.25,0.0 +bm25,589,10,17.21012,1.0,1.0,1.0,1.0 +bm25,593,10,9.63867,1.0,1.0,1.0,1.0 +bm25,597,10,6.51542,0.93252,1.0,1.0,1.0 +bm25,598,10,15.40742,0.35621,1.0,0.16667,0.0 +bm25,613,10,8.71104,1.0,1.0,1.0,1.0 +bm25,619,10,11.72787,0.2372,0.5,0.2,0.0 +bm25,623,10,10.82529,0.0,0.0,0.0,0.0 +bm25,628,10,13.77529,1.0,1.0,1.0,1.0 +bm25,636,10,10.60442,0.5,1.0,0.33333,1.0 +bm25,637,10,10.50217,0.28906,1.0,0.1,0.0 +bm25,641,10,7.96275,0.61315,0.5,1.0,1.0 +bm25,644,10,6.88983,0.63093,1.0,0.5,1.0 +bm25,649,10,10.94979,1.0,1.0,1.0,1.0 +bm25,659,10,6.20738,1.0,1.0,1.0,1.0 +bm25,660,10,5.57962,0.38685,1.0,0.2,0.0 +bm25,674,10,10.72238,1.0,1.0,1.0,1.0 +bm25,684,10,11.37117,1.0,1.0,1.0,1.0 +bm25,690,10,18.04729,0.0,0.0,0.0,0.0 +bm25,691,10,11.60492,1.0,1.0,1.0,1.0 +bm25,692,10,8.93612,0.63093,1.0,0.5,1.0 +bm25,693,10,9.63412,1.0,1.0,1.0,1.0 +bm25,700,10,9.9095,0.63093,1.0,0.5,1.0 +bm25,702,10,9.91521,0.5,1.0,0.33333,1.0 +bm25,715,10,11.86879,0.0,0.0,0.0,0.0 +bm25,716,10,8.08167,0.0,0.0,0.0,0.0 +bm25,718,10,7.53596,0.63093,1.0,0.5,1.0 +bm25,721,10,10.53037,1.0,1.0,1.0,1.0 +bm25,723,10,13.82679,1.0,1.0,1.0,1.0 +bm25,727,10,11.71587,0.63093,1.0,0.5,1.0 +bm25,728,10,9.38396,0.60526,1.0,0.5,1.0 +bm25,729,10,10.55496,0.63093,1.0,0.5,1.0 +bm25,742,10,7.17479,1.0,1.0,1.0,1.0 +bm25,743,10,3.70217,1.0,1.0,1.0,1.0 +bm25,744,10,12.30004,0.63093,1.0,0.5,1.0 +bm25,756,10,12.77854,0.30103,1.0,0.11111,0.0 +bm25,759,10,15.66808,1.0,1.0,1.0,1.0 +bm25,768,10,9.97662,0.0,0.0,0.0,0.0 +bm25,770,10,21.01017,1.0,1.0,1.0,1.0 +bm25,775,10,12.95421,0.0,0.0,0.0,0.0 +bm25,781,10,15.74463,0.63093,1.0,0.5,1.0 +bm25,783,10,16.54171,0.0,0.0,0.0,0.0 +bm25,784,10,16.49671,0.43068,1.0,0.25,0.0 +bm25,785,10,13.86817,0.63093,1.0,0.5,1.0 +bm25,793,10,5.15421,1.0,1.0,1.0,1.0 +bm25,800,10,15.65192,0.0,0.0,0.0,0.0 +bm25,805,10,7.55096,1.0,1.0,1.0,1.0 +bm25,808,10,8.37258,1.0,1.0,1.0,1.0 +bm25,811,10,12.50858,1.0,1.0,1.0,1.0 +bm25,814,10,20.60779,1.0,1.0,1.0,1.0 +bm25,820,10,7.60575,0.0,0.0,0.0,0.0 +bm25,821,10,6.85204,0.0,0.0,0.0,0.0 +bm25,823,10,6.684,1.0,1.0,1.0,1.0 +bm25,830,10,15.30358,0.0,0.0,0.0,0.0 +bm25,831,10,10.23262,0.0,0.0,0.0,0.0 +bm25,832,10,6.22742,1.0,1.0,1.0,1.0 +bm25,834,10,9.2725,0.0,0.0,0.0,0.0 +bm25,837,10,7.56838,1.0,1.0,1.0,1.0 +bm25,839,10,11.55612,1.0,1.0,1.0,1.0 +bm25,845,10,9.67804,1.0,1.0,1.0,1.0 +bm25,847,10,14.94446,1.0,1.0,1.0,1.0 +bm25,852,10,12.15612,1.0,1.0,1.0,1.0 +bm25,859,10,7.09625,0.63093,1.0,0.5,1.0 +bm25,870,10,3.496,0.0,0.0,0.0,0.0 +bm25,873,10,6.95583,0.0,0.0,0.0,0.0 +bm25,879,10,8.99162,1.0,1.0,1.0,1.0 +bm25,880,10,7.82067,1.0,1.0,1.0,1.0 +bm25,882,10,9.28521,1.0,1.0,1.0,1.0 +bm25,887,10,10.597,0.0,0.0,0.0,0.0 +bm25,903,10,9.42246,1.0,1.0,1.0,1.0 +bm25,904,10,17.77467,1.0,1.0,1.0,1.0 +bm25,907,10,14.29833,1.0,1.0,1.0,1.0 +bm25,911,10,13.03596,1.0,1.0,1.0,1.0 +bm25,913,10,6.38492,0.0,0.0,0.0,0.0 +bm25,914,10,6.88429,0.0,0.0,0.0,0.0 +bm25,921,10,10.50412,0.63093,1.0,0.5,1.0 +bm25,922,10,10.24363,1.0,1.0,1.0,1.0 +bm25,936,10,8.02858,1.0,1.0,1.0,1.0 +bm25,956,10,11.39104,0.5,1.0,0.33333,1.0 +bm25,957,10,8.93121,0.63093,1.0,0.5,1.0 +bm25,960,10,4.81125,1.0,1.0,1.0,1.0 +bm25,967,10,8.8885,0.91972,1.0,1.0,1.0 +bm25,971,10,16.11687,0.64446,1.0,0.33333,1.0 +bm25,975,10,8.04925,0.0,0.0,0.0,0.0 +bm25,982,10,15.16554,1.0,1.0,1.0,1.0 +bm25,985,10,10.83192,1.0,1.0,1.0,1.0 +bm25,993,10,7.28842,0.63093,1.0,0.5,1.0 +bm25,1012,10,8.28333,1.0,1.0,1.0,1.0 +bm25,1014,10,8.42367,1.0,1.0,1.0,1.0 +bm25,1019,10,6.9365,0.63093,1.0,0.5,1.0 +bm25,1020,10,18.88788,0.5,1.0,0.33333,1.0 +bm25,1021,10,22.59346,0.5,1.0,0.33333,1.0 +bm25,1024,10,14.55538,1.0,1.0,1.0,1.0 +bm25,1029,10,20.55846,0.76536,0.66667,1.0,1.0 +bm25,1041,10,13.37721,0.44131,1.0,0.25,0.0 +bm25,1049,10,9.45513,0.0,0.0,0.0,0.0 +bm25,1062,10,5.57379,1.0,1.0,1.0,1.0 +bm25,1086,10,14.82925,1.0,1.0,1.0,1.0 +bm25,1088,10,10.967,0.0,0.0,0.0,0.0 +bm25,1089,10,11.94017,1.0,1.0,1.0,1.0 +bm25,1099,10,3.31392,0.5,1.0,0.33333,1.0 +bm25,1100,10,2.72496,0.63093,1.0,0.5,1.0 +bm25,1104,10,19.36808,1.0,1.0,1.0,1.0 +bm25,1107,10,7.67954,1.0,1.0,1.0,1.0 +bm25,1110,10,7.24754,0.0,0.0,0.0,0.0 +bm25,1121,10,11.20708,0.63093,1.0,0.5,1.0 +bm25,1130,10,15.22621,1.0,1.0,1.0,1.0 +bm25,1132,10,15.96667,0.2372,0.5,0.2,0.0 +bm25,1137,10,6.43829,1.0,1.0,1.0,1.0 +bm25,1140,10,9.25054,0.0,0.0,0.0,0.0 +bm25,1144,10,15.98325,1.0,1.0,1.0,1.0 +bm25,1146,10,8.66808,1.0,1.0,1.0,1.0 +bm25,1150,10,10.75138,1.0,1.0,1.0,1.0 +bm25,1163,10,9.78467,1.0,1.0,1.0,1.0 +bm25,1175,10,8.64913,0.0,0.0,0.0,0.0 +bm25,1179,10,9.10583,0.30103,1.0,0.11111,0.0 +bm25,1180,10,9.19733,0.43068,1.0,0.25,0.0 +bm25,1185,10,24.34592,1.0,1.0,1.0,1.0 +bm25,1187,10,22.34067,1.0,1.0,1.0,1.0 +bm25,1191,10,10.12829,0.0,0.0,0.0,0.0 +bm25,1194,10,19.3935,0.31546,1.0,0.125,0.0 +bm25,1196,10,11.14196,0.0,0.0,0.0,0.0 +bm25,1197,10,11.85383,0.0,0.0,0.0,0.0 +bm25,1199,10,18.74658,0.0,0.0,0.0,0.0 +bm25,1200,10,19.50367,0.0,0.0,0.0,0.0 +bm25,1202,10,12.13917,1.0,1.0,1.0,1.0 +bm25,1204,10,12.32467,1.0,1.0,1.0,1.0 +bm25,1207,10,16.69225,0.43068,1.0,0.25,0.0 +bm25,1213,10,10.9275,0.0,0.0,0.0,0.0 +bm25,1216,10,9.81242,1.0,1.0,1.0,1.0 +bm25,1221,10,14.78133,0.0,0.0,0.0,0.0 +bm25,1225,10,7.2,1.0,1.0,1.0,1.0 +bm25,1226,10,12.82425,0.0,0.0,0.0,0.0 +bm25,1232,10,15.12758,1.0,1.0,1.0,1.0 +bm25,1241,10,8.09846,0.0,0.0,0.0,0.0 +bm25,1245,10,9.23325,0.31546,1.0,0.125,0.0 +bm25,1259,10,22.27188,0.63093,1.0,0.5,1.0 +bm25,1262,10,11.34475,1.0,1.0,1.0,1.0 +bm25,1266,10,20.61804,0.5,1.0,0.33333,1.0 +bm25,1270,10,11.65025,1.0,1.0,1.0,1.0 +bm25,1271,10,18.09092,1.0,1.0,1.0,1.0 +bm25,1272,10,12.72087,0.5,1.0,0.33333,1.0 +bm25,1273,10,10.02258,1.0,1.0,1.0,1.0 +bm25,1274,10,18.0415,0.64478,1.0,0.5,1.0 +bm25,1278,10,13.97362,0.0,0.0,0.0,0.0 +bm25,1279,10,11.32787,0.0,0.0,0.0,0.0 +bm25,1280,10,11.57125,0.0,0.0,0.0,0.0 +bm25,1281,10,9.52158,0.0,0.0,0.0,0.0 +bm25,1282,10,12.92404,1.0,1.0,1.0,1.0 +bm25,1290,10,9.53083,1.0,1.0,1.0,1.0 +bm25,1292,10,9.299,1.0,1.0,1.0,1.0 +bm25,1298,10,19.19871,1.0,1.0,1.0,1.0 +bm25,1303,10,6.64908,1.0,1.0,1.0,1.0 +bm25,1316,10,9.31392,0.0,0.0,0.0,0.0 +bm25,1319,10,9.09987,0.43068,1.0,0.25,0.0 +bm25,1320,10,12.85742,1.0,1.0,1.0,1.0 +bm25,1332,10,18.21812,0.0,0.0,0.0,0.0 +bm25,1335,10,8.85867,1.0,1.0,1.0,1.0 +bm25,1336,10,7.68296,1.0,1.0,1.0,1.0 +bm25,1337,10,9.58479,1.0,1.0,1.0,1.0 +bm25,1339,10,10.65229,1.0,1.0,1.0,1.0 +bm25,1344,10,23.08617,0.63093,1.0,0.5,1.0 +bm25,1352,10,10.24183,1.0,1.0,1.0,1.0 +bm25,1359,10,16.14542,1.0,1.0,1.0,1.0 +bm25,1362,10,7.22592,0.5,1.0,0.33333,1.0 +bm25,1363,10,9.61308,1.0,1.0,1.0,1.0 +bm25,1368,10,7.10329,0.0,0.0,0.0,0.0 +bm25,1370,10,7.01379,0.38685,1.0,0.2,0.0 +bm25,1379,10,13.80154,0.33425,0.5,0.33333,1.0 +bm25,1382,10,6.91583,0.0,0.0,0.0,0.0 +bm25,1385,10,5.06821,1.0,1.0,1.0,1.0 +bm25,1389,10,7.20125,1.0,1.0,1.0,1.0 +bm25,1395,10,18.47513,0.0,0.0,0.0,0.0 +dense+rerank,1,10,1009.78521,0.0,0.0,0.0,0.0 +dense+rerank,3,10,880.62825,1.0,1.0,1.0,1.0 +dense+rerank,5,10,1007.98917,1.0,1.0,1.0,1.0 +dense+rerank,13,10,1011.46154,0.0,0.0,0.0,0.0 +dense+rerank,36,10,1009.82292,0.38896,1.0,0.14286,0.0 +dense+rerank,42,10,1014.894,1.0,1.0,1.0,1.0 +dense+rerank,48,10,1016.75271,0.5,1.0,0.33333,1.0 +dense+rerank,49,10,1038.481,1.0,1.0,1.0,1.0 +dense+rerank,50,10,991.08075,1.0,1.0,1.0,1.0 +dense+rerank,51,10,1022.65988,1.0,1.0,1.0,1.0 +dense+rerank,53,10,1022.17125,1.0,1.0,1.0,1.0 +dense+rerank,54,10,1025.97067,0.30103,1.0,0.11111,0.0 +dense+rerank,56,10,1018.45629,1.0,1.0,1.0,1.0 +dense+rerank,57,10,1022.86125,1.0,1.0,1.0,1.0 +dense+rerank,70,10,1025.20042,0.0,0.0,0.0,0.0 +dense+rerank,72,10,1028.25692,1.0,1.0,1.0,1.0 +dense+rerank,75,10,1014.26921,1.0,1.0,1.0,1.0 +dense+rerank,94,10,1031.948,0.30103,1.0,0.11111,0.0 +dense+rerank,99,10,1026.93721,1.0,1.0,1.0,1.0 +dense+rerank,100,10,1043.66467,1.0,1.0,1.0,1.0 +dense+rerank,113,10,1027.17021,1.0,1.0,1.0,1.0 +dense+rerank,115,10,1030.895,1.0,1.0,1.0,1.0 +dense+rerank,118,10,1035.47325,1.0,1.0,1.0,1.0 +dense+rerank,124,10,1031.44537,1.0,1.0,1.0,1.0 +dense+rerank,127,10,1025.029,0.63093,1.0,0.5,1.0 +dense+rerank,128,10,1023.42808,0.0,0.0,0.0,0.0 +dense+rerank,129,10,1037.30796,1.0,1.0,1.0,1.0 +dense+rerank,130,10,1037.259,1.0,1.0,1.0,1.0 +dense+rerank,132,10,1023.97725,0.0,0.0,0.0,0.0 +dense+rerank,133,10,1039.20421,0.63173,0.8,0.5,1.0 +dense+rerank,137,10,1022.014,1.0,1.0,1.0,1.0 +dense+rerank,141,10,1027.7245,0.91972,1.0,1.0,1.0 +dense+rerank,142,10,1032.56412,1.0,1.0,1.0,1.0 +dense+rerank,143,10,1025.20546,1.0,1.0,1.0,1.0 +dense+rerank,146,10,1030.94404,1.0,1.0,1.0,1.0 +dense+rerank,148,10,1029.52858,1.0,1.0,1.0,1.0 +dense+rerank,163,10,1024.19913,0.63093,1.0,0.5,1.0 +dense+rerank,171,10,1046.07667,1.0,1.0,1.0,1.0 +dense+rerank,179,10,1029.89525,0.90522,1.0,1.0,1.0 +dense+rerank,180,10,1031.1865,1.0,1.0,1.0,1.0 +dense+rerank,183,10,1027.42908,1.0,1.0,1.0,1.0 +dense+rerank,185,10,1035.24508,0.43068,1.0,0.25,0.0 +dense+rerank,198,10,1029.42463,0.31546,1.0,0.125,0.0 +dense+rerank,208,10,1031.07163,1.0,1.0,1.0,1.0 +dense+rerank,212,10,998.84887,0.5,1.0,0.33333,1.0 +dense+rerank,213,10,1027.85304,0.28906,1.0,0.1,0.0 +dense+rerank,216,10,1065.31479,1.0,1.0,1.0,1.0 +dense+rerank,217,10,1038.96763,1.0,1.0,1.0,1.0 +dense+rerank,218,10,1051.94746,1.0,1.0,1.0,1.0 +dense+rerank,219,10,1059.66946,1.0,1.0,1.0,1.0 +dense+rerank,230,10,1049.01242,1.0,1.0,1.0,1.0 +dense+rerank,232,10,1032.11767,0.63093,1.0,0.5,1.0 +dense+rerank,233,10,1013.41137,1.0,1.0,1.0,1.0 +dense+rerank,236,10,1025.96229,1.0,1.0,1.0,1.0 +dense+rerank,237,10,883.48454,0.43068,1.0,0.25,0.0 +dense+rerank,238,10,1027.94438,0.0,0.0,0.0,0.0 +dense+rerank,239,10,1029.40808,0.0,0.0,0.0,0.0 +dense+rerank,248,10,1030.393,1.0,1.0,1.0,1.0 +dense+rerank,249,10,1027.50258,1.0,1.0,1.0,1.0 +dense+rerank,261,10,1034.60942,0.38685,0.5,0.5,1.0 +dense+rerank,268,10,1037.65529,0.63093,1.0,0.5,1.0 +dense+rerank,269,10,1032.08454,0.63093,1.0,0.5,1.0 +dense+rerank,274,10,1025.18788,1.0,1.0,1.0,1.0 +dense+rerank,275,10,1033.79604,0.87108,1.0,1.0,1.0 +dense+rerank,279,10,1034.352,1.0,1.0,1.0,1.0 +dense+rerank,294,10,773.76529,1.0,1.0,1.0,1.0 +dense+rerank,295,10,893.09112,1.0,1.0,1.0,1.0 +dense+rerank,298,10,1078.06512,1.0,1.0,1.0,1.0 +dense+rerank,300,10,1045.70908,0.63093,1.0,0.5,1.0 +dense+rerank,303,10,1028.88892,0.0,0.0,0.0,0.0 +dense+rerank,312,10,1050.47208,0.0,0.0,0.0,0.0 +dense+rerank,314,10,1044.88875,0.0,0.0,0.0,0.0 +dense+rerank,324,10,1042.01067,0.0,0.0,0.0,0.0 +dense+rerank,327,10,1050.5515,1.0,1.0,1.0,1.0 +dense+rerank,338,10,1044.91204,1.0,1.0,1.0,1.0 +dense+rerank,343,10,1048.49312,0.57064,1.0,0.33333,1.0 +dense+rerank,350,10,1093.99613,1.0,1.0,1.0,1.0 +dense+rerank,354,10,1056.26571,1.0,1.0,1.0,1.0 +dense+rerank,362,10,1053.59479,1.0,1.0,1.0,1.0 +dense+rerank,380,10,1055.99121,1.0,1.0,1.0,1.0 +dense+rerank,384,10,1055.28479,0.30103,1.0,0.11111,0.0 +dense+rerank,385,10,1058.63146,0.69343,1.0,0.5,1.0 +dense+rerank,386,10,1062.16042,1.0,1.0,1.0,1.0 +dense+rerank,388,10,1064.88875,0.38685,1.0,0.2,0.0 +dense+rerank,399,10,1101.25975,1.0,1.0,1.0,1.0 +dense+rerank,410,10,1106.26446,1.0,1.0,1.0,1.0 +dense+rerank,411,10,1209.19117,1.0,1.0,1.0,1.0 +dense+rerank,415,10,1212.10088,0.5,1.0,0.33333,1.0 +dense+rerank,421,10,1178.74292,0.0,0.0,0.0,0.0 +dense+rerank,431,10,1151.02987,0.0,0.0,0.0,0.0 +dense+rerank,436,10,803.41158,1.0,1.0,1.0,1.0 +dense+rerank,437,10,1218.13425,0.0,0.0,0.0,0.0 +dense+rerank,439,10,1237.85858,1.0,1.0,1.0,1.0 +dense+rerank,440,10,1232.10083,1.0,1.0,1.0,1.0 +dense+rerank,443,10,1245.38896,1.0,1.0,1.0,1.0 +dense+rerank,452,10,927.36317,0.21841,0.5,0.16667,0.0 +dense+rerank,475,10,1213.05554,0.38685,1.0,0.2,0.0 +dense+rerank,478,10,1223.80658,1.0,1.0,1.0,1.0 +dense+rerank,491,10,1142.10729,1.0,1.0,1.0,1.0 +dense+rerank,501,10,1101.59171,1.0,1.0,1.0,1.0 +dense+rerank,502,10,1151.67671,0.0,0.0,0.0,0.0 +dense+rerank,507,10,1152.23425,0.63093,1.0,0.5,1.0 +dense+rerank,508,10,1185.733,0.0,0.0,0.0,0.0 +dense+rerank,513,10,1083.54892,0.63093,1.0,0.5,1.0 +dense+rerank,514,10,1110.5025,0.5,1.0,0.33333,1.0 +dense+rerank,516,10,1070.86037,1.0,1.0,1.0,1.0 +dense+rerank,517,10,1046.80954,0.43068,1.0,0.25,0.0 +dense+rerank,521,10,1038.95092,1.0,1.0,1.0,1.0 +dense+rerank,525,10,1056.20404,1.0,1.0,1.0,1.0 +dense+rerank,527,10,1050.16238,0.31546,1.0,0.125,0.0 +dense+rerank,528,10,1052.59063,0.63093,1.0,0.5,1.0 +dense+rerank,532,10,1054.50946,1.0,1.0,1.0,1.0 +dense+rerank,533,10,1041.59721,1.0,1.0,1.0,1.0 +dense+rerank,535,10,1049.55913,0.0,0.0,0.0,0.0 +dense+rerank,536,10,1056.55658,1.0,1.0,1.0,1.0 +dense+rerank,539,10,1051.97612,1.0,1.0,1.0,1.0 +dense+rerank,540,10,1055.22467,0.30657,0.5,0.33333,1.0 +dense+rerank,544,10,1044.48067,0.43068,1.0,0.25,0.0 +dense+rerank,549,10,1048.63983,1.0,1.0,1.0,1.0 +dense+rerank,551,10,1011.91825,0.5,1.0,0.33333,1.0 +dense+rerank,552,10,1053.00833,1.0,1.0,1.0,1.0 +dense+rerank,554,10,1057.57121,0.28906,1.0,0.1,0.0 +dense+rerank,560,10,1074.47058,0.0,0.0,0.0,0.0 +dense+rerank,569,10,1057.66204,1.0,1.0,1.0,1.0 +dense+rerank,575,10,924.02242,1.0,1.0,1.0,1.0 +dense+rerank,577,10,1044.16792,0.0,0.0,0.0,0.0 +dense+rerank,578,10,1048.03192,1.0,1.0,1.0,1.0 +dense+rerank,587,10,1051.20596,0.63093,1.0,0.5,1.0 +dense+rerank,589,10,1056.30283,1.0,1.0,1.0,1.0 +dense+rerank,593,10,1041.22596,1.0,1.0,1.0,1.0 +dense+rerank,597,10,1034.44325,0.88546,1.0,1.0,1.0 +dense+rerank,598,10,1045.58262,0.33333,1.0,0.14286,0.0 +dense+rerank,613,10,1050.21254,1.0,1.0,1.0,1.0 +dense+rerank,619,10,1050.5825,0.0,0.0,0.0,0.0 +dense+rerank,623,10,1046.09221,0.0,0.0,0.0,0.0 +dense+rerank,628,10,1042.64804,1.0,1.0,1.0,1.0 +dense+rerank,636,10,1042.54079,1.0,1.0,1.0,1.0 +dense+rerank,637,10,1048.40804,0.63093,1.0,0.5,1.0 +dense+rerank,641,10,1058.58867,0.79039,1.0,1.0,1.0 +dense+rerank,644,10,1057.93171,1.0,1.0,1.0,1.0 +dense+rerank,649,10,1051.12779,1.0,1.0,1.0,1.0 +dense+rerank,659,10,1042.94983,0.28906,1.0,0.1,0.0 +dense+rerank,660,10,1043.31842,0.0,0.0,0.0,0.0 +dense+rerank,674,10,1043.51096,1.0,1.0,1.0,1.0 +dense+rerank,684,10,1039.49625,0.43068,1.0,0.25,0.0 +dense+rerank,690,10,1051.59071,0.0,0.0,0.0,0.0 +dense+rerank,691,10,1055.66558,0.63093,1.0,0.5,1.0 +dense+rerank,692,10,1045.65325,1.0,1.0,1.0,1.0 +dense+rerank,693,10,1080.75179,1.0,1.0,1.0,1.0 +dense+rerank,700,10,1089.86021,0.33333,1.0,0.14286,0.0 +dense+rerank,702,10,1088.93808,0.31546,1.0,0.125,0.0 +dense+rerank,715,10,1103.40958,0.35621,1.0,0.16667,0.0 +dense+rerank,716,10,1073.89871,0.0,0.0,0.0,0.0 +dense+rerank,718,10,1057.48979,0.63093,1.0,0.5,1.0 +dense+rerank,721,10,1052.98346,1.0,1.0,1.0,1.0 +dense+rerank,723,10,1050.5595,1.0,1.0,1.0,1.0 +dense+rerank,727,10,1044.05017,0.31546,1.0,0.125,0.0 +dense+rerank,728,10,1041.46067,0.81753,1.0,1.0,1.0 +dense+rerank,729,10,1043.86679,1.0,1.0,1.0,1.0 +dense+rerank,742,10,1034.13146,1.0,1.0,1.0,1.0 +dense+rerank,743,10,1080.24021,1.0,1.0,1.0,1.0 +dense+rerank,744,10,1056.01192,0.63093,1.0,0.5,1.0 +dense+rerank,756,10,1056.23717,0.38685,1.0,0.2,0.0 +dense+rerank,759,10,1051.62525,1.0,1.0,1.0,1.0 +dense+rerank,768,10,1065.31187,0.0,0.0,0.0,0.0 +dense+rerank,770,10,1071.00158,1.0,1.0,1.0,1.0 +dense+rerank,775,10,994.81846,0.33333,1.0,0.14286,0.0 +dense+rerank,781,10,1050.62417,0.63093,1.0,0.5,1.0 +dense+rerank,783,10,992.15396,0.28906,1.0,0.1,0.0 +dense+rerank,784,10,1040.354,1.0,1.0,1.0,1.0 +dense+rerank,785,10,1044.82996,0.31546,1.0,0.125,0.0 +dense+rerank,793,10,1045.74504,1.0,1.0,1.0,1.0 +dense+rerank,800,10,837.01958,0.63093,1.0,0.5,1.0 +dense+rerank,805,10,1040.66029,1.0,1.0,1.0,1.0 +dense+rerank,808,10,1052.40621,1.0,1.0,1.0,1.0 +dense+rerank,811,10,1038.72037,1.0,1.0,1.0,1.0 +dense+rerank,814,10,1057.4765,1.0,1.0,1.0,1.0 +dense+rerank,820,10,899.62283,1.0,1.0,1.0,1.0 +dense+rerank,821,10,1043.27717,1.0,1.0,1.0,1.0 +dense+rerank,823,10,1055.30708,1.0,1.0,1.0,1.0 +dense+rerank,830,10,1109.13358,0.0,0.0,0.0,0.0 +dense+rerank,831,10,1085.01771,0.0,0.0,0.0,0.0 +dense+rerank,832,10,1071.36817,1.0,1.0,1.0,1.0 +dense+rerank,834,10,1056.29379,0.0,0.0,0.0,0.0 +dense+rerank,837,10,1043.38492,0.63093,1.0,0.5,1.0 +dense+rerank,839,10,1057.03883,1.0,1.0,1.0,1.0 +dense+rerank,845,10,1030.12971,1.0,1.0,1.0,1.0 +dense+rerank,847,10,1055.88079,1.0,1.0,1.0,1.0 +dense+rerank,852,10,1045.92458,0.63093,1.0,0.5,1.0 +dense+rerank,859,10,1062.50446,0.63093,1.0,0.5,1.0 +dense+rerank,870,10,1056.54183,0.0,0.0,0.0,0.0 +dense+rerank,873,10,1049.40113,0.41775,0.6,0.33333,1.0 +dense+rerank,879,10,1051.30821,1.0,1.0,1.0,1.0 +dense+rerank,880,10,1046.69387,1.0,1.0,1.0,1.0 +dense+rerank,882,10,1047.11029,1.0,1.0,1.0,1.0 +dense+rerank,887,10,1056.34846,0.0,0.0,0.0,0.0 +dense+rerank,903,10,1052.52658,1.0,1.0,1.0,1.0 +dense+rerank,904,10,1051.22383,1.0,1.0,1.0,1.0 +dense+rerank,907,10,1056.709,1.0,1.0,1.0,1.0 +dense+rerank,911,10,1052.24979,1.0,1.0,1.0,1.0 +dense+rerank,913,10,1045.73546,0.0,0.0,0.0,0.0 +dense+rerank,914,10,1052.66225,0.0,0.0,0.0,0.0 +dense+rerank,921,10,1046.57725,1.0,1.0,1.0,1.0 +dense+rerank,922,10,1044.06958,1.0,1.0,1.0,1.0 +dense+rerank,936,10,1050.46521,1.0,1.0,1.0,1.0 +dense+rerank,956,10,1050.40467,1.0,1.0,1.0,1.0 +dense+rerank,957,10,995.35312,1.0,1.0,1.0,1.0 +dense+rerank,960,10,1046.92425,1.0,1.0,1.0,1.0 +dense+rerank,967,10,1051.28567,1.0,1.0,1.0,1.0 +dense+rerank,971,10,1053.23054,0.95583,1.0,1.0,1.0 +dense+rerank,975,10,1056.99604,0.0,0.0,0.0,0.0 +dense+rerank,982,10,934.63658,1.0,1.0,1.0,1.0 +dense+rerank,985,10,1050.292,1.0,1.0,1.0,1.0 +dense+rerank,993,10,1055.72954,1.0,1.0,1.0,1.0 +dense+rerank,1012,10,1036.77721,1.0,1.0,1.0,1.0 +dense+rerank,1014,10,1048.80454,1.0,1.0,1.0,1.0 +dense+rerank,1019,10,1051.51621,0.63093,1.0,0.5,1.0 +dense+rerank,1020,10,1057.53304,0.63093,1.0,0.5,1.0 +dense+rerank,1021,10,1065.147,0.63093,1.0,0.5,1.0 +dense+rerank,1024,10,999.06146,1.0,1.0,1.0,1.0 +dense+rerank,1029,10,1071.04708,0.76536,0.66667,1.0,1.0 +dense+rerank,1041,10,1052.85879,0.21841,0.5,0.16667,0.0 +dense+rerank,1049,10,1061.30125,0.43068,1.0,0.25,0.0 +dense+rerank,1062,10,1055.89254,1.0,1.0,1.0,1.0 +dense+rerank,1086,10,1056.32996,1.0,1.0,1.0,1.0 +dense+rerank,1088,10,1058.19762,1.0,1.0,1.0,1.0 +dense+rerank,1089,10,871.75217,1.0,1.0,1.0,1.0 +dense+rerank,1099,10,1051.88263,0.43068,1.0,0.25,0.0 +dense+rerank,1100,10,1050.04917,0.43068,1.0,0.25,0.0 +dense+rerank,1104,10,1042.07292,1.0,1.0,1.0,1.0 +dense+rerank,1107,10,1073.97188,1.0,1.0,1.0,1.0 +dense+rerank,1110,10,1053.66287,0.0,0.0,0.0,0.0 +dense+rerank,1121,10,1057.65308,1.0,1.0,1.0,1.0 +dense+rerank,1130,10,1055.15321,1.0,1.0,1.0,1.0 +dense+rerank,1132,10,1050.99688,0.61315,0.5,1.0,1.0 +dense+rerank,1137,10,1051.76625,1.0,1.0,1.0,1.0 +dense+rerank,1140,10,1048.69758,1.0,1.0,1.0,1.0 +dense+rerank,1144,10,1048.85033,1.0,1.0,1.0,1.0 +dense+rerank,1146,10,1054.718,1.0,1.0,1.0,1.0 +dense+rerank,1150,10,1048.47825,1.0,1.0,1.0,1.0 +dense+rerank,1163,10,1058.74937,1.0,1.0,1.0,1.0 +dense+rerank,1175,10,1055.11,0.0,0.0,0.0,0.0 +dense+rerank,1179,10,1023.39337,0.0,0.0,0.0,0.0 +dense+rerank,1180,10,1058.73417,1.0,1.0,1.0,1.0 +dense+rerank,1185,10,1047.73508,1.0,1.0,1.0,1.0 +dense+rerank,1187,10,1044.537,1.0,1.0,1.0,1.0 +dense+rerank,1191,10,1040.05912,0.0,0.0,0.0,0.0 +dense+rerank,1194,10,1007.79904,1.0,1.0,1.0,1.0 +dense+rerank,1196,10,1057.92121,0.43068,1.0,0.25,0.0 +dense+rerank,1197,10,1058.79554,0.43068,1.0,0.25,0.0 +dense+rerank,1199,10,1048.97621,0.0,0.0,0.0,0.0 +dense+rerank,1200,10,1047.61004,0.5,1.0,0.33333,1.0 +dense+rerank,1202,10,1058.24333,0.63093,1.0,0.5,1.0 +dense+rerank,1204,10,1059.14821,0.5,1.0,0.33333,1.0 +dense+rerank,1207,10,1060.33683,1.0,1.0,1.0,1.0 +dense+rerank,1213,10,1056.00646,0.0,0.0,0.0,0.0 +dense+rerank,1216,10,1057.46604,1.0,1.0,1.0,1.0 +dense+rerank,1221,10,1052.6145,0.33333,1.0,0.14286,0.0 +dense+rerank,1225,10,1061.51688,1.0,1.0,1.0,1.0 +dense+rerank,1226,10,1073.70125,0.0,0.0,0.0,0.0 +dense+rerank,1232,10,1051.75704,1.0,1.0,1.0,1.0 +dense+rerank,1241,10,1054.59496,1.0,1.0,1.0,1.0 +dense+rerank,1245,10,1053.15892,0.0,0.0,0.0,0.0 +dense+rerank,1259,10,1056.88067,1.0,1.0,1.0,1.0 +dense+rerank,1262,10,1040.11592,1.0,1.0,1.0,1.0 +dense+rerank,1266,10,1061.7545,0.5,1.0,0.33333,1.0 +dense+rerank,1270,10,1054.80183,1.0,1.0,1.0,1.0 +dense+rerank,1271,10,1054.43375,1.0,1.0,1.0,1.0 +dense+rerank,1272,10,1057.07196,0.43068,1.0,0.25,0.0 +dense+rerank,1273,10,1099.08588,1.0,1.0,1.0,1.0 +dense+rerank,1274,10,1085.61283,0.93252,1.0,1.0,1.0 +dense+rerank,1278,10,1101.09987,0.0,0.0,0.0,0.0 +dense+rerank,1279,10,1072.58883,0.28906,1.0,0.1,0.0 +dense+rerank,1280,10,1075.45696,0.33333,1.0,0.14286,0.0 +dense+rerank,1281,10,1081.66979,0.28906,1.0,0.1,0.0 +dense+rerank,1282,10,1096.15275,1.0,1.0,1.0,1.0 +dense+rerank,1290,10,1067.55358,1.0,1.0,1.0,1.0 +dense+rerank,1292,10,1158.37479,0.63093,1.0,0.5,1.0 +dense+rerank,1298,10,1062.02579,1.0,1.0,1.0,1.0 +dense+rerank,1303,10,1079.98346,1.0,1.0,1.0,1.0 +dense+rerank,1316,10,1070.04983,0.0,0.0,0.0,0.0 +dense+rerank,1319,10,1094.80675,1.0,1.0,1.0,1.0 +dense+rerank,1320,10,1086.365,1.0,1.0,1.0,1.0 +dense+rerank,1332,10,1080.40342,0.0,0.0,0.0,0.0 +dense+rerank,1335,10,1087.58567,1.0,1.0,1.0,1.0 +dense+rerank,1336,10,1145.05983,1.0,1.0,1.0,1.0 +dense+rerank,1337,10,1165.06654,0.63093,1.0,0.5,1.0 +dense+rerank,1339,10,1156.70183,1.0,1.0,1.0,1.0 +dense+rerank,1344,10,1162.43854,0.5,1.0,0.33333,1.0 +dense+rerank,1352,10,1155.15558,1.0,1.0,1.0,1.0 +dense+rerank,1359,10,1112.25717,1.0,1.0,1.0,1.0 +dense+rerank,1362,10,1117.11279,0.0,0.0,0.0,0.0 +dense+rerank,1363,10,1129.41171,0.0,0.0,0.0,0.0 +dense+rerank,1368,10,1121.50154,1.0,1.0,1.0,1.0 +dense+rerank,1370,10,1116.48679,0.63093,1.0,0.5,1.0 +dense+rerank,1379,10,1131.67321,0.83268,1.0,1.0,1.0 +dense+rerank,1382,10,1111.33404,1.0,1.0,1.0,1.0 +dense+rerank,1385,10,1107.30067,1.0,1.0,1.0,1.0 +dense+rerank,1389,10,1107.24467,1.0,1.0,1.0,1.0 +dense+rerank,1395,10,1116.58317,1.0,1.0,1.0,1.0 +hybrid+rerank,1,10,1690.34683,0.0,0.0,0.0,0.0 +hybrid+rerank,3,10,1663.17963,1.0,1.0,1.0,1.0 +hybrid+rerank,5,10,2141.90463,1.0,1.0,1.0,1.0 +hybrid+rerank,13,10,1632.50358,0.0,0.0,0.0,0.0 +hybrid+rerank,36,10,1477.19075,0.38896,1.0,0.14286,0.0 +hybrid+rerank,42,10,1819.62575,1.0,1.0,1.0,1.0 +hybrid+rerank,48,10,1855.7785,0.5,1.0,0.33333,1.0 +hybrid+rerank,49,10,1457.37962,1.0,1.0,1.0,1.0 +hybrid+rerank,50,10,1465.62933,1.0,1.0,1.0,1.0 +hybrid+rerank,51,10,1695.55913,0.63093,1.0,0.5,1.0 +hybrid+rerank,53,10,1798.67438,1.0,1.0,1.0,1.0 +hybrid+rerank,54,10,1379.39375,0.30103,1.0,0.11111,0.0 +hybrid+rerank,56,10,1581.02387,1.0,1.0,1.0,1.0 +hybrid+rerank,57,10,1592.53313,1.0,1.0,1.0,1.0 +hybrid+rerank,70,10,1331.27075,0.0,0.0,0.0,0.0 +hybrid+rerank,72,10,1828.47325,1.0,1.0,1.0,1.0 +hybrid+rerank,75,10,1344.687,1.0,1.0,1.0,1.0 +hybrid+rerank,94,10,1928.595,0.0,0.0,0.0,0.0 +hybrid+rerank,99,10,1697.64908,1.0,1.0,1.0,1.0 +hybrid+rerank,100,10,1461.74667,1.0,1.0,1.0,1.0 +hybrid+rerank,113,10,1497.256,1.0,1.0,1.0,1.0 +hybrid+rerank,115,10,1697.56617,1.0,1.0,1.0,1.0 +hybrid+rerank,118,10,1567.30225,1.0,1.0,1.0,1.0 +hybrid+rerank,124,10,1492.595,1.0,1.0,1.0,1.0 +hybrid+rerank,127,10,1656.09896,0.63093,1.0,0.5,1.0 +hybrid+rerank,128,10,1994.6905,0.31546,1.0,0.125,0.0 +hybrid+rerank,129,10,1625.13992,1.0,1.0,1.0,1.0 +hybrid+rerank,130,10,1641.42063,1.0,1.0,1.0,1.0 +hybrid+rerank,132,10,1599.44179,0.0,0.0,0.0,0.0 +hybrid+rerank,133,10,1601.84958,0.52963,0.6,0.5,1.0 +hybrid+rerank,137,10,1683.69062,1.0,1.0,1.0,1.0 +hybrid+rerank,141,10,1469.11325,0.91972,1.0,1.0,1.0 +hybrid+rerank,142,10,1644.45954,1.0,1.0,1.0,1.0 +hybrid+rerank,143,10,1625.70862,1.0,1.0,1.0,1.0 +hybrid+rerank,146,10,1879.03033,1.0,1.0,1.0,1.0 +hybrid+rerank,148,10,1422.42854,1.0,1.0,1.0,1.0 +hybrid+rerank,163,10,1497.94917,0.63093,1.0,0.5,1.0 +hybrid+rerank,171,10,1627.85637,1.0,1.0,1.0,1.0 +hybrid+rerank,179,10,1368.27292,0.90522,1.0,1.0,1.0 +hybrid+rerank,180,10,1649.27733,1.0,1.0,1.0,1.0 +hybrid+rerank,183,10,1502.49862,1.0,1.0,1.0,1.0 +hybrid+rerank,185,10,1633.284,0.43068,1.0,0.25,0.0 +hybrid+rerank,198,10,1814.69242,0.0,0.0,0.0,0.0 +hybrid+rerank,208,10,1558.63638,1.0,1.0,1.0,1.0 +hybrid+rerank,212,10,1501.03254,0.5,1.0,0.33333,1.0 +hybrid+rerank,213,10,2009.69663,0.0,0.0,0.0,0.0 +hybrid+rerank,216,10,1597.61579,1.0,1.0,1.0,1.0 +hybrid+rerank,217,10,1641.32692,1.0,1.0,1.0,1.0 +hybrid+rerank,218,10,1519.35046,1.0,1.0,1.0,1.0 +hybrid+rerank,219,10,1623.26508,1.0,1.0,1.0,1.0 +hybrid+rerank,230,10,1609.73804,1.0,1.0,1.0,1.0 +hybrid+rerank,232,10,1704.26804,0.63093,1.0,0.5,1.0 +hybrid+rerank,233,10,1604.00508,1.0,1.0,1.0,1.0 +hybrid+rerank,236,10,1502.15475,1.0,1.0,1.0,1.0 +hybrid+rerank,237,10,1558.14254,0.43068,1.0,0.25,0.0 +hybrid+rerank,238,10,1622.97658,0.0,0.0,0.0,0.0 +hybrid+rerank,239,10,1497.04475,0.0,0.0,0.0,0.0 +hybrid+rerank,248,10,1788.19658,1.0,1.0,1.0,1.0 +hybrid+rerank,249,10,1772.78421,1.0,1.0,1.0,1.0 +hybrid+rerank,261,10,1568.953,0.38685,0.5,0.5,1.0 +hybrid+rerank,268,10,1503.70842,0.63093,1.0,0.5,1.0 +hybrid+rerank,269,10,1533.65446,0.63093,1.0,0.5,1.0 +hybrid+rerank,274,10,2015.24946,1.0,1.0,1.0,1.0 +hybrid+rerank,275,10,1509.49758,0.87108,1.0,1.0,1.0 +hybrid+rerank,279,10,1675.03204,1.0,1.0,1.0,1.0 +hybrid+rerank,294,10,1386.86683,1.0,1.0,1.0,1.0 +hybrid+rerank,295,10,1353.05829,1.0,1.0,1.0,1.0 +hybrid+rerank,298,10,1548.23258,1.0,1.0,1.0,1.0 +hybrid+rerank,300,10,1593.17396,0.63093,1.0,0.5,1.0 +hybrid+rerank,303,10,1668.83375,0.0,0.0,0.0,0.0 +hybrid+rerank,312,10,1570.61696,0.0,0.0,0.0,0.0 +hybrid+rerank,314,10,1616.47808,0.0,0.0,0.0,0.0 +hybrid+rerank,324,10,1798.236,0.0,0.0,0.0,0.0 +hybrid+rerank,327,10,1896.63313,1.0,1.0,1.0,1.0 +hybrid+rerank,338,10,1788.01517,1.0,1.0,1.0,1.0 +hybrid+rerank,343,10,1835.86392,0.57064,1.0,0.33333,1.0 +hybrid+rerank,350,10,1520.61237,1.0,1.0,1.0,1.0 +hybrid+rerank,354,10,1658.01758,1.0,1.0,1.0,1.0 +hybrid+rerank,362,10,1793.48775,1.0,1.0,1.0,1.0 +hybrid+rerank,380,10,1713.59392,1.0,1.0,1.0,1.0 +hybrid+rerank,384,10,1727.60933,0.28906,1.0,0.1,0.0 +hybrid+rerank,385,10,1714.05408,0.65092,1.0,0.5,1.0 +hybrid+rerank,386,10,1901.73992,1.0,1.0,1.0,1.0 +hybrid+rerank,388,10,1703.62562,0.30103,1.0,0.11111,0.0 +hybrid+rerank,399,10,1743.89142,1.0,1.0,1.0,1.0 +hybrid+rerank,410,10,1549.47975,1.0,1.0,1.0,1.0 +hybrid+rerank,411,10,1564.31171,1.0,1.0,1.0,1.0 +hybrid+rerank,415,10,1511.91488,0.5,1.0,0.33333,1.0 +hybrid+rerank,421,10,1773.69542,0.0,0.0,0.0,0.0 +hybrid+rerank,431,10,1509.30646,0.0,0.0,0.0,0.0 +hybrid+rerank,436,10,1604.97517,1.0,1.0,1.0,1.0 +hybrid+rerank,437,10,1546.01308,0.0,0.0,0.0,0.0 +hybrid+rerank,439,10,1572.35896,1.0,1.0,1.0,1.0 +hybrid+rerank,440,10,1583.81908,1.0,1.0,1.0,1.0 +hybrid+rerank,443,10,1310.079,1.0,1.0,1.0,1.0 +hybrid+rerank,452,10,1605.10508,0.59124,1.0,0.5,1.0 +hybrid+rerank,475,10,1511.23017,0.38685,1.0,0.2,0.0 +hybrid+rerank,478,10,1671.933,1.0,1.0,1.0,1.0 +hybrid+rerank,491,10,1825.91842,1.0,1.0,1.0,1.0 +hybrid+rerank,501,10,1637.45437,1.0,1.0,1.0,1.0 +hybrid+rerank,502,10,1787.87733,0.0,0.0,0.0,0.0 +hybrid+rerank,507,10,1668.59312,0.63093,1.0,0.5,1.0 +hybrid+rerank,508,10,1483.12242,0.0,0.0,0.0,0.0 +hybrid+rerank,513,10,1865.63358,0.63093,1.0,0.5,1.0 +hybrid+rerank,514,10,1470.73238,0.5,1.0,0.33333,1.0 +hybrid+rerank,516,10,1363.75638,1.0,1.0,1.0,1.0 +hybrid+rerank,517,10,1685.67113,0.33333,1.0,0.14286,0.0 +hybrid+rerank,521,10,1871.518,1.0,1.0,1.0,1.0 +hybrid+rerank,525,10,1711.78587,1.0,1.0,1.0,1.0 +hybrid+rerank,527,10,1645.8545,0.28906,1.0,0.1,0.0 +hybrid+rerank,528,10,1809.01596,0.5,1.0,0.33333,1.0 +hybrid+rerank,532,10,2100.74863,1.0,1.0,1.0,1.0 +hybrid+rerank,533,10,2079.8475,1.0,1.0,1.0,1.0 +hybrid+rerank,535,10,1876.50279,0.0,0.0,0.0,0.0 +hybrid+rerank,536,10,1805.63346,1.0,1.0,1.0,1.0 +hybrid+rerank,539,10,1464.42629,1.0,1.0,1.0,1.0 +hybrid+rerank,540,10,1375.39204,0.30657,0.5,0.33333,1.0 +hybrid+rerank,544,10,1664.91262,0.38685,1.0,0.2,0.0 +hybrid+rerank,549,10,1557.98296,1.0,1.0,1.0,1.0 +hybrid+rerank,551,10,1576.363,0.5,1.0,0.33333,1.0 +hybrid+rerank,552,10,1577.83858,1.0,1.0,1.0,1.0 +hybrid+rerank,554,10,1450.17929,0.0,0.0,0.0,0.0 +hybrid+rerank,560,10,1543.48946,0.0,0.0,0.0,0.0 +hybrid+rerank,569,10,1248.09533,1.0,1.0,1.0,1.0 +hybrid+rerank,575,10,1454.08058,1.0,1.0,1.0,1.0 +hybrid+rerank,577,10,1715.92512,0.0,0.0,0.0,0.0 +hybrid+rerank,578,10,1724.33383,1.0,1.0,1.0,1.0 +hybrid+rerank,587,10,1728.49325,0.63093,1.0,0.5,1.0 +hybrid+rerank,589,10,1725.08646,1.0,1.0,1.0,1.0 +hybrid+rerank,593,10,1866.03004,1.0,1.0,1.0,1.0 +hybrid+rerank,597,10,1594.26563,0.83855,1.0,1.0,1.0 +hybrid+rerank,598,10,1320.50767,0.33333,1.0,0.14286,0.0 +hybrid+rerank,613,10,1761.6005,1.0,1.0,1.0,1.0 +hybrid+rerank,619,10,1880.877,0.38685,0.5,0.5,1.0 +hybrid+rerank,623,10,1316.29812,0.0,0.0,0.0,0.0 +hybrid+rerank,628,10,1687.91996,1.0,1.0,1.0,1.0 +hybrid+rerank,636,10,1635.60342,1.0,1.0,1.0,1.0 +hybrid+rerank,637,10,1517.78958,0.63093,1.0,0.5,1.0 +hybrid+rerank,641,10,1571.23842,0.79039,1.0,1.0,1.0 +hybrid+rerank,644,10,1715.478,1.0,1.0,1.0,1.0 +hybrid+rerank,649,10,1739.30046,1.0,1.0,1.0,1.0 +hybrid+rerank,659,10,1865.32358,0.0,0.0,0.0,0.0 +hybrid+rerank,660,10,1985.53888,0.0,0.0,0.0,0.0 +hybrid+rerank,674,10,1623.71546,1.0,1.0,1.0,1.0 +hybrid+rerank,684,10,1603.59062,0.43068,1.0,0.25,0.0 +hybrid+rerank,690,10,1972.92329,1.0,1.0,1.0,1.0 +hybrid+rerank,691,10,1588.88267,0.63093,1.0,0.5,1.0 +hybrid+rerank,692,10,1604.21233,1.0,1.0,1.0,1.0 +hybrid+rerank,693,10,1552.95138,1.0,1.0,1.0,1.0 +hybrid+rerank,700,10,1605.17125,0.31546,1.0,0.125,0.0 +hybrid+rerank,702,10,1551.72721,0.31546,1.0,0.125,0.0 +hybrid+rerank,715,10,1730.01296,0.35621,1.0,0.16667,0.0 +hybrid+rerank,716,10,1731.37371,0.0,0.0,0.0,0.0 +hybrid+rerank,718,10,1535.01096,0.63093,1.0,0.5,1.0 +hybrid+rerank,721,10,1641.04879,1.0,1.0,1.0,1.0 +hybrid+rerank,723,10,1561.72233,1.0,1.0,1.0,1.0 +hybrid+rerank,727,10,1608.30433,0.30103,1.0,0.11111,0.0 +hybrid+rerank,728,10,1538.25829,0.80657,1.0,1.0,1.0 +hybrid+rerank,729,10,1758.11854,1.0,1.0,1.0,1.0 +hybrid+rerank,742,10,1831.20804,1.0,1.0,1.0,1.0 +hybrid+rerank,743,10,1752.13462,1.0,1.0,1.0,1.0 +hybrid+rerank,744,10,1652.14629,0.63093,1.0,0.5,1.0 +hybrid+rerank,756,10,1454.80171,0.38685,1.0,0.2,0.0 +hybrid+rerank,759,10,1336.06417,1.0,1.0,1.0,1.0 +hybrid+rerank,768,10,1926.42008,0.0,0.0,0.0,0.0 +hybrid+rerank,770,10,1597.45517,1.0,1.0,1.0,1.0 +hybrid+rerank,775,10,1565.13962,0.31546,1.0,0.125,0.0 +hybrid+rerank,781,10,1463.081,0.63093,1.0,0.5,1.0 +hybrid+rerank,783,10,1598.29554,0.0,0.0,0.0,0.0 +hybrid+rerank,784,10,1535.22008,1.0,1.0,1.0,1.0 +hybrid+rerank,785,10,1638.12021,0.30103,1.0,0.11111,0.0 +hybrid+rerank,793,10,1576.78125,1.0,1.0,1.0,1.0 +hybrid+rerank,800,10,1455.34883,0.63093,1.0,0.5,1.0 +hybrid+rerank,805,10,1618.78054,1.0,1.0,1.0,1.0 +hybrid+rerank,808,10,1632.24037,1.0,1.0,1.0,1.0 +hybrid+rerank,811,10,1767.779,1.0,1.0,1.0,1.0 +hybrid+rerank,814,10,1795.03508,1.0,1.0,1.0,1.0 +hybrid+rerank,820,10,1652.90887,0.33333,1.0,0.14286,0.0 +hybrid+rerank,821,10,1733.16929,0.38685,1.0,0.2,0.0 +hybrid+rerank,823,10,1614.94825,1.0,1.0,1.0,1.0 +hybrid+rerank,830,10,1661.95029,0.0,0.0,0.0,0.0 +hybrid+rerank,831,10,1840.14204,0.0,0.0,0.0,0.0 +hybrid+rerank,832,10,1686.15583,1.0,1.0,1.0,1.0 +hybrid+rerank,834,10,1682.01338,0.33333,1.0,0.14286,0.0 +hybrid+rerank,837,10,1725.19283,0.63093,1.0,0.5,1.0 +hybrid+rerank,839,10,1581.16892,1.0,1.0,1.0,1.0 +hybrid+rerank,845,10,1265.99979,1.0,1.0,1.0,1.0 +hybrid+rerank,847,10,1369.46033,1.0,1.0,1.0,1.0 +hybrid+rerank,852,10,1787.83513,0.63093,1.0,0.5,1.0 +hybrid+rerank,859,10,1649.25204,0.5,1.0,0.33333,1.0 +hybrid+rerank,870,10,1769.55496,0.0,0.0,0.0,0.0 +hybrid+rerank,873,10,1631.75996,0.25202,0.4,0.2,0.0 +hybrid+rerank,879,10,1647.56833,1.0,1.0,1.0,1.0 +hybrid+rerank,880,10,1701.94954,1.0,1.0,1.0,1.0 +hybrid+rerank,882,10,1627.49617,1.0,1.0,1.0,1.0 +hybrid+rerank,887,10,1704.10858,0.0,0.0,0.0,0.0 +hybrid+rerank,903,10,1475.55458,1.0,1.0,1.0,1.0 +hybrid+rerank,904,10,1623.40579,1.0,1.0,1.0,1.0 +hybrid+rerank,907,10,1662.79946,1.0,1.0,1.0,1.0 +hybrid+rerank,911,10,1778.45513,1.0,1.0,1.0,1.0 +hybrid+rerank,913,10,1619.43537,0.0,0.0,0.0,0.0 +hybrid+rerank,914,10,1627.98646,0.0,0.0,0.0,0.0 +hybrid+rerank,921,10,1669.01054,1.0,1.0,1.0,1.0 +hybrid+rerank,922,10,1726.29742,1.0,1.0,1.0,1.0 +hybrid+rerank,936,10,1530.33063,1.0,1.0,1.0,1.0 +hybrid+rerank,956,10,1610.44837,1.0,1.0,1.0,1.0 +hybrid+rerank,957,10,1578.66179,1.0,1.0,1.0,1.0 +hybrid+rerank,960,10,1948.51175,1.0,1.0,1.0,1.0 +hybrid+rerank,967,10,1691.40587,0.91972,1.0,1.0,1.0 +hybrid+rerank,971,10,1305.33792,0.95583,1.0,1.0,1.0 +hybrid+rerank,975,10,1916.27133,0.0,0.0,0.0,0.0 +hybrid+rerank,982,10,1799.29987,1.0,1.0,1.0,1.0 +hybrid+rerank,985,10,1632.269,1.0,1.0,1.0,1.0 +hybrid+rerank,993,10,1547.30479,1.0,1.0,1.0,1.0 +hybrid+rerank,1012,10,1785.36804,1.0,1.0,1.0,1.0 +hybrid+rerank,1014,10,1618.63438,1.0,1.0,1.0,1.0 +hybrid+rerank,1019,10,1530.83346,0.63093,1.0,0.5,1.0 +hybrid+rerank,1020,10,1694.47658,0.5,1.0,0.33333,1.0 +hybrid+rerank,1021,10,1704.28233,0.5,1.0,0.33333,1.0 +hybrid+rerank,1024,10,1608.99721,1.0,1.0,1.0,1.0 +hybrid+rerank,1029,10,1579.66533,0.76536,0.66667,1.0,1.0 +hybrid+rerank,1041,10,1335.16754,0.21841,0.5,0.16667,0.0 +hybrid+rerank,1049,10,1659.15575,0.43068,1.0,0.25,0.0 +hybrid+rerank,1062,10,1551.96438,1.0,1.0,1.0,1.0 +hybrid+rerank,1086,10,1930.545,1.0,1.0,1.0,1.0 +hybrid+rerank,1088,10,1773.41837,1.0,1.0,1.0,1.0 +hybrid+rerank,1089,10,1680.24225,1.0,1.0,1.0,1.0 +hybrid+rerank,1099,10,1450.25967,0.43068,1.0,0.25,0.0 +hybrid+rerank,1100,10,1464.4175,0.43068,1.0,0.25,0.0 +hybrid+rerank,1104,10,1811.18633,1.0,1.0,1.0,1.0 +hybrid+rerank,1107,10,1519.94804,1.0,1.0,1.0,1.0 +hybrid+rerank,1110,10,2046.98217,0.0,0.0,0.0,0.0 +hybrid+rerank,1121,10,1482.7945,1.0,1.0,1.0,1.0 +hybrid+rerank,1130,10,1720.12188,1.0,1.0,1.0,1.0 +hybrid+rerank,1132,10,1417.71421,0.61315,0.5,1.0,1.0 +hybrid+rerank,1137,10,1630.49829,1.0,1.0,1.0,1.0 +hybrid+rerank,1140,10,1955.00629,1.0,1.0,1.0,1.0 +hybrid+rerank,1144,10,1949.4045,1.0,1.0,1.0,1.0 +hybrid+rerank,1146,10,1651.19912,1.0,1.0,1.0,1.0 +hybrid+rerank,1150,10,1603.90238,1.0,1.0,1.0,1.0 +hybrid+rerank,1163,10,1679.45542,1.0,1.0,1.0,1.0 +hybrid+rerank,1175,10,1622.81954,0.0,0.0,0.0,0.0 +hybrid+rerank,1179,10,1582.64921,0.30103,1.0,0.11111,0.0 +hybrid+rerank,1180,10,1394.64054,1.0,1.0,1.0,1.0 +hybrid+rerank,1185,10,1845.24621,1.0,1.0,1.0,1.0 +hybrid+rerank,1187,10,1693.83871,1.0,1.0,1.0,1.0 +hybrid+rerank,1191,10,1835.64971,0.0,0.0,0.0,0.0 +hybrid+rerank,1194,10,1694.44237,1.0,1.0,1.0,1.0 +hybrid+rerank,1196,10,1734.18804,0.43068,1.0,0.25,0.0 +hybrid+rerank,1197,10,1737.33938,0.43068,1.0,0.25,0.0 +hybrid+rerank,1199,10,1640.59642,0.0,0.0,0.0,0.0 +hybrid+rerank,1200,10,1764.73183,0.43068,1.0,0.25,0.0 +hybrid+rerank,1202,10,1671.38258,0.63093,1.0,0.5,1.0 +hybrid+rerank,1204,10,1647.47796,0.5,1.0,0.33333,1.0 +hybrid+rerank,1207,10,1729.82046,1.0,1.0,1.0,1.0 +hybrid+rerank,1213,10,1542.11083,0.0,0.0,0.0,0.0 +hybrid+rerank,1216,10,1791.5335,1.0,1.0,1.0,1.0 +hybrid+rerank,1221,10,1758.01571,0.31546,1.0,0.125,0.0 +hybrid+rerank,1225,10,1829.31787,1.0,1.0,1.0,1.0 +hybrid+rerank,1226,10,1733.47192,0.0,0.0,0.0,0.0 +hybrid+rerank,1232,10,1689.51496,1.0,1.0,1.0,1.0 +hybrid+rerank,1241,10,1643.97725,1.0,1.0,1.0,1.0 +hybrid+rerank,1245,10,1801.04813,0.0,0.0,0.0,0.0 +hybrid+rerank,1259,10,1923.79054,1.0,1.0,1.0,1.0 +hybrid+rerank,1262,10,1436.9475,1.0,1.0,1.0,1.0 +hybrid+rerank,1266,10,1347.66458,0.5,1.0,0.33333,1.0 +hybrid+rerank,1270,10,1762.14208,1.0,1.0,1.0,1.0 +hybrid+rerank,1271,10,1647.90567,1.0,1.0,1.0,1.0 +hybrid+rerank,1272,10,1683.95938,0.43068,1.0,0.25,0.0 +hybrid+rerank,1273,10,1393.79933,1.0,1.0,1.0,1.0 +hybrid+rerank,1274,10,1550.68075,0.93252,1.0,1.0,1.0 +hybrid+rerank,1278,10,1833.2625,0.0,0.0,0.0,0.0 +hybrid+rerank,1279,10,1812.83996,0.0,0.0,0.0,0.0 +hybrid+rerank,1280,10,1609.96417,0.0,0.0,0.0,0.0 +hybrid+rerank,1281,10,1809.89883,0.0,0.0,0.0,0.0 +hybrid+rerank,1282,10,2039.98075,1.0,1.0,1.0,1.0 +hybrid+rerank,1290,10,1904.31025,1.0,1.0,1.0,1.0 +hybrid+rerank,1292,10,2074.32188,0.63093,1.0,0.5,1.0 +hybrid+rerank,1298,10,2024.84079,1.0,1.0,1.0,1.0 +hybrid+rerank,1303,10,1865.04258,1.0,1.0,1.0,1.0 +hybrid+rerank,1316,10,1392.68217,0.0,0.0,0.0,0.0 +hybrid+rerank,1319,10,1570.11488,1.0,1.0,1.0,1.0 +hybrid+rerank,1320,10,1504.762,1.0,1.0,1.0,1.0 +hybrid+rerank,1332,10,1586.30592,0.0,0.0,0.0,0.0 +hybrid+rerank,1335,10,1620.59988,1.0,1.0,1.0,1.0 +hybrid+rerank,1336,10,1571.67879,1.0,1.0,1.0,1.0 +hybrid+rerank,1337,10,1537.57113,0.63093,1.0,0.5,1.0 +hybrid+rerank,1339,10,1827.51762,1.0,1.0,1.0,1.0 +hybrid+rerank,1344,10,1634.433,0.5,1.0,0.33333,1.0 +hybrid+rerank,1352,10,1669.91625,1.0,1.0,1.0,1.0 +hybrid+rerank,1359,10,1870.84125,1.0,1.0,1.0,1.0 +hybrid+rerank,1362,10,1996.43229,0.31546,1.0,0.125,0.0 +hybrid+rerank,1363,10,1760.20621,0.5,1.0,0.33333,1.0 +hybrid+rerank,1368,10,1424.03308,1.0,1.0,1.0,1.0 +hybrid+rerank,1370,10,1395.4115,0.63093,1.0,0.5,1.0 +hybrid+rerank,1379,10,1558.45071,0.83268,1.0,1.0,1.0 +hybrid+rerank,1382,10,1516.68617,1.0,1.0,1.0,1.0 +hybrid+rerank,1385,10,1638.36687,1.0,1.0,1.0,1.0 +hybrid+rerank,1389,10,1784.29037,1.0,1.0,1.0,1.0 +hybrid+rerank,1395,10,1979.69754,1.0,1.0,1.0,1.0 +app_default,1,0,733.20454,0.0,0.0,0.0,0.0 +app_default,3,3,505.39179,1.0,1.0,1.0,1.0 +app_default,5,1,688.36579,1.0,1.0,1.0,1.0 +app_default,13,2,593.80225,0.0,0.0,0.0,0.0 +app_default,36,3,452.57854,0.0,0.0,0.0,0.0 +app_default,42,1,602.50663,1.0,1.0,1.0,1.0 +app_default,48,2,631.14687,0.0,0.0,0.0,0.0 +app_default,49,2,588.62612,1.0,1.0,1.0,1.0 +app_default,50,3,381.027,1.0,1.0,1.0,1.0 +app_default,51,3,628.34879,0.63093,1.0,0.5,1.0 +app_default,53,3,675.37029,1.0,1.0,1.0,1.0 +app_default,54,3,459.59721,0.0,0.0,0.0,0.0 +app_default,56,2,597.06925,1.0,1.0,1.0,1.0 +app_default,57,2,598.9425,1.0,1.0,1.0,1.0 +app_default,70,3,636.37083,0.0,0.0,0.0,0.0 +app_default,72,0,663.37458,0.0,0.0,0.0,0.0 +app_default,75,1,553.61029,1.0,1.0,1.0,1.0 +app_default,94,0,664.89525,0.0,0.0,0.0,0.0 +app_default,99,1,622.66167,1.0,1.0,1.0,1.0 +app_default,100,3,492.17342,1.0,1.0,1.0,1.0 +app_default,113,3,567.51837,1.0,1.0,1.0,1.0 +app_default,115,1,523.17608,1.0,1.0,1.0,1.0 +app_default,118,3,512.12129,1.0,1.0,1.0,1.0 +app_default,124,3,505.94554,1.0,1.0,1.0,1.0 +app_default,127,2,571.01575,0.63093,1.0,0.5,1.0 +app_default,128,0,655.10829,0.0,0.0,0.0,0.0 +app_default,129,1,596.07642,1.0,1.0,1.0,1.0 +app_default,130,1,562.64933,1.0,1.0,1.0,1.0 +app_default,132,3,580.29558,0.0,0.0,0.0,0.0 +app_default,133,3,645.80563,0.38357,0.4,0.5,1.0 +app_default,137,1,618.19954,1.0,1.0,1.0,1.0 +app_default,141,3,515.97075,0.91972,1.0,1.0,1.0 +app_default,142,3,577.79725,1.0,1.0,1.0,1.0 +app_default,143,3,578.94833,1.0,1.0,1.0,1.0 +app_default,146,3,572.133,1.0,1.0,1.0,1.0 +app_default,148,3,507.77192,1.0,1.0,1.0,1.0 +app_default,163,3,434.76396,0.63093,1.0,0.5,1.0 +app_default,171,3,403.39275,1.0,1.0,1.0,1.0 +app_default,179,3,495.08771,0.63668,0.5,1.0,1.0 +app_default,180,2,577.05,1.0,1.0,1.0,1.0 +app_default,183,3,457.0035,1.0,1.0,1.0,1.0 +app_default,185,3,589.28112,0.0,0.0,0.0,0.0 +app_default,198,0,640.36529,0.0,0.0,0.0,0.0 +app_default,208,3,665.19121,1.0,1.0,1.0,1.0 +app_default,212,3,486.08725,0.5,1.0,0.33333,1.0 +app_default,213,3,471.2155,0.0,0.0,0.0,0.0 +app_default,216,3,584.49412,1.0,1.0,1.0,1.0 +app_default,217,3,632.33163,1.0,1.0,1.0,1.0 +app_default,218,3,562.23613,1.0,1.0,1.0,1.0 +app_default,219,3,528.8825,1.0,1.0,1.0,1.0 +app_default,230,0,530.09213,0.0,0.0,0.0,0.0 +app_default,232,3,431.33471,0.63093,1.0,0.5,1.0 +app_default,233,1,485.59163,1.0,1.0,1.0,1.0 +app_default,236,1,516.60725,1.0,1.0,1.0,1.0 +app_default,237,3,530.44721,0.0,0.0,0.0,0.0 +app_default,238,1,523.708,0.0,0.0,0.0,0.0 +app_default,239,1,466.02017,0.0,0.0,0.0,0.0 +app_default,248,1,558.29462,1.0,1.0,1.0,1.0 +app_default,249,1,588.07687,1.0,1.0,1.0,1.0 +app_default,261,3,439.31875,0.38685,0.5,0.5,1.0 +app_default,268,2,561.36033,0.63093,1.0,0.5,1.0 +app_default,269,1,528.43379,0.0,0.0,0.0,0.0 +app_default,274,3,479.78033,1.0,1.0,1.0,1.0 +app_default,275,3,564.52642,0.70392,0.66667,1.0,1.0 +app_default,279,3,538.09088,1.0,1.0,1.0,1.0 +app_default,294,1,465.33,1.0,1.0,1.0,1.0 +app_default,295,3,480.55908,1.0,1.0,1.0,1.0 +app_default,298,3,536.16496,1.0,1.0,1.0,1.0 +app_default,300,2,591.19346,0.63093,1.0,0.5,1.0 +app_default,303,2,629.44483,0.0,0.0,0.0,0.0 +app_default,312,2,508.87317,0.0,0.0,0.0,0.0 +app_default,314,2,582.22833,0.0,0.0,0.0,0.0 +app_default,324,0,648.96188,0.0,0.0,0.0,0.0 +app_default,327,3,552.10033,1.0,1.0,1.0,1.0 +app_default,338,3,522.32446,1.0,1.0,1.0,1.0 +app_default,343,3,562.31375,0.69343,1.0,0.5,1.0 +app_default,350,2,487.29304,1.0,1.0,1.0,1.0 +app_default,354,2,587.89683,1.0,1.0,1.0,1.0 +app_default,362,3,480.62929,1.0,1.0,1.0,1.0 +app_default,380,2,611.84133,1.0,1.0,1.0,1.0 +app_default,384,3,550.49179,0.0,0.0,0.0,0.0 +app_default,385,3,563.23646,0.38685,0.5,0.5,1.0 +app_default,386,1,558.40821,1.0,1.0,1.0,1.0 +app_default,388,0,556.52208,0.0,0.0,0.0,0.0 +app_default,399,3,470.85896,1.0,1.0,1.0,1.0 +app_default,410,3,499.22233,1.0,1.0,1.0,1.0 +app_default,411,3,436.39442,1.0,1.0,1.0,1.0 +app_default,415,3,429.13171,0.5,1.0,0.33333,1.0 +app_default,421,0,655.7795,0.0,0.0,0.0,0.0 +app_default,431,3,500.44513,0.0,0.0,0.0,0.0 +app_default,436,1,551.4485,1.0,1.0,1.0,1.0 +app_default,437,3,492.74621,0.0,0.0,0.0,0.0 +app_default,439,2,531.03688,1.0,1.0,1.0,1.0 +app_default,440,1,566.88171,1.0,1.0,1.0,1.0 +app_default,443,3,578.05283,1.0,1.0,1.0,1.0 +app_default,452,3,587.24954,0.38685,0.5,0.5,1.0 +app_default,475,0,498.27433,0.0,0.0,0.0,0.0 +app_default,478,1,486.17067,1.0,1.0,1.0,1.0 +app_default,491,1,588.54358,1.0,1.0,1.0,1.0 +app_default,501,2,577.56225,1.0,1.0,1.0,1.0 +app_default,502,0,650.00442,0.0,0.0,0.0,0.0 +app_default,507,3,586.47975,0.63093,1.0,0.5,1.0 +app_default,508,1,522.59488,0.0,0.0,0.0,0.0 +app_default,513,3,581.99758,0.63093,1.0,0.5,1.0 +app_default,514,3,562.79188,0.5,1.0,0.33333,1.0 +app_default,516,3,471.5795,1.0,1.0,1.0,1.0 +app_default,517,2,512.03079,0.0,0.0,0.0,0.0 +app_default,521,1,531.67233,1.0,1.0,1.0,1.0 +app_default,525,3,560.00142,1.0,1.0,1.0,1.0 +app_default,527,1,560.80738,0.0,0.0,0.0,0.0 +app_default,528,3,543.16775,0.5,1.0,0.33333,1.0 +app_default,532,1,604.21917,1.0,1.0,1.0,1.0 +app_default,533,1,605.69304,1.0,1.0,1.0,1.0 +app_default,535,3,571.154,0.0,0.0,0.0,0.0 +app_default,536,2,577.85871,1.0,1.0,1.0,1.0 +app_default,539,3,513.30983,1.0,1.0,1.0,1.0 +app_default,540,3,498.99783,0.30657,0.5,0.33333,1.0 +app_default,544,2,542.42237,0.0,0.0,0.0,0.0 +app_default,549,3,604.64687,1.0,1.0,1.0,1.0 +app_default,551,2,590.98946,0.0,0.0,0.0,0.0 +app_default,552,3,406.93217,1.0,1.0,1.0,1.0 +app_default,554,3,441.94529,0.0,0.0,0.0,0.0 +app_default,560,3,553.04862,0.0,0.0,0.0,0.0 +app_default,569,3,402.05775,1.0,1.0,1.0,1.0 +app_default,575,1,528.98325,1.0,1.0,1.0,1.0 +app_default,577,0,590.97329,0.0,0.0,0.0,0.0 +app_default,578,2,582.02183,1.0,1.0,1.0,1.0 +app_default,587,1,569.14137,0.0,0.0,0.0,0.0 +app_default,589,1,592.30412,1.0,1.0,1.0,1.0 +app_default,593,2,575.74646,1.0,1.0,1.0,1.0 +app_default,597,3,483.0865,0.46928,0.33333,1.0,1.0 +app_default,598,3,426.38871,0.0,0.0,0.0,0.0 +app_default,613,1,587.03792,1.0,1.0,1.0,1.0 +app_default,619,0,610.13404,0.0,0.0,0.0,0.0 +app_default,623,3,495.89167,0.0,0.0,0.0,0.0 +app_default,628,3,558.18567,1.0,1.0,1.0,1.0 +app_default,636,3,488.96254,1.0,1.0,1.0,1.0 +app_default,637,1,583.0355,0.0,0.0,0.0,0.0 +app_default,641,2,520.60612,0.61315,0.5,1.0,1.0 +app_default,644,2,608.25008,1.0,1.0,1.0,1.0 +app_default,649,0,612.34404,0.0,0.0,0.0,0.0 +app_default,659,0,613.62863,0.0,0.0,0.0,0.0 +app_default,660,0,647.96233,0.0,0.0,0.0,0.0 +app_default,674,3,512.40746,1.0,1.0,1.0,1.0 +app_default,684,3,490.65658,0.0,0.0,0.0,0.0 +app_default,690,0,612.91492,0.0,0.0,0.0,0.0 +app_default,691,3,585.44546,0.63093,1.0,0.5,1.0 +app_default,692,1,577.90583,1.0,1.0,1.0,1.0 +app_default,693,1,618.75125,1.0,1.0,1.0,1.0 +app_default,700,1,552.43517,0.0,0.0,0.0,0.0 +app_default,702,2,442.92125,0.0,0.0,0.0,0.0 +app_default,715,3,641.64833,0.0,0.0,0.0,0.0 +app_default,716,2,645.26362,0.0,0.0,0.0,0.0 +app_default,718,0,554.26196,0.0,0.0,0.0,0.0 +app_default,721,1,582.77854,1.0,1.0,1.0,1.0 +app_default,723,1,489.44754,1.0,1.0,1.0,1.0 +app_default,727,2,516.77304,0.0,0.0,0.0,0.0 +app_default,728,3,521.30813,0.61315,0.5,1.0,1.0 +app_default,729,3,582.1855,1.0,1.0,1.0,1.0 +app_default,742,1,600.66879,1.0,1.0,1.0,1.0 +app_default,743,3,566.70842,1.0,1.0,1.0,1.0 +app_default,744,3,434.27308,0.63093,1.0,0.5,1.0 +app_default,756,3,423.10846,0.5,1.0,0.33333,1.0 +app_default,759,3,432.97463,1.0,1.0,1.0,1.0 +app_default,768,1,567.64821,0.0,0.0,0.0,0.0 +app_default,770,3,589.38787,1.0,1.0,1.0,1.0 +app_default,775,3,428.68171,0.0,0.0,0.0,0.0 +app_default,781,3,415.79283,0.63093,1.0,0.5,1.0 +app_default,783,3,530.27879,0.0,0.0,0.0,0.0 +app_default,784,3,625.71062,1.0,1.0,1.0,1.0 +app_default,785,1,574.86817,0.0,0.0,0.0,0.0 +app_default,793,2,547.97892,1.0,1.0,1.0,1.0 +app_default,800,3,524.10225,0.63093,1.0,0.5,1.0 +app_default,805,3,455.57392,1.0,1.0,1.0,1.0 +app_default,808,1,573.54813,1.0,1.0,1.0,1.0 +app_default,811,3,610.32904,1.0,1.0,1.0,1.0 +app_default,814,2,586.37763,1.0,1.0,1.0,1.0 +app_default,820,0,575.716,0.0,0.0,0.0,0.0 +app_default,821,0,573.37792,0.0,0.0,0.0,0.0 +app_default,823,3,490.18575,1.0,1.0,1.0,1.0 +app_default,830,1,579.52158,0.0,0.0,0.0,0.0 +app_default,831,0,573.00075,0.0,0.0,0.0,0.0 +app_default,832,3,542.56733,1.0,1.0,1.0,1.0 +app_default,834,1,533.65104,0.0,0.0,0.0,0.0 +app_default,837,3,484.09596,0.63093,1.0,0.5,1.0 +app_default,839,3,470.99662,1.0,1.0,1.0,1.0 +app_default,845,3,472.88933,1.0,1.0,1.0,1.0 +app_default,847,2,460.43387,1.0,1.0,1.0,1.0 +app_default,852,3,495.161,0.63093,1.0,0.5,1.0 +app_default,859,3,504.44163,0.5,1.0,0.33333,1.0 +app_default,870,3,594.065,0.0,0.0,0.0,0.0 +app_default,873,3,637.78308,0.0,0.0,0.0,0.0 +app_default,879,1,526.40275,1.0,1.0,1.0,1.0 +app_default,880,1,547.42004,1.0,1.0,1.0,1.0 +app_default,882,1,550.87038,1.0,1.0,1.0,1.0 +app_default,887,0,544.01946,0.0,0.0,0.0,0.0 +app_default,903,3,498.48375,1.0,1.0,1.0,1.0 +app_default,904,2,516.17963,1.0,1.0,1.0,1.0 +app_default,907,3,551.54942,1.0,1.0,1.0,1.0 +app_default,911,3,556.79658,1.0,1.0,1.0,1.0 +app_default,913,3,509.12217,0.0,0.0,0.0,0.0 +app_default,914,3,594.22575,0.0,0.0,0.0,0.0 +app_default,921,3,545.33179,1.0,1.0,1.0,1.0 +app_default,922,1,572.2675,1.0,1.0,1.0,1.0 +app_default,936,1,515.96842,1.0,1.0,1.0,1.0 +app_default,956,3,580.22071,1.0,1.0,1.0,1.0 +app_default,957,1,445.98421,1.0,1.0,1.0,1.0 +app_default,960,1,610.69554,1.0,1.0,1.0,1.0 +app_default,967,3,553.55858,0.91972,1.0,1.0,1.0 +app_default,971,3,461.53379,0.63668,0.5,1.0,1.0 +app_default,975,3,609.90442,0.0,0.0,0.0,0.0 +app_default,982,1,625.72958,1.0,1.0,1.0,1.0 +app_default,985,3,552.55021,1.0,1.0,1.0,1.0 +app_default,993,3,420.21429,1.0,1.0,1.0,1.0 +app_default,1012,3,486.00092,1.0,1.0,1.0,1.0 +app_default,1014,1,574.44254,1.0,1.0,1.0,1.0 +app_default,1019,2,554.49246,0.63093,1.0,0.5,1.0 +app_default,1020,2,458.52125,0.63093,1.0,0.5,1.0 +app_default,1021,1,456.13667,0.0,0.0,0.0,0.0 +app_default,1024,2,484.17679,1.0,1.0,1.0,1.0 +app_default,1029,3,559.41017,0.76536,0.66667,1.0,1.0 +app_default,1041,3,320.37983,0.0,0.0,0.0,0.0 +app_default,1049,0,577.58263,0.0,0.0,0.0,0.0 +app_default,1062,3,427.16171,1.0,1.0,1.0,1.0 +app_default,1086,1,574.55629,1.0,1.0,1.0,1.0 +app_default,1088,3,564.99867,1.0,1.0,1.0,1.0 +app_default,1089,2,486.17533,1.0,1.0,1.0,1.0 +app_default,1099,3,513.71017,0.0,0.0,0.0,0.0 +app_default,1100,3,483.7295,0.0,0.0,0.0,0.0 +app_default,1104,2,520.21688,1.0,1.0,1.0,1.0 +app_default,1107,3,501.03942,1.0,1.0,1.0,1.0 +app_default,1110,0,614.15142,0.0,0.0,0.0,0.0 +app_default,1121,3,515.26125,1.0,1.0,1.0,1.0 +app_default,1130,3,611.07217,1.0,1.0,1.0,1.0 +app_default,1132,3,519.10904,0.61315,0.5,1.0,1.0 +app_default,1137,3,491.55225,1.0,1.0,1.0,1.0 +app_default,1140,0,633.58754,0.0,0.0,0.0,0.0 +app_default,1144,2,488.77712,1.0,1.0,1.0,1.0 +app_default,1146,1,575.26413,1.0,1.0,1.0,1.0 +app_default,1150,3,550.10925,1.0,1.0,1.0,1.0 +app_default,1163,2,516.11504,1.0,1.0,1.0,1.0 +app_default,1175,0,549.69533,0.0,0.0,0.0,0.0 +app_default,1179,1,568.69596,0.0,0.0,0.0,0.0 +app_default,1180,3,423.28321,1.0,1.0,1.0,1.0 +app_default,1185,1,548.736,1.0,1.0,1.0,1.0 +app_default,1187,3,590.75942,1.0,1.0,1.0,1.0 +app_default,1191,0,637.21863,0.0,0.0,0.0,0.0 +app_default,1194,1,478.32679,1.0,1.0,1.0,1.0 +app_default,1196,0,528.41475,0.0,0.0,0.0,0.0 +app_default,1197,0,400.2415,0.0,0.0,0.0,0.0 +app_default,1199,2,552.77871,0.0,0.0,0.0,0.0 +app_default,1200,0,646.4515,0.0,0.0,0.0,0.0 +app_default,1202,3,579.13192,0.63093,1.0,0.5,1.0 +app_default,1204,1,616.33096,0.0,0.0,0.0,0.0 +app_default,1207,1,612.00862,1.0,1.0,1.0,1.0 +app_default,1213,3,545.74004,0.0,0.0,0.0,0.0 +app_default,1216,1,603.49908,1.0,1.0,1.0,1.0 +app_default,1221,0,589.767,0.0,0.0,0.0,0.0 +app_default,1225,2,569.15562,1.0,1.0,1.0,1.0 +app_default,1226,1,595.40621,0.0,0.0,0.0,0.0 +app_default,1232,1,546.81338,1.0,1.0,1.0,1.0 +app_default,1241,3,641.78383,1.0,1.0,1.0,1.0 +app_default,1245,0,550.13638,0.0,0.0,0.0,0.0 +app_default,1259,3,551.55738,1.0,1.0,1.0,1.0 +app_default,1262,3,393.352,1.0,1.0,1.0,1.0 +app_default,1266,3,535.13137,0.5,1.0,0.33333,1.0 +app_default,1270,1,516.51958,1.0,1.0,1.0,1.0 +app_default,1271,1,524.04,1.0,1.0,1.0,1.0 +app_default,1272,0,591.89271,0.0,0.0,0.0,0.0 +app_default,1273,3,527.02804,1.0,1.0,1.0,1.0 +app_default,1274,3,463.9885,0.76536,0.66667,1.0,1.0 +app_default,1278,1,611.8225,0.0,0.0,0.0,0.0 +app_default,1279,1,600.22658,0.0,0.0,0.0,0.0 +app_default,1280,3,493.88092,0.0,0.0,0.0,0.0 +app_default,1281,1,556.12492,0.0,0.0,0.0,0.0 +app_default,1282,1,583.09358,1.0,1.0,1.0,1.0 +app_default,1290,1,539.25137,1.0,1.0,1.0,1.0 +app_default,1292,3,572.14746,0.63093,1.0,0.5,1.0 +app_default,1298,1,552.56762,1.0,1.0,1.0,1.0 +app_default,1303,0,600.61162,0.0,0.0,0.0,0.0 +app_default,1316,1,478.24679,0.0,0.0,0.0,0.0 +app_default,1319,3,400.16317,1.0,1.0,1.0,1.0 +app_default,1320,3,556.84804,1.0,1.0,1.0,1.0 +app_default,1332,3,582.25817,0.0,0.0,0.0,0.0 +app_default,1335,3,470.08375,1.0,1.0,1.0,1.0 +app_default,1336,3,514.15354,1.0,1.0,1.0,1.0 +app_default,1337,3,488.52421,0.63093,1.0,0.5,1.0 +app_default,1339,1,610.17933,1.0,1.0,1.0,1.0 +app_default,1344,3,569.877,0.5,1.0,0.33333,1.0 +app_default,1352,3,500.59688,1.0,1.0,1.0,1.0 +app_default,1359,3,496.08721,1.0,1.0,1.0,1.0 +app_default,1362,0,599.38275,0.0,0.0,0.0,0.0 +app_default,1363,0,572.44475,0.0,0.0,0.0,0.0 +app_default,1368,3,506.26629,1.0,1.0,1.0,1.0 +app_default,1370,2,506.14983,0.63093,1.0,0.5,1.0 +app_default,1379,3,460.56846,0.39038,0.25,1.0,1.0 +app_default,1382,3,549.27888,1.0,1.0,1.0,1.0 +app_default,1385,2,577.18729,1.0,1.0,1.0,1.0 +app_default,1389,3,610.73508,1.0,1.0,1.0,1.0 +app_default,1395,0,570.14354,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1,3,635.71329,0.0,0.0,0.0,0.0 +app_default_no_cutoff,3,3,521.00129,1.0,1.0,1.0,1.0 +app_default_no_cutoff,5,3,606.03963,1.0,1.0,1.0,1.0 +app_default_no_cutoff,13,3,539.86817,0.0,0.0,0.0,0.0 +app_default_no_cutoff,36,3,417.53783,0.0,0.0,0.0,0.0 +app_default_no_cutoff,42,3,556.59346,1.0,1.0,1.0,1.0 +app_default_no_cutoff,48,3,580.50246,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,49,3,541.15279,1.0,1.0,1.0,1.0 +app_default_no_cutoff,50,3,343.82392,1.0,1.0,1.0,1.0 +app_default_no_cutoff,51,3,572.0615,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,53,3,611.76092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,54,3,422.85133,0.0,0.0,0.0,0.0 +app_default_no_cutoff,56,3,556.18342,1.0,1.0,1.0,1.0 +app_default_no_cutoff,57,3,554.6595,1.0,1.0,1.0,1.0 +app_default_no_cutoff,70,3,573.16179,0.0,0.0,0.0,0.0 +app_default_no_cutoff,72,3,603.43579,1.0,1.0,1.0,1.0 +app_default_no_cutoff,75,3,515.15092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,94,3,611.64217,0.0,0.0,0.0,0.0 +app_default_no_cutoff,99,3,571.79,1.0,1.0,1.0,1.0 +app_default_no_cutoff,100,3,451.94492,1.0,1.0,1.0,1.0 +app_default_no_cutoff,113,3,514.91692,1.0,1.0,1.0,1.0 +app_default_no_cutoff,115,3,485.00688,1.0,1.0,1.0,1.0 +app_default_no_cutoff,118,3,473.75358,1.0,1.0,1.0,1.0 +app_default_no_cutoff,124,3,456.73542,1.0,1.0,1.0,1.0 +app_default_no_cutoff,127,3,513.45537,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,128,3,604.68342,0.0,0.0,0.0,0.0 +app_default_no_cutoff,129,3,557.28571,1.0,1.0,1.0,1.0 +app_default_no_cutoff,130,3,525.69092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,132,3,546.90575,0.0,0.0,0.0,0.0 +app_default_no_cutoff,133,3,614.23583,0.38357,0.4,0.5,1.0 +app_default_no_cutoff,137,3,574.99517,1.0,1.0,1.0,1.0 +app_default_no_cutoff,141,3,490.88271,0.91972,1.0,1.0,1.0 +app_default_no_cutoff,142,3,565.99592,1.0,1.0,1.0,1.0 +app_default_no_cutoff,143,3,556.53654,1.0,1.0,1.0,1.0 +app_default_no_cutoff,146,3,555.38633,1.0,1.0,1.0,1.0 +app_default_no_cutoff,148,3,483.89308,1.0,1.0,1.0,1.0 +app_default_no_cutoff,163,3,426.53933,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,171,3,393.60967,1.0,1.0,1.0,1.0 +app_default_no_cutoff,179,3,489.35275,0.63668,0.5,1.0,1.0 +app_default_no_cutoff,180,3,553.73683,1.0,1.0,1.0,1.0 +app_default_no_cutoff,183,3,448.48137,1.0,1.0,1.0,1.0 +app_default_no_cutoff,185,3,575.35133,0.0,0.0,0.0,0.0 +app_default_no_cutoff,198,3,613.39575,0.0,0.0,0.0,0.0 +app_default_no_cutoff,208,3,599.45238,1.0,1.0,1.0,1.0 +app_default_no_cutoff,212,3,476.73183,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,213,3,455.64279,0.0,0.0,0.0,0.0 +app_default_no_cutoff,216,3,580.79617,1.0,1.0,1.0,1.0 +app_default_no_cutoff,217,3,606.84142,1.0,1.0,1.0,1.0 +app_default_no_cutoff,218,3,546.31671,1.0,1.0,1.0,1.0 +app_default_no_cutoff,219,3,519.58096,1.0,1.0,1.0,1.0 +app_default_no_cutoff,230,3,516.67004,1.0,1.0,1.0,1.0 +app_default_no_cutoff,232,3,432.03042,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,233,3,470.18717,1.0,1.0,1.0,1.0 +app_default_no_cutoff,236,3,511.62829,1.0,1.0,1.0,1.0 +app_default_no_cutoff,237,3,522.31733,0.0,0.0,0.0,0.0 +app_default_no_cutoff,238,3,520.39283,0.0,0.0,0.0,0.0 +app_default_no_cutoff,239,3,462.30367,0.0,0.0,0.0,0.0 +app_default_no_cutoff,248,3,549.78917,1.0,1.0,1.0,1.0 +app_default_no_cutoff,249,3,604.22692,1.0,1.0,1.0,1.0 +app_default_no_cutoff,261,3,423.295,0.38685,0.5,0.5,1.0 +app_default_no_cutoff,268,3,542.53779,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,269,3,513.80867,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,274,3,463.47829,1.0,1.0,1.0,1.0 +app_default_no_cutoff,275,3,551.33892,0.70392,0.66667,1.0,1.0 +app_default_no_cutoff,279,3,527.96458,1.0,1.0,1.0,1.0 +app_default_no_cutoff,294,3,471.46846,1.0,1.0,1.0,1.0 +app_default_no_cutoff,295,3,461.75604,1.0,1.0,1.0,1.0 +app_default_no_cutoff,298,3,522.83638,1.0,1.0,1.0,1.0 +app_default_no_cutoff,300,3,590.6725,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,303,3,609.35796,0.0,0.0,0.0,0.0 +app_default_no_cutoff,312,3,500.33904,0.0,0.0,0.0,0.0 +app_default_no_cutoff,314,3,579.90804,0.0,0.0,0.0,0.0 +app_default_no_cutoff,324,3,636.62704,0.0,0.0,0.0,0.0 +app_default_no_cutoff,327,3,545.78458,1.0,1.0,1.0,1.0 +app_default_no_cutoff,338,3,512.23408,1.0,1.0,1.0,1.0 +app_default_no_cutoff,343,3,553.14658,0.69343,1.0,0.5,1.0 +app_default_no_cutoff,350,3,479.855,1.0,1.0,1.0,1.0 +app_default_no_cutoff,354,3,582.00033,1.0,1.0,1.0,1.0 +app_default_no_cutoff,362,3,495.32704,1.0,1.0,1.0,1.0 +app_default_no_cutoff,380,3,610.13288,1.0,1.0,1.0,1.0 +app_default_no_cutoff,384,3,546.36171,0.0,0.0,0.0,0.0 +app_default_no_cutoff,385,3,557.71071,0.38685,0.5,0.5,1.0 +app_default_no_cutoff,386,3,550.62713,1.0,1.0,1.0,1.0 +app_default_no_cutoff,388,3,548.08254,0.0,0.0,0.0,0.0 +app_default_no_cutoff,399,3,457.8035,1.0,1.0,1.0,1.0 +app_default_no_cutoff,410,3,498.20204,1.0,1.0,1.0,1.0 +app_default_no_cutoff,411,3,420.0825,1.0,1.0,1.0,1.0 +app_default_no_cutoff,415,3,425.37042,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,421,3,635.80571,0.0,0.0,0.0,0.0 +app_default_no_cutoff,431,3,500.29392,0.0,0.0,0.0,0.0 +app_default_no_cutoff,436,3,550.24767,1.0,1.0,1.0,1.0 +app_default_no_cutoff,437,3,484.9915,0.0,0.0,0.0,0.0 +app_default_no_cutoff,439,3,524.67325,1.0,1.0,1.0,1.0 +app_default_no_cutoff,440,3,563.40442,1.0,1.0,1.0,1.0 +app_default_no_cutoff,443,3,597.12187,1.0,1.0,1.0,1.0 +app_default_no_cutoff,452,3,585.18833,0.38685,0.5,0.5,1.0 +app_default_no_cutoff,475,3,498.95112,0.0,0.0,0.0,0.0 +app_default_no_cutoff,478,3,475.30729,1.0,1.0,1.0,1.0 +app_default_no_cutoff,491,3,580.3835,1.0,1.0,1.0,1.0 +app_default_no_cutoff,501,3,573.872,1.0,1.0,1.0,1.0 +app_default_no_cutoff,502,3,654.09642,0.0,0.0,0.0,0.0 +app_default_no_cutoff,507,3,551.71929,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,508,3,513.39979,0.0,0.0,0.0,0.0 +app_default_no_cutoff,513,3,574.74721,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,514,3,553.91954,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,516,3,456.1065,1.0,1.0,1.0,1.0 +app_default_no_cutoff,517,3,513.84625,0.0,0.0,0.0,0.0 +app_default_no_cutoff,521,3,521.45813,1.0,1.0,1.0,1.0 +app_default_no_cutoff,525,3,651.56392,1.0,1.0,1.0,1.0 +app_default_no_cutoff,527,3,610.57825,0.0,0.0,0.0,0.0 +app_default_no_cutoff,528,3,567.26283,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,532,3,723.83821,1.0,1.0,1.0,1.0 +app_default_no_cutoff,533,3,771.60742,1.0,1.0,1.0,1.0 +app_default_no_cutoff,535,3,641.54092,0.0,0.0,0.0,0.0 +app_default_no_cutoff,536,3,704.78371,1.0,1.0,1.0,1.0 +app_default_no_cutoff,539,3,616.73883,1.0,1.0,1.0,1.0 +app_default_no_cutoff,540,3,622.02683,0.30657,0.5,0.33333,1.0 +app_default_no_cutoff,544,3,699.25354,0.0,0.0,0.0,0.0 +app_default_no_cutoff,549,3,829.39008,1.0,1.0,1.0,1.0 +app_default_no_cutoff,551,3,859.17833,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,552,3,673.44667,1.0,1.0,1.0,1.0 +app_default_no_cutoff,554,3,673.80837,0.0,0.0,0.0,0.0 +app_default_no_cutoff,560,3,843.41258,0.0,0.0,0.0,0.0 +app_default_no_cutoff,569,3,538.40483,1.0,1.0,1.0,1.0 +app_default_no_cutoff,575,3,776.94079,1.0,1.0,1.0,1.0 +app_default_no_cutoff,577,3,855.66792,0.0,0.0,0.0,0.0 +app_default_no_cutoff,578,3,1034.15608,1.0,1.0,1.0,1.0 +app_default_no_cutoff,587,3,756.52879,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,589,3,727.54312,1.0,1.0,1.0,1.0 +app_default_no_cutoff,593,3,696.33487,1.0,1.0,1.0,1.0 +app_default_no_cutoff,597,3,585.11642,0.46928,0.33333,1.0,1.0 +app_default_no_cutoff,598,3,537.59738,0.0,0.0,0.0,0.0 +app_default_no_cutoff,613,3,763.85529,1.0,1.0,1.0,1.0 +app_default_no_cutoff,619,3,724.80225,0.61315,0.5,1.0,1.0 +app_default_no_cutoff,623,3,569.92996,0.0,0.0,0.0,0.0 +app_default_no_cutoff,628,3,622.77954,1.0,1.0,1.0,1.0 +app_default_no_cutoff,636,3,543.31158,1.0,1.0,1.0,1.0 +app_default_no_cutoff,637,3,652.73967,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,641,3,602.29967,0.61315,0.5,1.0,1.0 +app_default_no_cutoff,644,3,684.29633,1.0,1.0,1.0,1.0 +app_default_no_cutoff,649,3,689.24883,1.0,1.0,1.0,1.0 +app_default_no_cutoff,659,3,671.98113,0.0,0.0,0.0,0.0 +app_default_no_cutoff,660,3,712.41996,0.0,0.0,0.0,0.0 +app_default_no_cutoff,674,3,568.31663,1.0,1.0,1.0,1.0 +app_default_no_cutoff,684,3,541.36375,0.0,0.0,0.0,0.0 +app_default_no_cutoff,690,3,672.24033,0.0,0.0,0.0,0.0 +app_default_no_cutoff,691,3,668.58988,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,692,3,639.22471,1.0,1.0,1.0,1.0 +app_default_no_cutoff,693,3,674.62821,1.0,1.0,1.0,1.0 +app_default_no_cutoff,700,3,603.55233,0.0,0.0,0.0,0.0 +app_default_no_cutoff,702,3,563.05546,0.0,0.0,0.0,0.0 +app_default_no_cutoff,715,3,912.75921,0.0,0.0,0.0,0.0 +app_default_no_cutoff,716,3,882.02054,0.0,0.0,0.0,0.0 +app_default_no_cutoff,718,3,737.05358,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,721,3,718.89129,1.0,1.0,1.0,1.0 +app_default_no_cutoff,723,3,597.07179,1.0,1.0,1.0,1.0 +app_default_no_cutoff,727,3,650.31713,0.0,0.0,0.0,0.0 +app_default_no_cutoff,728,3,652.01612,0.61315,0.5,1.0,1.0 +app_default_no_cutoff,729,3,719.66271,1.0,1.0,1.0,1.0 +app_default_no_cutoff,742,3,704.16863,1.0,1.0,1.0,1.0 +app_default_no_cutoff,743,3,725.97458,1.0,1.0,1.0,1.0 +app_default_no_cutoff,744,3,591.43804,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,756,3,541.35271,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,759,3,539.349,1.0,1.0,1.0,1.0 +app_default_no_cutoff,768,3,676.71513,0.0,0.0,0.0,0.0 +app_default_no_cutoff,770,3,674.28225,1.0,1.0,1.0,1.0 +app_default_no_cutoff,775,3,493.72621,0.0,0.0,0.0,0.0 +app_default_no_cutoff,781,3,528.04767,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,783,3,613.75275,0.0,0.0,0.0,0.0 +app_default_no_cutoff,784,3,710.28812,1.0,1.0,1.0,1.0 +app_default_no_cutoff,785,3,666.88042,0.0,0.0,0.0,0.0 +app_default_no_cutoff,793,3,636.21321,1.0,1.0,1.0,1.0 +app_default_no_cutoff,800,3,614.47771,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,805,3,524.26717,1.0,1.0,1.0,1.0 +app_default_no_cutoff,808,3,662.65621,1.0,1.0,1.0,1.0 +app_default_no_cutoff,811,3,703.75129,1.0,1.0,1.0,1.0 +app_default_no_cutoff,814,3,661.23971,1.0,1.0,1.0,1.0 +app_default_no_cutoff,820,3,654.36079,0.0,0.0,0.0,0.0 +app_default_no_cutoff,821,3,649.09079,0.0,0.0,0.0,0.0 +app_default_no_cutoff,823,3,545.01687,1.0,1.0,1.0,1.0 +app_default_no_cutoff,830,3,670.95013,0.0,0.0,0.0,0.0 +app_default_no_cutoff,831,3,662.02488,0.0,0.0,0.0,0.0 +app_default_no_cutoff,832,3,640.15546,1.0,1.0,1.0,1.0 +app_default_no_cutoff,834,3,651.51137,0.0,0.0,0.0,0.0 +app_default_no_cutoff,837,3,545.53525,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,839,3,515.72683,1.0,1.0,1.0,1.0 +app_default_no_cutoff,845,3,527.81975,1.0,1.0,1.0,1.0 +app_default_no_cutoff,847,3,525.41358,1.0,1.0,1.0,1.0 +app_default_no_cutoff,852,3,546.6025,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,859,3,571.01792,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,870,3,671.13608,0.0,0.0,0.0,0.0 +app_default_no_cutoff,873,3,715.26554,0.0,0.0,0.0,0.0 +app_default_no_cutoff,879,3,639.05883,1.0,1.0,1.0,1.0 +app_default_no_cutoff,880,3,621.54567,1.0,1.0,1.0,1.0 +app_default_no_cutoff,882,3,610.59712,1.0,1.0,1.0,1.0 +app_default_no_cutoff,887,3,590.53813,0.0,0.0,0.0,0.0 +app_default_no_cutoff,903,3,536.09354,1.0,1.0,1.0,1.0 +app_default_no_cutoff,904,3,562.76675,1.0,1.0,1.0,1.0 +app_default_no_cutoff,907,3,594.50133,1.0,1.0,1.0,1.0 +app_default_no_cutoff,911,3,609.15238,1.0,1.0,1.0,1.0 +app_default_no_cutoff,913,3,567.46783,0.0,0.0,0.0,0.0 +app_default_no_cutoff,914,3,656.05371,0.0,0.0,0.0,0.0 +app_default_no_cutoff,921,3,595.16537,1.0,1.0,1.0,1.0 +app_default_no_cutoff,922,3,631.85567,1.0,1.0,1.0,1.0 +app_default_no_cutoff,936,3,570.17821,1.0,1.0,1.0,1.0 +app_default_no_cutoff,956,3,641.84262,1.0,1.0,1.0,1.0 +app_default_no_cutoff,957,3,488.94387,1.0,1.0,1.0,1.0 +app_default_no_cutoff,960,3,650.80092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,967,3,589.00696,0.91972,1.0,1.0,1.0 +app_default_no_cutoff,971,3,506.87408,0.63668,0.5,1.0,1.0 +app_default_no_cutoff,975,3,660.13117,0.0,0.0,0.0,0.0 +app_default_no_cutoff,982,3,677.33429,1.0,1.0,1.0,1.0 +app_default_no_cutoff,985,3,609.42704,1.0,1.0,1.0,1.0 +app_default_no_cutoff,993,3,472.98292,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1012,3,529.63233,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1014,3,620.51267,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1019,3,603.73325,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1020,3,505.13554,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1021,3,497.87025,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1024,3,524.48129,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1029,3,601.6055,0.76536,0.66667,1.0,1.0 +app_default_no_cutoff,1041,3,348.28146,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1049,3,629.385,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1062,3,467.17392,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1086,3,624.85937,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1088,3,637.00729,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1089,3,534.98958,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1099,3,554.2435,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1100,3,529.90146,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1104,3,561.87654,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1107,3,533.5735,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1110,3,657.67233,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1121,3,612.03417,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1130,3,672.46825,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1132,3,569.99504,0.61315,0.5,1.0,1.0 +app_default_no_cutoff,1137,3,531.50779,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1140,3,694.69517,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1144,3,535.84087,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1146,3,628.76187,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1150,3,606.38058,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1163,3,569.94554,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1175,3,606.73154,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1179,3,622.45175,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1180,3,461.61092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1185,3,574.16238,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1187,3,650.67346,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1191,3,710.82383,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1194,3,522.07358,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1196,3,584.3855,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1197,3,440.04983,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1199,3,602.16321,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1200,3,715.04758,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1202,3,629.65925,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1204,3,671.40712,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,1207,3,679.93213,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1213,3,602.48017,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1216,3,661.56887,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1221,3,641.21254,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1225,3,625.53958,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1226,3,641.12017,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1232,3,604.842,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1241,3,690.12404,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1245,3,608.24083,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1259,3,607.93188,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1262,3,430.65325,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1266,3,563.91825,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,1270,3,576.4635,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1271,3,567.05604,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1272,3,627.21433,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1273,3,577.49146,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1274,3,511.53508,0.76536,0.66667,1.0,1.0 +app_default_no_cutoff,1278,3,659.54475,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1279,3,658.80833,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1280,3,533.123,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1281,3,614.90979,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1282,3,629.69075,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1290,3,563.11067,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1292,3,624.14204,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1298,3,601.68817,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1303,3,656.80029,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1316,3,522.05925,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1319,3,436.52433,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1320,3,635.48033,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1332,3,640.48092,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1335,3,512.76413,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1336,3,562.01246,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1337,3,543.11992,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1339,3,662.82242,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1344,3,610.64796,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,1352,3,550.16521,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1359,3,538.80092,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1362,3,656.45929,0.0,0.0,0.0,0.0 +app_default_no_cutoff,1363,3,621.62487,0.5,1.0,0.33333,1.0 +app_default_no_cutoff,1368,3,553.87054,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1370,3,575.48871,0.63093,1.0,0.5,1.0 +app_default_no_cutoff,1379,3,506.80154,0.39038,0.25,1.0,1.0 +app_default_no_cutoff,1382,3,601.52771,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1385,3,622.31158,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1389,3,657.56117,1.0,1.0,1.0,1.0 +app_default_no_cutoff,1395,3,687.82096,1.0,1.0,1.0,1.0 diff --git a/evaluation/reranker_check.py b/evaluation/reranker_check.py new file mode 100644 index 0000000..18c8a77 --- /dev/null +++ b/evaluation/reranker_check.py @@ -0,0 +1,156 @@ +"""Would a stronger reranker help? Chosen on the SciFact training claims, checked once on the test claims. + +Rules, fixed before running: +- Candidates: the app's reranker (ms-marco-MiniLM-L-6-v2), the larger ms-marco-MiniLM-L-12-v2, + and bge-reranker-base. +- Each one goes into the app's default call (3 passages from 9 vector + 9 keyword candidates) + with the confidence cut-off switched off, on the index with all 5,183 abstracts. +- Choice on the 809 training claims by Hit@3 (a right abstract among the 3 passages). A + candidate replaces the app's reranker only if its paired 95% bootstrap interval against the + app's reranker is above 0 on the training claims. +- If one is chosen, the test claims (300) are run once: Hit@3 against the app's reranker, and how + well its confidence tells answerable from unanswerable claims (AUROC, using the index without + the answering abstracts, as in `evaluation.cutoff_tuning`). + +The app's reranker results on test come from `evaluation.cutoff_tuning` (same call, same index), +so run that first. + + python -m evaluation.reranker_check --embedder bge-small-en-v1.5 +""" + +import argparse +import json +import statistics +import sys +import tempfile +import time +from pathlib import Path + +import numpy as np +import torch +from sentence_transformers import CrossEncoder, SentenceTransformer + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "src")) + +from rag_engine import EMBEDDERS, RERANKER, RAGEngine # noqa: E402 + +from evaluation.compare import paired_bootstrap # noqa: E402 +from evaluation.cutoff_tuning import add_passages # noqa: E402 +from evaluation.dense_cutoff import auroc # noqa: E402 +from evaluation.retrieval_eval import load # noqa: E402 + +CANDIDATES = { + "ms-marco-MiniLM-L-6-v2": RERANKER, + "ms-marco-MiniLM-L-12-v2": ("cross-encoder/ms-marco-MiniLM-L-12-v2", "7b0235231ca2674cb8ca8f022859a6eba2b1c968"), + "bge-reranker-base": ("BAAI/bge-reranker-base", "2cfc18c9415c912f9d8155881c133215df768a70"), +} +CURRENT = "ms-marco-MiniLM-L-6-v2" +OUT_DIR = Path(__file__).resolve().parent + + +def load_reranker(name: str) -> CrossEncoder: + repo, revision = CANDIDATES[name] + # Raw scores: the app turns them into a confidence itself (sigmoid). + return CrossEncoder(repo, revision=revision, device="cpu", activation_fn=torch.nn.Identity()) + + +def run(engine: RAGEngine, questions: dict, relevant: dict) -> tuple[dict, list[float]]: + rows, ms = {}, [] + for qid, q in questions.items(): + t0 = time.perf_counter() + _, metas = engine.retrieve(q, confidence_threshold=0.0) + ms.append((time.perf_counter() - t0) * 1000) + rows[qid] = {"confidence": [m["confidence"] for m in metas], + "relevant": [relevant[qid].get(m["source"], 0) > 0 for m in metas]} + return rows, ms + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("--embedder", choices=sorted(EMBEDDERS), default="bge-small-en-v1.5") + args = parser.parse_args() + + current_rows = [json.loads(line) for line in + open(OUT_DIR / f"cutoff_per_claim_{args.embedder}.jsonl", encoding="utf-8")] + corpus, test_q, test_rel = load("scifact", "test") + _, train_q, train_rel = load("scifact", "train") + texts = [(t + "\n" + b).strip() for t, b in zip(corpus["title"].fillna("").astype(str), + corpus["text"].fillna("").astype(str))] + ids = corpus["_id"].astype(str).tolist() + spec = EMBEDDERS[args.embedder] + embedder = SentenceTransformer(spec["repo"], revision=spec["revision"], device="cpu") + + t_start = time.time() + result = {"embedder": args.embedder, "candidates": {k: list(v) for k, v in CANDIDATES.items()}, + "train": {}, "test": {}} + with tempfile.TemporaryDirectory() as tmp: + def engine(name, reranker): + return RAGEngine(db_path=tmp, collection_name=name, embedding_model=embedder, + reranker=reranker, embedder=args.embedder) + + full = engine("all", load_reranker(CURRENT)) + emb = full.embed_passages(texts) + add_passages(full, ids, texts, emb) + print(f"index built in {time.time() - t_start:.0f} s", flush=True) + + train_hits = {} + for name in CANDIDATES: + full.reranker = load_reranker(name) + rows, ms = run(full, train_q, train_rel) + train_hits[name] = np.array([float(any(rows[q]["relevant"])) for q in train_q]) + result["train"][name] = {"hit@3": round(float(train_hits[name].mean()), 4), + "ms_per_claim_median": round(statistics.median(ms), 1)} + print(name, result["train"][name], f"{time.time() - t_start:.0f} s", flush=True) + + # The app's reranker here must find the same right abstracts as in cutoff_tuning + # (same call, same index). Only the hits are compared, not the confidences. + stored = {r["query_id"]: any(r["relevant"]) for r in current_rows + if r["split"] == "train" and r["index"] == "answerable"} + assert all(stored[q] == bool(h) for q, h in zip(train_q, train_hits[CURRENT])), "results differ" + + for name in CANDIDATES: + if name != CURRENT: + mean, low, high = paired_bootstrap(train_hits[name] - train_hits[CURRENT]) + result["train"][name]["difference_vs_current"] = { + "difference": round(mean, 4), "interval_95": [round(low, 4), round(high, 4)]} + winners = [n for n in CANDIDATES if n != CURRENT + and result["train"][n]["difference_vs_current"]["interval_95"][0] > 0] + chosen = max(winners, key=lambda n: result["train"][n]["hit@3"]) if winners else CURRENT + result["chosen_on_train"] = chosen + + if chosen != CURRENT: + full.reranker = load_reranker(chosen) + answers = {d for rel in test_rel.values() for d, g in rel.items() if g > 0} + keep = [i for i, d in enumerate(ids) if d not in answers] + without = engine("without-test", full.reranker) + add_passages(without, [ids[i] for i in keep], [texts[i] for i in keep], emb[keep]) + ans, ms = run(full, test_q, test_rel) + una, _ = run(without, test_q, test_rel) + cur_ans = {r["query_id"]: r for r in current_rows if r["split"] == "test" and r["index"] == "answerable"} + cur_una = {r["query_id"]: r for r in current_rows if r["split"] == "test" and r["index"] == "unanswerable"} + new_hit = np.array([float(any(ans[q]["relevant"])) for q in test_q]) + cur_hit = np.array([float(any(cur_ans[q]["relevant"])) for q in test_q]) + mean, low, high = paired_bootstrap(new_hit - cur_hit) + + def top(r): + return max(r["confidence"], default=0.0) + + result["test"] = { + chosen: {"hit@3": round(float(new_hit.mean()), 4), + "auroc_answerable_vs_unanswerable": round(auroc([top(ans[q]) for q in test_q], + [top(una[q]) for q in test_q]), 4), + "ms_per_claim_median": round(statistics.median(ms), 1)}, + CURRENT: {"hit@3": round(float(cur_hit.mean()), 4), + "auroc_answerable_vs_unanswerable": round(auroc([top(cur_ans[q]) for q in test_q], + [top(cur_una[q]) for q in test_q]), 4)}, + "difference": {"difference": round(mean, 4), "interval_95": [round(low, 4), round(high, 4)], + "claims": len(test_q)}, + } + result["minutes"] = round((time.time() - t_start) / 60, 1) + (OUT_DIR / f"reranker_check_{args.embedder}.json").write_text(json.dumps(result, indent=2) + "\n") + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/evaluation/reranker_check_bge-small-en-v1.5.json b/evaluation/reranker_check_bge-small-en-v1.5.json new file mode 100644 index 0000000..4f8a8f4 --- /dev/null +++ b/evaluation/reranker_check_bge-small-en-v1.5.json @@ -0,0 +1,48 @@ +{ + "embedder": "bge-small-en-v1.5", + "candidates": { + "ms-marco-MiniLM-L-6-v2": [ + "cross-encoder/ms-marco-MiniLM-L-6-v2", + "233902d25c440f23af6f7d6e94d2946bac0bee0a" + ], + "ms-marco-MiniLM-L-12-v2": [ + "cross-encoder/ms-marco-MiniLM-L-12-v2", + "7b0235231ca2674cb8ca8f022859a6eba2b1c968" + ], + "bge-reranker-base": [ + "BAAI/bge-reranker-base", + "2cfc18c9415c912f9d8155881c133215df768a70" + ] + }, + "train": { + "ms-marco-MiniLM-L-6-v2": { + "hit@3": 0.759, + "ms_per_claim_median": 659.4 + }, + "ms-marco-MiniLM-L-12-v2": { + "hit@3": 0.7565, + "ms_per_claim_median": 1315.6, + "difference_vs_current": { + "difference": -0.0025, + "interval_95": [ + -0.0173, + 0.0136 + ] + } + }, + "bge-reranker-base": { + "hit@3": 0.7787, + "ms_per_claim_median": 2986.5, + "difference_vs_current": { + "difference": 0.0198, + "interval_95": [ + -0.0012, + 0.0408 + ] + } + } + }, + "test": {}, + "chosen_on_train": "ms-marco-MiniLM-L-6-v2", + "minutes": 72.7 +} diff --git a/evaluation/results.json b/evaluation/results.json index f3b48c6..4d37b3c 100644 --- a/evaluation/results.json +++ b/evaluation/results.json @@ -1,5 +1,5 @@ { - "scifact": { + "scifact/all-MiniLM-L6-v2": { "dataset": "scifact", "documents_in_corpus": 5183, "documents_stored": 5183, @@ -79,6 +79,90 @@ "no_passage_share": 0.0, "ms_per_question_median": 741.0 } + }, + "embedder": "all-MiniLM-L6-v2" + }, + "scifact/bge-small-en-v1.5": { + "dataset": "scifact", + "embedder": "bge-small-en-v1.5", + "documents_in_corpus": 5183, + "documents_stored": 5183, + "test_questions": 300, + "full_test_set": true, + "sources": { + "corpus": [ + "BeIR/scifact", + "b3b5335604bf5ee3c4447671af975ea25143d4f5" + ], + "qrels": [ + "BeIR/scifact-qrels", + "2938d17dc3b09882fdb8c12bbbe2e2dc0e75a029" + ], + "embedder": [ + "BAAI/bge-small-en-v1.5", + "5c38ec7c405ec4b44b94cc5a9bb96e735b38267a" + ], + "reranker": [ + "cross-encoder/ms-marco-MiniLM-L-6-v2", + "233902d25c440f23af6f7d6e94d2946bac0bee0a" + ] + }, + "machine": "arm64, Darwin 25.4.0, CPU only", + "packages": { + "chromadb": "1.5.9", + "sentence-transformers": "6.3.0", + "torch": "2.15.0", + "rank-bm25": "0.2.2" + }, + "setups": { + "dense": { + "ndcg@10": 0.7127, + "recall@10": 0.8362, + "mrr@10": 0.6822, + "hit@3": 0.73, + "no_passage_share": 0.0, + "ms_per_question_median": 18.3 + }, + "bm25": { + "ndcg@10": 0.6519, + "recall@10": 0.774, + "mrr@10": 0.6186, + "hit@3": 0.6933, + "no_passage_share": 0.0, + "ms_per_question_median": 10.2 + }, + "dense+rerank": { + "ndcg@10": 0.7039, + "recall@10": 0.8352, + "mrr@10": 0.6692, + "hit@3": 0.7233, + "no_passage_share": 0.0, + "ms_per_question_median": 1051.2 + }, + "hybrid+rerank": { + "ndcg@10": 0.6965, + "recall@10": 0.8272, + "mrr@10": 0.6635, + "hit@3": 0.7233, + "no_passage_share": 0.0, + "ms_per_question_median": 1639.5 + }, + "app_default": { + "ndcg@10": 0.6074, + "recall@10": 0.6449, + "mrr@10": 0.6083, + "hit@3": 0.67, + "no_passage_share": 0.1233, + "ms_per_question_median": 552.7 + }, + "app_default_no_cutoff": { + "ndcg@10": 0.6466, + "recall@10": 0.6966, + "mrr@10": 0.6444, + "hit@3": 0.7233, + "no_passage_share": 0.0, + "ms_per_question_median": 580.4 + } } } } diff --git a/evaluation/retrieval_eval.py b/evaluation/retrieval_eval.py index 4953c6e..adf278f 100644 --- a/evaluation/retrieval_eval.py +++ b/evaluation/retrieval_eval.py @@ -4,7 +4,7 @@ (`src/rag_engine.py`, ChromaDB backend). The steps that read files and split them into chunks are not part of this test. Then every test question goes through these set-ups: - dense vector search only (all-MiniLM-L6-v2 in ChromaDB) + dense vector search only (the chosen embedding model in ChromaDB) bm25 keyword search only (the app's BM25) dense+rerank the app's retrieve() with BM25 switched off: vector candidates, then the cross-encoder puts them in order @@ -20,7 +20,7 @@ and compared with the same call without the cut-off. Run from the repository root: - python -m evaluation.retrieval_eval --dataset scifact + python -m evaluation.retrieval_eval --dataset scifact --embedder bge-small-en-v1.5 """ import argparse @@ -35,12 +35,12 @@ import pandas as pd from huggingface_hub import hf_hub_download -from sentence_transformers import CrossEncoder, SentenceTransformer +from sentence_transformers import SentenceTransformer ROOT = Path(__file__).resolve().parents[1] sys.path.insert(0, str(ROOT / "src")) -from rag_engine import RAGEngine # noqa: E402 +from rag_engine import EMBEDDERS, RERANKER, RAGEngine, load_reranker # noqa: E402 from evaluation.metrics import hit_at_k, mrr_at_k, ndcg_at_k, recall_at_k # noqa: E402 @@ -55,8 +55,6 @@ "qrels": ("BeIR/fiqa-qrels", "252958f2d646e22cab6d0c72dd3f0d5de6d0655a"), }, } -EMBEDDER = ("sentence-transformers/all-MiniLM-L6-v2", "1110a243fdf4706b3f48f1d95db1a4f5529b4d41") -RERANKER = ("cross-encoder/ms-marco-MiniLM-L-6-v2", "233902d25c440f23af6f7d6e94d2946bac0bee0a") K = 10 APP_TOP_K = 3 OUT_DIR = Path(__file__).resolve().parent @@ -82,10 +80,11 @@ def get(repo, file, rev): return corpus, questions, relevant -def build_engine(corpus: pd.DataFrame, embedder, reranker, db_path: str, name: str) -> tuple[RAGEngine, int]: +def build_engine(corpus: pd.DataFrame, embedder_name: str, embedder, reranker, db_path: str, + name: str) -> tuple[RAGEngine, int]: """Store every document as one passage, with its id as the source name.""" engine = RAGEngine(db_path=db_path, collection_name=f"beir_{name}", - embedding_model=embedder, reranker=reranker) + embedding_model=embedder, reranker=reranker, embedder=embedder_name) titles = corpus["title"].fillna("").astype(str) bodies = corpus["text"].fillna("").astype(str) texts = [(t + "\n" + b).strip() for t, b in zip(titles, bodies)] @@ -95,7 +94,7 @@ def build_engine(corpus: pd.DataFrame, embedder, reranker, db_path: str, name: s step = 2000 for start in range(0, len(texts), step): chunk_texts, chunk_ids = texts[start:start + step], ids[start:start + step] - emb = embedder.encode(chunk_texts, batch_size=64).tolist() + emb = engine.embed_passages(chunk_texts).tolist() engine.collection.add(ids=chunk_ids, documents=chunk_texts, embeddings=emb, metadatas=[{"source": i} for i in chunk_ids]) print(f" stored {start + len(chunk_texts)}/{len(texts)}", flush=True) @@ -109,6 +108,7 @@ def sources(metas) -> list[str]: def main() -> None: parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) parser.add_argument("--dataset", choices=sorted(DATASETS), default="scifact") + parser.add_argument("--embedder", choices=sorted(EMBEDDERS), default="bge-small-en-v1.5") parser.add_argument("--limit", type=int, default=0, help="only the first N questions (quick check)") args = parser.parse_args() @@ -117,14 +117,15 @@ def main() -> None: questions = dict(list(questions.items())[:args.limit]) print(f"{args.dataset}: {len(corpus)} documents, {len(questions)} test questions", flush=True) - embedder = SentenceTransformer(EMBEDDER[0], revision=EMBEDDER[1], device="cpu") - reranker = CrossEncoder(RERANKER[0], revision=RERANKER[1], device="cpu") + spec = EMBEDDERS[args.embedder] + embedder = SentenceTransformer(spec["repo"], revision=spec["revision"], device="cpu") + reranker = load_reranker(device="cpu") with tempfile.TemporaryDirectory() as tmp: - engine, n_stored = build_engine(corpus, embedder, reranker, tmp, args.dataset) + engine, n_stored = build_engine(corpus, args.embedder, embedder, reranker, tmp, args.dataset) def dense(q): - res = engine.collection.query(query_embeddings=embedder.encode([q]).tolist(), n_results=K) + res = engine.collection.query(query_embeddings=engine.embed_query(q).tolist(), n_results=K) return res["ids"][0] def bm25(q): @@ -183,12 +184,13 @@ def app_no_cutoff(q): result = { "dataset": args.dataset, + "embedder": args.embedder, "documents_in_corpus": len(corpus), "documents_stored": n_stored, "test_questions": len(questions), "full_test_set": not args.limit, "sources": {"corpus": DATASETS[args.dataset]["corpus"], "qrels": DATASETS[args.dataset]["qrels"], - "embedder": EMBEDDER, "reranker": RERANKER}, + "embedder": (spec["repo"], spec["revision"]), "reranker": RERANKER}, "machine": f"{platform.machine()}, {platform.system()} {platform.release()}, CPU only", "packages": {p: version(p) for p in ["chromadb", "sentence-transformers", "torch", "rank-bm25"]}, "setups": summary, @@ -199,10 +201,10 @@ def app_no_cutoff(q): return all_results_path = OUT_DIR / "results.json" all_results = json.loads(all_results_path.read_text()) if all_results_path.exists() else {} - all_results[args.dataset] = result + all_results[f"{args.dataset}/{args.embedder}"] = result all_results_path.write_text(json.dumps(all_results, indent=2) + "\n") - per_q.round(5).to_csv(OUT_DIR / f"per_question_{args.dataset}.csv", index=False) - print(f"Saved {all_results_path} and per_question_{args.dataset}.csv") + per_q.round(5).to_csv(OUT_DIR / f"per_question_{args.dataset}_{args.embedder}.csv", index=False) + print(f"Saved {all_results_path} and per_question_{args.dataset}_{args.embedder}.csv") if __name__ == "__main__": diff --git a/main.py b/main.py index 76dd989..5dff803 100644 --- a/main.py +++ b/main.py @@ -124,8 +124,12 @@ backend="faiss", embedding_model=rag_engine_chroma.embedding_model, reranker=rag_engine_chroma.reranker, + embedder=rag_engine_chroma.embedder, ) print("FAISS engine initialized successfully.") + except ValueError as e: + logger.error(f"FAISS index not used, it does not match: {e}") + print(f"FAISS index not used ({e}), using ChromaDB only.") except Exception as e: logger.warning(f"FAISS not available, using ChromaDB only: {e}") print(f"FAISS not available ({e}), using ChromaDB only.") @@ -138,6 +142,17 @@ # Keep a default alias for backward compatibility rag_engine = rag_engine_chroma + # Each embedding model has its own index. After a change of model (for example the switch + # of the default to bge-small-en-v1.5) the new index starts empty, while the uploaded files + # are still on disk: say so instead of answering "cannot answer" without a reason. + _uploads = [f for f in UPLOAD_DIR.glob("*") if f.suffix.lower() in SUPPORTED_EXTENSIONS] + if _uploads and rag_engine_chroma.get_stats()["total_vectors"] == 0: + _msg = (f"The index for embedding model {rag_engine_chroma.embedder} is empty, but " + f"{len(_uploads)} uploaded file(s) are in {UPLOAD_DIR}. Upload them again to index " + f"them with this model, or set RAG_EMBEDDER to the model used before.") + logger.warning(_msg) + print(f"WARNING: {_msg}") + print("Initializing Ollama LLM...") llm_client = OllamaClient(model="qwen3:1.7b") print("Server ready!") diff --git a/requirements.txt b/requirements.txt index 9673ee9..3e1b42f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,7 +2,7 @@ fastapi uvicorn requests chromadb -sentence-transformers +sentence-transformers>=4.0 pypdf python-multipart ollama diff --git a/src/rag_engine.py b/src/rag_engine.py index a87edfa..2409aa1 100644 --- a/src/rag_engine.py +++ b/src/rag_engine.py @@ -17,6 +17,7 @@ from docx import Document from openpyxl import load_workbook from pptx import Presentation +import torch from sentence_transformers import CrossEncoder, SentenceTransformer try: @@ -35,6 +36,63 @@ BM25_AVAILABLE = False +# Embedding models the engine can use. Each expects its own prefix before a question (and +# sometimes before a passage). bge-small-en-v1.5 was chosen on the SciFact training claims +# (see evaluation/embedder_choice.py and evaluation/embedder_choice_scifact.json). +EMBEDDERS = { + "bge-small-en-v1.5": { + "repo": "BAAI/bge-small-en-v1.5", + "revision": "5c38ec7c405ec4b44b94cc5a9bb96e735b38267a", + "query_prefix": "Represent this sentence for searching relevant passages: ", + "passage_prefix": "", + }, + "all-MiniLM-L6-v2": { + "repo": "sentence-transformers/all-MiniLM-L6-v2", + "revision": "1110a243fdf4706b3f48f1d95db1a4f5529b4d41", + "query_prefix": "", + "passage_prefix": "", + }, +} +DEFAULT_EMBEDDER = "bge-small-en-v1.5" +RERANKER = ("cross-encoder/ms-marco-MiniLM-L-6-v2", "233902d25c440f23af6f7d6e94d2946bac0bee0a") + + +def load_reranker(repo: str = RERANKER[0], revision: str = RERANKER[1], + device: Optional[str] = None) -> CrossEncoder: + """The reranker giving raw scores; the engine turns them into a confidence with its own sigmoid. + + Without ``activation_fn``, sentence-transformers applies a sigmoid to models whose config + names no activation (bge-reranker-base, for one), and the engine's sigmoid would come on top. + """ + return CrossEncoder(repo, revision=revision, device=device, activation_fn=torch.nn.Identity()) + + +def resolve_embedder(name: Optional[str]) -> str: + """The ``EMBEDDERS`` key for a name or a Hugging Face repo id; empty means the default.""" + name = (name or "").strip() or DEFAULT_EMBEDDER + for key, spec in EMBEDDERS.items(): + if name in (key, spec["repo"]): + return key + raise ValueError(f"Unknown embedder {name!r}. Use one of {sorted(EMBEDDERS)}.") + + +def embedder_record(key: str) -> Dict[str, str]: + """What is stored with an index, so it is never searched with a different model.""" + spec = EMBEDDERS[key] + return {"embedder": key, "embedder_repo": spec["repo"], "embedder_revision": spec["revision"]} + + +def check_embedder_record(stored: Optional[Dict], key: str, where: str) -> None: + """Raise if an index was built with another model (indexes without a record are accepted).""" + wanted = embedder_record(key) + stored = {k: v for k, v in (stored or {}).items() if k in wanted} + if stored and stored != wanted: + raise ValueError( + f"The index in {where} was built with {stored}, not with {wanted}. " + f"Use the matching RAG_EMBEDDER, or move the index away to build a new one." + ) + + def bm25_tokens(text: str) -> List[str]: """Lower-case words for BM25, without punctuation, so that "cancer." matches "cancer". @@ -54,6 +112,7 @@ def __init__( backend: str = "chroma", embedding_model: Optional[SentenceTransformer] = None, reranker: Optional[CrossEncoder] = None, + embedder: Optional[str] = None, ): """Initialize the Enhanced RAG Engine. @@ -61,49 +120,91 @@ def __init__( db_path: Path to persistent storage. collection_name: Name of the ChromaDB collection (chroma only). backend: Vector store backend – ``"chroma"`` or ``"faiss"``. - embedding_model: Pre-loaded SentenceTransformer (avoids reload). + embedding_model: Pre-loaded SentenceTransformer (avoids reload); + it must be the model named by ``embedder``. reranker: Pre-loaded CrossEncoder reranker (avoids reload). + embedder: A key of ``EMBEDDERS`` or its repo id (default: the + ``RAG_EMBEDDER`` environment variable, else bge-small-en-v1.5). + Each embedder has its own index, because vectors from different + models cannot be mixed; the model is recorded with the index and + checked when the index is opened. """ self.backend = backend - - # --- Shared models (load once, reuse across backends) --- - if embedding_model is not None: - self.embedding_model = embedding_model - else: - print("Loading embedding model... (this happens only once)") - self.embedding_model = SentenceTransformer("all-MiniLM-L6-v2") - - if reranker is not None: - self.reranker = reranker - else: - print("Loading reranker model...") - self.reranker = CrossEncoder( - "cross-encoder/ms-marco-MiniLM-L-6-v2" - ) - - # --- Backend-specific initialisation --- + embedder = resolve_embedder(embedder if embedder is not None else os.getenv("RAG_EMBEDDER")) + self.embedder = embedder + spec = EMBEDDERS[embedder] + self.query_prefix = spec["query_prefix"] + self.passage_prefix = spec["passage_prefix"] + # The index built before this option existed used all-MiniLM-L6-v2 under the plain name. + if embedder != "all-MiniLM-L6-v2": + collection_name = f"{collection_name}-{embedder}" + db_path = os.path.join(db_path, embedder) if backend == "faiss" else db_path + + # --- Is the index on disk from this model? Checked before any model loads. --- if backend == "chroma": self.chroma_client = chromadb.PersistentClient(path=db_path) self.collection = self.chroma_client.get_or_create_collection( name=collection_name ) + meta = self.collection.metadata or {} + check_embedder_record(meta, embedder, f"ChromaDB collection {collection_name!r}") + if not meta.get("embedder"): + # New collection, or one made before the model was recorded. Distance settings + # ("hnsw:...") cannot be passed to modify(), so they are left out. + kept = {k: v for k, v in meta.items() if not k.startswith("hnsw:")} + self.collection.modify(metadata={**kept, **embedder_record(embedder)}) elif backend == "faiss": if not FAISS_AVAILABLE: raise ImportError( "faiss-cpu is required for the FAISS backend. " "Install with: pip install faiss-cpu" ) - self._init_faiss(db_path) + meta_path = os.path.join(db_path, "faiss_meta.json") + if os.path.exists(meta_path): + with open(meta_path, "r", encoding="utf-8") as fh: + check_embedder_record(json_mod.load(fh).get("embedder"), embedder, + f"FAISS folder {db_path!r}") else: raise ValueError( f"Unknown backend: {backend}. Use 'chroma' or 'faiss'." ) + # --- Shared models (load once, reuse across backends) --- + if embedding_model is not None: + self.embedding_model = embedding_model + else: + print(f"Loading embedding model {embedder}... (this happens only once)") + self.embedding_model = SentenceTransformer(spec["repo"], revision=spec["revision"]) + + if reranker is not None: + self.reranker = reranker + else: + print("Loading reranker model...") + self.reranker = load_reranker() + + # --- FAISS needs the model's vector size to open or create its index --- + if backend == "faiss": + self._init_faiss(db_path) + # --- BM25 cache (avoids rebuilding index on every query) --- self._bm25_cache: Optional[Dict] = None print(f"Models loaded. Backend: {backend}") + # ------------------------------------------------------------------ + # Embeddings (unit length, with the prefixes the model expects) + # ------------------------------------------------------------------ + + def embed_passages(self, texts: List[str]) -> np.ndarray: + return self.embedding_model.encode( + [self.passage_prefix + t for t in texts], normalize_embeddings=True + ) + + def embed_query(self, query: str) -> np.ndarray: + return self.embedding_model.encode( + [self.query_prefix + query], normalize_embeddings=True + ) + # ------------------------------------------------------------------ # FAISS helpers # ------------------------------------------------------------------ @@ -113,7 +214,7 @@ def _init_faiss(self, db_path: str) -> None: self.faiss_db_path = db_path os.makedirs(db_path, exist_ok=True) - self.embedding_dim = 384 # all-MiniLM-L6-v2 output dim + self.embedding_dim = self.embedding_model.get_sentence_embedding_dimension() index_path = os.path.join(db_path, "faiss.index") meta_path = os.path.join(db_path, "faiss_meta.json") @@ -125,6 +226,18 @@ def _init_faiss(self, db_path: str) -> None: self.faiss_documents: List[str] = sidecar["documents"] self.faiss_metadatas: List[Dict] = sidecar["metadatas"] self.faiss_ids: List[str] = sidecar["ids"] + check_embedder_record(sidecar.get("embedder"), self.embedder, f"FAISS folder {db_path!r}") + if self.faiss_index.d != self.embedding_dim: + raise ValueError( + f"The FAISS index in {db_path!r} holds vectors of size {self.faiss_index.d}, " + f"but {self.embedder} gives {self.embedding_dim}." + ) + if not (self.faiss_index.ntotal == len(self.faiss_documents) + == len(self.faiss_metadatas) == len(self.faiss_ids)): + raise ValueError( + f"The FAISS index in {db_path!r} does not match its metadata file " + f"({self.faiss_index.ntotal} vectors, {len(self.faiss_documents)} passages)." + ) print( f"Loaded FAISS index with " f"{self.faiss_index.ntotal} vectors" @@ -148,6 +261,7 @@ def _save_faiss(self) -> None: "documents": self.faiss_documents, "metadatas": self.faiss_metadatas, "ids": self.faiss_ids, + "embedder": embedder_record(self.embedder), }, fh, ) @@ -212,6 +326,7 @@ def get_stats(self) -> Dict: "total_vectors": self.faiss_index.ntotal, "index_type": type(self.faiss_index).__name__, "dimension": self.embedding_dim, + "embedder": self.embedder, } # chroma count = self.collection.count() @@ -219,6 +334,7 @@ def get_stats(self) -> Dict: "backend": "chroma", "total_vectors": count, "collection": self.collection.name, + "embedder": self.embedder, } # ------------------------------------------------------------------ @@ -992,7 +1108,7 @@ def ingest_file(self, file_path: str) -> str: f"{batch_id}_chunk_{i}" for i in range(len(texts)) ] - embeddings = self.embedding_model.encode(texts).tolist() + embeddings = self.embed_passages(texts).tolist() if self.backend == "faiss": self._faiss_add(texts, embeddings, metadatas, ids) @@ -1048,9 +1164,7 @@ def retrieve( Tuple of (documents, metadatas) with confidence. """ initial_k = top_k * 3 - query_emb = self.embedding_model.encode( - [query] - ).tolist() + query_emb = self.embed_query(query).tolist() # --- Stage 1a: Semantic (vector) search --- if self.backend == "faiss": @@ -1315,7 +1429,7 @@ def delete_document(self, source_name: str) -> str: # Rebuild FAISS index self.faiss_index = faiss.IndexFlatL2(self.embedding_dim) if new_docs: - embeddings = self.embedding_model.encode(new_docs).tolist() + embeddings = self.embed_passages(new_docs).tolist() vectors = np.array(embeddings, dtype=np.float32) self.faiss_index.add(vectors) @@ -1383,7 +1497,7 @@ def _score_results( confidence = 1 / (1 + math.exp(-score)) # ChromaDB default metric is L2 (squared). # Convert to cosine similarity: cos_sim = 1 - (L2^2 / 2) - # (valid for normalised embeddings like all-MiniLM-L6-v2) + # (valid for unit-length embeddings; embed_passages/embed_query normalise them) cosine_sim = max(0.0, 1.0 - (dist / 2.0)) euclidean = math.sqrt(max(0.0, dist)) enriched = dict(meta) if meta else {} @@ -1450,7 +1564,7 @@ def analyze_query(self, query: str) -> dict: bm25_label = " + BM25 keyword" if BM25_AVAILABLE else "" strategy = ( f"Hybrid search: semantic embeddings" - f" (all-MiniLM-L6-v2){bm25_label}" + f" ({self.embedder}){bm25_label}" f" → cross-encoder reranking" ) result: dict = { diff --git a/tests/test_embedders.py b/tests/test_embedders.py new file mode 100644 index 0000000..7ce0e3a --- /dev/null +++ b/tests/test_embedders.py @@ -0,0 +1,183 @@ +"""How the engine handles its embedding model, with a small fake model (nothing is downloaded). + +Needs the full requirements (the engine imports its libraries); the FAISS tests also need faiss-cpu. +""" + +import sys +from pathlib import Path + +import numpy as np +import pytest + +pytest.importorskip("chromadb") +pytest.importorskip("sentence_transformers") +sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) + +import rag_engine # noqa: E402 +from rag_engine import EMBEDDERS, RAGEngine, resolve_embedder # noqa: E402 + +BGE, MINILM = "bge-small-en-v1.5", "all-MiniLM-L6-v2" + + +class FakeModel: + """Turns a text into a few numbers that are not unit length; remembers what it was given.""" + + def __init__(self, dim: int = 4): + self.dim, self.seen = dim, [] + + def encode(self, texts, normalize_embeddings=False, **_): + self.seen.extend(texts) + out = np.array([[len(t) % 7 + 1.0, t.count("a") + 1.0, t.count("e") + 2.0, 3.0, 1.0][: self.dim] + for t in texts]) + return out / np.linalg.norm(out, axis=1, keepdims=True) if normalize_embeddings else out + + def get_sentence_embedding_dimension(self): + return self.dim + + +class FakeReranker: + def predict(self, pairs, **_): + return np.zeros(len(pairs)) + + +def engine(tmp_path, embedder=BGE, backend="chroma", model=None, name="docs"): + return RAGEngine(db_path=str(tmp_path), collection_name=name, backend=backend, + embedding_model=model or FakeModel(), reranker=FakeReranker(), embedder=embedder) + + +def test_query_prefix_goes_on_questions_only(tmp_path): + model = FakeModel() + e = engine(tmp_path, model=model) + e.embed_passages(["a passage"]) + e.embed_query("a question") + assert model.seen == ["a passage", EMBEDDERS[BGE]["query_prefix"] + "a question"] + + +def test_vectors_are_unit_length(tmp_path): + e = engine(tmp_path) + assert np.allclose(np.linalg.norm(e.embed_passages(["one", "two words"]), axis=1), 1.0) + assert np.allclose(np.linalg.norm(e.embed_query("three"), axis=1), 1.0) + + +def test_each_model_has_its_own_collection_and_the_old_name_stays(tmp_path): + assert engine(tmp_path, BGE).collection.name == f"docs-{BGE}" + assert engine(tmp_path, MINILM).collection.name == "docs" + + +def test_collection_records_its_model(tmp_path): + meta = engine(tmp_path).collection.metadata + assert meta["embedder"] == BGE and meta["embedder_revision"] == EMBEDDERS[BGE]["revision"] + + +def test_collection_built_with_another_revision_is_refused(tmp_path): + e = engine(tmp_path) + e.collection.modify(metadata={**e.collection.metadata, "embedder_revision": "0" * 40}) + with pytest.raises(ValueError, match="was built with"): + engine(tmp_path) + + +def test_mismatch_is_found_before_any_model_loads(tmp_path, monkeypatch): + e = engine(tmp_path) + e.collection.modify(metadata={**e.collection.metadata, "embedder_revision": "0" * 40}) + + def boom(*_, **__): + raise AssertionError("a model was loaded") + + monkeypatch.setattr(rag_engine, "SentenceTransformer", boom) + monkeypatch.setattr(rag_engine, "CrossEncoder", boom) + with pytest.raises(ValueError, match="was built with"): + RAGEngine(db_path=str(tmp_path), collection_name="docs", embedder=BGE) + + +def test_collection_with_distance_settings_gets_its_record(tmp_path): + import chromadb + + chromadb.PersistentClient(path=str(tmp_path)).get_or_create_collection( + f"docs-{BGE}", metadata={"hnsw:space": "l2"}) + assert engine(tmp_path).collection.metadata["embedder"] == BGE + + +def test_old_collection_without_a_record_is_accepted_and_recorded(tmp_path): + import chromadb + + chromadb.PersistentClient(path=str(tmp_path)).get_or_create_collection("docs").add( + ids=["1"], documents=["old"], embeddings=[[0.5, 0.5, 0.5, 0.5]]) + e = engine(tmp_path, MINILM) + assert e.collection.count() == 1 + assert e.collection.metadata["embedder"] == MINILM + + +def test_unknown_model_is_refused_before_any_model_loads(tmp_path, monkeypatch): + def boom(*_, **__): + raise AssertionError("a model was loaded") + + monkeypatch.setattr(rag_engine, "SentenceTransformer", boom) + monkeypatch.setattr(rag_engine, "CrossEncoder", boom) + with pytest.raises(ValueError, match="Unknown embedder"): + RAGEngine(db_path=str(tmp_path), embedder="no-such-model") + + +def test_model_names_from_the_environment(tmp_path, monkeypatch): + assert resolve_embedder(None) == BGE + assert resolve_embedder(" ") == BGE + assert resolve_embedder("BAAI/bge-small-en-v1.5") == BGE + monkeypatch.setenv("RAG_EMBEDDER", f" {MINILM} ") + e = RAGEngine(db_path=str(tmp_path), embedding_model=FakeModel(), reranker=FakeReranker()) + assert e.embedder == MINILM + + +def test_each_model_is_pinned(): + for spec in EMBEDDERS.values(): + assert len(spec["revision"]) == 40 + + +def test_reranker_gives_raw_scores(monkeypatch): + import torch + + seen = {} + monkeypatch.setattr(rag_engine, "CrossEncoder", lambda *a, **kw: seen.update(kw)) + rag_engine.load_reranker() + assert isinstance(seen["activation_fn"], torch.nn.Identity) + + +# --- FAISS --- + +def faiss_engine(tmp_path, embedder=BGE, dim=4): + pytest.importorskip("faiss") + return engine(tmp_path, embedder, backend="faiss", model=FakeModel(dim)) + + +def add_and_save(e, texts): + e._faiss_add(texts, e.embed_passages(texts).tolist(), [{"source": t} for t in texts], texts) + e._save_faiss() + + +def test_faiss_has_a_folder_per_model_and_the_size_of_the_model(tmp_path): + e = faiss_engine(tmp_path, dim=4) + assert e.faiss_db_path == str(tmp_path / BGE) + assert e.faiss_index.d == 4 + assert faiss_engine(tmp_path, MINILM).faiss_db_path == str(tmp_path) + + +def test_faiss_index_reopens_with_its_passages(tmp_path): + add_and_save(faiss_engine(tmp_path), ["one", "two"]) + e = faiss_engine(tmp_path) + assert e.faiss_index.ntotal == 2 and e.faiss_ids == ["one", "two"] + + +def test_faiss_index_with_another_vector_size_is_refused(tmp_path): + add_and_save(faiss_engine(tmp_path, dim=4), ["one"]) + with pytest.raises(ValueError, match="vectors of size 4"): + faiss_engine(tmp_path, dim=5) + + +def test_faiss_index_built_with_another_revision_is_refused(tmp_path): + import json + + add_and_save(faiss_engine(tmp_path), ["one"]) + meta = tmp_path / BGE / "faiss_meta.json" + data = json.loads(meta.read_text()) + data["embedder"]["embedder_revision"] = "0" * 40 + meta.write_text(json.dumps(data)) + with pytest.raises(ValueError, match="was built with"): + faiss_engine(tmp_path)