Improving RAG Accuracy Through Multi-Retrieval Integration and Rank Base Retrieved Chunk Selection
DOI:
https://doi.org/10.70112/ajeat-2025.14.2.4327Keywords:
Retrieval-Augmented Generation (RAG), Vector Embeddings, Reciprocal Rank Fusion (RRF), Large Language Models (LLMs), Information RetrievalAbstract
In a standard RAG pipeline, source documents are split into smaller chunks, and embedding models generate vector representations for these chunks. The embeddings are stored in a vector database, which retrieves relevant chunks using vector similarity or keyword-based search. The retrieved chunks are then combined with the user query and passed to an LLM to generate the final response. Although effective, traditional RAG systems depend heavily on the choice of embedding model, retrieval method, and the number of retrieved chunks, all of which significantly impact accuracy and hallucination levels. Results show that the proposed RAG system significantly outperforms individual retrieval systems. It achieves a correctness score of 79.75% and a similarity score of 78.7%, surpassing all baseline RAG pipelines. Furthermore, experiments varying the number of retrieved chunks per retriever (from 1 to 10) reveal an interesting trend: performance peaks at several even-numbered retrieval counts, indicating local maxima in correctness and similarity when using even numbers of retrieved documents before applying Reciprocal Rank Fusion (RRF). Overall, this study demonstrates that combining multiple retrieval mechanisms with RRF yields more accurate, contextually aligned, and consistent outputs compared to traditional single-retriever RAG implementations. The proposed framework enhances RAG reliability without fine-tuning and provides empirically validated insights into the impact of retrieval volume on performance. The work repository is publicly maintained at: https://github.com/Surajxyz/RAG_PAPER
References
[1] H. Soudani, E. Kanoulas, and F. Hasibi, “Fine-tuning vs. retrieval-augmented generation for less popular knowledge,” in Proc. 2024 Annu. Int. ACM SIGIR Conf. Res. Dev. Inf. Retrieval Asia Pac. Reg., Dec. 2024, pp. 12–22.
[2] A. Salemi and H. Zamani, “Evaluating retrieval quality in retrieval-augmented generation,” in Proc. 47th Int. ACM SIGIR Conf. Res. Dev. Inf. Retrieval, Jul. 2024, pp. 2395–2400.
[3] H. Yu, A. Gan, K. Zhang, S. Tong, Q. Liu, and Z. Liu, “Evaluation of retrieval-augmented generation: A survey,” in Proc. CCF Conf. Big Data, Singapore, 2024, pp. 102–120.
[4] J. Chen, H. Lin, X. Han, and L. Sun, “Benchmarking large language models in retrieval-augmented generation,” in Proc. AAAI Conf. Artif. Intell., vol. 38, no. 16, Mar. 2024, pp. 17754–17762.
[5] Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, et al., “Active retrieval-augmented generation,” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), Dec. 2023, pp. 7969–7992.
[6] H. Li, Y. Su, D. Cai, Y. Wang, and L. Liu, “A survey on retrieval-augmented text generation,” arXiv preprint, arXiv:2202.01110, 2022.
[7] Z. Rackauckas, “RAG-Fusion: A new take on retrieval-augmented generation,” arXiv preprint, arXiv:2402.03367, 2024.
[8] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Trans. Assoc. Comput. Linguist., vol. 12, pp. 157–173, 2024.
[9] B. Peng, Y. Zhu, Y. Liu, X. Bo, H. Shi, C. Hong, et al., “Graph retrieval-augmented generation: A survey,” ACM Trans. Inf. Syst., 2024.
[10] H. Han, Y. Wang, H. Shomer, K. Guo, J. Ding, Y. Lei, et al., “Retrieval-augmented generation with graphs (GraphRAG),” arXiv preprint, arXiv:2501.00309, 2024.
[11] R. T. de Lima, S. Gupta, C. B. Ramis, L. Mishra, M. Dolfi, P. Staar, and P. Vagenas, “Know your RAG: Dataset taxonomy and generation strategies for evaluating RAG systems,” in Proc. 31st Int. Conf. Comput. Linguistics: Ind. Track, Jan. 2025, pp. 39–57.
[12] Z. Wang, J. Araki, Z. Jiang, M. R. Parvez, and G. Neubig, “Learning to filter context for retrieval-augmented generation,” arXiv preprint, arXiv:2311.08377, 2023.
[13] Z. Jiang, X. Ma, and W. Chen, “LongRAG: Enhancing retrieval-augmented generation with long-context LLMs,” arXiv preprint, arXiv:2406.15319, 2024.
[14] Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang, “LightRAG: Simple and fast retrieval-augmented generation,” arXiv preprint, arXiv:2410.05779, 2024.
[15] X. Wang, Z. Wang, X. Gao, F. Zhang, Y. Wu, Z. Xu, et al., “Searching for best practices in retrieval-augmented generation,” in Proc. Conf. Empir. Methods Nat. Lang. Process. (EMNLP), Nov. 2024, pp. 17716–17736.
[16] C. M. Chan, C. Xu, R. Yuan, H. Luo, W. Xue, Y. Guo, and J. Fu, “RQ-RAG: Learning to refine queries for retrieval-augmented generation,” arXiv preprint, arXiv:2404.00610, 2024.
[17] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Adv. Neural Inf. Process. Syst., vol. 33, 2020, pp. 9459–9474.
[18] Y. Yu, W. Ping, Z. Liu, B. Wang, J. You, C. Zhang, et al., “RankRAG: Unifying context ranking with retrieval-augmented generation in LLMs,” in Adv. Neural Inf. Process. Syst., vol. 37, 2024, pp. 121156–121184.
[19] X. Cheng, X. Wang, X. Zhang, T. Ge, S. Q. Chen, F. Wei, et al., “XRAG: Extreme context compression for retrieval-augmented generation with one token,” in Adv. Neural Inf. Process. Syst., vol. 37, 2024, pp. 109487–109516.
[20] K. Zhu, Y. Luo, D. Xu, Y. Yan, Z. Liu, S. Yu, et al., “RAGEval: Scenario-specific RAG evaluation dataset generation framework,” in Proc. 63rd Annu. Meet. Assoc. Comput. Linguistics (ACL), vol. 1, Jul. 2025, pp. 8520–8544.
[21] S. Es, J. James, L. E. Anke, and S. Schockaert, “RAGAS: Automated evaluation of retrieval-augmented generation,” in Proc. 18th Conf. Eur. Chapter Assoc. Comput. Linguistics: Syst. Demonstrations, Mar. 2024, pp. 150–158.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Centre for Research and Innovation
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
