Outlier detection in information retrieval: A systematic review of methods, challenges, and applications

Abstract

Information Retrieval (IR) systems are designed to efficiently retrieve relevant information from large datasets. The presence of outlier’s data points that significantly deviate from normal patterns poses significant challenges in ensuring accuracy, relevance and efficiency in retrieval processes. Outlier detection plays a critical role in improving IR performance by identifying and handling noisy, redundant or irrelevant data that may affect ranking algorithms, query processing, and recommendation systems. This systematic review explores various outlier detection techniques in Information Retrieval, categorizing them into statistical, machine learning-based and deep learning approaches. It examines the effectiveness of these methods in different IR applications, including web search engines, recommender systems and social media analytics. Furthermore, the study highlights key challenges such as high-dimensional data handling, interpretability of models and computational efficiency. The review also discusses recent advancements in hybrid models and anomaly-aware ranking algorithms that aim to mitigate the impact of outliers on retrieval performance. By synthesizing existing literature, this study provides insights into the evolving landscape of outlier detection in IR and identifies future research directions, including the integration of explainable AI, real-time anomaly detection and domain-specific anomaly handling techniques. The findings serve as a foundation for researchers and practitioners seeking to enhance the robustness and reliability of IR systems in the presence of outliers.