Javed A. Aslam
ABSTRACT
We present a unified framework for simultaneously solving both the pooling problem (the construction of efficient document pools for the evaluation of retrieval systems) and metasearch (the fusion of ranked lists returned by retrieval systems in order to increase performance). The implementation is based on the Hedge algorithm for online learning, which has the advantage of convergence to bounded error rates approaching the performance of the best linear combination of the underlying systems. The choice of a loss function closely related to the average precision measure of system performance ensures that the judged document set performs well, both in constructing a metasearch list and as a pool for the accurate evaluation of retrieval systems. Our experimental results on TREC data demonstrate excellent performance in all measures---evaluation of systems, retrieval of relevant documents, and generation of metasearch lists.
- G. V. Cormack, C. R. Palmer, and C. L. A. Clarke. Efficient construction of large test collections. In Croft et~al. \citesigir98, pages 282--289. Google Scholar
Digital Library
- W. B. Croft, A. Moffat, C. J. van Rijsbergen, R. Wilkinson, and J. Zobel, editors. Proceedings of the 21th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia, Aug. 1998. ACM Press, New York. Google Scholar
- Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119--139, Aug. 1997. Google ScholarDigital Library
- J. Zobel. How reliable are the results of large-scale retrieval experiments? In Croft et al. {2}, pages 307--314. Google ScholarDigital Library
Index Terms
- A unified model for metasearch and the efficient evaluation of retrieval systems via the hedge algorithm
Recommendations
A unified model for metasearch, pooling, and system evaluation
CIKM '03: Proceedings of the twelfth international conference on Information and knowledge managementWe present a unified model which, given the ranked lists of documents returned by multiple retrieval systems in response to a given query, simultaneously solves the problems of (1) fusing the ranked lists of documents in order to obtain a high-quality ...
Building efficient and effective metasearch engines
Frequently a user's information needs are stored in the databases of multiple search engines. It is inconvenient and inefficient for an ordinary user to invoke multiple search engines and identify useful documents from the returned results. To support ...
Rank aggregation using ant colony approach for metasearch
Metasearch engines provide a plethora of information to the user through World Wide Web. They are the prominent sources of query-based search and centralized human---world interactions. Metasearch engine shows a list of Web sites to a particular query ...
Comments