Skip to main content
SpringerLink
Log in

Theoretical and empirical evaluation of data reduction for exact Kemeny Rank Aggregation

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Kemeny Rank Aggregation is a consensus finding problem important in many areas ranging from classical voting over web search and databases to bioinformatics. The underlying decision problem Kemeny Score is NP-complete even in case of four input rankings to be aggregated into a “median ranking”. We analyze efficient polynomial-time data reduction rules with provable performance bounds that allow us to find even all optimal median rankings. We show that our reduced instances contain at most candidates where \(d_a\) denotes the average Kendall’s tau distance between the input votes. On the theoretical side, this improves a corresponding result for a “partial problem kernel” from quadratic to linear size. In this context we provide a theoretical analysis of a commonly used data reduction. On the practical side, we provide experimental results with data based on web search and sport competitions, e.g., computing optimal median rankings for real-world instances with more than 100 candidates within milliseconds. Moreover, we perform experiments with randomly generated data based on two random distribution models for permutations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Let \(r=x_1>x_2>\dots >x_m\). Then, \(\pi (r)\) is the ranking \(\pi (x_1)>\pi (x_2)>\dots >\pi (x_m)\).

  2. Obtained from the German server http://www.sportschau.de/sp/wintersport/.

  3. http://en.wikipedia.org/wiki/List_of_national_capitals, http://en.wikipedia.org/w/index.php?title=Forbes_list_of_billionaires_%282008%29&oldid=335542552

  4. http://www.boost.org/

  5. http://www.gnu.org/software/glpk/

  6. http://www.ibm.com/software/integration/optimization/cplex-optimizer/

  7. http://www.gurobi.com/

References

  1. Ailon, N., Charikar, M., & Newman, A. (2008). Aggregating inconsistent information: ranking and clustering. Journal of the ACM, 55(5), 1–27.

    Article  MathSciNet  Google Scholar 

  2. Ali, A., & Meilă, M. (2012). Experiments with Kemeny ranking: What works when? Mathematical Social Sciences, 64(1), 28–40.

    Article  MATH  MathSciNet  Google Scholar 

  3. Bartholdi, J, I. I. I., Tovey, C. A., & Trick, M. A. (1989). Voting schemes for which it can be difficult to tell who won the election. Social Choice and Welfare, 6, 157–165.

    Article  MATH  MathSciNet  Google Scholar 

  4. Betzler, N. (2010). A multivariate complexity analysis of voting problems. PhD thesis, Friedrich-Schiller-Universität Jena.

  5. Betzler, N., Bredereck, R., Chen, J., & Niedermeier, R. (2012). Studies in computational aspects of voting—a parameterized complexity perspective. In The multivariate algorithmic revolution and beyond. LNCS (Vol. 7370, pp. 318–363). New York: Springer.

  6. Betzler, N., Fellows, M. R., Guo, J., Niedermeier, R., & Rosamond, F. A. (2009). Fixed-parameter algorithms for Kemeny rankings. Theoretical Computer Science, 410(45), 4554–4570.

    Article  MATH  MathSciNet  Google Scholar 

  7. Betzler, N., Guo, J., Komusiewicz, C., & Niedermeier, R. (2011). Average parameterization and partial kernelization for computing medians. Journal of Computer and System Sciences, 77(4), 774–789.

    Article  MATH  MathSciNet  Google Scholar 

  8. Biedl, T., Brandenburg, F. J., & Deng, X. (2009). On the complexity of crossings in permutations. Discrete Mathematics, 309(7), 1813–1823.

    Article  MATH  MathSciNet  Google Scholar 

  9. Brandenburg, F., Gleißner, A., & Hofmeier, A. (2012). Comparing and aggregating partial orders with Kendall tau distances. In Proceedings of the 6th international workshop on algorithms and computation (WALCOM ’12). LNCS (Vol. 7157, pp. 88–99). New York: Springer.

  10. Brandt, F., Brill, M., & Seedig, H. G. (2011). On the fixed-parameter tractability of composition-consistent tournament solutions. In Proceedings of the 22nd international joint conference on artificial intelligence (IJCAI ’11) (pp. 85–90). Menlo Park, CA: AAAI Press.

  11. Bredereck, R., (2010). Fixed-parameter algorithms for computing Kemeny scores—Theory and practice. Studienarbeit, Universität Jena, CoRR abs/1001.4003.

  12. Cohen, W. W., Schapire, R. E., & Singer, Y. (1999). Learning to order things. Journal of Artificial Intelligence Research, 10(1), 243–270.

    MATH  MathSciNet  Google Scholar 

  13. Conitzer, V., Davenport, A., & Kalagnanam, J. (2006). Improved bounds for computing Kemeny rankings. In Proceedings of the 21st national conference on artificial intelligence (AAAI’06) (pp. 620–626). Menlo Park, CA: AAAI Press.

  14. Davenport, A., & Kalagnanam, J. (2004). A computational study of the Kemeny rule for preference aggregation. In Proceedings of the 19th national conference on artificial intelligence (AAAI’04) (pp. 697–702). Menlo Park, CA: AAAI Press.

  15. Downey, R. G., & Fellows, M. R. (1999). Parameterized complexity. New York: Springer.

    Book  Google Scholar 

  16. Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the Web. In Proceedings of the 10th international World Wide Web conference (WWW’01) (pp. 613–622).

  17. Fagin, R., Kumar, R., & Sivakumar, D. (2003). Efficient similarity search and classification via rank aggregation. In Proceedings of the 22nd ACM SIGMOD international conference on management of data (SIGMOD’03) (pp. 301–312). New York: ACM.

  18. Fernau, H., Fomin, F. V., Lokshtanov, D., Mnich, M., Philip, G., & Saurabh, S. (2011). Ranking and drawing in subexponential time. In Proceedings of the 21st international conference on combinatorial algorithms. LNCS (Vol. 6460, pp. 337–348). New York: Springer.

  19. Fligner, M. A., & Verducci, J. S. (1986). Distance based ranking models. Journal of the Royal Statistical Society, 48(3), 359–369.

    MATH  MathSciNet  Google Scholar 

  20. Flum, J., & Grohe, M. (2006). Parameterized complexity theory. Berlin: Springer.

    Google Scholar 

  21. Hemaspaandra, E., Spakowski, H., & Vogel, J. (2005). The complexity of Kemeny elections. Theoretical Computer Science, 349(3), 382–391.

    Article  MATH  MathSciNet  Google Scholar 

  22. Jackson, B. N., Schnable, P. S., & Aluru, S. (2008). Consensus genetic maps as median orders from inconsistent sources. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 5(2), 161–171.

    Article  Google Scholar 

  23. Karpinski, M., & Schudy, W. (2010). Faster algorithms for feedback arc set tournament, Kemeny rank aggregation and betweenness tournament. In Proceedings of the 21st international symposium on algorithms and computation (ISAAC’10). LNCS (Vol. 6506, pp. 3–14). Berlin: Springer.

  24. Kenyon-Mathieu, C., & Schudy, W. (2007). How to rank with few errors. In Proceedings of the 39th annual ACM symposium on theory of computing (STOC’07) (pp. 95–103). New York: ACM.

  25. Luce, R. D. (2005). Individual choice behavior: A theoretical analysis. New York: Dover Publications.

    Book  Google Scholar 

  26. Mahajan, M., Raman, V., & Sikdar, S. (2009). Parameterizing above or below guaranteed values. Journal of Computer and System Sciences, 75, 137–153.

    Article  MATH  MathSciNet  Google Scholar 

  27. Mallows, C. (1957). Non-null ranking models. I. Biometrika, 44(1/2), 114–130.

    Article  MATH  MathSciNet  Google Scholar 

  28. Mandhani, B., & Meilǎ, M. (2009). Tractable search for learning exponential models of rankings. Journal of Machine Learning Research—Proceedings Track, 5, 392–399.

    Google Scholar 

  29. Meilǎ, M., & Bao, L. (2010). An exponential model for infinite rankings. Journal of Machine Learning Research, 11, 3481–3518.

    Google Scholar 

  30. Meilǎ, M., Phadnis, K., Patterson, A., & Bilmes, J. (2007). Consensus ranking under the exponential model. In Proceedings of the 23rd Conference on Uncertainty in artificial intelligence (UAI’07) (pp. 285–294).

  31. Niedermeier, R. (2006). Invitation to fixed-parameter algorithms. Oxford: Oxford University Press.

    Book  MATH  Google Scholar 

  32. Nishimura, N., & Simjour, N. (2013). Parameterized enumeration of (locally-) optimal aggregations. In 13th International symposium on algorithms and data structures (WADS’13). LNCS (Vol. 8037, pp. 512–523). New York: Springer.

  33. Placket, R. L. (1975). The analysis of permutations. Applied Statistics, 24, 193–202.

    Article  Google Scholar 

  34. Raman, V., & Saurabh, S. (2007). Improved fixed parameter tractable algorithms for two “edge” problems: MAXCUT and MAXDAG. Information Processing Letters, 104(2), 65–72.

    Article  MATH  MathSciNet  Google Scholar 

  35. Schalekamp, F., & van Zuylen, A. (2009). Rank aggregation: Together we’re strong. In Proceedings of the 11th workshop on algorithm engineering and experiments (ALENEX’09) (pp. 38–51). Philadelphia: SIAM.

  36. Simjour, N. (2009). Improved parameterized algorithms for the Kemeny aggregation problem. Proceedings of the 4th international workshop on parameterized and exact computation (IWPEC’09). LNCS (Vol. 5917, pp. 312–323). New York: Springer.

  37. Simjour, N. (2013). Parameterized enumeration of neighbour strings and Kemeny aggregations. PhD thesis, University of Waterloo.

  38. Truchon, M. (1998). An extension of the Condorcet criterion and Kemeny orders. Technical report, cahier 98–15 du Centre de Recherche en Économie et Finance Appliquées, Université Laval, Québec, Candada.

  39. Young, H. P. (1995). Optimal voting rules. Journal of Economic Perspectives, 9(1), 51–64.

    Article  Google Scholar 

  40. Young, H. P., & Levenglick, A. (1978). A consistent extension of Condorcet’s election principle. SIAM Journal on Applied Mathematics, 35(2), 285–300.

    Article  MATH  MathSciNet  Google Scholar 

  41. van Zuylen, A., & Williamson, D. P. (2009). Deterministic pivoting algorithms for constrained ranking and clustering problems. Mathematics of Operations Research, 34(3), 594–620.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We are grateful to the anonymous referees of the Fifth International Symposium on Parameterized and Exact Computation (IPEC-2010) and of the Third International Workshop on Computational Social Choice (COMSOC-2010) for constructive feedback helping to improve this work. We are indebted to three anonymous referees of JAAMAS for providing numerous insightful remarks that helped to significantly improve the paper. In particular, the more efficient and effective data reduction rule exploiting the extended Condorcet property helped to improve our theoretical and practical results. We thank Christian Komusiewicz for pointing us to an improved (compared to the conference version) analysis for the bound of Theorem 1, and our student research assistant Leila Arras for her great support in doing implementations and experiments with synthetic data. Nadja Betzler and Robert Bredereck were supported by the DFG, research project PAWS, NI 369/10.

Author information

Authors and Affiliations

  1. Institut für Softwaretechnik und Theoretische Informatik, TU Berlin, Berlin, Germany

    Nadja Betzler, Robert Bredereck & Rolf Niedermeier

Authors
  1. Nadja Betzler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Bredereck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rolf Niedermeier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Bredereck.

Additional information

A preliminary version of this work appeared under the title “Partial Kernelization for Rank Aggregation: Theory and Experiments” in Proceedings of the 5th International Symposium on Parameterized and Exact Computation (IPEC-2010), Chennai, India, December 2010, volume 6478 in Lecture Notes in Computer Science, pp. 26–37, Springer. This journal version expands on the conference version by siginificantly revising the theoretical part, extending the range of test data, and by using state-of-the-art ILP-solvers. This work was started while all authors were with Friedrich-Schiller-Universität Jena, Germany.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 382 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Betzler, N., Bredereck, R. & Niedermeier, R. Theoretical and empirical evaluation of data reduction for exact Kemeny Rank Aggregation. Auton Agent Multi-Agent Syst 28, 721–748 (2014). https://doi.org/10.1007/s10458-013-9236-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-013-9236-y

Keywords

Search

Navigation