Skip to main content
SpringerLink
Log in

Fairness in Learning-Based Sequential Decision Algorithms: A Survey

  • Chapter
  • First Online:
Handbook of Reinforcement Learning and Control

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 325))

Abstract

Algorithmic fairness in decision-making has been studied extensively in static settings where one-shot decisions are made on tasks such as classification. However, in practice most decision-making processes are of a sequential nature, where decisions made in the past may have an impact on future data. This is particularly the case when decisions affect the individuals or users generating the data used for future decisions. In this survey, we review existing literature on the fairness of data-driven sequential decision-making. We will focus on two types of sequential decisions: (1) past decisions have no impact on the underlying user population and thus no impact on future data; (2) past decisions have an impact on the underlying user population, and therefore the future data, which can then impact future decisions. In each case the impact of various fairness interventions on the underlying population is examined.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Based on the context, this criterion can also refer to equal false negative rate (FNR), false positive rate (FPR), or true negative rate (TNR).

  2. 2.

    Note that such an ideal decision rule assumes the knowledge of y, which is not actually observable. In this sense this decision rule, which has 0 error, is not practically feasible. Our understanding is that the goal in [34] is to analyze what happens in such an ideal scenario when applying the perfect decision.

  3. 3.

    In [32] the assumption that such a perfect decision rule with 0 error is feasible is formally stated as “realizability”.

  4. 4.

    \(\Phi ^t\) is a t-fold composition of \(\Phi \).

  5. 5.

    \(\tau _{TLM}=1\) only ensures a worker’s eligibility to be hired in the PLM (a necessary condition); whether the worker is indeed hired in the PLM is determined by the hiring strategy in the PLM.

References

  1. Agarwal, A., Beygelzimer, A., Dudik, M., Langford, J., Wallach, H.: A reductions approach to fair classification. In: International Conference on Machine Learning, pp. 60–69 (2018)

    Google Scholar 

  2. Aneja, A.P., Avenancio-León, C.F.: No credit for time served? incarceration and credit-driven crime cycles (2019)

    Google Scholar 

  3. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)

    Article  Google Scholar 

  4. Bechavod, Y., Ligett, K., Roth, A., Waggoner, B., Steven, Z.: Equal opportunity in online classification with partial feedback. Adv. Neural Inf. Process. Syst. 32, 8972–8982 (2019)

    Google Scholar 

  5. Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., Roth, A.: A convex framework for fair regression (2017). arXiv:1706.02409

  6. Blum, A., Gunasekar, S., Lykouris,T., Srebro, N.: On preserving non-discrimination when combining expert advice. In: Advances in Neural Information Processing Systems, pp. 8376–8387 (2018)

    Google Scholar 

  7. Bolukbasi, T., Chang, K.-W., Zou, J.Y., Saligrama, V., Kalai, A.T.: Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in Neural Information Processing Systems 29, 4349–4357 (2016)

    Google Scholar 

  8. Calders, T., Žliobaitė, I.: Why unbiased computational processes can lead to discriminative decision procedures. In: Discrimination and Privacy in the Information Society, pp. 43–57. Springer (2013)

    Google Scholar 

  9. Chaney, A.J.B., Stewart, B.M., Engelhardt, B.E.: How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp. 224–232. ACM (2018)

    Google Scholar 

  10. Chen, Y., Cuellar, A., Luo, H., Modi, J., Nemlekar, H., Nikolaidis, S.: Fair contextual multi-armed bandits: Theory and experiments (2019). arXiv:1912.08055, 2019

  11. Dressel, J., Farid, H.: The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4(1), eaao5580 (2018)

    Google Scholar 

  12. Ensign, D., Friedler, S.A., Neville, S., Scheidegger, C., Venkatasubramanian, S.: Runaway feedback loops in predictive policing. In: Conference of Fairness, Accountability, and Transparency (2018)

    Google Scholar 

  13. Fuster, A., Goldsmith-Pinkham, P., Ramadorai, T., Walther, A.: Predictably unequal? the effects of machine learning on credit markets. The Effects of Machine Learning on Credit Markets (2018)

    Google Scholar 

  14. Gillen, S., Jung, C., Kearns, M., Roth, A.: Online learning with an unknown fairness metric. In: Advances in Neural Information Processing Systems, pp. 2600–2609 (2018)

    Google Scholar 

  15. Gordaliza, P., Del Barrio, E., Fabrice, G., Jean-Michel, L.: Obtaining fairness using optimal transport theory. In: International Conference on Machine Learning, pp. 2357–2365 (2019)

    Google Scholar 

  16. Gupta, S., Kamble, V.: Individual fairness in hindsight. In: Proceedings of the 2019 ACM Conference on Economics and Computation, pp. 805–806. ACM (2019)

    Google Scholar 

  17. Hardt, M., Price, E., Srebro, N. et al.: Equality of opportunity in supervised learning. In: Advances in neural information processing systems, pp. 3315–3323 (2016)

    Google Scholar 

  18. Harwell, D.: Amazon’s alexa and google home show accent bias, with Chinese and Spanish hardest to understand (2018). http://bit.ly/2QFA1MR

  19. Hashimoto, T., Srivastava, M., Namkoong, H., Liang, P.: Fairness without demographics in repeated loss minimization. In: Dy, J., Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 80, pp. 1929–1938. PMLR (2018)

    Google Scholar 

  20. Heidari, H., Krause, A.: Preventing disparate treatment in sequential decision making. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 2248–2254 (2018)

    Google Scholar 

  21. Heidari, H., Nanda, V., Gummadi, K.: On the long-term impact of algorithmic decision policies: effort unfairness and feature segregation through social learning. In: International Conference on Machine Learning, pp. 2692–2701 (2019)

    Google Scholar 

  22. Hu, L., Chen, Y.: A short-term intervention for long-term fairness in the labor market. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 1389–1398. International World Wide Web Conferences Steering Committee (2018)

    Google Scholar 

  23. Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Roth, A.: Fairness in reinforcement learning. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1617–1626. JMLR. org, (2017)

    Google Scholar 

  24. Joseph, M., Kearns, M., Morgenstern, J., Neel, S., Roth, A.: Meritocratic fairness for infinite and contextual bandits. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp 158–163. ACM, (2018)

    Google Scholar 

  25. Joseph, M., Kearns, M., Morgenstern, J.H., Roth, A.: Fairness in learning: classic and contextual bandits. In: Advances in Neural Information Processing Systems, pp. 325–333 (2016)

    Google Scholar 

  26. Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33(1), 1–33 (2012)

    Article  Google Scholar 

  27. Kannan, S., Roth, A., Ziani, J.: Downstream effects of affirmative action. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 240–248. ACM (2019)

    Google Scholar 

  28. Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2017)

    Google Scholar 

  29. Lahoti, P., Gummadi, K.P., Weikum, G.: ifair: learning individually fair data representations for algorithmic decision making. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 1334–1345. IEEE, (2019)

    Google Scholar 

  30. Li, F., Liu, J., Ji, B.: Combinatorial sleeping bandits with fairness constraints. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1702–1710. IEEE (2019)

    Google Scholar 

  31. Liu, L.T., Dean, S., Rolf, E., Simchowitz, M., Hardt, M.: Delayed impact of fair machine learning. In: Dy, J., Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 80, pp. 3150–3158. PMLR (2018)

    Google Scholar 

  32. Liu, L.T., Wilson, A., Haghtalab, N., Tauman Kalai, A., Borgs, C., Jennifer Chayes. The disparate equilibria of algorithmic decision making when individuals invest rationally. arXiv preprint arXiv:1910.04123, 2019

  33. Liu, Y., Radanovic, G., Dimitrakakis, C., Mandal, D., Parkes, D.C.: Calibrated fairness in bandits (2017). arXiv:1707.01875

  34. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning (2019). arXiv:1908.09635

  35. Obermeyer, Z., Mullainathan, S.: Dissecting racial bias in an algorithm that guides health decisions for 70 million people. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, p. 89. ACM, 2019

    Google Scholar 

  36. Patil, V., Ghalme, G., Nair, V., Narahari, Y.: Achieving fairness in the stochastic multi-armed bandit problem (2019). arXiv:1907.10516

  37. Shankar, S., Halpern, Y., Breck, E., Atwood, J., Wilson, J., Sculley, D.: No classification without representation: Assessing geodiversity issues in open data sets for the developing world. stat 1050, 22 (2017)

    Google Scholar 

  38. Valera, I., Singla, A., Gomez Rodriguez, M.: Enhancing the accuracy and fairness of human decision making. In: Advances in Neural Information Processing Systems 31, pp. 1774–1783. Curran Associates, Inc. (2018)

    Google Scholar 

  39. Wen, M. Bastani, O., Topcu, U.: Fairness with dynamics (2019). arXiv:1901.08568

  40. Zafar, M.B., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness beyond disparate treatment & disparate impact: lclassification without disparate mistreatment. In: Proceedings of the 26th International Conference on World Wide Web, pp. 1171–1180. International World Wide Web Conferences Steering Committee (2017)

    Google Scholar 

  41. Zafar, M.B., Valera, I., Gomez-Rodriguez, M., Gummadi, K.P.: Fairness constraints: a flexible approach for fair classification. J. Mach. Learn. Res. 20(75), 1–42 (2019)

    MathSciNet  MATH  Google Scholar 

  42. Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. In:International Conference on Machine Learning, pp. 325–333 (2013)

    Google Scholar 

  43. Zhang, X., Mahdi Khalili, M., Liu, M.: Long-term impacts of fair machine learning. Ergonom, Des (2019)

    Book  Google Scholar 

  44. Zhang, X., Mahdi Khalili, M., Tekin, C., Liu, M.: Group retention when using machine learning in sequential decision making: the interplay between user dynamics and fairness. Adv. Neural Inf. Process. Syst. 32, 15243–15252 (2019)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the NSF under grants CNS-1616575, CNS-1646019, CNS-1739517, CNS-2040800, and by the ARO under contract W911NF1810208.

Author information

Authors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Xueru Zhang & Mingyan Liu

Authors
  1. Xueru Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xueru Zhang .

Editor information

Editors and Affiliations

  1. The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA

    Kyriakos G. Vamvoudakis

  2. Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX, USA

    Yan Wan

  3. Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX, USA

    Frank L. Lewis

  4. Army Research Office, Durham, NC, USA

    Derya Cansever

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, X., Liu, M. (2021). Fairness in Learning-Based Sequential Decision Algorithms: A Survey. In: Vamvoudakis, K.G., Wan, Y., Lewis, F.L., Cansever, D. (eds) Handbook of Reinforcement Learning and Control. Studies in Systems, Decision and Control, vol 325. Springer, Cham. https://doi.org/10.1007/978-3-030-60990-0_18

Download citation

Publish with us

Policies and ethics

Search

Navigation