Abstract
The rapid growth of ride-hailing platforms has created a highly competitive market where businesses struggle to make profits, demanding the need for better operational strategies. However, real-world experiments are risky and expensive for these platforms as they deal with millions of users daily. Thus, a need arises for a simulated environment where they can predict users’ reactions to changes in the platform-specific parameters such as trip fares and incentives. Building such a simulation is challenging, as these platforms exist within dynamic environments where thousands of users regularly interact with one another. This paper presents a framework to mimic and predict user, specifically driver, behaviors in ride-hailing services. We use a data-driven hybrid reinforcement learning and imitation learning approach for this. First, the agent utilizes behavioral cloning to mimic driver behavior using a real-world data-set. Next, reinforcement learning is applied on top of the pre-trained agents in a simulated environment, to allow them to adapt to changes in the platform. Our framework provides an ideal playground for ride-hailing platforms to experiment with platform-specific parameters to predict drivers’ behavioral patterns.
H. Jayasinghe, T. Jayatilaka and R. Gunawardena—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bailey, W.A., Clark, T.D.: A simulation analysis of demand and fleet size effects on taxicab service rates. In: Proceedings of the 19th Conference on Winter Simulation, pp. 838–844 (1987)
Bailey, W.A., Clark, T.D.: Taxi management and route control: a systems study and simulation experiment. In: Proceedings of the 24th Conference on Winter Simulation, pp. 1217–1222 (1992)
Bellemare, M.G., Dabney, W., Munos, R.: A distributional perspective on reinforcement learning. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 449–458 (2017)
Gao, Y., Jiang, D., Xu, Y.: Optimize taxi driving strategies based on reinforcement learning. Int. J. Geogr. Inf. Sci. 32, 1677–1696 (2018)
Garg, N., Nazerzadeh, H.: Driver surge pricing. arXiv:1905.07544 (2021)
Goecks, V.G., Gremillion, G.M., Lawhern, V.J., Valasek, J., Waytowich, N.R.: Integrating behavior cloning and reinforcement learning for improved performance in dense and sparse reward environments. In: AAMAS, pp. 465–473 (2020)
Hausknecht, M.J., Stone, P.: Deep recurrent Q-learning for partially observable MDPs. In: AAAI Fall Symposia (2015)
Konda, V.R., Tsitsiklis, J.N.: Actor-critic algorithms. In: Solla, S.A., Leen, T.K., Müller, K. (eds.) Advances in Neural Information Processing Systems 12, pp. 1008–1014 (2000)
Lin, K., Zhao, R., Xu, Z., Zhou, J.: Efficient large-scale fleet management via multi-agent deep reinforcement learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1774–1783 (2018)
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, vol. 48, pp. 1928–1937 (2016)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 7540, 529–533 (2015)
Rossi, A., Barlacchi, G., Bianchini, M., Lepri, B.: Modelling taxi drivers’ behaviour for the next destination prediction. IEEE Trans. Intell. Transp. Syst. 21(7), 2980–2989 (2019)
Shou, Z., Di, X., Ye, J., Zhu, H., Zhang, H., Hampshire, R.: Optimal passenger-seeking policies on e-hailing platforms using Markov decision process and imitation learning. Transp. Res. Part C Emerg. Technol. 111, 91–113 (2020)
Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Proceedings of the 12th International Conference on Neural Information Processing Systems, pp. 1057–1063 (1999)
Torabi, F., Warnell, G., Stone, P.: Behavioral cloning from observation. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI) (2018)
Zheng, S., Trott, A., Srinivasa, S., Naik, N., Gruesbeck, M., Parkes, D.C.: The AI economist: improving equality and productivity with AI-driven tax policies (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jayasinghe, H., Jayatilaka, T., Gunawardena, R., Thayasivam, U. (2021). Data-Driven Simulation of Ride-Hailing Services Using Imitation and Reinforcement Learning. In: Fujita, H., Selamat, A., Lin, J.CW., Ali, M. (eds) Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices. IEA/AIE 2021. Lecture Notes in Computer Science(), vol 12798. Springer, Cham. https://doi.org/10.1007/978-3-030-79457-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-79457-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79456-9
Online ISBN: 978-3-030-79457-6
eBook Packages: Computer ScienceComputer Science (R0)