Skip to main content
SpringerLink
Log in

Supporting customer-oriented marketing with artificial intelligence: automatically quantifying customer needs from social media

  • Research Paper
  • Published:
Electronic Markets Aims and scope Submit manuscript

Abstract

The elicitation and monitoring of customer needs is an important task for businesses, allowing them to design customer-centric products and services and control marketing activities. While there are different approaches available, most lack in automation, scalability and monitoring capabilities. In this work, we demonstrate the feasibility towards an automated prioritization and quantification of customer needs from social media data. To do so, we apply a supervised machine learning approach on the example of previously labeled Twitter data from the domain of e-mobility. We descriptively code over 1000 German tweets and build eight distinct classification models, so that a resulting artifact can independently determine the probabilities of a tweet containing each of the eight previously defined needs. To increase the scope of application, we deploy the machine learning models as part of a web service for public use. The resulting artifact can provide valuable insights for need elicitation and monitoring when analyzing user-generated content on a large scale.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. e-tankstelle, eauto, elektroauto, elektrofahrzeug, elektromobilitaet, elektromobilität, ladesaeule, ladesäule

  2. ecar, electric mobility, EV vehicle, e-mobility, emobility

  3. bmw i3, egolf, eup, fortwo electric drive, miev, nissan leaf, opel ampera, peugeot ion, renault zoe, tesla model s

  4. In case one tweet contains multiple needs that belong to the same need category, only one instance is used in the implementation. Therefore, the effective frequencies of tweets containing need categories is slightly different than the total amounts shown in Table 3.

References

  • Andrews, N. O., & Fox, E. A. (2007). Recent developments in document clustering.

    Google Scholar 

  • Balakrishnan, V., & Lloyd-Yemoh, E. (2014). Stemming and lemmatization: A comparison of retrieval performances. Lecture Notes on Software Engineering, 2(3), 262.

    Google Scholar 

  • Barberá, P., & Rivero, G. (2015). Understanding the political representativeness of twitter users. Social Science Computer Review, 33(6), 712–729.

    Google Scholar 

  • Bergstra, J. S., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Advances in neural information processing systems (pp. 2546–2554).

    Google Scholar 

  • Bermingham, A., & Smeaton, A. F. (2011). On using twitter to monitor political sentiment and predict election results.

    Google Scholar 

  • Bifet, A., & Frank, E. (2010). Sentiment knowledge discovery in twitter streaming data. In B. Pfahringer, G. Holmes, & A. Hoffmann (Eds.), Discovery science (Vol. 6332, pp. 1–15). Berlin Heidelberg: Springer.

    Google Scholar 

  • Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: Analyzing text with the natural language toolkit. “ O’Reilly Media, Inc.”

  • Blindheim, J., Wulvik, A., & Steinert, M. (2016). Using secondary video material for user observation in the needfinding process for new product development and design. In Proceedings of the DESIGN 2016 14th International Design Conference.

  • Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145–1159.

    Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Google Scholar 

  • Cavnar, W. B., & Trenkle, J. M. (1994). N-gram-based text categorization. Ann Arbor MI, 48113(2), 161–175.

    Google Scholar 

  • Cawley, G. C., & Talbot, N. L. C. (2010). On over-fitting in model selection and subsequent selection Bias in performance evaluation. Journal of Machine Learning Research, 11, 2079–2107 Retrieved from http://jmlr.csail.mit.edu/papers/v11/cawley10a.html%5Cn, http://www.jmlr.org/papers/volume11/cawley10a/cawley10a.pdf. Accessed 2019-02-01.

    Google Scholar 

  • Chawla, N. V. (2005). Data mining for imbalanced datasets: An overview. In Data mining and knowledge discovery handbook (pp. 853–867). Springer.

  • Chen, R., & Lazer, M. (2013). Sentiment analysis of twitter feeds for the prediction of stock market movement. Stanford. Edu. Retrieved January, 25, 2013.

  • Chen, H., & Zimbra, D. (2010). AI and opinion mining. IEEE Intelligent Systems, 25(3), 74–80.

    Google Scholar 

  • Christopher, M., Payne, A., & Ballantyne, D. (1991). Relationship marketing: Bringing quality customer service and marketing together.

    Google Scholar 

  • Cieliebak, M., Deriu, J., Egger, D., & Uzdilli, F. (2017). A twitter Corpus and benchmark resources for German sentiment analysis. Fifth International Workshop on Natural Language Processing for Social Media, (April), 45–51. https://doi.org/10.18653/v1/W17-1106

  • Correa, T., Hinsley, A. W., & de Zúñiga, H. G. (2010). Who interacts on the web?: The intersection of users’ personality and social media use. Computers in Human Behavior, 26(2), 247–253. https://doi.org/10.1016/j.chb.2009.09.003 .

    Article  Google Scholar 

  • Cuthbertson, R., Furseth, P. I., & Ezell, S. J. (2015). Innovating in a service-driven economy: Insights, application, and practice. Palgrave Macmillan UK Retrieved from https://books.google.de/books?id=kXWkCgAAQBAJ. Accessed 2019-02-01.

  • Edvardsson, B., Kristensson, P., Magnusson, P., & Sundström, E. (2012). Customer integration within service development - a review of methods and an analysis of insitu and exsitu contributions. Technovation, 32, 419–429. https://doi.org/10.1016/j.technovation.2011.04.006 .

    Article  Google Scholar 

  • Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (Vol. 96, pp. 226–231).

    Google Scholar 

  • Finin, T., Murnane, W., Karandikar, A., Keller, N., Martineau, J., & Dredze, M. (2010). Annotating named entities in twitter data with crowdsourcing. Proceedings of the NAACL HLT 2010 workshop on creating speech and language data with Amazon’s mechanical Turk.

  • Garimella, K., Weber, I., & De Choudhury, M. (2016). Quote rts on twitter: Usage of the new feature for political discourse. In Proceedings of the 8th ACM conference on web science (pp. 200–204).

  • Gerber, M. S. (2014). Predicting crime using twitter and kernel density estimation. Decision Support Systems, 61, 115–125.

    Google Scholar 

  • Gollapudi, S. (2016). Practical machine learning. Packt Publishing. Retrieved from https://books.google.de/books?id=WmsdDAAAQBAJ. Accessed 2019-02-01.

  • Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for Maximim impact. MIS Quarterly, 37(2), 337–355. https://doi.org/10.2753/MIS0742-1222240302 .

    Article  Google Scholar 

  • Grinberg, M. (2018). Flask web development: Developing web applications with python. O’Reilly Media, Inc.

  • Haas, D., Wang, J., Wu, E., & Franklin, M. J. (2015). Clamshell: Speeding up crowds for low-latency data labeling. Proceedings of the VLDB Endowment, 9(4), 372–383.

    Google Scholar 

  • Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating ( ROC ) Curvel characteristic. Radiology, 143(1), 29–36. https://doi.org/10.1148/radiology.143.1.7063747 .

    Article  Google Scholar 

  • Harding, J. A., Popplewell, K., Fung, R. Y. K., & Omar, A. R. (2001). Intelligent information framework relating customer requirements and product characteristics. Computers in Industry, 44(1), 51–65. https://doi.org/10.1016/S0166-3615(00)00074-9 .

    Article  Google Scholar 

  • Hauser, J. R., & Griffin, A. (1993). The voice of the customer. Marketing Science, 12, 1–27.

    Google Scholar 

  • Hirt, R., Kühl, N., & Satzger, G. (2017). An end-to-end process model for supervised machine learning classification: from problem to deployment in information systems. In Designing the Digital Transformation: DESRIST 2017 Research in Progress Proceedings of the 12th International Conference on Design Science Research in Information Systems and Technology. Karlsruhe, Germany. 30 May-1 Jun.

  • Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support vector classification.

  • Hu, M., & Liu, B. (2004a). Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining (pp. 168–177).

  • Hu, M., & Liu, B. (2004b). Mining opinion features in customer reviews. 19th National Conference on Artifical intelligence, 755–760. https://doi.org/10.1145/1014052.1014073

  • Hu, M., & Liu, B. (2006). Opinion feature extraction using class sequential rules. In Proceedings of the AAAI spring symposium: Computational approaches to analyzing weblogs (pp. 61–66).

    Google Scholar 

  • Hull, E., Jackson, K., & Dick, J. (2010). Requirements engineering. Springer Science & Business Media.

  • James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning. New York: Springer. Retrieved from https://link.springer.com/content/pdf/10.1007%2F978-1-4614-7138-7.pdf. Accessed 2019-02-01.

  • Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260.

    Google Scholar 

  • Jovic, A., Brkic, K., & Bogunovic, N. (2014). An overview of free software tools for general data mining. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2014 37th International Convention on (pp. 1112–1117).

  • Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.3115/v1/P14-1062

  • Kaplan, A. M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 53(1), 59–68. https://doi.org/10.1016/j.bushor.2009.09.003 .

    Article  Google Scholar 

  • Khalid, H. M., & Helander, M. G. (2006). Customer emotional needs in product design. Concurrent Engineering, 14(3), 197–206.

    Google Scholar 

  • Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media? Get serious! Understanding the functional building blocks of social media. Business Horizons, 54(3), 241–251.

    Google Scholar 

  • Kotler, P., & Armstrong, G. (2001). Principles of marketing. World wide web internet and web. Information Systems, 42, 105. https://doi.org/10.2307/1250103 .

    Article  Google Scholar 

  • Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31, 249–268. https://doi.org/10.1115/1.1559160 .

    Article  Google Scholar 

  • Kuechler, W., & Vaishnavi, V. (2012). A framework for theory development in design science research: Multiple perspectives. Journal of the Association for Information Systems. https://doi.org/10.1201/b18448-6 .

  • Kuehl, N., Scheurenbrand, J., & Satzger, G. (2016). “Needmining: Identifying micro blog data containing customer needs”. Research Papers. 185. https://aisel.aisnet.org/ecis2016_rp/185.

  • Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. In Applied predictive modeling. https://doi.org/10.1007/978-1-4614-6849-3 .

    Chapter  Google Scholar 

  • Kühl, N., Goutier, M., Ensslen, A., & Jochem, P. (2018). Literature vs. Twitter: Empirical insights on customer needs in e-mobility. Journal of Cleaner Production. https://doi.org/10.1016/j.jclepro.2018.12.003.

  • Kvaløy, O., Nieken, P., & Schöttner, A. (2015). Hidden benefits of reward: A field experiment on motivation and monetary incentives. European Economic Review, 76, 188–199.

    Google Scholar 

  • Lee, D., Jeong, O.-R., & Lee, S.-G. (2008). Opinion mining of customer feedback data on the web. Proceedings of the 2nd international conference on ubiquitous information management and communication ICUIMC 08, 230. https://doi.org/10.1145/1352793.1352842

  • Lemaître, G., Nogueira, F., & Aridas, C. K. (2017). Imbalanced-learn: A Python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17), 1–5.

    Google Scholar 

  • Limehouse, D. (1999). Know your customer. Work Study, 48(3), 100–102. https://doi.org/10.1108/00438029910262518 .

    Article  Google Scholar 

  • Ling, C. X., Huang, J., & Zhang, H. (2003). AUC: A statistically consistent and more discriminating measure than accuracy. In IJCAI international joint conference on artificial intelligence (pp. 519–524).

  • Loria, S. (2017). TextBlob.

  • March, S. T., & Smith, G. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266 Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=epref&AN=DSS.AE.BEA.MARCH.DNSRIT&site=ehost-live. Accessed 2019-02-01.

    Google Scholar 

  • Marshall, T. C., Lefringhausen, K., & Ferenczi, N. (2015). The big five, self-esteem, and narcissism as predictors of the topics people write about in Facebook status updates. Personality and Individual Differences, 85, 35–40. https://doi.org/10.1016/j.paid.2015.04.039 .

    Article  Google Scholar 

  • Maynard, D., Bontcheva, K., & Rout, D. (2012). Challenges in developing opinion mining tools for social media. Proceedings of the@ NLP can u tag# Usergeneratedcontent, 15–22.

  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013a). Efficient estimation of word representations in vector space, 1–12.

  • Mikolov, T., Yih, W., & Zweig, G. (2013b). Linguistic regularities in continuous space word representations. In Proceedings of the annual conference of the north American chapter of the Association for Computational Linguistics: Human language technologies (HLT-NAACL) (Vol. 13, pp. 746–751).

  • Misopoulos, F., Mitic, M., Kapoulas, A., & Karapiperis, C. (2014). Uncovering customer service experiences with twitter: The case of airline industry. Management Decision, 52(4), 705–723.

    Google Scholar 

  • Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of machine learning. MIT Press.

  • Montgomery, D. C. (2013). Design and Analysis of Experiments (Eighth Edi). Hoboken: Wiley.

    Google Scholar 

  • Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 553–570). Springer.

  • Nair, S., Rao, N., Mishra, S., & Patil, A. (2017). India’s charging infrastructure — Biggest single point impediment in EV adaptation in India. In 2017 IEEE transportation electrification conference (ITEC-India) (pp. 1–6). https://doi.org/10.1109/ITEC-India.2017.8333884

  • Oke, A. (2007). Innovation types and innovation management practices in service companies. International Journal of Operations & Production Management, 27(6), 564–587.

    Google Scholar 

  • Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on empirical methods in natural language processing-Volume 10 (pp. 79–86).

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in {P}ython. Journal of Machine Learning Research, 12, 2825–2830.

    Google Scholar 

  • Peffers, K., Rothenberger, M., Tuunanen, T., & Vaezi, R. (2012). Design science research evaluation. Design Science Research in Information Systems. Advances in Theory and Practice, 398–410. https://doi.org/10.1007/978-3-642-29863-9_29 .

  • Perrin, A. (2015). Social Media Usage: 2005–2015: 65% of Adults Now Use Social Networking Sites–a Nearly Tenfold Jump in the Past Decade. Pew Research Center, (October), 2005–2015.

  • Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10(3), 61–74.

    Google Scholar 

  • Porter, M. F. (2001). Snowball: A language for stemming algorithms.

  • Powers, D. M. (2011). Evaluation: From precision, recall and F-measure to ROC, Informedness, Markedness and correlation. Journal of Machine Learning Technologies, 2(1), 37–63.

    Google Scholar 

  • Saldaña, J. (2015). The coding manual for qualitative researchers. The coding manual for qualitative researchers. https://doi.org/10.1017/CBO9781107415324.004

  • Scheffler, T., Gontrum, J., Wegel, M., & Wendler, S. (2015). Mapping German tweets to geographic regions. Working Paper - University of Potsdam.

  • Scheurenbrand, J., Engel, C., Peters, F., & Kühl, N. (2015). Holistically defining E-mobility: A modern approach to systematic literature reviews. Karlsruhe Service Summit, 17–27. https://doi.org/10.5445/KSP/1000045634 .

  • Schlagwein, D., Fischbach, K., & Schoder, D. (2011). Social information systems: Review, framework, and research agenda. International Conference on Information Systems. https://doi.org/10.12980/APJTB.4.2014C1020 .

  • Sheldon, K. M., Elliot, A. J., Kim, Y., & Kasser, T. (2001). What is satisfying about satisfying events? Testing 10 candidate psychological needs. Journal of Personality and Social Psychology, 80(2), 325–339.

    Google Scholar 

  • Silva, C., & Ribeiro, B. (2003). The importance of stop word removal on recall values in text categorization. In Neural networks, 2003. Proceedings of the international joint conference on (Vol. 3, pp. 1661–1666).

    Google Scholar 

  • Singhal, A. (2001). Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24(4), 35–43.

    Google Scholar 

  • Srivastava, A. N., & Sahami, M. (2009). Text mining: Classification, clustering, and applications. CRC Press.

  • Srivastava, R. K., Shervani, T. A., & Fahey, L. (1999). Marketing, business processes, and shareholder value: An organizationally embedded view of marketing activities and the discipline of marketing. The Journal of Marketing, 168–179.

  • St Louis, C., & Zorlu, G. (2012). Can twitter predict disease outbreaks. BMJ, 344, e2353.

    Google Scholar 

  • Steinwart, I., & Christmann, A. (2008). Support vector machines. Springer Science & Business Media.

  • Stieglitz, S., Dang-Xuan, L., Bruns, A., & Neuberger, C. (2014). Social media analytics. Business & Information Systems Engineering, 6(2), 89–96.

    Google Scholar 

  • Subramaniam, L. V., Faruquie, T. A., Ikbal, S., Godbole, S., & Mohania, M. K. (2009). Business intelligence from voice of customer. In IEEE international conference on data engineering (pp. 1391–1402).

  • Timoshenko, A., & Hauser, J. R. (2018). Identifying customer needs from user-generated content.

  • Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010). Predicting elections with twitter: What 140 characters reveal about political sentiment. Proceedings of the fourth international AAAI conference on weblogs and social media, 178–185. https://doi.org/10.1074/jbc.M501708200 , 280

  • Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 417–424).

  • Twitter (2016). About Twitter. Retrieved August 2, 2016, from https://about.twitter.com/de/company/press/milestones, last accessed 2016-08-02.

  • Twitter (2018). Twitter Streaming API. Retrieved March 22, 2018, from https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/connecting. Accessed 2019-02-01.

  • Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7(1), 91.

    Google Scholar 

  • Webb, G. I., Pazzani, M. J., & Billsus, D. (2001). Machine learning for user modeling. User Modeling and User-Adapted Interaction, 11(1), 19–29.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology (KIT), Karlsruhe Service Research Institute (KSRI), Kaiserstr. 89, 76133, Karlsruhe, Germany

    Niklas Kühl, Marius Mühlthaler & Marc Goutier

Authors
  1. Niklas Kühl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marius Mühlthaler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marc Goutier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niklas Kühl.

Additional information

Responsible Editors: Rainer Schmidt, Rainer Alt and Selmin Nurcan

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Social Information Systems Minitrack - HICCS 51

Appendix

Appendix

Fig. 5
figure 5

Example of a JSON response from the deployed REST API

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kühl, N., Mühlthaler, M. & Goutier, M. Supporting customer-oriented marketing with artificial intelligence: automatically quantifying customer needs from social media. Electron Markets 30, 351–367 (2020). https://doi.org/10.1007/s12525-019-00351-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12525-019-00351-0

Keywords

JEL classification

Search

Navigation