Abstract
We present an online attribute inference attack by leveraging Facebook picture metadata (i) alt-text generated by Facebook to describe picture contents, and (ii) comments containing words and emojis posted by other Facebook users. Specifically, we study the correlation of the picture’s owner with Facebook generated alt-text and comments used by commenters when reacting to the image. We concentrate on gender attribute that is highly relevant for targeted advertising or privacy breaking. We explore how to launch an online gender inference attack on any Facebook user by handling online newly discovered vocabulary using the retrofitting process to enrich a core vocabulary built during offline training. Our experiments show that even when the user hides most public data (e.g., friend list, attribute, page, group), an attacker can detect user gender with AUC (area under the ROC curve) from 87% to 92%, depending on the picture metadata availability. Moreover, we can detect with high accuracy sequences of words leading to gender disclosure, and accordingly, enable users to derive countermeasures and configure their privacy settings safely.
This work is supported by DIGITRUST (http://lue.univ-lorraine.fr/fr/article/digitrust/).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abdelberi, C., Ács, G., Kâafar, M.A.: You are what you like! Information leakage through users’ interests. In: 19th Annual Network and Distributed System Security Symposium, NDSS, San Diego, California, USA. The Internet Society (2012)
Alipour, B., Imine, A., Rusinowitch, M.: Gender inference for Facebook picture owners. In: Gritzalis, S., Weippl, E.R., Katsikas, S.K., Anderst-Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) TrustBus 2019. LNCS, vol. 11711, pp. 145–160. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27813-7_10
Alipour, B., Imine, A., Rusinowitch, M.: You are what emojis say about your pictures: language-independent gender inference attack on Facebook. In: The 35th ACM/SIGAPP Symposium On Applied Computing, Brno, Czech Republic, pp. 1826–1834. ACM (2020)
Belinic, T.: Personality profile of social media users how to get maximum from it, April 2009. https://medium.com/krakensystems-blog/personality-profile-of-social-media-users-how-to-get-maximum-from-it-5e8b803efb30
Cadwalladr, C., Graham Harrison, E.: How Cambridge analytica turned Facebook ‘likes’ into a lucrative political tool, May 2018. https://www.theguardian.com/technology/2018/mar/17/facebook-cambridge-analytica-kogan-data-algorithm
Chen, Z., Lu, X., Ai, W., Li, H., Mei, Q., Liu, X.: Through a gender lens: learning usage patterns of emojis from large-scale Android users. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW, Lyon, France, pp. 763–772. ACM (2018)
Cheung, M., She, J.: An analytic system for user gender identification through user shared images. TOMCCAP 13(3), 1–20 (2017). Article no. 30
De Choudhury, M., Sharma, S.S., Logar, T., Eekhout, W., Nielsen, R.C.: Gender and cross-cultural differences in social media disclosures of mental illness. In: Proceedings of the Conference on Computer Supported Cooperative Work and Social Computing, CSCW, Portland, OR, USA, pp. 353–369. ACM (2017)
Church, K.W., Hanks, P.: Word association norms, mutual information, and lexicography. Comput. Linguist. 16(1), 22–29 (1990)
Eisner, B., Rocktäschel, T., Augenstein, I., Bošnjak, M., Riedel, S.: emoji2vec: learning emoji representations from their description. arXiv preprint arXiv:1609.08359 (2016)
Farahbakhsh, R., Han, X., Cuevas, Á., Crespi, N.: Analysis of publicly disclosed information in Facebook profiles. CoRR, abs/1705.00515 (2017)
Faruqui, M., Dodge, J., Jauhar, S.K., Dyer, C., Hovy, E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. arXiv preprint arXiv:1411.4166 (2014)
Flekova, L., Carpenter, J., Giorgi, S., Ungar, L.H., Preotiuc-Pietro, D.: Analyzing biases in human perception of user age and gender from text. In: Proceedings of the 54th Annual Meeting of the Association for Computational, Berlin, Germany, pp. 843–854. ACL (2016)
Gong, N.Z., Liu, B.: You are who you know and how you behave: attribute inference attacks via users’ social friends and behaviors. In: 25th Security Symposium, Austin, TX, USA, pp. 979–995. USENIX (2016)
Karimi, F., Wagner, C., Lemmerich, F., Jadidi, M., Strohmaier, M.: Inferring gender from names on the web: a comparative evaluation of gender detection methods. In: Proceedings of the 25th International Conference on World Wide Web, WWW, Montreal, Canada, pp. 53–54. ACM (2016)
Lenton, A.: Facebook wants you to search photos of your female friends at the beach, but not your male mates, February 2019. https://www.whimn.com.au/talk/people/facebook-wants-you-to-search-photos-of-your-female-friends-at-the-beach-but-not-your-male-mates/news-story/bbc21ee6883bd07bfbbbe76a0c8ca54c
Ludu, P.S.: Inferring gender of a Twitter user using celebrities it follows. CoRR, abs/1405.6667 (2014)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Nguyen, D., et al.: Why gender and age prediction from tweets is hard: lessons from a crowdsourcing experiment. In: Proceedings of the 25th International Conference on Computational Linguistics, COLING, Dublin, Ireland, pp. 1950–1961. ACL (2014)
Pagliardini, M., Gupta, P., Jaggi, M.: Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507 (2017)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should i trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016)
Santamaría, L., Mihaljevic, H.: Comparison and benchmark of name-to-gender inference services. PeerJ Comput. Sci. 4, e156 (2018)
Sap, M., et al.: Developing age and gender predictive lexica over social media. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, Doha, Qatar, pp. 1146–1151. ACL (2014)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pijani, B.A., Imine, A., Rusinowitch, M. (2020). Online Attacks on Picture Owner Privacy. In: Hartmann, S., Küng, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2020. Lecture Notes in Computer Science(), vol 12392. Springer, Cham. https://doi.org/10.1007/978-3-030-59051-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-59051-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59050-5
Online ISBN: 978-3-030-59051-2
eBook Packages: Computer ScienceComputer Science (R0)