Skip to main content
SpringerLink
Log in

Answering why-not questions on semantic multimedia queries

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Linked data is a promising way to publish media data as resources on the Web and interlink them with other resources. While significant amounts of image, audio and video fragments have been tagged and exposed as linked data, searching and explaining the unexpected query results have been rarely studied. To improve the functionality and usability of SPARQL-based multimedia search engines, we focus on explaining missing items in the query results, or the so-called why-not question in this paper. We first formalize why-not questions on multimedia SPARQL queries and then propose a novel explanation model to answer why-not questions. Our model adopts a to generate logical explanations at the basic graph pattern level, the filter constraint level, or the multimedia function level, respectively, which helps users refine their initial queries. Extensive experimental results on two real-world RDF datasets show that the proposed model and algorithms can provide high-quality explanations both in terms of effectiveness and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. Mona Lisa was painted in 1503. Mary, mother of Jesus was indeed painted by Leonardo da Vinci, but its was only a part of the Virgin of the Rocks.

  2. http://www.w3.org/TR/sparql11-query/#QueryForms

  3. https://github.com/tkurz/sparql-mm/tree/master/ns/2.0.0/ontology, a class model to describe spatio-temporal multimedia fragments and support ontology agnostic functions.

  4. Planet Earth II is a 2016 multinational nature documentary series produced by the BBC.

  5. http://wiki.dbpedia.org/Datasets, released in Sep., 2014

  6. http://queens.db.toronto.edu/oktie/linkedmdb/

  7. http://americanart.si.edu/collections/search/lod/about/

  8. https://datahub.io/dataset/bio2rdf-drugbank

  9. http://kfm.skyclass.net/anna/mmqueryset.html

  10. The Cronbach’s alpha is a standard measure of internal con-sistency reliability. A Cronbach’s alpha of 0.7 and higher may be regarded as satisfactory.

References

  1. Bhowmick SS, Sun A, Truong BQ (2013) Why not, wine?: towards answering why-not questions in social image search. In: Proceedings of the 21st ACM international conference on multimedia. ACM, pp 917–926

  2. Bidoit N, Herschel M, Tzompanaki K (2014) Query-based why-not provenance with nedexplain. In: Extending database technology (EDBT)

  3. Bizer C, Heath T, Berners-Lee T (2009) Linked data-the story so far. Semantic services, interoperability and web applications: emerging concepts, pp 205–227

    Chapter  Google Scholar 

  4. Calvanese D, Ortiz M, Simkus M, Stefanoni G (2013) Reasoning about explanations for negative query answers in dl-lite. J Artif Intell Res 48:635–669

    Article  MathSciNet  Google Scholar 

  5. Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920

    Article  MathSciNet  Google Scholar 

  6. Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann AG (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197

    Article  Google Scholar 

  7. Chang X, Yang Y (2016) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2016.2582746

    Article  MathSciNet  Google Scholar 

  8. Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632

    Article  Google Scholar 

  9. Chapman A, Jagadish H (2009) Why not?. In: Proceedings of the 2009 ACM SIGMOD international conference on management of data. ACM, pp 523–534

  10. Chen L, Lin X, Hu H, Jensen CS, Xu J (2015) Answering why-not questions on spatial keyword top-k queries. In: IEEE 31st international conference on data engineering (ICDE), 2015. IEEE, pp 279– 290

  11. Cui Y, Widom J (2003) Lineage tracing for general data warehouse transformations. VLDB J 12(1):41–58

    Article  Google Scholar 

  12. Damásio CV, Analyti A, Antoniou G (2012) Provenance for sparql queries. In: Proc. ISWC. Springer, pp 625–640

  13. Dividino R, Sizov S, Staab S, Schueler B (2009) Querying for provenance, trust, uncertainty and other meta knowledge in rdf. Web Semant Sci Serv Agents World Wide Web 7(3):204– 219

    Article  Google Scholar 

  14. Gao Y, Liu Q, Chen G, Zheng B, Zhou L (2015) Answering why-not questions on reverse top-k queries. Proc VLDB Endowment 8(7):738–749

    Article  Google Scholar 

  15. Hausenblas M, Troncy R, Raimond Y, Bürger T (2009) Interlinking multimedia: how to apply linked data principles to multimedia fragments. In: Proceedings of the 2nd international workshop on linked data on the web (LDOW). Madrid

  16. He Z, Lo E (2014) Answering why-not questions on top-k queries. IEEE Trans Knowl Data Eng 26(6):1300–1315

    Article  Google Scholar 

  17. Herschel M, Hernández MA (2010) Explaining missing answers to spjua queries. Proc VLDB Endowment 3(1-2):185–196

    Article  Google Scholar 

  18. Hu C, Xu Z, Liu Y, Mei L, Chen L, Luo X (2014) Semantic link network-based model for organizing multimedia big data. IEEE Trans Emerg Topics Comput 2(3):376–387

    Article  Google Scholar 

  19. Huang J, Chen T, Doan A, Naughton JF (2008) On the provenance of non-answers to queries over extracted data. Proc VLDB Endowment 1(1):736–747

    Article  Google Scholar 

  20. Islam MS, Liu C, Li J (2015) Efficient answering of why-not questions in similar graph matching. IEEE Trans Knowl Data Eng 27(10):2672–2686

    Article  Google Scholar 

  21. Islam MS, Zhou R, Liu C (2013) On answering why-not questions in reverse skyline queries. In: IEEE 29th international conference on data engineering (ICDE), 2013. IEEE, pp 973– 984

  22. Kurz T, Schlegel K, Kosch H (2015) Enabling access to linked media with sparql-mm. In: Proceedings of the 24th international conference on world wide web. ACM, pp 721–726

  23. Li Y, Wald M, Omitola T, Shadbolt N, Wills G (2012) Synote: weaving media fragments and linked data. In: Proceedings of the 5th workshop on linked data on the web (LDOW 2012). Lyon

  24. Luo M, Chang X, Nie L, Yang Y, Hauptmann A, Zheng Q (2017) An adaptive semi-supervised feature analysis for video semantic recognition. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2017.2647904

    Article  Google Scholar 

  25. Luo M, Nie F, Chang X, Yang Y, Hauptmann AG, Zheng Q (2017) Adaptive unsupervised feature selection with structure regularization. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2017.2650978

    Article  Google Scholar 

  26. Ma Z, Chang X, Xu Z, Sebe N, Hauptmann AG (2017) Joint attributes and event analysis for multimedia event detection. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2017.2709308

  27. Pérez J, Arenas M, Gutierrez C (2009) Semantics and complexity of sparql. ACM Trans Database Syst 34:1–45

    Article  Google Scholar 

  28. Shvaiko P, Euzenat J (2005) A survey of schema-based matching approaches. In: Journal on data semantics IV. Springer, pp 146–171

  29. Ten Cate B, Civili C, Sherkhonov E, Tan WC (2015) High-level why-not explanations using ontologies. In: Proceedings of the 34th ACM SIGMOD-SIGACT-SIGAI symposium on principles of database systems. ACM, pp 31–43

  30. Theoharis Y, Fundulaki I, Karvounarakis G, Christophides V (2011) On provenance of queries on semantic web data. IEEE Internet Comput 15(1):31–39

    Article  Google Scholar 

  31. The W3C SPARQL Working Group (2013) SPARQL 1.1 overview. http://www.w3.org/TR/sparql11-overview/

  32. Tran QT, Chan CY (2010) How to conquer why-not questions. In: Proceedings of the 2010 ACM SIGMOD international conference on management of data. ACM, pp 15–26

  33. Zhang D, Han J, Jiang L, Ye S, Chang X (2017) Revealing event saliency in unconstrained video collection. IEEE Trans Image Process 26(4):1746–1758

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is sponsored by The Fundamental Theory and Applications of Big Data with Knowledge Engineering under the National Key Research and Development Program of China with grant number 2016YFB1000903; National Science Foundation of China under Grant Nos.61672419, 61532004, and 61532015; MOE Research Center for Online Education Funds under Grant No.2016YB165; Ministry of Education Innovation Research Team No.IRT17R86.

Author information

Authors and Affiliations

  1. MOEKLINNS Lab, Xi’an Jiaotong University, Xi’an, China

    Meng Wang & Jun Liu

  2. The University of Queensland, Brisbane, Australia

    Weitong Chen & Xue Li

  3. Griffith Universtiy, Gold Coast Campus, Southport, Australia

    Sen Wang & Bela Stantic

Authors
  1. Meng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weitong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bela Stantic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Chen, W., Wang, S. et al. Answering why-not questions on semantic multimedia queries. Multimed Tools Appl 77, 3405–3429 (2018). https://doi.org/10.1007/s11042-017-5151-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5151-6

Keywords

Search

Navigation