Kateryna Tymoshenko
ABSTRACT
In this paper, we extensively study the use of syntactic and semantic structures obtained with shallow and deeper syntactic parsers in the answer passage reranking task. We propose several dependency-based structures enriched with Linked Open Data (LD) knowledge for representing pairs of questions and answer passages. We use such tree structures in learning to rank (L2R) algorithms based on tree kernel. The latter can represent questions and passages in a tree fragment space, where each substructure represents a powerful syntactic/semantic feature. Additionally since we define links between structures, tree kernels also generate relational features spanning question and passage structures. We derive very important findings, which can be useful to build state-of-the-art systems: (i) full syntactic dependencies can outperform shallow models also using external knowledge and (ii) the semantic information should be derived by effective and high-coverage resources, e.g., LD, and incorporated in syntactic structures to be effective. We demonstrate our findings by carrying out an extensive comparative experimentation on two different TREC QA corpora and one community question answer dataset, namely Answerbag. Our comparative analysis on well-defined answer selection benchmarks consistently demonstrates that our structural semantic models largely outperform the state of the art in passage reranking.
- E. Aktolga, J. Allan, and D. A. Smith. Passage reranking for question answering using syntactic structures and answer types. In ECIR, 2011. Google Scholar
Digital Library
- M. W. Bilotti, J. L. Elsas, J. Carbonell, and E. Nyberg. Rank learning for factoid question answering with linguistic and semantic constraints. In CIKM, 2010. Google ScholarDigital Library
- M. W. Bilotti and E. Nyberg. Improving text retrieval precision and answer accuracy in question answering systems. In (IR4QA) Workshop at COLING, 2008. Google ScholarDigital Library
- C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. Cyganiak, and S. Hellmann. Dbpedia - a crystallization point for the web of data. Web Semantics, 7(3), 2009. Google ScholarDigital Library
- C. Bizer and A. Schultz. The berlin sparql benchmark. IJSWIS, 5(2):1--24, 2009.Google ScholarCross Ref
- B. Bohnet. Top accuracy and fast dependency parsing is not a contradiction. In COLING, 2010. Google ScholarDigital Library
- J. D. Choi and M. Palmer. Getting the most out of transition-based dependency parsing. In ACL, 2011. Google ScholarDigital Library
- D. Croce, A. Moschitti, and R. Basili. Structured lexical similarity via convolution kernels on dependency trees. In Proceedings of EMNLP, 2011. Google ScholarDigital Library
- A. Csomai and R. Mihalcea. Linking documents to encyclopedic knowledge. IEEE Intelligent Systems, 23(5):34--41, 2008. Google ScholarDigital Library
- R. E. de Castilho and I. Gurevych. A broad-coverage collection of portable NLP components for building shareable analysis pipelines. In OIAF4HLT Workshop (COLING), 2014.Google ScholarCross Ref
- J. Fan, D. Ferrucci, D. Gondek, and A. Kalyanpur. Prismatic: Inducing knowledge from a large scale lexicalized relation resource. In NAACL HLT, 2010. Google ScholarDigital Library
- C. Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, 1998.Google ScholarCross Ref
- D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty. Building Watson: An Overview of the DeepQA Project. AI Magazine, 31(3), 2010.Google Scholar
- M. Heilman and N. A. Smith. Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In NAACL, 2010. Google ScholarDigital Library
- A. Hickl, J. Williams, J. Bensley, K. Roberts, Y. Shi, and B. Rink. Question answering with lcc chaucer at trec 2006. In TREC, 2006.Google Scholar
- J. Hoffart, F. Suchanek, K. Berberich, and G. Weikum. Yago2: a spatially and temporally enhanced knowledge base from wikipedia. Artificial Intelligence, 2012. Google ScholarDigital Library
- J. Jeon, W. B. Croft, and J. H. Lee. Finding similar questions in large question and answer archives. In CIKM, 2005. Google ScholarDigital Library
- T. Joachims. Optimizing search engines using clickthrough data. In ACM SIGKDD KDD, 2002. Google ScholarDigital Library
- B. Katz and J. Lin. Selectively using relations to improve precision in question answering. In Workshop on NLP for QA (EACL), 2003.Google Scholar
- X. Li and D. Roth. Learning question classifiers. In Proceedings of ACL, 2002. Google ScholarDigital Library
- C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. The Stanford CoreNLP natural language processing toolkit. In ACL, 2014.Google ScholarCross Ref
- M. C. McCord, J. W. Murdock, and B. Boguraev. Deep parsing in Watson. IBM Journal, 56(3), 2012. Google ScholarDigital Library
- D. Milne and I. Witten. An open-source toolkit for mining wikipedia. In NZCSRSC, volume 9, 2009.Google Scholar
- A. Moschitti. Efficient convolution kernels for dependency and constituent syntactic trees. In ECML, 2006. Google ScholarDigital Library
- J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kübler, S. Marinov, and E. Marsi. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 2007.Google ScholarCross Ref
- M. Pasca. Open-Domain Question Answering from Large Text Collections. CSLI Publications, 2003.Google Scholar
- V. Punyakanok and D. Roth. The use of classifiers in sequential inference. In NIPS, pages 995--1001, 2001.Google Scholar
- F. Radlinski and T. Joachims. Query chains: Learning to rank from implicit feedback. CoRR, 2006.Google Scholar
- A. Severyn and A. Moschitti. Automatic feature engineering for answer selection and extraction. In EMNLP, 2013.Google Scholar
- A. Severyn and A. Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In SIGIR, 2015. Google ScholarDigital Library
- A. Severyn, M. Nicosia, and A. Moschitti. Building structures from classifiers for passage reranking. In CIKM, 2013. Google ScholarDigital Library
- A. Severyn, M. Nicosia, and A. Moschitti. Learning adaptable patterns for passage reranking. CoNLL-2013, page 75, 2013.Google Scholar
- J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004. Google ScholarCross Ref
- D. Shen and M. Lapata. Using semantic roles to improve question answering. In EMNLP-CoNLL, 2007.Google Scholar
- S. Small, T. Strzalkowski, T. Liu, S. Ryan, R. Salkin, N. Shimizu, P. Kantor, D. Kelly, and N. Wacholder. Hitiqa: Towards analytical question answering. In COLING, 2004. Google ScholarDigital Library
- M. Surdeanu, M. Ciaramita, and H. Zaragoza. Learning to rank answers on large online QA collections. In ACL, 2008.Google Scholar
- K. Tymoshenko, A. Moschitti, and A. Severyn. Encoding semantic resources in syntactic structures for passage reranking. In Proceedings of EACL, 2014.Google ScholarCross Ref
- E. M. Voorhees. Overview of the TREC 2001 Question Answering Track. In Proceedings of TREC, 2001.Google Scholar
- E. M. Voorhees. Overview of the trec 2003 question answering track. In TREC 2003, 2003.Google Scholar
- D. Wang and E. Nyberg. A long short-term memory model for answer sentence selection in question answering. In ACL, July 2015.Google ScholarCross Ref
- M. Wang and C. D. Manning. Probabilistic tree-edit models with structured latent variables for textual entailment and question answering. In ACL, 2010. Google ScholarDigital Library
- M. Wang, N. A. Smith, and T. Mitamura. What is the jeopardy model? a quasi-synchronous grammar for qa. In EMNLP-CoNLL, 2007.Google Scholar
- P. C. Xuchen Yao, Benjamin Van Durme and C. Callison-Burch. Answer extraction as sequence tagging with tree edit distance. In NAACL, 2013.Google Scholar
- X. Yao, B. Van Durme, and P. Clark. Answer extraction as sequence tagging with tree edit distance. In Proceedings of NAACL-HLT, pages 858--867, 2013.Google Scholar
- W.-t. Yih, M.-W. Chang, C. Meek, and A. Pastusiak. Question answering using enhanced lexical semantic models. In ACL, 2013.Google Scholar
Index Terms
- Assessing the Impact of Syntactic and Semantic Structures for Answer Passages Reranking
Recommendations
Shallow and Deep Syntactic/Semantic Structures for Passage Reranking in Question-Answering Systems
In this article, we extensively study the use of syntactic and semantic structures obtained with shallow and full syntactic parsers for answer passage reranking. We propose several dependency and constituent-based structures, also enriched with Linked ...
Passage Reranking for Question Answering Using Syntactic Structures and Answer Types
ECIR 2011: Proceedings of the 33rd European Conference on Advances in Information Retrieval - Volume 6611Passage Retrieval is a crucial step in question answering systems, one that has been well researched in the past. Due to the vocabulary mismatch problem and independence assumption of bag-of-words retrieval models, correct passages are often ranked ...
Building structures from classifiers for passage reranking
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementThis paper shows that learning to rank models can be applied to automatically learn complex patterns, such as relational semantic structures occurring in questions and their answer passages. This is achieved by providing the learning algorithm with a ...
Comments