Webis-Context-sensitive-Word-Search-Queries-2022

This is the dataset created for Language Models as Context-sensitive Word Search Engines at the In2Writing workshop at ACL22.

Cite 

@inproceedings{wiegmann:2022,
    title =     "Language Models as Context-sensitive Word Search Engines",
    author =    "Wiegmann, Matti and V{\"{o}}lske, Michael and Potthast, Martin and Stein, Benno",
    booktitle = "Proceedings of the 1st Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022)",
    month =     may,
    year =      "2022",
    address =   "Online",
    publisher = "Association for Computational Linguistics",

Datasets

This repository contains two datasets with word search queries. Each word search query consists of a token n-gram with one wildcard token ([MASK]). The answers to each query are the most likely token to replace the mask. All queries originate from wikitext-103 and CLOTH, the respected source is annotated for each query.

The original-token dataset lists exactly one top answer for each query. The ranked-answers dataset lists multiple, sorted answers in three relevance categories, where 3 is the most relevant. Please refer to the citation for more details.