We organised this shared task in the framework of the 16th International Workshop on Semantic Evaluation (SemEval-2022). PreTENS is a task on the evaluation of the acceptability of simple sentences containing constructions whose two arguments are presupposed to be or not to be in an ordered taxonomic relation. It addresses three languages: English, Italian and French.

The task has been articulated into the two following sub-tasks:

1. a binary classification sub-task, which consists in predicting the acceptability label assigned to each sentence of the test set;
2. a regression sub-task, which consists in predicting the average score assigned by human annotators on a seven point Likert-scale with respect to the subset of data evaluated via crowdsourcing.

For detailed documentation about the task, see the PreTENS website.

The training and test datasets are available on GitHub here.

References

Zamparelli R., Chowdhury S., Brunato D., Chesi C., Dell’Orletta F., Hasan M. A., Venturi G. (2022). SemEval-2022 Task 3: PreTENS-Evaluating Neural Networks on Presuppositional Semantic Knowledge”. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), 14-15 July 2022, Seattle, United States.