Addressing lexical ambiguity in multiple languages via novel manually curated lexical-semantic resources

Interacting in natural language is one of the most important human abilities. In the field of Artificial Intelligence (AI), this ability has been regarded as a key criterion for determining whether an entity can be considered intelligent. Enabling machines to understand and interact in natural language has therefore been a longstanding goal of AI and, more specifically, of Natural Language Understanding (NLU), the field of AI concerned with this challenge. Over the past few decades, considerable efforts have been devoted to this goal, and recent advances in NLU have brought significant progress towards its achievement. A crucial step in this direction is enabling machines to effectively handle lexical ambiguity, a pervasive feature of natural language that affects virtually all areas of NLU. Specifically, Word Sense Disambiguation (WSD) is the NLU task aimed at identifying the meaning of a word or expression in a given context. Depending on the approach adopted, WSD relies heavily on lexical-semantic resources such as semantic networks, large-scale sense-labeled datasets, and manually curated benchmarks. However, despite the efforts made so far, significant limitations in currently available lexical-semantic resources hinder models’ ability to handle lexical ambiguity. In light of these limitations, this thesis investigates the challenges that lexical ambiguity poses to the development and evaluation of NLU systems, focusing on the role of linguistic resources and computational approaches. A key issue addressed in this thesis is the knowledge acquisition bottleneck, i.e., the difficulty of acquiring large amounts of reliable annotated data. Specifically, we focus on the scarcity of sense-labeled training data across languages, which significantly limits the effectiveness of supervised WSD approaches. To address this challenge, we manually create multilingual datasets for word alignment (WA) and propose MultiMirror, a novel approach to cross-lingual label projection that leverages WA to automatically generate large amounts of sense-labeled data for multilingual WSD. Across most evaluated settings, MultiMirror outperforms competing approaches and enables a simple neural classifier based on multilingual BERT (mBERT) to achieve state-of-the-art performance. Furthermore, we introduce XL-WA, a collection of manually curated datasets for WA covering 14 language pairs, designed to support the training and evaluation of word alignment systems and future research on cross-lingual label projection in additional languages. Beyond the availability of training data, the development of effective multilingual WSD systems also requires reliable benchmarks to assess their disambiguation capabilities. However, manually curated evaluation resources remain scarce, making it difficult to compare systems fairly and, in some cases, preventing their evaluation altogether, particularly for mid- and low-resource languages. To address this problem, we introduce the ELEXIS parallel sense-annotated dataset, a novel manually curated semantic dataset available in 10 European languages. Traditionally, such benchmarks are annotated with senses drawn from open-access sense inventories such as BabelNet. However, the limited coverage of these inventories in terms of both senses and languages, together with longstanding issues concerning sense granularity, can make the evaluation of WSD systems based on these inventories problematic. An alternative is intrinsic evaluation, which assesses a system’s ability to distinguish meanings without relying on a predefined sense inventory. To this end, we propose Multilingual and Cross-lingual Word-in-Context (MCL-WiC), a novel manually created benchmark for inventory-independent evaluation, covering five languages through monolingual and cross-lingual tasks. The challenges of developing and evaluating WSD systems are further complicated by the variety of linguistic phenomena involved in lexical ambiguity. While research has largely focused on individual words, many expressions involve multiple words whose meaning cannot always be inferred from the meanings of their individual components. Idiomatic expressions are a particularly challenging example, yet their automatic identification remains under-investigated, and manually curated multilingual resources for this task are scarce. To address this gap, we create new evaluation benchmarks in four languages and silver training datasets in 10 languages for the automatic identification of idioms, and propose ID10M, a novel approach to automatically identifying idiomaticity in context. We demonstrate that our system effectively distinguishes between the literal and idiomatic uses of a given multi-word expression in context and across multiple languages. Taken together, these resources and approaches address different aspects of lexical ambiguity, from its identification and evaluation to the development of multilingual systems. An important further question is whether and how addressing lexical ambiguity can improve downstream tasks. To investigate this, the final contribution of this thesis focuses on Machine Translation (MT), a key downstream application of NLU, examining the impact of lexical ambiguity on translation quality. Indeed, lexical ambiguity is a major source of errors in MT and, to the best of our knowledge, no fully manually curated benchmarks are currently available for studying its impact across multiple language combinations. To address this gap, we introduce DiBiMT, a fully manually curated evaluation benchmark for investigating semantic biases in MT across five language combinations. DiBiMT enables a systematic assessment of how lexical ambiguity affects translation, providing a basis for further research into semantic biases in multilingual MT systems. Overall, the contributions of this thesis demonstrate the importance of combining high-quality multilingual lexical-semantic resources with computational approaches to address the challenges posed by lexical ambiguity. By advancing the creation of training and evaluation resources, developing methods for identifying word senses and idiomatic expressions across languages, and investigating the effects of lexical ambiguity on downstream applications such as MT, this work contributes to a broader understanding of how lexical meaning can be represented, interpreted, and evaluated computationally.