Estrazione della conoscenza da immagini nel dominio del patrimonio culturale

In its broadest definition, cultural heritage is used to indicate natural, tangible, and intangible elements that are endowed with cultural significance. The latter define the personal and social realm of individuals and determine their relationship with the local area. Cultural heritage encompasses places, physical objects, and abstract forms that enable individuals, peoples, and societies to create a common sense of belonging and display their cultural diversity. Countries and international organizations, UNESCO above all, are aware of the importance of cultural heritage from a social and anthropological perspective and promote actions aimed at safeguarding and valorizing it. In compliance with these objectives, digital technologies offer important opportunities, as demonstrated by the substantial funding at European and national levels. In Italy, the funding from the National Recovery and Resilience Plan for Culture (PNRR Culture) is meant to exploit the potential of current technologies to carry out activities finalized for the promotion of the national cultural heritage. Moreover, for a country such as Italy which owns a considerable number of cultural sites1, realizing such activities may constitute an impulse for tourism and, consequentially, the enhancement of territory. Even though the digital ecosystem allows remarkable possibilities, some aspects must be taken into account. The latter concerns the negative ethical, social, and legal consequences that, in some instances, can inevitably undermine the cultural heritage and existence of populations. The preliminary understanding of technologies and their impact on relations between society, economy, and culture is necessary for the correct safeguarding and promotion of cultural heritage. Currently, with the development of systems based on Artificial Intelligence (AI) becoming increasingly sophisticated, there is a greater awareness regarding the dangers related. In particular, attention is mostly paid to the misuse or illegitimacy of such systems that can negatively impact on individuals. In recent years, various institutions have intensified their efforts toward defining and implementing policies for the standardization of AI-based systems to ensure their “trustworthy”2 use. The cultural context can benefit greatly from using AI technologies, specifically Machine and Deep Learning, to create activities that highlight the unique characteristics of cultural heritage. Contrary to what has been said in the past, an interconnection between technologists and domain experts is necessary to define the correct methodology for the proper application of this kind of technologies. As witnessed by the AI Act, this is an awareness of the change in the trend of thinking on artificial intelligence: you do not need to have only computer scientists able to develop models but to integrate knowledge with domain experts to make effective solutions towards a greater sectorial. In this scenario, the present doctoral thesis aims to formalize a methodology for the application of different technologies and standards for the extraction of hidden knowledge by images of cultural heritage. The method here proposed reflects the main notion of Digital Humanities as a transversal discipline for the definition of practices and policies for the use of computational tools in the Humanities Sciences. In the cultural domain, images represent one of the main means of representing and disseminating heritage. For example, augmented or virtual reality (AR/VR) are highly interactive solutions that require, however, on-site users. Instead, the International Image Interoperability Framework (IIIF) is an optimal solution for the simple and fast sharing of cultural material via the Internet network. One of the main advantages characteristics of the framework is the possibility to represent both the media and descriptions allowing users to navigate between the visual and informative content. The importance of emphasizing this function lies in the fact that images are valuable sources of information, consisting of both text and visual objects, that can conceal knowledge that can be extracted. The results achieved today with technological development have allowed the implementation of sophisticated algorithms based on convolutional neural networks (CNNs) able to investigate semantically the content of images. The latter may automatically compute different operations such as recognition, extraction, and analysis of information with extreme precision and accuracy. By departing from these considerations, this doctoral thesis focuses on the possibility of adapting convolutional neural networks to the cultural domain for the automatic extraction of knowledge from images. In addition to that, the International Image Interoperability Framework is chosen as a model to provide a standardized description of resources. The ultimate goal is to define a methodology for the promotion and valorization of cultural objects via the extraction and representation of the knowledge they contain. In particular, the aim put in place has a double purpose: the first one concerns the contextualization of convolutional neural networks to the domain of cultural heritage for the automatic recognition and extraction of elements of various genres. The second purpose regards the identification of modalities for the automatic definition of an IIIF Manifest model to provide a standardized description of cultural resources. The work is focused on the application of image classification and object detection techniques to recognize and extract informative elements from cultural resources. The information here extracted constitute an additional value from which to derive optimum benefit for describing resources in IIIF. This doctoral thesis begins with a general introduction to the cultural heritage domain analyzing the importance for societies and individuals. In particular, some considerations are placed inherent to the actual national digitization scenario, focusing on the activities carried out by the Italian Digital Library within the “Piano nazionale di digitalizzazione del patrimonio culturale”3. The central parts of the work regard the analysis circa IIIF and convolutional neural networks, defining their characteristics and current applications. Especially for what concerns the International Image Interoperability Framework, applications in LAM (Libraries, Archives, and Museums) domains are presented. Finally, the doctoral work focuses on the definition of a methodology for implementing an automatic process of knowledge extraction from images of the cultural heritage domain. The process implies the automatic realization of an IIIF model for making resources available with their informative content enriched with the elements extractable by convolutional neural networks. The discussion is placed at a theoretical level, defining the methodology, which refers to technologies, techniques, and tools that can enhance cultural resources under a descriptive plan. The process outlined here aims to promote cultural heritage through the knowledge contained in the images, opening up new challenges and opportunities for the entire community of actors involved in the cultural market. Capitalizing knowledge means taking advantage of the information elements in cultural heritage images. This operation allows, in a context that is increasingly focused on linked open data (LOD), the development of an informative base that can connect the various cultural sectors. Then, if properly managed information from each institution, be it archives, museums, or libraries, can be integrated to help enhance cultural heritage.