OPUS

OPUS - Open AI solutions for historical masonry annotation

Understanding historical masonries in cultural heritage assets is a challenging task that involves different disciplinary fields and the work of experts.
The annotation of historical masonry is a process of association between the graphically represented element and any relevant knowledge-based information. This autoptic analysis is vital for supporting the documentation, analysis and diagnosis of heritage structures. The conventional approach is based on the manual drawing of annotated regions over photos or orthographic images of the structure. This makes the process prohibitively time consuming and extremely costly, limiting the ability to manage large-scale monitoring.
The OPUS project aims to develop and test an integrated AI-based system for the assisted and automatic annotation of historical masonries. The project will tackle the annotation task using both assisted AI-based tools and automated CNN (Convolutional Neural Networks) semantic segmentation models, working at different levels and granularity: the mapping of construction features, namely techniques and materials, and the mapping of individual constituent elements.
The aim will be reached through the use of TagLab, an open-source AI-powered semantic segmentation software developed by the project’s partners. This tool, originally designed to work on marine biology, will be extended to work on masonries and improved by tweaking and extending its functionalities. A key part of the project’s approach is to maintain the role of the expert pivotal. Our aim is not to create a novel process of masonry annotation, but to complement the current workflow with AI technology, able to support and facilitate the manual annotation, and to automate the large-scale tasks.
The project will work on the theoretical aspects of the masonry annotation, formalizing the annotation process and mapping to an AI framework, defining the classes of interest and input data, and establishing integrated human-AI annotation workflows. From a methodological viewpoint, the project will promote a transdisciplinary approach to the study of historical masonry. From a technical viewpoint, the project will explore the use of AI technology in the cultural heritage field, developing practical solutions to open research issues: advanced user interface and interaction, use of multi-band data, improving the learning process on real-world data.
The OPUS project will adopt a case study research approach, as it can generate an in-depth, multi-faceted understanding of masonries located in a geographical area, while mitigating challenges and complexities derived from the variety of local building cultures. The project will reach out to the community working on historical masonry (academic, public and private), distributing the produced datasets as open-data and the developed software as open source, organizing training workshops and dissemination events.

---

Bando PRIN 2022 Decreto Direttoriale n. 104 del 02-02-2022
Principal Investigator: Marco Callieri
Codice progetto: 202277KJ8F
Settore ERC: SH6
Durata: 24 mesi