Within the RePAIR project, the team from Bonn was responsible for the robotic system, devising planning algorithms for the robot’s two moving arms that allowed it to place fragments of frescoes with a high degree of precision. Maren Bennewitz is Professor for Humanoid Robots and Vice Rector for Digitalization and Information Management. She says: “We brought our many years of experience in movement planning and robot-driven manipulation to bear in the RePAIR project. It was geared toward finding out whether robots can be capable of handling the laborious sorting and jigsaw-like reassembly work so that the experts can turn their attention to those tasks that need a human touch.” The University of Bonn was also in charge of integrating and evaluating the entire robotic platform and made sure that all the various components fitted together perfectly, with doctoral student Nils Dengler spending a total of five weeks in Pompeii earlier in the year for this purpose.
Coordinated by Ca’ Foscari University of Venice, the EU-funded project used cutting-edge robotics and artificial intelligence (AI) methods to solve archaeological challenges. The Italian Institute of Technology (IIT), the Archaeological Park of Pompeii, Ben-Gurion University of the Negev and the Instituto Superior Técnico in Lisbon were also among the project partners.
“Robo-Puzzler” powered by expertise from Bonn
The key element of RePAIR was a robotic platform that was equipped with two robot arms and soft gripper hands and that grabbed fragments of murals and put them back down where the AI-powered jigsaw solver had calculated they should go. And it is precisely here, where smart planning met physical execution, that the research from the University of Bonn came in: “Our algorithms calculated what movements the robot’s two arms needed to make in order to pick the fragments up safely and place them carefully in the position we’d determined for them,” Nils Dengler explains.
A fresco jigsaw from Pompeii
The innovative system was trialled in the Archaeological Park of Pompeii, which is home, among other things, to ceiling paintings from the “House of the Painters at Work” and frescoes from the Schola Armaturarum (“House of the Gladiators”). These works were smashed to pieces by the eruption of Mount Vesuvius in 79 CE as well as Allied bombardment during the Second World War and a building collapse in 2010. Using a 3D scanning system developed specially for the task, the fragments are converted into digital images that the AI then uses to attempt to reassemble the “jigsaw.” Specifically, it suggests combinations of fragments that would fit and calculates how larger segments of each painting could be restored out of hundreds or thousands of tiny pieces.
“What makes it challenging is that, unlike a standard jigsaw puzzle, it doesn’t come in a box with a picture on it,” Maren Bennewitz says. “Many pieces are badly damaged or missing entirely, and you often get fragments from different works mixed up together. So it’s all the more important that the robotics and the AI work closely with the expert archaeologists.”
Taking the strain off the archaeologists
Up until now, reconstructing fragmented finds has been a task that has required a lot of patience and that has often found itself neglected in the busy day-to-day life of a museum. By largely automating the time-consuming steps of digitization, calculating suggestions and physical reconstruction, however, RePAIR has enabled archaeologists to concentrate more on their scientific evaluations. And the knowledge developed in the project is not restricted solely to Pompeii: in repositories and storerooms all over the world, there are mountains of fragments of ceramics, murals or pieces of architecture still awaiting reassembly due to time constraints.
“It’s fascinating to see how robotics is not only solving problems in factories and the logistics industry but can also help preserve our cultural heritage,” says Professor Bennewitz, summing up. “RePAIR is also a shining example of successful interdisciplinary collaboration in Europe. For us here in Bonn, the project played an important part in our efforts to incorporate our robotics research even more closely into applications that can benefit society.”
The RePAIR project was funded as part of the EU’s Horizon 2020 grant program under grant agreement no. 964854.
You can find more information about it here:
This video shows the robot at work:
https://youtu.be/AUN5aGMb6Eo
Contact:
Prof. Dr. Maren Bennewitz
Telefon: 0228/73-54164
E-Mail: maren@cs.uni-bonn.de
