Abstract
In this paper, a service robot solution is presented for the analysis, surveying and restoration of fresco paintings. The proposed design approach integrates robot design and restoration operation. It aims to merge them into a feasible solution that can be both practical and feasible for restorers. The simulation results are reported to show a successful design solution, which has been conceived with the constraints of a low-cost user-oriented design and the consideration of cultural heritage.
Introduction
The problem of restoring damaged and missing areas in artistic works is of great importance in the field of restoration. These restoration problems have similarities in many areas of cultural heritage (architecture, painting and sculpture). Thus, the basic concepts are common but the practice can differ in achieving suitable results that permit the future preservation of a work with a clear indication of passed time. The theory of restoration, as established by Cesare Brandi, [1], requires that the restoration aims to maintain and not renew a historical work. Thus, modern means of restoration must be adapted to damaged historical works and not vice versa. Brandi's approach can be summarized in three points, [1] as follows:
An intervention of a restoration must be visible but it must not disturb the whole output of the restored work.
The appearance and material of a work is irreplaceable.
A restoration intervention will not prevent future interventions but it will facilitate them.
The disciplines of conservation and restoration of cultural heritage goods are under a significant evolution. Activities and applications from other disciplines and experiences are considered. Collaboration with teams from other scientific and technological areas is useful when looking to reach multidisciplinary approaches. These collaborations produce more successful results both in practical and theoretical terms. In general, restoration activity on a historical good is composed of several phases, with direct and indirect operations that are often completed with chromatic reintegration of lost parts. Colour reintegration can be achieved using several techniques, depending on the material ruptures, time deterioration and damage causes, with the aim of restoring the original readability of the artistic artefact [7, 22].
Currently, there are several techniques that can be classified as points, segments or glazing operations, depending on the execution operations. All of them are based on laboriously slow manual actions and procedures that are determined by the experience and expertise of the restorer. Beside the above human factors, there are also many external factors that influence the colour perception of the restored results. The proposed robotic system has been conceived as a means to facilitate operations and make them efficient in any restoration strategy or methodology that a restorer considers to be necessary for her/his intervention. The proposed case study is based on the technology of inkjet printing with a solution that can be installed on a robotic mobile structure. Today, inkjet printing is of high quality and extremely durable, and can be considered suitable for restoration interventions [8, 18].
The design results, along with the proposed robotic system and its simulation, have been implemented as an integrated activity among the authors, who come from different areas of engineering, architecture and the arts. All of the authors have elaborated on the criteria and design solutions. Coming from different areas of expertise, they have different perspectives. However, they have the common target of creating a proper robotic system that can help the restoration of frescos in architectural frames.
The simulation results are reported, as well as the design results. The operation feasibility of the proposed solution for the restoration of fresco paintings is also reported. First experiences with a preliminary prototype are also presented to show an implementation plan of the proposed approach, with reference to a specific case of study.
Goods of Cultural Heritage and Restoration
Cultural heritage can be understood as the legacy of physical artefacts and intangible attributes of a group or society that are inherited from past generations, maintained in the present and bestowed for the benefit of future generations. Cultural heritage includes tangible culture (such as historical cities, buildings, monuments, landscapes, books, works of art and artefacts), intangible culture (such as folklore, music, traditions, language and knowledge) and natural heritage (including culturally-significant landscapes and biodiversity). Thus, cultural heritage attention can be given to objects and facts that are on very different levels of the scale - from the very small to the very huge - as well as in terms of time duration. Small objects, such as artworks and other cultural masterpieces, are generally collected in museums, art galleries and exhibition sites, as well as in archives for storage. The significance of artworks and other cultural masterpieces can be understood against the interpretation of socio-economic, political, ethnic, religious and philosophical values of a particular or wide group of people, in addition to an appreciation of pure artistic views.
Goods of cultural heritage, as products with values in history, arts, society developments, and science/technical achievements, are usually considered as objects to be preserved, as indicated by several national and international law prescriptions for cultural heritage. The main documents are the Chart of Athens in 1931 - the first international act for the preservation of cultural heritage, and then, the Chart of Venice in 1964 and the Chart of Nara in 1994. The elaboration of international recognized agreements, such as those in Aja on 14 May 1954, Granada in 1985, Washington in 1987 and Paris on 17 October 2003, are also considered important documents. The United Nations Educational, Scientific and Cultural Organization (UNESCO) seeks to encourage the identification, protection and preservation of cultural and natural heritage sites around the world with outstanding value for humanity. This is well clarified in an international treaty called the Convention Concerning the Protection of the World Cultural and Natural Heritage [21], which was adopted by UNESCO in 1972.
Keeping present cultural heritage for the future is also connected with technical actions such as identification, analysis, preservation and restoration. Each of these areas of intervention not only includes technical actions and expertise but also, requires more cultural evaluations in order to respect the concept of cultural heritage [1; 19]. Indeed, preservation means understanding the value of a product. Therefore, a proper intervention requires a deep knowledge of the object or fact by looking at all of the above-mentioned aspects with a wide interdisciplinary approach.
General Considerations of Service Robots
Beside the standard definitions by International organizations like ISO, IFR, IFToMM and IFAC, service robots are introduced with several different views, depending on the application areas and discipline frames. The IFR definition states: ‘a robot which operates semi or fully autonomously to perform services useful to the well-being of humans and equipment, excluding manufacturing operations’ with a classification of ‘servicing humans, servicing equipment, and other performing function and/or service robots that cannot be classified in the above two groups’ [6].
In general, a service robot for restoration activity can be still understood as a robotic system, [2]. It has a certain level of autonomy in performing service operations with given restoration tasks within a specified environment and interaction with human users in cultural heritage frames.
The level of autonomy of a service robot in cultural heritage frames can be considered as a function of the service restoration/preservation/identification task. A service robot's autonomy can be achieved by a versatile structure design and suitable flexible operation. The level of autonomy is due to the required supervision or interaction with a human operator or user working in a specific field of intervention of cultural goods. In some cases, a full autonomy with artificial intelligence is necessary for the service restoration/identification goals, mainly when the service robot is expected to operate fully autonomously, even in unstructured environments in crowed or outdoor spaces. In some other cases, due to the well-structured configuration of the environment, the autonomy can be conveniently designed just by using proper sensors and suitable trajectory planning. In other cases, as it has a strong interaction with the human user, the autonomy is fully constrained by and for the proper interaction with a human user indicating the operations that interact with a cultural heritage good.
In general, service operations can be understood as a set of actions and behaviours towards a service task, [3]. They are the ultimate goal of the design and operation of a service robot.
The above-mentioned short considerations can be useful in understanding the multidisciplinary integration that is required to design and successfully operate a service robot in applications within areas of cultural heritage in one or more activities of intervention on cultural goods. The extent of multidisciplinary is much more wide than in any other application field, since it includes technical aspects, human attitudes (of operators and/or users), human-machine interactions and environment issues, as summarized in Fig. 1. Indeed, in developing and operating service robots, involved competences from fields of human life and environment considerations become increasingly necessary, in addition to technical expertise. Thus, for example, psychologists and historians (and many others) are welcomed in R&D teams for designing service robots, as well as architects and restorers for practical implementations in interventions on cultural heritage.
Referring to the technical aspects, Fig. 2 summarizes a general mechatronic character of a service robot as a traditional robot. However, it places specific emphasis on the above-mentioned aspects in terms of interactions with the environment and human beings, as well as a careful consideration of the tasks to be performed. These interactions should be understood not only in terms of engineering issues (mainly mechanical ones) but also, by looking at more general aspects, for example, the psychological attitudes, social impacts and cultural interests. A consideration of the environment and cultural heritage goods should also include how a service robot affects or is affected by them. This will require analysing and designing a variety of conditions and situations with more technical aspects and constraints. In particular, activities in cultural heritage frames should also evaluate how the human operators will operate and interact with the robot for the desired tasks as a function of the importance of the goods that are involved or targeted in the activities.
In addition, service robots for restoration activities in cultural heritage can be considered efficient and successful when, ultimately, the cost, both in design and operation, can be properly sized as a function of the service task, and mainly as a function of the affordable budget of users and operators in frames of cultural heritage. The affordable cost will also depend on the importance of the good under restoration with those actions that are permitted by the restoring conditions. Thus, from the beginning of a design process, economic evaluation and management will be included in both R&D and the design of service robots Fig. 2b).
Multidisciplinary area integration for service robotics in cultural heritage frames
Mechatronic scheme for service robots: a) a technical mechatronic viewpoint; b) a user/market oriented viewpoint
In general, the challenges for service robots in cultural heritage frames can be understood as:
Operating together with human users, with suitable behaviours and careful user-friendly operation in interaction with cultural heritage goods.
Operating service tasks with proper easy-operation modes at user-oriented cost for actions on cultural heritage goods.
It can be particularly challenging to make a service robot acceptable from both a psychological and technical viewpoint for a novel application in a professional area, which traditionally does not work with or use technical means like the frames for restoration of historical goods in cultural heritage. Thus, there are challenges in designing acceptable systems by users of cultural heritage areas. This may require the adjustment and inclusion of specific features of novel applications, even if they are not thought to be essential or functional for the design and operation of new service robots for restoration activities. It is also challenging to convince operators and users, like restorers, art historians and architects from such novel application areas, to cooperate in developing solutions or even identifying the main problems for the design and operation of new service robots for restoration activities. It is also essential to plan the level of interventions of the service robots, which will not exclude the restorers from significant action in the restoration activities. This is because they are responsible for the decisions and expertise, which cannot be accomplished by a service robot. This must be clarified with the consideration that service robots are intended to help and not substitute humans in restoration activities with those intellectual and essential characters. In addition, most of the time, there are difficulties in the cultural barriers, which make it difficult to understand each other. In particular, it is challenging to identify specific issues in proper engineering models that can be understood by the new operators and users, who are not from technical engineering fields.
Fig. 3 summarizes these aspects by outlining a general approach for designing service robots by considering specific aspects and challenges in the areas of cultural heritage for the activities of identification, restoration, and preservation. In particular, the main flow of the technical design activity is indicated in the central streamline, with reference to the data consideration of both the technical and non-technical aspects for technical constraints/issues, analysis of service operations and goals and system programming. Then, the final checks are worked out by the operators, restorers and robot users. Technical care in the design activity is indicated by the system design and operation planning. It is strongly influenced by aspects and activities that are grouped in the two lateral blocks concerning the interactions with human beings and environment, respectively, within the restoration frames. Each indicated item refers to the aspects that, even with non-technical concerns, must be included in the development of proper engineering models and problem formulation, as synthetically indicated in the box for task features and constraints. The reported list of topics in the lateral blocks is not exhaustive. However, it aims to outline the many different aspects that should be considered as useful guidelines for successful service robots in cultural heritage applications, with specific focus on restoration activities.
A flowchart for designing service robots for activity in the restoration of historical goods
Special emphasis has been indicated with relation to the acceptance of the operators, restorers and users, which will require a reiteration of the considerations of all of the aspects and the design process itself. The meeting of the scheduled aims can be verified both in terms of the accomplishments of the operation tasks and restoration characteristics with the prescribed constraints. These are given by the general considerations from Brandi's theory and specific needs from the specific interventions.
The conservation/restoration activities on historical architectonic and artistic goods are generally carried out by manual operations, with the help of assisting devices. In addition, the activity is differentiated in architecture and history of art, not only for the differences of the goods in terms of materials and structure types but also, for the scale of the dimensions of the objects that are under consideration. In general, these are buildings or large parts of architecture and small components of buildings or decorative details, as well as specific artistic products of history of art. As an important part of the identification and preservation activities, a survey aims to detect the status of the historic goods both in terms of the dimensions and details for different interests in an historical analysis. Thus, there are several aspects that may differentiate the activity in architecture survey and history of art. Nevertheless, there are also several aspects in common, mainly in the approaches for the activity in getting knowledge and experiences of the objects that are under examination in the survey and analysis activity. They are related to tasks that can often be repetitive for acquiring data from historic goods both in terms of dimensions and figures. These repetitive tasks are performed with the assistance of devices that help to achieve proper repetitions and acquisition data with regular characteristics. However, the assisting devices are often developed or even adapted for a specific case without a general frame of operation. Thus, each use can require several attempts. For non-technical operators, like restorers, who, in general, do not have machinery experience, there can be difficulties in understanding how to operate a service robot.
The proposed guidelines for designing and operating a service robot in the activities for analysis, survey and restoration can be organized with a general plan using the following outline, [4]:
First phase: the definition of the design requirements and operation characteristics for restoration tasks.
Second phase: the development of the design methodologies and operation strategies for designing mechanical structure and functionality of robotic systems that are dedicated for cultural heritage frames.
Third phase: the development of the specific operation strategies for the designed robotic systems in restoration activities.
Fourth phase: the construction of prototypes and test-bed applications.
Fifth phase: tests and validations with training of restorers/users on field.
The plan of each phase can be carried out with an activity that can overlap and even continue in the next phases, as described in the following. The cores of the design activity are phases 2 and 3, which, in principle, will have activity that is extended during the whole plan.
In particular, the engineering design process is strongly affected by the task features and constraints. These can be formulated from the consideration of survey and restoration operators and for the environment care, as is shortly indicated in the lateral boxes of the flowchart in Fig. 3. Safety issues are fundamental not only towards personnel operating/using those service robots but also, towards the environment and, particularly, the cultural goods that are the goals or frames of the service tasks. The design activity can be carried out through simulations and experiences to define the procedures for the analysis and determination of the operation strategies. Simulations can be carried out with numerical computations, laboratory experiences, within the different specific cultural frames and, finally, with integrated views.
A new application for service robots requires an investigation of the variety of situations and conditions in cultural heritage applications with the aim to design new suitable solutions and develop appropriate operation capability with user-oriented features for restorers in architecture and history of art. The restoration of fresco paintings is discussed as an illustrative example to show the feasibility of the above-mentioned considerations for the design and implementation of service robots in a specific application in cultural heritage frames.
Frescos are usually pictorial decorative artistic products that are produced on large surfaces like walls and dome surfaces. When they are damaged, the main problem for a proper intervention of restoration consists in the large damaged and missing areas that make it difficult to understand the fresco's original message. In other cases, although the aim of the scenes is still clear, the damaged and missing areas can be large or small but with a strong perturbation of the whole. Even when the damages or missing areas are small, but largely distributed, they can have a significant effect on the readability of scenes. In all of these cases, interventions are needed and they aim to restore the continuity or at least limit the interruptions. However, at the same time, they should not modify or be more relevant or impressive than the original remains, according to Brandi's theory.
In addition, frescos do not only have a decorative function. They also aim to be more relevant, for example, for devotion activity, memory or political purposes additional values are given to the cultural heritage. Thus, a restorer must evaluate both the historic-aesthetic aspects and functioning purposes of the pictorial representation with proper criteria and actions that must be decided for a plan of interventions. There are several methods for the reintegration of fresco paintings. The most used ones can be considered as rigatino or tratteggio, puntinismo and low colouring, [1, 7, 19, 22]. Each of these methods is aimed to restore the continuity of a figure but to make the restoration intervention evident. Nevertheless, their drawbacks can be identified in the fact that a final result much depends on the skill and expertise of the restorer in order to achieve a proper interpretation of the restoration effect. Rigatino or tratteggio (a sort of dashing) is performed by drawing short, thin parallel line segments to reconstruct the images and their colours. Similarly, puntinismo (a sort of dotting) makes use of small points that are near to each other. Low-colouring is used to fill the missed coloured parts with colours of a lower tone.
A team of restorers in Valencia has recently outlined a novel approach for reintegrating lost parts of frescos. Their outline uses printing reintegration by an inkjet printer in preliminary experiences for the restoration work of the fresco by the painter Antonio Palomino in the church of Santos Juanes in Valencia, [9, 10, 17]. The reintegration and its colour are decided on the basis of careful considerations of archival photos, which are available for the damaged fresco that is under intervention. These are based on the techniques that are available for computer-based image treatments both for image recognition and colour identification. Thus, the inkjet printing can be performed through temporary supports or even directly on the fresco surface as a printing of the image data through standard computer-printer transmissions. In particular, the inkjet printer will use colours that have been chosen by restorers after their analysis of the chromatic aspects and colour chemistry of the fresco that is under restoration, with Brand's conditions that the restored parts will be clearly recognized. It is worth noting that the inkjet printing can still be performed with restoration techniques of rigatino, tratteggio, puntinismo, or low-colouring, but with a relevant aspect to give a uniform result all over its application.
Preliminary studies and attempts were successfully carried out with experiences using informatics means, as reported in [13–16]. In particular, a procedure for fresco reintegration using inkjet printing has been conceived through a collaboration among teams of restorers from Technical University of Valencia, technicians from Hewlett-Packard, and consultants from ARSU PAER company.
The procedure has been set up with the following three steps:
The acquisition of images and corresponding computer-based elaborations.
The composition of the image to be restored.
The reintegration by inkjet printing.
After this, an acquisition of the image elaborations is carried out to reproduce the scene with all of the fragments and missing areas. Then, the restorers work on the restoration of the full scene by reintegrating the parts that have been previously composed. To do this, they use archived data and specific elaborations in image definition of the lost fresco segments. During the composition, the restorer decides that the adjustments, both in place and colour, are also in agreement with Brandi's theory. The last step relates to the printing of the defined parts using the inkjet printer. The lost portions of the frescos are printed on specific supports, which are finally installed as a reintegration of the fresco. It can be observed that the final result much depends on the proper interfaces among the equipment units, as well as on the data transfer, in order to achieve a reintegration that is decided by the restorers.
Characteristics for Implementing Inkjet Printing
Inkjet printers have rapidly evolved to become efficient solutions, both in printing performance and resolution. Thus, they can be properly used in restoration activities when they are correctly adapted to the conditions of a specific application. In this work, the aim is to use a conventional inkjet printer that can be properly located to act directly on a fresco surface. The location aim has been considered for a robotic system with proper capability of locomotion and then manipulation of the printing end-effector. These capabilities depend on the parameters and actions in the inkjet printing, which are analysed as constraints for a proper positioning of the printing device when they are installed on a robotic mobile system.
The considered problems for a prototype design are related to all of the parameters and physical factors that influence a printing process. In addition, it is a great challenge to adapt these printers to the conditions of a scaffold working environment and wall surfaces morphology with irregular curves. Sensing and actuation in robotic systems must be designed with the operation features that avoid damaging the fresco or printing system.
Ink injection in printers occurs during the lateral movement of the print head. When there are irregularities in the printing surfaces, it is necessary to adjust the distance between the surface and print head. The distance of the printer head from the ink receiving surface is determined by the speed of the printer carriage, Fig. 4, but never exceeding a distance of 2 mm. The distance d1 refers to an optimal setup to achieve a normal printing on papers. In the case of fresco printing, the distance must be modified depending on the kind of mortar. This increase to d2, d3, or d4 affects the final deposition of the ink drops. The lateral movement of the carriage along the guide permits the injection of ink both in the outward and return motion, Fig. 4b). The carriage speed W depends on the print quality, which is usually inversely proportional to the speed. With a distance d1, the ink drops are well deposited in both directions. However, when the distance between the head and surface increases to d2, d3, or d4, there is an increased mismatch in the overlap of each deposition. This phenomenon will affect the sharpness and image quality so that significant colour vibrations can be appreciated at a certain distance.
Position of ink droplets as a function of distances d1, d2, d3, and d4 and printer head speed W
In addition, the effect of gravity is important because, according to the orientation of the jet tip, it will have a greater or lesser influence on the final position of the ink drops. Typically, in printers, the print is performed by directing their nozzles along the gravity direction (g +) for a correct projection and positioning of the ink drops, Fig. 5a). In this case, the path p is not so much modified by the increase of the distance d1, and the definition of the resulting image is better than in the case of Fig. 5b). If the head is oriented to the opposite direction of gravity, the mistakes in the final position of the ink drops will be greater as the distance increases. This is because the path p described by the drop is more strongly influenced, Fig. 5b).
Ink trajectory due to gravity influence: a) positive gravity b) negative gravity
If the printer head is located with the injection direction perpendicular to gravity and the printing is against a plane parallel to the nozzle surface, bleeding during the drying of drops will occur due to the pull of gravity, Fig. 6.
Influence of speed and distance run of drops on the drop position and ink bleeding
However, when the distance between the tip and surface is increased to d2, the gravity will also influence the path p, which is described by a drop, Fig. 6b). As a result, in addition to the bleeding, there will be an error in the position of the drops. The bleeding also results in an increased interference of the drops with the surrounding ones. As such, certain chromatic aberrations can be produced as a detectable error in the form of colour noise and loss of image resolution. The relevance of this phenomenon depends on the head position and capacity to receive support in absorbing and drying the colour drops. This is why, in practical implementations, mortars for mural painting reintegration are used, with a suitable porosity and absorptive capacity to rapidly dry inks.
Another factor that influences ink penetration is the ink pressure by which the head can inject. For a distance d1, the printed area will be different, depending on the ink injection pressure P1 or Pd, Fig. 7. The ink spread is wider at lower pressure Pd and the ability to penetrate the support will be lower. It is very important to keep this factor in mind because, in the event that the wall surface requires increasing the head distance in order to avoid damages to the head, more pressure is needed to allow the ink to properly reach the surface.
Effect of the pressure on the printing area (P is pressure with P1 > P)
In addition to the above-described factors, the ink nature and condition of the receiving surface have a great influence on the final quality of the printing. For this reason, it is fundamental that a robotic system can adapt the printer head position to optimal conditions for proper printing, with the above-considered factors. Referring to Fig. 8, for a correct position of the drops, it is necessary that the wall surface is exactly parallel to the head nozzle. If the printing surface is inclined to an angle θ with respect to the orthogonal posture, there will be a deformation of the drops and consequently of the image.
A model for inkjet position: a) with main parameters; b) shape of drop deposition. (W: head velocity; a: ink cone angle; V: ink volume; p: main direction of ink trajectory; d: distance from the surface; a and b: surface orientation).
A final specific goal of this project is the implementation of a prototype for the restoration of frescos in the church of Santos Juanes in Valencia. Here, an initial reintegration phase has been worked out by transferring reconstructed images after being printed with inkjet systems in a laboratory. The frescos that were painted by Antonio Palomino are in a dome surface, Fig. 9. In 1936, over 60% of the dome's surface was damaged by a fire.
The case of study for implementation of the proposed robotic system in the dome of the church of the Santos Juanes in Valencia: a) location of the damaged frescos, b) painting's position with respect to scaffolding
A robotic system has been designed for an inkjet commercial printer to implement specific fresco reintegration on mural surfaces in curved walls. In order to achieve this goal, a robotic system has been designed with the capability to adapt to the different working environment conditions and wall surface geometries. To do this, it independently operates with its actuators and sensors to determine a suitable robot position with respect to the environment and original painting. The purpose of the robotized system is to work out direct fresco reintegration in agreement with the criteria that are chosen by the restorers. These criteria aim to facilitate and substantially reduce the working time, and ensure quality uniformity, consistency and printing accuracy.
The robot has a total size of 1,900 × 1,000 × 1,000 mm in order to be movable within the scaffold structures. The main structure works as a support. It is composed of aluminium rods with suitable connections of the same material in a modular mechanical design for a fairly easy change of the overall size, Fig. 10.
The modular structure of the proposed robotic system with printer: a) with different tall size; b) a 3D view
The design of the robotic structure and simulation of its operation have been elaborated in SolidWorks frames. In particular, the CAD design has been worked out in detail due to the cooperation of the engineers and restorers, not only to achieve a synergy of ideas but also, to develop a solution that can be easily understood by the restorers and art historian operators. The simulation has been worked out to check the feasibility of the operation and to characterize the performance of the robot system. The results that are shown in Figs. 10 to 16 are aimed to draw the structure solution and to characterize the operation feasibility. The design of the robotic system is characterized by the mobile elements, which are actuated by linear actuators from market solutions. The overall weight of the robotic system can be estimated as 30 Kg, whereas the structure that is made of light material, like aluminium alloy, weighs about 20 Kg. The printer and its carriage system is about 3 Kg and the other equipment (PC, batteries and electric devices, etc.) that are located at the structure base can be estimated to weigh 7 Kg. The architecture of the robotic system looks like an actuated scaffold as per the mobile structure. Meanwhile, the robot features are performed by servocontrolled actuators whose action is under an intelligent supervision. This is achieved through sensors data within a proper operation programming of the locomotion, printer motion and printing.
In addition, the robot system has suitable mobility in order to change the printer position with respect to the printing surface. It is very important that the printer head never crashes against the wall surface and maintains a prescribed distance from the printing surface during the reintegration operations. This can be achieved by providing the robotic system with sensors that are specifically devoted to the control and monitoring of the head distance from the printing surface. Together with these sensors, the robot structure will have a shape that easily permits a proper head position for the dome surface. Thus, these are the constraints that gave the shape of the robotic system, as in Fig. 10.
Even in the case of flat surfaces, different conditions can occur, for example, when the wall is sloped in a vertical direction. In this case, four infrared sensors can be used to see the surface of the wall that is tracked with a point-to-point strategy. In addition, each wheel is equipped with a motor that can adjust in height and move the structure to adapt it to the wall slope, Fig. 11. In the case of a sloped area A, a sensor detects the distance d1 and the height at the r wheel will be adjusted to the wall inclination. In the case of the wall B, the sensor will detect the new distance d2 and orders the height and position modification to the motor of wheel r. Similarly, other sensors can be placed along the robot structure to detect the wall slope and perform robot adjustments in several different situations.
In order to adapt the robotic system to curved surfaces of a dome, the printer head will be installed on the structure with a curvature that is similar to the receiving surface, Fig. 12a). In order to change the position of the curved guide where the printer is installed, electrical pistons are activated with the aim of driving the printer head close to the surface of the dome Fig. 12b). The curved rods of the robotic structure have a fixed profile and they cannot be modified in shape. Thus, adjustment towards the dome surface can be achieved by the simultaneous action of the pistons supporting the curved structure and motors of the wheels. As such, the curved structure can be close to the tangent surface of the dome.
Simulation of a robotic system operation to fit a sloped surface activating a single actuator
Simulation of the robotic system operation adjusting it to the dome surface with the activation of a single actuator: a) 3D view; b) lateral view
Four sensors can be used to detect the proper positioning of the robot frame carrying the printer. These can be used to control the motion of the system to reach a proper distance between the printer head and printing surface. At the bottom of the robot structure, a weight has been placed, as shown in Fig. 10. This gives greater stability to the system and prevents lateral movement of the carriage, which may affect the correct positioning of the robotic system. Taking advantage of this weight requirement, all of the equipment for controlling the system is placed there.
The above functioning has been simulated with the designed system to show the feasibility of the above-mentioned solutions. The total duration of the simulated operation for the case of vertical walls in Fig. 13a) takes about 30 seconds, among which the first 10 are used to move the robotic system to the wall. This motion is obtained by activating the wheel motors that are stopped when the structure is at a certain distance from the wall, which is detected by sensors. Once the system is located in the correct position and distance against the wall, the operation of printing starts. This works for 20 seconds by moving the printer head along the printing surface. On sloped walls, the sensors detect the surface distance that is maintained by adjusting the head position through the height of the rear driven wheels. In the second part of the simulation, the wall is perpendicular to the ground and the robotic system again adjusts the position of the print head. The overall simulation is summarized in the snapshots in Fig. 13a).
In the case of a robotic system working against the curved surfaces, the displacement of the whole robot is also obtained by the driven wheels that are at its base. However, unlike the previous case, the secondary curved structure at the top is responsible for keeping the printer close to the dome surface through the action of electrical pistons, Fig. 13b). In order to correctly locate the printer carriage at an initial position, four infrared sensors are attached on the carriage extremities. These are used to measure the correct position of the ink nozzle so that a corresponding piston will be activated. Once the printer is positioned in the right place, the printing starts moving the printing carrier along the curved structure. If necessary, depending on the distance variations from the wall, the electrical pistons are activated to adjust the top curved structure to the dome surface. The ink injection carriage makes the same lateral movement as in the previous cases. The overall simulation is summarized in the snapshots in Fig. 13b).
The above-mentioned simulations have been worked out using SolidWorks softwareσ; and CosmosMotionσ; tool box. The numerical results can be understood as a characterization of the operation feasibility and restorers can also use them for programming the use of the system. The computed features of the robotized system are the position, acceleration and speed of the printer head. The operation has been simulated by taking into account the situations of the ink injection, which have been discussed in section 5. The computations results during the simulation for the case with a wall are represented in Fig. 14 as plots of displacements as a function of time. The complete simulated operation has a duration of 30 seconds in two parts:
Performance simulation sequence of the robotic system for tasks against: a) sloped wall; b) curved vault surface
The first 10 seconds when the robot only moves in the direction of X axis to reach the wall and be placed at the correct distance using the infrared approximation sensors. This shift is represented in animation sequences, Fig. 13a) and b).
The second part, with a duration of 20 seconds, is composed of two movements of the printer. Along the Z-axis, there is a downwards motion of the printer carriage, which is coordinated with the displacement along the Y direction so that the printer can span the printing surface, Fig. 14a).
The simulated operation is similar for the case of printing against a dome surface, as shown in Fig. 14b).
Motion of the printer carriage in the robotized system: a) for vertical wall; b) for dome curved surface
In Fig. 15, the numerical results are reported with motion characteristics. The first 10 seconds refer to the movement of the structure towards the wall. This is represented by a parabolic output of the velocity magnitude between the start and final conditions. After stopping the system in front of the wall, the printing process runs with the carriage movement along the Y and Z axes. This is indicated in the plots where the parameter evolution switches to an oscillatory motion along the Y axis, which represents the alternating motion of the carriage. Typical motions of the ink jet printer require about 1 second to complete one line printing with a length of about 600 mm. The ink jet carrier moves at a speed of about 100 mm/s. The highest acceleration has been estimated to equal about 4 m/s2. Higher accelerations can produce reduced accuracy of the printing process. The motion is obtained with the proper smooth action that is suitable for ink injection with the characteristics that avoid the problems in section 5. The simulation has considered the gross motion of the printer head and not directly the ink drop motion. Nevertheless, this is considered through the constraints on the distance of the ink nozzle from the printing surface and the carriage velocity while printing.
Numerical results of a simulation in terms of the speed of the printer carriage: a) magnitude; b) x-component; c) y-component; d) z-component
In addition, alternatives were designed for a system adjustment with manual operations for the cases of irregularities in the scaffolding and for different low-cost solutions. One of the possibilities has been designed with feet that a suitable piston system can adjust to the surface plane, Fig. 16a). Another possible solution is shown in Fig. 16b), with a manual system that is composed by screwed feet, which are adjustable singularly with a proper screw motion.
Alternative solutions of low-cost adjustment and stability of the robotized structure without actuators: (a) solution with pistons; (b) manual solution
In summary, specific characteristics of the proposed robotic system may allow an automatic and intelligent operation with flexibility and versatility of the operation in a multitude of tasks that have been developed for fresco reintegration. The advantages of the proposed robotization can be summarized in the design of a low-cost system with a flexible operation, which can be adjusted to a specific implementation for different restoration situations and tasks.
Experiences have been carried out to develop the proposed robotized inkjet printing from lab testing up to the preliminary application in a restoration task, [23]. The results and attempts are summarized in Figs. 17 to 22.
The feasibility of the technological solutions has been investigated with tests using an adaptation of a HP printer, Fig. 17, with successive arrangements for its installation in a designed scaffold-like frame as a robotic structure, Figs. 20 to 22. In particular, Fig. 17 shows a laboratory layout of the printer under operation with a table for testing the quality of the inkjet printing, according to the analysis of the parameter influencing the colour reintegration effects in section 5. The result that is reported in Fig. 18a) and b) shows the influence of the distance of the inkjet nozzle from the printing surface. From a comparison with the original image of Fig. 18c), it is noted how efficient the inkjet printing can be when performed at the proper distance of the inkjet nozzle and how improper image resolution can be obtained when the distance is too large, as the image in Fig. 18a) shows. The lab testing has also proven the operation efficiency and has checked the effects of the printer parameters with a practical characterization. This serves as a validation of the analysis in section 5.
A laboratory test of fresco printing with the inkjet printer
After the tests and trials at the laboratory levels, the technique of inkjet printing has been applied to the reintegration of a fresco in the dome of the church of Santos Juanes in Valencia, as introduced in section 4. An example of the successful application with a procedure that fully respects Brandi's theory of restoration is reported in Fig. 19. After an image study and reconstruction of the missing areas in the fresco painting, the reintegration has been worked out on a suitable paper panel in a restoration laboratory. Then, the paper panel has been manually transported at the restoration area, Fig. 19a). Here, a manual operation has again achieved the correct placing of the panel, Fig. 19b) for a reintegration of the fresco, Fig. 19c).
Test results of ink jet printing as function of printer head distance from the printing surface: a) at 5 mm; b) at 2 mm; c) original figure
In Figs. 20 to 22, attempts are shown to build the proposed robotic system with a structure that can be used within scaffold frames in restoration areas, like the one that was experienced at the church of Santos Juanes in Valencia. In particular, Fig. 20 shows a first arrangement of the carriage bringing the inkjet printer head in a rod frame with fairly simple actuation on the top of the frame. In this solution, the prototype was used to check the operation feasibility and transportability of the system.
A fresco reintegration with inkjet printed paper at the dome of church of the Santos Juanes in Valencia: a) a printed fragment; b) its integration in the fresco under restoration; c) final result
A laboratory prototype of the robotic system with the inkjet printer
Fig. 21 shows a test of inkjet printing on a paper panel in an inclined plane using the characteristic motion of the carriage, as designed in the proposed robotic systems. The test is made at a full-scale to also investigate the speed limits that could give good results. This kind of characterization is still under investigation as a function of the final robotic solution that is under development. The scaffold-like structure for the robot frame has been tested with preliminary solutions like the one that is shown in Fig. 22. Here, the efforts were directed towards a modular design of the frame that could be fairly easily adjusted to the geometry of the printing surface, much more than in the proposed robot solution. Nevertheless, the preliminary prototype of the robot rod frame in Fig. 22 has confirmed the feasibility of the robot design when the successful operation of the inkjet printing is also considered with the properly adjusted carriage motion of the above-mentioned tests.
A Laboratory test of the inkjet printing on an inclined wall surface
A curved structure for the robotic frame in a laboratory experience
This paper has presented the results of a collaboration between art restorers, survey architects and robotics engineers in order to design a structure and operation of a robotic system for the restoration activity in reintegrating the colour parts in a fresco on the walls of a cathedral. The problems of fresco restoration have been considered with regard to colour reintegration with inkjet printing and using a robotic system both for robotizing the process and facilitating the tasks on the surfaces, which can be inconvenient to human operators. Additionally, the robotic solution has been designed to ensure better precision and repeatability in a well-defined restoration process, which is decided by restorers. The feasibility of the proposed design and its operation have been simulated with reference to a real case of study for a restoration project of frescos in the cathedral of Santos Juanes in Valencia, Spain. Laboratory experiences and a preliminary prototype are presented to discuss the feasibility of the proposed design. Satisfactory results are obtained that give the hope for future implementation and development both of the designed system and multidisciplinary collaboration.