Sage Journals: Discover world-class research

Abstract

The excellent performance of fruit and vegetable picking robots is usually contributed by the reasonable structure of end-effector and recognition–localization methods with high accuracy. As a result, efforts are focused on two aspects, and diverse structures of end-effector, target recognition methods as well as their combinations are yielded continuously. A good understanding for the working principle, advantages, limitations, and the adaptability in respective fields is helpful to design picking robots. Therefore, depending on different grasping ways, separating methods, structures, materials, and driving modes, main characteristics existing in traditional schemes will be depicted firstly. According to technical routes, advantages, potential applications, and challenges, underactuated manipulators and soft manipulators representing future development are then summarized systematically. Secondly, partial recognition and localization methods are also demonstrated. Specifically, current recognition manners adopting the single-feature, multi-feature fusion and deep learning are explained in view of their advantages, limitations, and successful instances. In the field of 3D localization, active vision based on the structured light, laser scanning, time of flight, and radar is reflected through the respective applications. Also, another 3D localization method called passive vision is also evaluated by advantages, limitations, the degree of automation, reconstruction effects, and the application scenario, such as monocular vision, binocular vision, and multiocular vision. Finally portrayed from structural development, recognition, and localization methods, it is possible to develop future end-effectors for fruit and vegetable picking robots with superior characteristics containing the less driving element, rigid–flexible–bionic coupling soft manipulators, simple control program, high efficiency, low damage, low cost, high versatility, and high recognition accuracy in all-season picking tasks.

Keywords

End-effectors picking methods underactuated manipulator soft manipulator target recognition

Introduction

As a crucial role of the agricultural production chain, fruit and vegetable picking usually depends on a lot of manpower, material resources, and time due to the strong seasonality, high labor intensity, and high cost.¹ Following the development of economy, the rural labor force is gradually transferred to other sectors of society in some developing countries, and the world is experiencing the problem of an aging population. The shortage of labor force will become a fact.² Therefore, traditional picking methods would not meet requirements of modern agricultural production. Owing to improvements in labor productivity, picking efficiency, cost, and so on, picking robots have been widely adopted in fields of fruit and vegetable productions.³ In order to develop picking robots in agricultural fields, international scholars or engineers have proposed their schemes, thus yielding so many picking robots and end-effectors in respective fields such as the strawberry,^4
–6 citrus,^7
–9 sweet orange,^10
–12 apple,^13

–18 kiwi fruit,^19
–21 tomato,^22

–28 eggplant,^29
–31 and cucumber^32

–35.

Especially working as the terminal element in the direct contact with fruits and vegetables, the end-effector is decisive to obtain the excellent performance of a picking robot relying on the universality, practicability, picking efficiency, and intact rate.³⁶ For satisfying requirements of picking tasks, end-effectors with different grasping and separating methods have been extensively utilized in the agricultural field.³⁷ Of those, three types of end-effectors with the sucking, clamping, and sucking-clamping functions are widely adopted in fields of picking strawberries,⁴ apples,^13

–16 tomatoes,^22

–27 and so on, which characterizes a picking way of grabbing. After grasping, separating methods should be considered. Typically serving as main separating components, end-effectors assembling rigidly and flexibly separated elements are adopted to pick the apple,^17,18 kiwi fruit,^19,20 citrus,^9,10and others.^{5,28
–30,33}

It would report from existing literature that a lot of end-effectors have been successfully used to pick corresponding fruits or vegetables. However, almost applied instances have presented limitations in the picking efficiency, damage rate, flexibility, recognition accuracy, and so on, and these disadvantages are usually caused by two aspects. For example, manipulators should be more flexible and present the complexity at the design stage since fruits and vegetables are easily damaged, scratched, and adhered, especially grasping hardly. The unstructured or semi-structured environments in agriculture are evident. Owing to differences in morphological characteristics of fruits and their growth environments, it is also not easy to identify and locate fruits quickly, thus affecting the picking efficiency. More importantly, future development is not quite clear although so many schemes have been proposed, tested, and applied.

Therefore, structural characteristics and target recognition methods for end-effectors of picking robots will be systematically summarized so that the future direction of structural development and recognition methods matched with them are clearer. The details will be portrayed in the following sections gradually.

Characteristics of end-effectors considering different grasping and separating ways

Besides considering the appearance, structures, and mechanical characteristics of stems of fruits or vegetables, the realization of picking methods is particularly important at the design stage.³⁸ Specifically, fruits and vegetables should be grabbed by an end-effector firstly, and then a separating component is utilized.³⁹ For achieving a grabbing function, diverse clamping mechanisms have been designed according to requirements of different fruits and vegetables, thus yielding sucking, clamping, and sucking-clamping types of end-effectors.³⁷ Also, a suitable separating manner should be considered by the binding force at the junction of the fruit and its stem.^37,40 For example, a flexible mode of separation can be adopted to pick the apple, kiwi fruit, and tomato production due to the relatively small binding force between fruits and their stems. For the relatively large force at the junction, rigid separation methods depending on scissors, rotating blades, and the cutting element of laser are usually utilized in picking operations of the citrus, cucumber, eggplant, and others.⁴¹ Specific methods and their technical characteristics will be presented in the following sections.

Characteristics of applied instances with different grasping methods

End-effectors with a sucking function

Generally generating negative pressures by a vacuum suction cup or a tube, an end-effector will absorb fruits directly. Such a grabbing method yields lots of elements called suction effectors.³⁷ Owing to the fast-picking speed and a low damage rate, these effectors are widely utilized to pick a single fruit with a spherical shape (hemispherical shape) and lightweight. There are successful instances although their suction force is also limited. For example, Naoshi Kondo et al. in Japan designed a suction effector for picking strawberries, as shown in Figure 1.⁴ With the suction force produced by the draft fan, a suction cup directly sucks fruits into the end-effector. However, this effector cannot judge the maturity of strawberries. It is also difficult to adjust the large suction force. Therefore, immature fruits may be inhaled and picked together by such an effector.⁴² As portrayed in Figure 2, another suction type of effector was proposed for the citrus in Japan,⁸ and fruits can be absorbed and fixed by a pneumatic suction cup. Fruit stems were cut off by rotating the circular cutter for one cycle. Also depending on the suction force of the suction cup, it is suggested to pick the citrus with small sizes and weights.⁴³ Johan Baeten et al. developed a suction cup type of an end-effector for picking apples in Belgium.¹³ As shown in Figure 3, the cup was manufactured by the silica gel, which is beneficial to reduce mechanical damage greatly. Essentially, apples can be sucked by a negative pressure generated by a vacuum pump. After this suction behavior, fruits will be separated from stems through rotating and swinging the mechanical wrist. It is concluded from existing literature that its picking efficiency could be raised to 80%,^13,17 thus satisfying a range from 40% to 86% in a filed of picking fruits.⁴⁴ However, it is also easy to suck branches and leaves into the suction cup.

Figure 1.

An end-effector for picking strawberries.

Figure 2.

An end-effector for picking citrus.

Figure 3.

An end-effector for picking apples in Belgium.

Also considering the suction effector, a picking robot for apples was designed by Abundant Robotics in America, as depicted in Figure 4.¹⁴ Evidently, apples will be transported to a collection apparatus through the suction inlet and pipelines if the suction force provided by a picker is large enough. Adopting a vacuum technology rather than the mechanical arm, the picking efficiency of such a robot is higher.⁴⁵

Figure 4.

An end-effector for picking apples in America.

As shown in Figure 5, an end-effector for the hanging line cultivated tomato was also established by Feng in China. Specifically, the airflow with the negative pressure provided by a vacuum generator can suck the tomato hidden in leaves,²² and then fruits will be captured into the sleeve as long as the telescopic cylinder extends out of the sleeve easily. Therefore, the achieving rate is up to 83.9% by such a picking method. Unfortunately, it also depends on high requirements for light at the picking stage. The successful rate will be decreased if the light is quite weak.⁴⁶

Figure 5.

An end-effector for picking tomato in China.

End-effectors with a clamping function

Depending on a gripper, other end-effectors different from suction types are actively researched by international scholars. Usually, the greater clamping weight and the more stable behavior are also achieved if more fingers in a gripper are considered.^37,47 However, the corresponding driving system will be more complicated. Therefore, scholars try their best knowledge to design different fingers according to fruit sizes, shapes, and weights.⁴⁸ Of those, Kondo N et al. in Japan developed an end-effector for picking tomatoes with upper and lower fingers, as shown in Figure 6.²³ Upper and lower fingers are opened firstly to wrap the tomato if branches of the tomato cluster are detected. Lower fingers are then used to clamp stems and upper fingers cut them orderly.⁴⁹ Davidson et al. in America proposed a picking robot utilizing the end-effectors with three underactuated fingers.¹⁵ As shown in Figure 7, two joints are assembled in each finger, and their artificial tendons provide the power. The effector can pick fruits by imitating behaviors of human hands. Therefore, such an effector not only generates the grasping force on fruits through pulling and swinging actions but also easily separates fruits from their stalks.^50,51

Figure 6.

An end-effector for picking tomato designed by Kondo N et al.

Figure 7.

An end-effector for picking apples developed by Davidson et al.

As portrayed in Figure 8, a clamping scheme of end-effector for picking apples was also established by Zhao De-an et al. in Jiangsu University of China.¹⁶ In such a clamping structure, two fingers are adopted. Damage to the apple could be avoided by designing an arc surface stuffed by the rubber-sponge material and arranging a pressure sensor in these fingers.⁵² An end-effector for picking the kiwi fruit was also considered by Rong Hao in Northwest University of Agriculture and Forestry Science and Technology, as shown in Figure 9.¹⁹ This effector adopts a fruit holder and a stem holder, and the lead screw nut can be moved to the middle position by the driving function of a stepping motor. Therefore, combining these abilities, the fruit and its stem can be clamped well. After fixing the fruit and stem, the movement of the lead screw nut is also achieved by the driving effect of a motor, thus obtaining the forward movement of the stem holder and cutting them. However, the precise positioning function is necessary, and many disadvantages of this effector are also evident, such as the relatively low accuracy and the poor adaptability for grasping.

Figure 8.

An end-effector for picking apples designed by Zhao De-an et al.

Figure 9.

An end-effector for picking the kiwi fruit proposed by Rong Hao.

End-effectors with the sucking-clamping function

Combining characteristics of above two grasping ways, another type of effector always includes the suction element and the holder. At the picking stage, fruits will be sucked first, and then the holder is adopted to clamp them.^37

–40 Owing to these considerations, diverse effectors have been developed. For example, containing four flexible fingers and a suction cup, an early actuator was proposed by Okayama University in Japan, as depicted in Figure 10.²⁵ Such an arrangement of structure can absorb fruits into soft fingers, and then these fingers are forced to bend if a certain tension is achieved by pulling the cable, which obtains high accuracy for grasping and also reduces the efficiency. Picking tomatoes, a manipulator with four fingers was developed by PP Ling et al. in Ohio State University, as shown in Figure 11.²⁶ With a compound vacuum effect, a suction cup is installed in the palm of fingers, thus absorbing fruits by generating the negative pressure of an air pump. Completing the suction, the cable and tendon are driven by a stepping motor so that fruits can be further clamped. After picking, it can be reset by a torque spring easily. Therefore, such an actuator characterizes a good universality and an excellent adaptability to different sizes although its control method is relatively complex.²⁶

Figure 10.

An end-effector for picking the tomato proposed by Okayama University.

Figure 11.

An end-effector for picking the tomato proposed by Ohio State University.

Utilizing a vacuum suction element and fingers, a robot for picking tomatoes was designed by Jizhan Liu et al. in Jiangsu University of China, as shown in Figure 12.²⁷ Absorbing fruits and then pulling them back, this effector can separate the single tomato from its cluster. Ending these actions, the tomato will be clamped well by wrapping fingers. While there are above evident advantages, it is possible to damage them because of the friction (or collision) between fruits and the suction cup.⁵³ Also, excessive suction may damage surfaces if tomatoes are ripe.

Figure 12.

An end-effector for picking tomatoes proposed by Jizhan Liu et al.

Adopting the sucking, clamping, and cutting elements, an end-effector for picking navel oranges was designed by Liming Xu et al. in China Agricultural University, pictured in Figure 13.¹⁰ This end-effector is mainly composed of an adsorption device, a gripping device, and a shearing element. The suction cup will be moved to a position near the citrus by the pushing behavior of the lower linear cylinder, further grasping the fruit by the clamping device. The controlling methods must be more complicated and the cost will be also raised although the fruit could be grabbed accurately.

Figure 13.

An end-effector for picking citrus proposed by Liming Xu et al.

Separating methods and their instances

Besides decisive functions for grasping methods, separating methods are also crucial to obtain appropriate structures and the satisfactory performance.^40

–45 While some separating methods have been demonstrated in above actuators, there is no systematic analysis about separating methods and characteristics of their instances. Therefore, different separating instances will be further explained in the next sections gradually.

Rigid separation type

Depending on the scissor, blade, saw, laser, and others, fruit stems can be further cut off by some devices after grasping fruits, and such a separating way is a so-called rigid one with them.⁵⁴ Adopting this cutting method, so many components have been developed to separate fruits from their stems actively.³⁹ Of those, assembling the scissors, mechanical fingers, color camera, and ultrasonic sensor together, an end-effector for picking the eggplant, as shown in Figure 14, was yielded in Japan. At the picking stage, the fruit position can be accurately recognized by visual global and ultrasonic local detection.²⁹ The end-effector reaches the predetermined position with the help of a mechanical arm after detection. Continuously, fruit stems will be clamped by the mechanical finger and cut off by scissors.⁵⁵

Figure 14.

An end-effector for picking eggplant proposed in Japan.

Also adopting a mechanical cutting method, lots of effectors have been developed. For example, including three active fingers, three passive fingers, and a cutting element, a noncontact effector for picking strawberries driven by a cable was proposed by Ya Xiong, as shown in Figure 15.⁵ Six fingers will be opened simultaneously and swallow strawberries. Composed of two curved blades, a cutting component can rotate quickly to cut off the stem. Finally, strawberries will fall into a container inside the holder. Therefore, a shorter time, about 7.5s,⁵⁶ has been achieved in such a continuous way.

Figure 15.

An end-effector for picking strawberries from Ya Xiong.

According to the principle of bionics and the mechanical cutting method, an effector for picking citrus was designed by Shun Fu of Chongqing University of Technology, as depicted in Figure 16.⁹ Specifically, such an effector contains a suction cup and a shearing element with a symmetrical hinge four-bar mechanism, and fruit stems can be bitten and cut off by the upper and lower blades.

Figure 16.

An end-effector for picking citrus proposed by Shun Fu in Chongqing University of Technology.

Introducing the occlusal amputation achieved by the blade and a fruit catcher with the dentation structure, a new picking effector with the tubular structure was designed by Guoli Li in Nanjing Agricultural University of China.¹⁷ As shown in Figure 17, the toothed fruit catcher will be rotated so that apples can be pulled into the catcher, thus hugging stems. Evidently, such an actuator presents several advantages such as the good universality, high efficiency, and low damage rate. However, it is not a better one for fruits hidden in branches and leaves as well as clustered fruits.⁵⁷ For round fruits such as apples and pears, a picking actuator also combining a mechanical cutting way was established by Shukui Han in Beihua Institute of Aerospace Technology.^58,59 As depicted in Figure 18, a stepping motor is worked as the driving unit, and the lead screw nut is the transmission mechanism. The grabbing and shearing devices will be driven by such a transmission mechanism so that grasping and cutting behaviors would be carried out orderly and continuously. Therefore, the relatively simple structure and the lower difficulty in control also brings more opportunities for application.

Figure 17.

An end-effector with the tubular shape proposed by Guoli Li in China.

Figure 18.

An end-effector for picking round fruits designed by Shukui Han.

Based on the hot cutting method, an end-effector was developed for picking cucumber in Netherland, as portrayed in Figure 19.^32
–34 The vision system of this actuator actively detects positions of main stems, and the electric hot knife can surround a main stem from three directions. Also, a V-type guider on the knife will slide upward along the main stem. Completing these programs, the current will be turned on at the moment of contact between a fruit stem and the knife, thus generating the appropriate heat to burn the stem. Such a thermal cutting method is useful to inhibit the bacteria on the cutting wound. However, more requirements for the length of fruit stems and spaced widths must be considered. Therefore, it is also difficult to achieve a satisfactory result.

Figure 19.

An end-effector for picking the cucumber presented in the Netherland.

Flexible separation type

Besides above schemes for the relatively large force of cutting stems, some stems will be separated from fruits by slightly pulling, folding, screwing, rotating, overturning, and other actions.^37,39,54 Generally, this way is the so-called flexible separation which is suitable to pick the apple, kiwi fruit, tomato, and others.^18,20,28 For example, with the suction and torsion ways, American scholars such as Schertz and Brown had developed an end-effector for picking citrus in 1968.^7,60 The fruit can be actively absorbed into a rubber hose by this initial robot, and then the stem of a fruit will be further cut off through rotating the wrist. While the application for picking the citrus was achieved, such a robot did not meet requirements for different sizes of fruits.

Also considering the rotation, the FFROBOTICS company (Israel) designed a robot for picking apples in 2017, as shown in Figure 20.^18,61 An underactuated gripper assembled in the effector will be rotated 90° after grasping the apple, further cutting off the stem. In 2020, this company proposed another machine for picking apples combining many mechanical arms. Evenly locating at left and right sides, 4–12 mechanical arms can pick apples synchronously, and the gripper with three fingers is utilized to grasp and separate fruits from stems.⁴⁵ Such an arrangement will achieve a high efficiency, such as 10,000 apples per hour.

Figure 20.

An end-effector for picking apples designed by the FFROBOTICS company.

Depending on the rotation of the asymmetric four-bar mechanism, a harvesting robot for the kiwi fruit was yielded in the New Zealand, as shown in Figure 21.^20,62,63 In addition to the asymmetric four-bar mechanism, such a robot contains four special mechanical arms and each of them has an end-effector. Receiving a working order, fruits will be clamped and rotated upward around fruit stems by these fingers, thus picking fruits from their stems slightly. Completing these programs, picked fruits will slide into a basket through bellows as long as the fingers are opened cooperatively, which is helpful to achieve the continuous actions without the additional placement and improve the efficiency greatly.

Figure 21.

The end-effector for picking kiwi fruit developed in the New Zealand.

In Beijing University of Technology of China, a picking robot for tomatoes under the greenhouse environment was proposed by Lili Wang, as shown in Figure 22.^28,64 Specially, with the bionic design of three finger types of a claw, this end-effector is composed of the wrist motor, a driving motor of the finger, mechanical fingers, the pressure sensor, and the silicone pad of the palm. Fruits can be grabbed and twisted by the hand claw, thus achieving the separation. Combining above components and methods, it usually takes an average of 15s to pick a single tomato, and the success rate is greater than 86.7%.⁶⁴ However, the grasping force of the hand claw should be monitored during the entire picking program so that ripe tomatoes will not be damaged. Therefore, it will take more time and bring complex methods of control for picking delicate and fragile fruits.

Figure 22.

The end-effector for picking tomatoes designed by Lili Wang in China.

Summary of traditional structures

For obtaining a referenced scheme from existing instances if traditional structures are anticipated to further utilized in some fields and guide how we can improve performances, brief comparisons are given in Table 1.

Table 1.

Comparisons for partial instances.

Fruits or vegetables	Structural characteristics	Separating methods	Environment	Harvest success (%)	Cycle time (s)	Recognition accuracy (%)	Research Location and References
Strawberry	Suction type	Pull	Greenhouse	41.3	10–15	—	Japan, Naoshi Kondo⁴²
Strawberry	—	Cut	Lab	70	7.5	90	USA, Ya Xiong⁵⁶
Citrus	Suction type	Cut	Field	—	—	—	Japan, Naoshi Kondo⁴³
Citrus	Bite type	Cut	Lab	90	3	—	China, Shun Fu^58,59
Apple	Suction type	Twist	Field	80	8–10	—	Belgium, Johan Baeten¹³
Apple	Clamp type	Cut off	Field	77	15	—	China, Zhao De-an¹⁶
Apple	Finger type	Twist	Lab	56	7	—	USA, Davidson^50,51
Apple	Suction type	—	Field	—	1	—	USA, Abundant company¹⁴
Apple	Tube-type	Bite and cut	Lab	82.14	4.5	—	China, Guoli Li¹⁷
Apple	Clamp type	Rotate and twist	Field	—	2.7	85–94	Israel, FFROBOTICS company¹⁸
Tomato	Sucking-clamping type	Pull	Lab	91	15	—	Japan, M. Monta²⁵
Tomato	Sucking-clamping type	Twist	Lab	85	—	—	UAS, Peter P. Ling²⁶
Tomato	Clamp type	Cut	Greenhouse	50	15	90	Japan, Kondo²³
Tomato	Sucking-clamping type	Pull	Greenhouse	83.3	3	—	China, Jizhan Liu⁵³
Tomato	Suction type	Spin and twist	Greenhouse	83.9	24	—	China, Qingcheng Feng²²
Tomato	Finger clamp type	Twist	Greenhouse	86.7	15	99	China, Lili Wang²⁸
Orange	Sucking-clamping type	Cut	Field	94.28	1.76	—	China, Liming Xu¹⁰
Eggplant	—	Cut	Field	62.5	64.1	85	Japan, Hayashi²⁹
Cucumber	—	Laser cut	Greenhouse	80	45	82	Holland, IMAG³²
Kiwi fruit	Clamp type	Break	Field	51	5.5	76.1	New Zealand, Williams²⁰
Kiwi fruit	Clamp type	Pull	Field	86	17	—	China, Rong Hao¹⁹

Concluded from existing instances including above ones, traditional rigid manipulators are usually utilized in the field of picking robots. There are deficiencies in the volume, weight, flexibility, adaptability, picking performance, and so on. In the future, the weight reduction, flexibility, and less mechanical constraints should be considered if these schemes are expected to be improved. Soft manipulators with the variable stiffness and less driven elements are highly anticipated to replace these traditional schemes gradually. Separating methods are also variable for picking different fruits and vegetables. Current separating manners characterize limitations in the versatility and picking efficiency. There is also a challenge in combining rigid and flexible separation methods so that a good method and corresponding separation devices could be adopted to pick diverse fruits. The average cycle for picking fruits or vegetables is generally long. For example, it takes 17 s¹⁹ to harvest a kiwi, 45 s³² to pick a cucumber, and 1 min²⁹ to harvest an eggplant. The performance of picking robots working in orchards needs to be further improved, which requires an integrated development of high-performance manipulator and smart control methods such as the image processing, target recognition with the high accuracy, quick response, and low cost.

Developments of underactuated and soft manipulators

As above effectors, traditional schemes also bring partial deficiencies in the grasping and separating methods, hardly achieving a good universality. An end-effector with numerous excellent performances will represent the future development, such as the simple structure, good flexibility, low damage rate, and strong universality. Of those, these defects of traditional manipulators will be overcome or improved following the emergence of underactuated and flexible ones, thus achieving an ability to grasp and separate fruits adaptively. The following chapters will briefly introduce two types of manipulators.

Characteristics of underactuated manipulators and their applied instances

Owing to the increasing requirement for a good universality, lots of underactuated manipulators have been developed.⁶⁵ As a new manipulator, it can also work normally under dynamic constraints of its mechanism if the number of driving elements in the mechanical structure is less than that of the degree of freedom of the mechanism.⁶⁶ Therefore, compared with a fully driven manipulator, these underactuated ones characterize evident advantages, such as fewer driving elements, the low complexity of the control system, the greater adaptability, the stable ability of grasping, and the small probability of damaging fruits.⁶⁷

Combining these concepts, diverse underactuated manipulators have been proposed by international scholars and engineers. For example, as portrayed in Figure 23, the gripper of the underactuated robot for the international space station was designed by Gosselin in Canada.^68,69 Essentially, adopting a three-finger structure, the manipulator achieves 10 degrees of freedom, and two motors are utilized to drive them. Further depending on the mechanical limitation and the spring, the adaptive grab for the target is also feasible through the unpowered joint. Presenting the superior stability, such a manipulator can grasp vigorously and pinch small objects with fingertips. Therefore, this scheme applied in the aviation field has an important reference for designing a robot in the agricultural field.

Figure 23.

The underactuated manipulator for the international space station.

Considering the pneumatic and underactuated ways, underactuated fingers have been developed by V Begoc in Italy, as shown in Figure 24.^70,71 Specifically, there are two fingers controlled by four cylinders pneumatically, and each finger contains three joints. Also, six degrees of freedom in the circuit can be achieved by a driving source of air. Therefore, these driving fingers can be used to grasp objects with diverse shapes adaptively, thus meeting requirements for grasping or pinching. However, owing to relatively large sizes, it is also difficult to apply them in picking fruits, and improvements are necessary. Besides these schemes in other fields, many underactuated manipulators with a useful reference for picking fruits or vegetables are proposed, such as supervised descent method (SDM)⁷² and Kungliga Tekniska Högskolan (KTH)⁷³ manipulators.

Figure 24.

The underactuated manipulator proposed by V Begoc in Italy.

Directly designing for picking fruits or vegetables, an end-effector with the underactuated element was achieved by Bo Wei in Chongqing University of Posts and Telecommunications.⁷⁴ As depicted in Figure 25, this actuator can be utilized to pick citrus with different sizes and ovality adaptively through the grasping and deflecting control for three fingers.

Figure 25.

The underactuated manipulator for picking citrus proposed by Bo Wei in China.

Also considering the underactuated mode, a new actuator with multi-degrees of freedom was devised by Huanjun Sun in Sichuan Agricultural University.⁷⁵ As depicted in Figure 26, fingers can be opened and closed by controlling a single motor, and it presents remarkable characteristics such as the reliable mechanism and a simple control method. However, the grasping force is small, which is not helpful to obtain a good universality. As shown in Figure 27, a bionic manipulator combining the adaptive and underactuated function was designed by Zhenhao Li in Xi’an University of Engineering.⁷⁶ In this structure, three joints and three degrees of freedom are adopted so that artificial fingers are consistent with the physiological structure of human fingers, thus grasping fruits with different sizes, shapes, and qualities adaptively. Therefore, the underactuated manipulator can be also used to pick fruits.

Figure 26.

The underactuated manipulator for picking tomato proposed by Huanjun Sun in China.

Figure 27.

The underactuated manipulator with the adaptive ability developed by Zhenhao Li in China.

Characteristics of soft manipulators and their applied instances

As mentioned in above sections, it is still not easy to grasp fruits and cut off stems for different sizes, shapes, and types since a rigid manipulator cannot wrap fruit adaptively. This limitation is usually generated by a small deformation range, the less degree of freedom, low flexibility, and poor compliance in a rigid structure although flexible materials have been adopted in inner layers of partial fingers.⁷⁷ Fortunately, usually made of flexible materials, soft manipulators can be deformed greatly with infinite degrees of freedom, which is beneficial to grasp fragile and vulnerable objects adaptively by imitating bending actions of human hands.⁷⁸ Therefore, flexible materials present an improved way for picking fruits and vegetables, and corresponding soft machines in agriculture and other fields will characterize good references for establishing manipulators directly or indirectly.⁷⁹ Crucial aspects of soft manipulators such as materials, driving modes, and the variable stiffness will be analyzed in the next sections.

Materials for soft manipulators

Made of soft materials, diverse flexible manipulators characterize the good flexibility and adaptability, and their safe interactions with the environment are also available.⁸⁰ More importantly, different performances are reflected through manufacturing materials in manipulators or robots.⁸¹ Therefore, so many materials such as high elastomers, plastics, particles, and fabrics have been considered in soft robots.⁸² Of those, the silica gel with a high elasticity is the most representative material for making soft robots and their manipulators.⁸³ The details of common soft materials and properties are given in Table 2.^84

–88

Table 2.

The performance of common flexible materials.

Material	Performance
Rubber^84,85	1. Strong adsorption capacity, good thermal stability and stable chemical properties 2. Shore hardness range of 10–80HA 3. Excellent tensile strength, tear strength, and elongation
Shape memory alloy (SMA)⁸⁶	1. Shape memory 2. Hyper elasticity 3. Variable stiffness
Ion-exchange polymer metal composite (IPMC)⁸⁷	1. Low driving voltage (less than 3 V) 2. Fast response (larger than 10 Hz in water) 3. Low power consumption, low density, and good flexibility
Hydrogel⁸⁸	1. Responsive swelling behavior under the change of the external environment 2. As the shaping and driving materials of soft robots

Adopting these soft materials, diverse soft manipulators for picking fruits have been designed.^89
–91 For example, combining a three-finger clamping structure and rubber sheets with the spirally arranged way, a soft manipulator with the bending ability was developed by Muscato et al. in Italy, as shown in Figure 28.⁹⁰ Its fingers made of a rubber material can further reduce the damaging rate for picking citrus. Considering a network structure of the soft silica gel, a soft manipulator with two fingers was proposed by Octinon company in Belgium, as portrayed in Figure 29.⁹¹ Such fingers can store more compression and deformation energy, and flexibly picking action of strawberries has been achieved.

Figure 28.

A soft manipulator developed by Muscato et al. in Italy.

Figure 29.

A soft manipulator developed by Octinon company.

Obviously, compared with rigid manipulators, flexible ones present an advantage in grasping easily damaged fruits.^36,92 In addition to existing schemes directly applying in the agricultural field, there are lots of soft robots in other fields have been widely considered, such as the shape memory alloy (SMA) bionic turtle (Figure 30),⁹³ the bionic robot fish made of IPMC (Figure 31),⁹⁴ and the worm bionic robot with the gel material.⁹⁵ Essentially, these artificial robots or bionic machines with novel materials present a good reference for picking robots and their end-effectors.

Figure 30.

The SMA bionic turtle. SMA: shape memory alloy.

Figure 31.

The bionic robot fish made of ionexchange polymer metal composite (IPMC).

Different driving modes

Besides the above excellent performance of flexible materials such as the high flexibility and the large capacity of deformation, the appropriate driving mode is quite crucial to obtain bending, stretching, and twisting behaviors.^96,97 Therefore, international researchers have tried their best knowledge to select a better driving mode, such as the pneumatic element,⁹⁸ pull wire,⁹⁹ SMA,¹⁰⁰ and electroactive polymer (EAP)¹⁰¹ driving manners. In detail, characteristics of four driving modes are shown in Table 3.

Table 3.

Comparisons for different driving modes.

Type	Driving principle	Advantages	Disadvantages
Pneumatic element⁹⁸	Flow airs	Simple control Rapid response Good interaction Large deformation	Miniaturize hardly Additional compressor
Pull wire⁹⁹	Pulling wires	Simple control High safety	Deformation Limitation
SMA¹⁰⁰	Deformations of material (HT)	Compact structure High power Large force Large deformation	Long response Aging easily
EAP¹⁰¹	Deformations of material (HP)	Compact structure Small mass Fast response Large deformation	Poor safety Difficult control

HT: high temperature; HP: high pressure; SMA: shape memory alloy; EAP: electroactive polymer.

Specifically, a soft manipulator with the pneumatic element usually actively changes the air pressure inside the driving mechanism, thus achieving bending and extension.^92,102 Combining the concept of pneumatic grid, a flexible pneumatic manipulator was proposed by Li Wen in Beijing University of Aeronautics and Astronautics. As mentioned in Figure 32,¹⁰³ the effective length of it can be adjusted according to sizes and shapes of objects, further grasping quickly and accurately. Additionally, this type of manipulator presents lots of advantages including but not limited to the good universality, the convenient manufacture, the economical application, and the stable performance. Therefore, it would be adopted to pick fruits and vegetables without evident damage although the additional air source for driving is also necessary.¹⁰⁴

Figure 32.

The soft manipulator with the pneumatic element proposed by Li Wen in China.

Besides the pneumatic element, pull wires are also used to obtain the deformation of the flexible manipulator through an external power. Usually following line channels and fixed points, these wires are embedded inside the soft manipulator in advance.^92,105 As shown in Figure 33, an applied instance of this type had been adopted to pick fruits. Evidently, the simple arc contact is only available, and it is also difficult to wrap and grasp fruits with irregular shapes.^101,106 Fruits with larger weights will be picked easily if soft materials with relatively large Young’s modulus are also considered.¹⁰⁷ However, it is not easy to achieve the miniaturization and integration because of a large volume of its external power source.¹⁰⁸

Figure 33.

The soft manipulator with a driving element controlled by wires and an external power.

In addition to above traditional ways, some novel driving elements have been applied in picking fruits or bring a good reference to design corresponding robots. For example, the SMA will be deformed under the external force, thus passively characterizing the certain flexibility.⁹⁶ Also, its original shape can be restored after heating, further eliminating the deformation at low temperature.^109
–111 Combining these factors, excellent performances including the shape memory, hyper elasticity, and variable stiffness will be obtained. Therefore, adopting the SMA, the good picking effect on small fruits and vegetables with exquisite skins should be achieved. Besides, it can be operated in a narrow space following flexible behaviors for picking.⁹² However, it is not easy to grasp fruits with large mass because of the flexible materials and the relatively small bearing capacity.

Another innovative material called EAP can be deformed under the electricity.^103,112 Therefore, applying the voltage, a manipulator made of this material would be deformed. Also, such a manipulator usually generates a large deformation and the output force.¹¹³ As portrayed in Figure 34, a soft manipulator made of the material presents a good ability of the self-adaptive adjustment according to shapes of fruits or vegetables, thus wrapping and grasping them stably and quickly.^114,115 However, the mechanical strength of the EAP is small, and it can be disturbed easily by surrounding environments. It is also not suitable for picking fruits and vegetables with large weights and sizes.⁹²

Figure 34.

The soft manipulator made and controlled by the EAP material. EAP: electroactive polymer.

Therefore, improvements for a good versatility in picking fruits with different sizes, shapes, and weights should be further considered if these materials and driving modes are adopted.

Variable stiffness technology

While evident advantages have been achieved in partial schemes as mentioned in above instances, soft manipulators generally present the poor anti-interference and the limited grasping ability because of the low rigidity.^92,116 Therefore, the variable stiffness is also proposed by many researchers so that obvious disadvantages can be overcome.^117

–121 Essentially, adding a structure with variable stiffness on the original manipulator benefits the switch between the rigidity and the flexibility, which is a good way to improve the anti-interference strength and increase the ability of grasping. Currently, particle blockage, layered interference, SMA, dielectric elastomer, and so on are extensively adopted to achieve the variable stiffness.^116,121,122 For example, a soft manipulator with variable stiffness combining a principle of passive particle interference was designed by Yingtian Li et al.^116,117 As portrayed in Figure 35, the proposed end-effector is composed of three flexible manipulators which are surrounded by their limiting membranes. These soft manipulators will be expanded as long as the inflation is successful, thus causing particles in the cavity to squeeze each other and improving the stiffness of the entire structure. It is concluded from the existing data that the stiffness has been increased by more than six times if the pressure increases from 20 kPa to 80 kPa.¹¹⁶

Figure 35.

The soft manipulator with the variable stiffness proposed by Yingtian Li et al.

Besides these schemes for picking fruits directly, there are numerous instances applied in other fields which present good references for developing picking robots in the agricultural field. Of those, achieving the variable stiffness and the layer interference, a snake-like manipulator was designed by Kim et al.^116,118 As depicted in Figure 36, its gripper contains a plastic layer with a spiral thin-walled structure, and it is wrapped by the rubber. The friction between different layers will be adjusted through applying the vacuum pressure, thus achieving the variable stiffness, a compact structure, and lightweight. It has been implemented in the minimally invasive surgery and other fields.¹²³

Figure 36.

A snake-like manipulator with the variable stiffness and the layer interference.

Adopting the SMA, a soft manipulator was yielded in Seoul University.^116,119 As shown in Figure 37, the gripper is composed of three driving fingers, and each finger is manufactured by the flexible material and a material with variable stiffness. Fruits can be grabbed adaptively under a state of low stiffness. At a state of high stiffness, the grasping force of a soft manipulator will be strengthened greatly, thus improving the success rate of grasping. It is experimentally concluded that the adjustment range of stiffness can reach up to 55 times.¹²⁴

Figure 37.

A soft manipulator made of the SMA. SMA: shape memory alloy.

Considering the large deformation mechanism of materials driven by the voltage, the dielectric elastomer was applied to the bionic robotic fish as the artificial muscle.¹²⁰ Essentially combining electronic devices, the rigid structure, and a flexible material together, the rigid and flexible integration between materials and the structure is achieved perfectly. Therefore, moving forward quickly, steering with a small radius, shifting right, and other actions benefit the excellent performance of such a robotic fish, which also presents a good reference to design a soft manipulator with the variable stiffness.^125,126

It would report from the above analysis that flexible manipulators working as end-effectors have presented lots of advantages, especially achieving a good flexibility and the high versatility. However, there are also defects in stress, strain, life, and other aspects, thus yielding the low stiffness, easy deformation, and poor load capacity. Relatively heavy auxiliary devices should be assembled additionally in both pneumatic and pulling wire driving modes. The driving force is still small in SMA and EAP driving modes because of limitations of their own material characteristics. As a result, end-effectors are suggested to develop following the integration of intelligent materials, soft manipulators, the variable stiffness, the bionic principle, and underactuated operation.

Target recognition methods

Besides structural characteristics of end-effector, the recognition and location method for fruits or vegetables is another decisive factor to improve the picking efficiency, the success rate, and the quality of them,¹²⁷ which is also helpful to reduce labor costs and promote application. Therefore, a large number of studies have been concentrated on how to quickly identify fruits and determine their exact locations, thus achieving diverse fruit recognition and location manners in recent years. The specific features of these methods and their applications will be summarized in the next sections.

Essentially, the fruit recognition is the primary task during operation of a picking robot. This task is always affected by interference conditions since the picking robot has been working in the unstructured environment. As a result, automatic recognition for target fruits or vegetables presents particular difficulties in vision control. Obtaining more appropriate methods in this field, international scholars have proposed numerous methods which can be classified into three types, such as the single-feature vision, multi-feature fusion method, and deep learning method.¹²⁸ Their details will be given in the subsequent sections.

Single-feature vision method

Color features are widely adopted in fruit recognition, and such a method is usually effective if the certain color difference between the target and the background can be distinguished. Sufficiently considering such a difference, two methods have been established such as the color difference method and intelligent learning method.¹²⁹ Of those, determining a suitable color component or establishing a color operator in different color spaces, the color difference between the target fruit and the surrounding environment would be highlighted, which characterizes the color difference method. Combining this principle, Patel¹³⁰ and Thendral¹³¹ obtained a feasible recognition method for ripe citrus utilizing a component in the Lab color space. Zhao De’an¹³² and Lü Jidong¹³³ achieved the recognition method of apple adopting the R-G operator.

While successful instances including above ones are accessible, the flexibility and segmentation recognition are difficult to further improve. Consequently, an intelligent learning-based approach was proposed by many scholars.^{134

–143} Specifically, the clustering, modeling, and decision trees at given color spaces or components have been adopted to found fruits rather than the simple method of threshold segmentation.^134,135 For example, the K-means clustering algorithm with a color space transformation model was used to distinguish ripe apples.¹³⁶ The automatic recognition and extraction for apples were achieved by dynamic threshold segmentation method of Otsu.¹³⁷ Adopting an improved Otsu algorithm, strawberry, tomato, pomegranate, and persimmon were effectively detected, and the accuracy raised to 95%.¹³⁸ However, segmenting the image based on different thresholds reduces the robust, and results are sensitive to the external environment. The detection accuracy will also be fluctuated easily. A multi-class color recognition method considering the double Otsu segmentation was also determined,¹³⁹ thus improving the detecting efficiency. Combining the HIS and YCbCr color features, K-means clustering algorithm was utilized to detect grape and achieve a success rate of 88.33%.¹⁴⁰ Further adopting a classification algorithm of adaptive boosting (AdaBoost), the rate was increased to 96.56%.¹⁴⁰ Lychee images under the night environment had been collected, and then they were converted to a YIQ color space.¹⁴¹ Continuously extracting their I5 component images and applying the fuzzy C-means (FCM) clustering analysis, fruits and stalks were successfully segmented from the background.¹⁴¹ Detecting kiwi calyxes in RGB and HSV color spaces, aggregated kiwis were separated effectively, and the success rate was about 93.7%.¹⁴² Combining segmentation thresholds drawn from the Otsu algorithm, ripe tomatoes had been identified by converting RGB color space to HIS color space.¹⁴³

Evidently portraying in the above instances, colors are effective to detect fruits or vegetables under specific conditions. However, the detecting accuracy will be affected by many factors including the difference of maturity, color, variety, background, external illumination, and uncertainty of background. Avoiding these problems, other methods are also considered for detecting fruits or vegetables. Of those, depending on distinguishable or distinctive shapes, some types of fruits would be well identified such as apples, oranges, and cucumbers. The unique shape is helpful to detect these fruits since apples and oranges are more round in shape than their branches and leaves, and cucumbers present a special shape of strip.¹⁴⁴ Considering this point, there are lots of applications in recognition. For example, target contours of round or long fruits had been detected successfully through the combination of the Canny algorithm and the Hough transform.^145,146 Boundary lines between target fruits and the background of images were extracted by the edge detection algorithm, and accurate identifications for long fruits had been achieved by such a combination.¹⁴⁷ A matching library with multi-templates containing 65 images of cucumber was established by Bao et al., and cucumber recognition was realized through a template-matched algorithm.¹⁴⁸ There are lots of instances for detecting round fruits.^149,150 For example, fruits in the cluster were detected effectively by shape analysis, and such a method could identify fruits under conditions of light changes and occlusion.¹⁴⁹ An automatic algorithm of recognition was developed according to convexly geometric features, and a recognition rate raised to 94% for apples in trees.¹⁵⁰

Generally, geometric features are not easily affected by light conditions and are suitable for feature extraction in the orchard environment. Unfortunately, fruits are easily blocked by branches, leaves, and clusters. Shapes, sizes, and other geometric parameters of fruits are varied. Therefore, a relatively low accuracy becomes a fact.

In addition to colors and shapes, textures represent important characteristics of fruits and vegetables, which is helpful to separate target fruits from their background regions. The texture of the fruit surface is not changed by color, and texture features are usually utilized to detect fruits with similar color of branches and leaves, such as green apples^151,152 and green citrus.^153
–155 According to this principle, a fruit recognition method combining the color data was proposed by Zhao. As a result, red and green apples had been successfully detected even if some fruits were clustered by others or blocked by branches and leaves.¹⁵¹ Combining color, texture, and 3D shape features, apples were detected under uneven illumination, partial occlusion of fruits, and similar background.¹⁵² However, such a method is influenced by threshold values. Adopting the circular Gabor texture and scanning an entire image, green citrus had been detected with a recognition accuracy of 75.3%.¹⁵³ Further improving accuracy, an identification method for green fruits was proposed by Chaivivatrakul, and the success rate was raised to 90% by analyzing textures.¹⁵⁴ Considering the texture and intensity distribution, an innovative method for detecting green citrus was demonstrated by Jun Lu et al. in 2018.¹⁵⁵

Concluded from existing instances containing above ones, recognition methods combining the single feature are effective under simple conditions. Disturbed by changes of light conditions, the fruit occlusion, and similar backgrounds in an unstructured environment, shortcomings in these single-feature methods have emerged gradually, such as low accuracy and weak self-adaptability.¹⁵⁶

Multi-feature fusion method

Overcoming evident defects of single-feature methods, multi-feature fusion methods combining colors, shapes, and textures have been proposed so that the success rate could be improved effectively.¹⁵⁷

Of those, the color threshold segmentation and edge perimeter were integrated together to detect citrus, thus obtaining a success rate of 90%.¹⁵⁸ Fusing shape and color figures, the recognition difficulty in clustering and occlusion of tomatoes was solved by Yin et al.¹⁵⁹ Numerous types of fruits had been detected exactly with an accuracy of 90% by a new algorithm including the color, intensity, edge, and direction.¹⁶⁰ According to color and texture features, green apples were detected successfully utilizing the K-nearest neighbor (KNN) and regional growth method.¹⁶¹ Adopting this method, the success rate was about 85.0% under direct sunlight, and it was raised to 95.0% without direct sunlight.¹⁶¹ An analysis algorithm with the principal component was established by Nuske et al.¹⁶² In this method, texture, color, and shape characteristics were considered collectively to find grape berries. Introducing color and texture features, detection rates of two types of red grapes were 88% and 91.6%, respectively.¹⁶³ Combining color characteristics and the sum of absolute transformed difference, the accuracy for detecting unripe green citrus is 83.4%.¹⁶⁴ Also considering colors, shapes, and textures, the recognition rate of citrus was only 84.4% in an instance.¹⁶⁵ Utilizing colors, shapes, and the support vector machine, eight types of fruits were identified with an accuracy of 83.33%.¹⁶⁶

It is drawn from the above literature that multi-feature fusion methods significantly improve the accuracy and robustness of fruit recognition. However, disturbances such as the light condition, occlusion, and fruit clustering could not be eliminated in natural environment.

Deep learning method

Essentially, above traditional methods are affected easily by illumination. Their fitting accuracy and processing ability for complex scenes are also limited. Avoiding these problems, the deep learning has been widely used to detect, position, and segment fruits from images.^{167

–173} For example, the VGG16 network was slightly adjusted and trained for obtaining an image net model.¹⁶⁷ The sweet pepper, rock melon, apple, avocado, mango, and orange were effectively recognized by this model.¹⁶⁷ Different parts of a tomato plant were detected quickly by deep convolutional neural networks (CNN).¹⁶⁸ In detail, 10 types of network models with classification for parts of a tomato plant were established through structural optimization and VGGNet, thus achieving the accuracy of 81.64%, 84.48%, and 53.94% for fruits, flowers, and stems, respectively.¹⁶⁸ Utilizing the Le Net convolutional neural network, kiwi fruit in clusters had been detected under complex conditions.¹⁶⁹ Recognition rates for occluded, overlapping, adjacent, and independent fruits were about 78.97%, 83.11%, 91.01%, and 94.78%, respectively.¹⁶⁹ Evidently, success rates for occluded and overlapping fruits should be further improved. The Faster R-CNN model was adopted to recognize green citrus in the natural environment. However, the comprehensive recognition rate was only 77.45%.¹⁷⁰ Improvements for YOLOv2 were beneficial to detect unripe mango, and the success rates reached 97.02%.¹⁷¹ Training YOLOv3, ResNet50, and ResNet152 deep networks, the recognition ability of DNN was validated.¹⁷² Of those, the highest recognition accuracy drawn from ResNet152 was achieved, thus obtaining a rate of 95.35% and 97.86% under the natural environment and overlapping situations individually. The identification accuracy for leaves and fruits was 85.12%.¹⁷² Li et al. segmented strawberry images using Otsu and completed the recognition of strawberries using Caffe Net for training, thus achieving a recognition rate of 95%.¹⁷³ Compared with traditional methods, the deep learning method characterizes better effects on fruits which are overlapped and covered by others. However, enough set for training is required, and defects are also obvious such as a large amount of calculation, long training time, and high cost.

Depicted from above sections, almost all recognition methods have deficiencies in the accuracy, efficiency and field versatility, and so on. For selecting and improving corresponding methods, brief comparisons are given in Table 4.

Table 4.

Comparison for fruit detection methods from images.

Methods	Feasible fields	Advantages	Limitations
Color-based^{130 –143}	Fruits with obvious color differences from the background such as apples, citrus, tomatoes, and so on	Distinguish fruits and backgrounds	Affected by differences of maturity, color, variety, background, external illumination, and background
Shape-based^151 –154	Round or long fruits or vegetables such as apples, oranges, and cucumbers	Obtain the outline information of fruit targets; not sensitive to light	Disturbed by branches, leaves clusters, shapes, sizes, and other geometric parameters
Texture-based^151 –155	Fruits with similar colors of branches and leaves such as green apples and green citrus	Separate fruits from their backgrounds	Influenced by changes of light conditions, the fruit occlusion, and similar backgrounds
Multi-feature fusion^{158 –166}	Suitable for identification of most fruits	Compensate limitations of single feature; improve the accuracy and robustness	Affected by the light condition, occlusion, and fruit clustering
Deep learning^{167 –173}	Suitable for fruit detection and recognition in complex natural environments	Solve the overlapping and occluded difficulties; achieve good robustness and versatility	Require a sufficient training set; the long training time, large computational load, and high cost

3D reconstruction technologies for positioning fruits

Above methods are generally adopted to determine which parts are fruit or vegetables according to images. It is still difficult to exactly obtain the specific position and deep information of each fruit through these methods. Appropriate methods for positioning fruits have been considered so that the intelligent control system could guide an end-effector to grasp and separate them effectively. Therefore, the 3D reconstruction method has been proposed in this field. Generally, target information is extracted from 2D images, and 3D reconstruction could be further achieved by lots of technologies for visual positioning.^{13,174

–208} Of those, active and passive vision are extensively utilized in the picking field,¹⁷⁴ and their details will be demonstrated in the next sections.

Active vision-based reconstruction technologies

Scanning surfaces of target fruits by optical instruments, exact information about the position and space of each fruit can be obtained, the 3D structure of the object will be further achieved through data analysis. Such a method is the so-called active vision method. Currently, the structured light method,¹⁷⁵ laser scanning method,^176,177 time of flight (TOF) method,¹⁷⁸ and radar technology¹⁷⁹ are usually utilized to establish corresponding technologies of 3D reconstruction. Of those, the structured light method is dependent on the principle of optical triangulation. Essentially, a triangular relationship can be determined among the light source, a light sensor, and the reflection point in a scanner.¹⁸⁰ It will be used to compute the depth information of the object, further reconstructing the 3D structure of fruits or vegetables.¹⁸⁰ Combining the near-infrared linear array of structured illumination, this method was successfully used to recognize and position apple stems and calyxes.¹⁸¹ Depending on the stereoscopic measurement principle of linear structured light, the 3D coordinate information about surface of ripe tomato was extracted effectively during optical fringe scanning process.¹⁸² Especially, the complex illumination in a greenhouse and the target occlusion would be solved by such a method. The reflection imaging principle of structured light was introduced into this field so that optical properties of fruit skin and flesh were measured, which is useful to identify product defects such as damage and decay.¹⁸³ Easily concluded from above principles and successful instances, the structured light method is beneficial to obtain exact positions and the deep information with characteristics of simplification, convenience, and high accuracy. However, this method is also relied on high requirements of the scene, and it is not suitable for the outdoor environment.

Besides the structured light method, a laser range finder can be adopted to capture the real scene actually, thus yielding a laser scanning method. Depending on such an operation, there are lots of instances for positioning fruits. For example, the information of orange trees was obtained by a laser ranging sensor.¹⁸⁴ Color and shape factors were fused to locate fruits in this instance. However, its processing speed could not meet requirements of real time. A laser vision system was developed for detecting the cherry in a tree,¹⁸⁵ and this method brought a limitation since it was also impacted by colors of fruit surfaces. Eliminating effects of different lights and complex backgrounds, an algorithm for automatic recognition of apples in trees was created by Juan Feng et al.¹⁸⁶ The laser vision system was adopted to obtain local 3D images of fruit trees with a good real-time performance. The recognition rate of fruit is higher than 93.75% if shaded areas of branches and leaves were less than 40%.¹⁸⁶ A novel laser vision system was designed for picking robots, further achieving identification and orientation with high accuracy in the unstructured environment of the orchard.¹⁸⁷ The recognition rate was over 90% and the response time was only 3.58s as long as the occlusion rate was less than 50%.¹⁸⁷ Easily portrayed from existing instances, laser scanning method is not only used to reconstruct 3D models of targets with simple shapes but also generates 3D models of irregular objects with high accuracy. However, time-consuming for a huge amount of point cloud registration is long, and equipment is also quite complex and expensive.

Another method called TOF has presented evident advantages since it can be used to directly capture geometric information of the scene from the 3D point cloud.¹⁸⁸ Specifically, an optical instrument emits pulsed light with a certain frequency to a target, and such an operation will be ended if the receiver receives the returned light. Finally, distance information of a target will be obtained by a combination of the number of pulses and the speed of light.¹⁸⁹ Considering these rules, a TOF camera was utilized to construct a 3D structure of an apple tree.¹⁹⁰ Reducing effects of picking environment, a sensing system of plant space was established by utilization of a TOF camera, and it was not sensitive to light conditions.¹⁹¹ Further combining RGB and TOF cameras, information of an entire tree was measured effectively.¹⁹² Essentially, high resolution of RGB camera and the high precision of TOF camera had been characterized collectively in this method, and positioning accuracy was improved greatly.

Unlike counting pulses of the TOF method, time differences between transmitted and received beams are also adopted to measure distances of a target, thus yielding another 3D reconstruction method with radar-based technologies. While direct instances in the picking field adopting radar technologies are relatively fewer, positioning applications in other sectors of the agricultural field will also present a useful reference for picking fruits or vegetables. For example, combining machine vision and a lidar, the motion of a spraying robot in a greenhouse was controlled accurately.¹⁹³ Relying on a lidar, a navigation system of the inspection robot was developed so that precise positioning and path planning could be achieved.¹⁹⁴ Evidently, a radar-based technology is quite suitable for navigation and path planning. With a huge field of view and a long distance, it must be affected by the external environment, the accuracy of 3D reconstruction is difficult to further improve. Also, the cost of a radar with high accuracy is expensive for applications in agriculture.

Passive vision-based 3D reconstruction technologies

As above instances, methods with active vision are generally affected by the environment and equipment. Therefore, a great deal of effort has been devoted to establish passive vision-based methods in recent years.^{13,196

–208} Generally, vision sensors (one or more cameras) are used to capture a series of images, and then the 3D structure of a target will be reconstructed, thus achieving a 3D reconstruction method with passive vision. According to the number of cameras, there are three types of passive vision methods, such as monocular vision, binocular vision, and multiocular vision.¹⁹⁵

A single camera is usually adopted to capture images. The target will be then recognized and reconstructed based on image information. Applying this method, numerous systems with monocular stereo vision had been established in this field.^13,196,197 For example, combining a camera and a light source with a high frequency, apples in an outdoor environment were detected effectively by a monocular vision system.¹³ A set of operating system nodes ensuring modularity and separation of concerned points were established for controlling crucial aspects synchronously, such as the motion, image acquisition, fruit detection, visual servo, and relative position of a monocular camera. This system was used to recognize and locate the sweet pepper by extracting visual information from the end-effector.¹⁹⁶ A monocular camera was assembled on the end-effector of a picking robot, and the effector was rotated to image and fruit centers after determining the center of mass of a fruit, further grasping fruits.¹⁹⁷

Reflected through existing instances, a positioning system with monocular camera presents evident advantages such as the simple equipment, flexible application, and low cost. However, the accuracy of 3D reconstruction is relatively poor.

To overcome shortcomings of monocular vision, a binocular vision system that is similar to human eyes has been considered in the field of picking fruits or vegetables.^198,199 Specially, two identical cameras in this system can be used to capture images of left and right sides in the same position from different angles, and then deep information of the object will be extracted by the principle of triangulation. Further depending on corresponding information, 3D models of a target fruits should be reconstructed effectively. Combining such a rule, a picking robot for tomatoes was designed in the University of Tokyo using a binocular vision system and a manipulator with six degrees of freedom.²⁰⁰ The success rate of picking was about 60%.²⁰⁰ The image of a cherry tree was collected by binocular vision system with the parallel optical axis.²⁰¹ Relying on the converging optical axis, a binocular vision system was adopted to capture left and right images of citrus, thus presenting a spatial location method for picking outdoor citrus.²⁰² A binocular vision system based on the embedded ARM was also developed for picking apples.²⁰³ In this system, a processing module of robot vision was established by a combination of a CMOS binocular camera and the ARM COTEX A9 processor.²⁰³ Therefore, fast location for a picking robot must be achieved after target extraction and stereo matching. Adding one or more cameras to the binocular vision system, an extension system called multiocular vision could be proposed.²⁰⁴ Actually, several images of an object will be obtained in view of different angles, and the blind area in measurement can be reduced greatly, especially obtaining a larger field of vision.²⁰⁵ However, the whole system must be more complex and computing loads is also increased greatly. Therefore, it is widely adopted in fields of vehicle autonomous driving, robot vision, mechanism control, and so on.^206
–208 Applications in other fields will also present a good reference for picking fruits or vegetables.

On the basis of above instances, characteristics of current positioning manners are briefly compared in Table 5.

Table 5.

Comparison for 3D reconstruction based on vision.

Method	Advantages	Disadvantages	Degree of automation	Reconstruction effect	Application scenarios
Structured light^180 –183	Simple, convenient, fast, and non-destructive; Accurate reconstruction result	Slow measurement speed; Not suitable for outdoor scenes	Some degree of automated reconstruction	Relatively high	Indoor scenes
Laser scan^184 –187	Accurate reconstruction results; Build 3D models for irregularly shaped objects	Algorithms need to be employed to fix the vulnerabilities; Long time-consuming and expensive	Some degree of automated reconstruction	Very good	Factory production and testing; Not widely available
TOF method^189 –192	High data acquisition frequency; Large vertical field of view; Extracting information directly (geometry information)	Large error in the depth measurement system; Poor contrast and low resolution of gray-scale images; Large search space and low efficiency	Some degree of automated reconstruction	Low accuracy of reconstruction effect	Widely used in human face detection, vehicle safety, and so on
Radar technology^193,194	Large field of view and long scanning distance; High sensitivity and low power; Consumption; Obtain depth information without calibrating external parameters	Greater influence by the environment; Larger computational volume and poor real-time performance	Some degree of automated reconstruction	Average reconstruction effect	Widely used in all walks of life
Monocular vision^13,196,197	Simple, convenient, flexible, reliable, inexpensive, wide range of utilization; Realize self-calibration of camera during reconstruction process; Short processing time	Inability to obtain depth information; Slower reconstruction speed	Automated reconstruction	Poor reconstruction effect	Available for various scenarios
Binocular vision^{198 –203}	The method is mature; Stably obtain a better reconstruction effect; Wide application	Large amount of operations; The reconstruction effect is reduced when the baseline distance is large; More expensive	Fully automated reconstruction	Better Baseline reconstruction under certain conditions	Outdoor scenes
Multiocular vision^204 –208	High recognition accuracy; Get a larger field of view and adaptability	Large computing capacity; Long reconstruction time and expensive	Fully automated reconstruction	Significantly lower reconstruction effect; Reduced measurement accuracy and limited speed for larger baseline distances	Various scenarios

TOF: time of flight.

Conclusion and future prospective

Reflected through structural schemes and their recognition as well as location methods, characteristics of them can be yielded that:

Grasping, separating, and recycling actions are carried out orderly. Specifically, fruits will be grabbed by a clamping or suction mechanism, and then the cutting mechanism is utilized to separate them. Finally, fruits will be recycled through mechanical arms, soft tubes, and others. Therefore, the picking efficiency might be reduced by the cumbersome action, the complex control, and uncoordinated movements between mechanisms and other factors.

The rigid structure is usually adopted in most of existing manipulators whichever grasping mode is considered. The clamping force and suction are not easy to control because of the hard material, and the mechanical damage to fruits and vegetables is also an inescapable fact. Besides these, the damage is also caused by the collision between the gripper and external environment at the picking stage. Therefore, for avoiding these damages, improvements in picking method, mechanical structure, material, driving mode, and control system should be considered. Rigid manipulator or underactuated ones working as a traditional way generally lacks a certain flexibility and the ability of force perception. It is still difficult to wrap and pick fruits adaptively.

Flexible materials have been adopted to in very few schemes so that some defects of the rigid structure could be overcome. However, the unstable behavior or the oscillation may be caused by these soft materials with a low stiffness. Especially, the soft manipulator will be prone to bending deformation, thus resulting in failure to grasp fruits. Therefore, a soft manipulator should be developed following a direction of variable stiffness, finally obtaining excellent performance combining good adaptability, a low damage rate, good versatility, and so on.

Most existing end-effectors are usually designed for specific fruits or vegetables, evidently presenting the poor universality in different types. In a word, it is possible to use them in a certain harvesting season, which undoubtedly reduces the utilization rate of picking robots.

The average cycle is also longer than what is anticipated in agricultural operation with high efficiency, and the recognition accuracy of end-effector is relatively low in easily practiced schemes. Large errors in recognizing and positioning are also generated by the vision system inevitably. Complex conditions including the light, color, shape, texture, and growth state bring difficulties in processing images. Therefore, a good vision system with the ability to overcome these problems is highly anticipated.

Consequently, the high efficiency, low damage, flexibility, adaptability, versatility, and high accuracy should be solved collectively in the future if harvesting robots are expected to be applied in diverse conditions with a low cost. Therefore, suggestions for developments are concentrated on the following aspects.

New structures for picking manipulator should be developed first so that a good adaptability for different sizes, shapes, and weights would be achieved possibly, thus meeting requirements for picking in all seasons and reducing the cost of utilization.

Further combining flexible materials, variable stiffness methods, bionic principles, and proper driving modes, a picking robot is suggested to achieve the simple structure, easily controlled program, low cost, less energy consumption, good stability, low damage rate, and miniaturization integration, which characterizes great challenges in promoting development of picking robots.

The accuracy and efficiency of recognizing and positioning systems with the low cost should be improved greatly. The excellent performance of a picking robot will be available if the manipulator with appropriate structure, the entire robot, intelligent recognition technologies, and the orchard environment can be controlled synchronously.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ziyue Li

References

Changlin

Tiezhong

. Research progress on picking robot for fruits and vegetables. J Anhui Agric Sci 2008; 36(13): 5394–5397.

Jie

Yanwen

. Research situation, problem and solution of fruit vegetable picking robots. J Mach Des 2010; 27(6): 1–5.

Xiuying

Tiezhong

. Fruit and vegetable harvesting robot research review. Robot 2005; 27(1): 90–96.

Arima

Kondo

Monta

. Strawberry harvesting robot on table-top culture: 2004. Ottawa, Canada: ASAE, 2004.

Xiong

Peng

Grimstad

, et al. Development and field evaluation of a strawberry harvesting robot with a cable-driven gripper. Comput Electron Agric 2019; 157: 392–402.

Defterli

. Review of robotic technology for strawberry production. Appl Eng Agric 2016; 32(3): 301–318.

Hannan

Burks

. Current developments in automated citrus harvesting. In: 2004 ASAE annual meeting, Ottawa, ON, Canada, 1–4 August 2004, p. 1. St. Joseph, MI: American Society of Agricultural and Biological Engineers, 2004.

Kondo

Monta

Noguchi

. Agricultural robots: mechanisms and practice. Kyoto; Melbourne: Kyoto University Press; Trans Pacific Press, 2011.

Shun

. Design and research on end-effector of citrus harvesting robot. Chongqing: Chongqing University of Technology, 2017.

10.

Liu

Zhang

, et al. Design and test of end-effector for navel orange picking robot. Trans Chin Soc Agricul Eng (Trans CSAE) 2018; 34(12): 53–61.

11.

Sanders

. Orange harvesting systems review. Biosyst Eng 2005; 90(2): 115–125.

12.

YOU

. Development of an adaptable vacuum based orange picking end effector. Agric Engi Int: CIGR J 2019; 21(1): 58–66.

13.

Baeten

Donne

Boeri

, et al. Autonomous fruit picking machine: a robotic apple harvester. In: Laugier

Siegwart

(eds) Field and service robotics. Springer tracts in advanced robotics, Vol. 42. Berlin, Heidelberg: Springer, 2008, pp. 531–539.

14.

Abundant Robotics and T&G Global, one of New Zealand’s largest apple growers, are collaborating on this major milestone. https://www.technologyreview.com/2017/05/03/152012/apple-picking-robot-prepares-to-compete-for-farm-jobs/.

15.

Davidson

Hohimer

, et al. Dual robot coordination for apple harvesting. In: 2017 ASABE annual international meeting, Spokane, WA, 16–19 July 2017. St. Joseph, MI: American Society of Agricultural and Biological Engineers, 2017, pp. 1–7.

16.

De-An

Jidong

Wei

, et al. Design and control of an apple harvesting robot. Biosyst Eng 2011; 110: 112–122.

17.

Guoli

. Design and research of apple picking robot with multiple end-effectors. Nanjing: Nanjing Agricultural University, 2017.

18.

Israeli robotics firm FFRobotics aims to release robotic apple picker by next year[EB/OL]. 2017-10-18[2021-01-15]. https://www.therobotreport.com/israeli-robotics-firmFFRobo-ticsaims-release-robotic-apple-picker-next-year .

19.

Hao

. Development of Kiwifruit picking end actuator based on separation and picking of fruit stems. Xianyang: Northwest A&F University, 2019.

20.

Williams

HAM

Jones

Nejat

, et al. Robotic kiwifruit harvesting using machine Vision, convolutional neural networks, and robotic arms. Biosyst Eng 2019; 181: 140–156.

21.

Scarfe

Flemmer

Bakker

, et al. Development of an autonomous kiwifruit picking robot. In: 2009 Fourth international conference on autonomous robots and agents, Wellington, New Zealand, 10–12 February 2009, pp. 380–384. IEEE. DOI: 10.1109/ICARA.2000.4804023.

22.

Feng

Wang

, et al. Design and test of tomatoes harvesting robot. In: 2015 IEEE international conference on information and automation, 2015, pp. 949–952. IEEE. DOI: 10.1109/ICInfA.2015.7279423.

23.

Kondo

Yuta

Iida

, et al. Development of an end-effector for a tomato cluster harvesting robot. Eng Agric Environ Food 2010; 3(1): 20–24.

24.

Chiu

Yang

Chen

. Development of the endeffector of a picking robot for greenhouse-grown tomatoes. Appl Eng in Agric 2013; 29(6): 1001–1009.

25.

Monta

Kondo

Ting

. End-effectors for tomato harvesting robot. In: Panigrahi

Ting

(eds) Artificial intelligence for biology and agriculture. Dordrecht: Springer, 1998, pp. 1–25.

26.

Ling

Ehsani

Ting

, et al. Sensing and end-effector for a robotic tomato harvester, Ottawa, ON, Canada, 1–4 August 2004, p. 1. St. Joseph, MI: American Society of Agricultural and Biological Engineers.

27.

Liu

, et al. Design and test of the vacuum suction device for tomato harvesting robot. Trans CSAM 2010; 41(10): 170–173, 184.

28.

Lili

. Research on key technology of tomato harvesting robot. Beijing University of Technology, 2017.

29.

Hayashi

Kubota

. Robotic harvesting technology for fruit vegetables in protected horticultural production. In: Information and technology for sustainable fruit and vegetable production, Fructi, Montpellier, France, 12–16 September 2005, Vol. 5, pp. 227–236.

30.

Hayashi

Ganno

Ishii

, et al. Robotic harvesting system for eggplants. Jpn Agric Res Q 2002; 36(3): 163–168.

31.

Changlin

Tiezhong

, et al. Design of end-effecter of eggplant harvesting robot. J Agric Mech Res 2008; 12(2008): 62–64.

32.

Van Henten

Hemming

Van Tuijl

BAJ

, et al. An autonomous robot for harvesting cucumbers in greenhouses. Auton Robots 2002; 13(3): 241–258.

33.

Van Henten

Van Tuijl

BAJ

Hemming

, et al. Field test of an autonomous cucumber picking robot. Biosyst Eng 2003; 86(3): 305–313.

34.

Van Henten

Hemming

Van Tuijl

BAJ

, et al. Collision-free motion planning for a cucumber picking robot. Biosyst Eng 2003; 86(2): 135–144.

35.

Yan

. Design on motion planning and Manipulator control system of the cucumber picking robot. Luoyang Inst of Sci and Technol 2017; 27(02): 56–60+78.(in Chinese with English abstract).

36.

Qinchuan

Ting

Chuanyu

, et al. Review of end-effectors in fruit and vegetable harvesting robot. Trans Chin Soc Agric Mach 2008; 39(3): 175–179.

37.

Xin

Chao

Long

, et al. Research progress in fruit picking robot end effectors. Forest Mach Woodwork Equip 2020; 48(9): 4–10.

38.

Seol

Lee

Son

. A review of end-effector for fruit and vegetable harvesting robot. J Korea Robot Soc 2020; 15(2): 91–99.

39.

Fengyun

Jizhan

Pingping

. Present situation and prospect of end-effector for harvesting robot. J Agric Mech Res 2011; 33(11): 10–14.

40.

Guoli

Changying

Lixin

. Research progress and analysis of end-effector for fruits and vegetables picking robot. J Chinese Agric Mech Res 2007; 3: 195–197.

41.

Tian

Qiu

Qin

, et al. Comparative study on manipulator of picking-robot at abroad and home. J Agric Mech Res 2007; (03): 195–197.

42.

Kondo

Ninomiya

Hayashi

, et al. A new challenge of robot for harvesting strawberry grown on table top culture. In: 2005 ASABE annual international meeting, Tampa, FL 17–20 July, 2005. St. Joseph, MI: American Society of Agricultural and Biological Engineers.

43.

Kondo

Tink

. Robotic for bioproduction systems. USA: America Society of Agricultural Engineering Publisher, 1998.

44.

Bac

Van Henten

Hemming

, et al. Harvesting robots for high-value crops: State-of-the-art review and challenges ahead. J of Field Robot 2014; 31(6): 888–911.

45.

Bogue

. Fruit picking robots: Has their time come? Ind Robot 2020; 47(2): 141–145.

46.

Xiaonan

Pinghui

Qingchun

, et al. Design and test of tomatoes harvesting robot. J Agric Mech Res 2016; 38(4): 94–98.

47.

Liping

Shuren

. The overview of fruit harvesting robot. Agric Equip Technol 2006; 32(1): 8–10.

48.

Lei

Diyi

Xiaoyi

. Researching on the fruit and vegetable harvesting robot. J Agric Mech Res 2011; 33(1): 224–227.

49.

. Research on key technology of bite-mode endeffector of citrus harvesting robot. Chongqing University, 2019.

50.

Davidson

Silwal

Hohimer

, et al. Proof-of-concept of a robotic apple harvester. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), Daejeon, Korea, 9–14 October 2016, pp. 634–639. IEEE. DOI: 10.1109/IROS.2016.7759119.

51.

Davidson

Hohimer

. Preliminary design of a robotic system for catching and storing fresh market apples. IFAC-Papers Online 2016; 49(16): 149–154.

52.

Lvzhong

Wenliang

Chengjun

, et al. Structure design and experiment of the end-effector for apple-harvesting robot. J Agric Mech Res 2009; 31(12): 65–67.

53.

Liu

. Hardware design of the end-effector for tomato-harvesting robot. Trans CSAM 2008; 39(3): 109–112.

54.

. Research on the fruit-vegetable picking robot end executor and manipulator. Shenyang: Shenyang University of Technology, 2014.

55.

Hayashi

Robotic harvesting technology for fruit vegetables in protected horticultural production. In: 7th Fruit, Nut and Vegetable Production Engineering Symposium, Montpellier, France, 12–16 September 2005.

56.

Xiong

Grimstad

, et al. An autonomous strawberry-harvesting robot: Design, development, integration, and field evaluation. J Field Robot 2020; 37(2): 202–224.

57.

Guoli

Changying

Baoxing

, et al. Kinematics analysis and experiment of apple harvesting robot manipulator with multiple end-effectors. Trans CSAM 2016; 47(12): 14–21.

58.

Shukui

Zikai

. Design and simulation analysis of the end effector of circular fruit picking robot. J Mach Des 2019; 36(S2): 82–85.

59.

Xuejing

. Design a citrus non-destructive picking end effector. Chongqing Three Gorges University, 2021.

60.

Schertz

Brown

. Basic considerations in mechanizing citrus harvest. Trans ASABE 1968; 11(3): 343–346.

61.

Tibbets

. Not too far from the tree. Mech Eng 2018; 140(2): 28–33.

62.

Williams

Ting

Nejati

, et al. Improvements to and large-scale evaluation of a robotic kiwifruit harvester. J Field Robot 2020; 37(2): 187–201.

63.

Williams

Nejati

Hussein

, et al. Autonomous pollination of individual kiwifruit flowers: toward a robotic kiwifruit pollinator. J Field Robot 2020; 37(2): 246–262.

64.

Lili

Jinwei

. Structural design and performance test of fruit and vegetable picking manipulator. Agric Eng 2017; 7(2): 107–113.

65.

Massa

Roccella

Carrozza

M C

, et al. Design and development of an underactuated prosthetic hand. In: Proceedings 2002 IEEE international conference on robotics and automation (Cat. No. 02CH37292), Washington, DC, USA, 11–15 May 2002, Vol. 4, pp. 3374–3379. IEEE. DOI: 10.1109/ROBOT.2002.1014232.

66.

Devi

Udupa

Sreedharan

. A novel underactuated multi-fingered soft robotic hand for prosthetic application. Robot Auto Syst 2018; 100: 267–277.

67.

Birglen

Laliberté

Gosselin

. Grasp stability of underactuated fingers. In: Underactuated robotic Hands. Berlin, Heidelberg: Springer, 2007, pp. 61–115.

68.

Birglen

Gosselin

CM.

On the force capability of underactuated fingers. In: 2003 IEEE international conference on robotics and automation (Cat. No. 03CH37422), Taipei, Taiwan, 14–19 September 2003, Vol. 1, pp. 1139–1145. IEEE. DOI: 10.1109/ROBOT.2003.1241746.

69.

Birglen

Gosselin

. Geometric design of three-phalanx underactuated fingers. J Mech Des 2006, 128(2): 356–364.

70.

Begoc

Krut

Dombre

, et al. Mechanical design of a new pneumatically driven underactuated hand. In: Proceedings 2007 IEEE international conference on robotics and automation, Rome, Italy, 10–14 April 2007, pp. 927–933. IEEE. DOI: 10.1109/ROBOT.2003.1241746.

71.

Fei

. Design and research for underactuated End-effector Manipulator. Zhejiang Sci-Tech University, 2014.

72.

Dollar

Howe

RD.

The highly adaptive SDM hand: design and performance evaluation. Int J Robot Res 2010; 29(5): 585–597.

73.

Tegin

Iliev

Skoglund

, et al. Real life grasping using an under-actuated robot hand-simulation and experiments. In: 2009 international conference on advanced robotics, Munich, Germany, 22–26 June 2009, pp. 1–8. IEEE.

74.

Jinyin

Yang

, et al. Design and experiment of underactuated end-effector for citrus picking. Trans CSAM 2021; 52(10): 120–128.

75.

Huanjun

. Design and experiment of the end actuator of intelligent greenhouse tomato picking system. Sichuan Agricultural University, 2019.

76.

Zhenhao

Shengqi

, et al. Structure design and kinematics simulation of new bionic manipulator. J Xi’an Polyt Univ 2020; 34(5): 56–62.

77.

Xue

Shanxiang

Shuang

, et al. The development status and its applications of soft manipulators. Manuf Auto 2019; 41(05): 85–92.

78.

Hughes

Culha

Giardina

, et al. Soft manipulators and grippers: a review. Front Robot AI 2016; 3: 69.

79.

Cao

Shang

Liang

, et al. A review on the soft robotics. J Mech Eng 2012; 48(3): 25–33.

80.

Elango

Faudzi

AAM

. A review article: investigations on soft materials for soft robot manipulations. Int J Adv Manuf Technol 2015; 80(5): 1027–1037.

81.

Case

White

Kramer

. Soft material characterization for robotic applications. Soft Robot 2015; 2(2): 80–87.

82.

Rus

Tolley

. Design, fabrication and control of soft robots. Nature 2015; 521(7553): 467–475.

83.

Hou

Wang

, et, al. Review on soft-bodied robots. Sci Technol Rev 2017; 35(18): 20–28.

84.

Yunxia

. The electrode modification of ionic polymer metal composite and improvement of PDMS adhesion performance. Nanjing: Nanjing University of Aeronautics and Astronautics, 2013.

85.

Cheng

Lobovsky

Keating

, et al. Design and analysis of a robust, low-cost, highly articulated manipulator enabled by jamming of granular media. In: IEEE international conference on robotics and automation, Saint Paul, MN, USA, 14–18 May 2012, pp. 4328–4333. IEEE. DOI: 10.1109/ICRA.2012.6225373.

86.

Kim

Laschi

Trimmer

. Soft robotics: a bioinspired evolution in robotics. Trends Biotechnol 2013; 31(5): 287–294.

87.

Nakabo

Mukai

Asaka

. Biomimetic soft robots using IPMC. In: Electroactive polymers for robotic applications. London: Springer, 2007, pp. 165–198.

88.

Kempaiah

Nie

. From nature to synthetic systems: shape transformation in soft materials. J Mater Chem B 2014; 2(17): 2357–2368.

89.

Hua

Chu

Chen

, et al. Design, analysis and experimental study of soft picking manipulator system for fruit harvesting. J Forestry Eng 2021; 6(3): 127–132.

90.

Muscato

Prestifilippo

Abbate

, et al. A prototype of an orange picking robot: past history, the new robot and experimental results. Ind Robot 2005; 32(2): 128–138.

91.

Pothering

. Tortga Ag Tech raises $2.4 million for strawberry picking robots, [2017-12-13], https://business.lesechos.fr/entrepreneurs/idees-de-business/0301580309302-octinion-lastart-up-qui-a-invente-le-robot-cueilleur-de-fraises-320507.php.

92.

Peng

Liu

Yang

, et al. Research progress on application of soft robotic gripper in fruit and vegetable picking. Trans Chin Soc Agric Eng (Trans CSAE) 2018; 34(9): 11–20.

93.

Kim

Song

Ahn

. A turtle-like swimming robot using a smart soft composite (SSC) structure. Smart Mater Struct 2012; 22(1): 014007.

94.

Hubbard

Fleming

Palmre

, et al. Monolithic IPMC fins for propulsion and maneuvering in bioinspired underwater robotics. Ocean Eng IEEE J 2014; 39(3): 540–551.

95.

Nakamaru

Maeda

Hara

, et al. Development of novel self-oscillating gel actuator for achievement of chemical robot. In: 2009. IROS 2009. IEEE/RSJ international conference on intelligent robots and systems, St. Louis, MO, USA, 10–15 October 2009, pp. 4319–4324. IEEE. DOI: 10.1109/IROS.2009.5354649.

96.

Zhang

Wang

Hong

, et al. Review of soft-bodied manipulator. J Mech Eng 2017; 53(13): 19–28.

97.

Sun

Zhang

Chen

. Overview of soft-bodied actuators. J Mach Des 2019; 36(2): 5–18.

98.

Shapiro

Wolf

Gabor

Bi-bellows: pneumatic bending actuator. Sens Actuator A Phys 2011; 167(2): 484–494.

99.

Manti

Hassan

Passetti

, et al. A bioinspired soft robotic gripper for adaptable and effective grasping. Soft Robot 2015; 2(3): 107–116.

100.

Rodrigue

Wang

Han

, et al. Comparison of mold designs for SMA-based twisting soft actuator. Sens Actuator A: Phys 2016; 237: 96–106.

101.

O’Halloran

O’malley

McHugh

. A review on dielectric elastomer actuators, technology, applications, and challenges. J Appl Phys 2008; 104(7): 9.

102.

Wang

. Research status of the soft robot. Micronanoelectron Technol 2019; 56(12): 948–955+991.

103.

Hao

Gong

Xie

, et al. Universal soft pneumatic robotic gripper with variable effective length. In: 2016 35th Chinese control conference (CCC), Chengdu, China, 27–29 July 2016, pp. 6109–6114. IEEE. DOI: 10.1109/ChiCC.2016.7554316.

104.

Marchese

Katzschmann

Rus

. A recipe for soft fluidic elastomer robots. Soft Robot 2015; 2(1): 7–25.

105.

Chao

. Dynamics and control of cable-driven silicone soft manipulator. Shanghai: Shanghai Jiaotong University, 2015.

106.

Lee

Jeong

, et al. Feasibility study of a slack enabling actuator for actuating tendon-driven soft wearable robot without pretension. In: IEEE international conference on robotics and automation. Seattle, WA, 26–30 May 2015, pp. 1229–l234. IEEE. DOI: 10.1109/ICRA.2015.7139348.

107.

Mishra

Del Dottore

Sadeghi

, et al. SIMBA: tendon-driven modular continuum arm with soft reconfigurable gripper. Front Robot AI 2017; 4: 4.

108.

Wang

Hao

Yang

. Soft robotics: structure, actuation, sensing and control. J Mech Eng 2017; 53(13): 1–12.

109.

Shu

Lagoudas

Hughes

, et al. Modeling of a flexible beam actuated by shape memory alloy wires. Smart Mater Struct 1997; 6(3): 265.

110.

Xiang

Yang

Sun

, et al. The design, hysteresis modeling and control of a novel SMA-fishing-line actuator. Smart Mater Struct 2017; 26(3): 037004.

111.

Zhang

Zou

Ding

. Research status of the soft robot driving. Robot 2018; 40(5): 648–659.

112.

Wang

Tang

. Review of soft robot. J Tongji Univ 2014; 42(10): 1596–1603.

113.

Mirfakhrai

Madden

JDW

Baughman

. Polymer artificial muscles. Mater Today 2007; 10(4): 30–38.

114.

Suo

. Theory of dielectric elastomers. Acta Mech Solida Sin 2010; 23(6): 549–578.

115.

Shintake

Rosset

Schubert

, et al. Versatile soft grippers with intrinsic electroadhesion based on multifunctional polymer actuators. Adv Mater 2016; 28(2): 231–238.

116.

Chao

. Research on a soft manipulator with variable stiffness. Nanjing: Nanjing Forestry University, 2021.

117.

Chen

Yang

, et al. Passive particle jamming and its stiffening of soft robotic grippers. IEEE Trans Robot 2017; 33(2): 446–455.

118.

Kim

Cheng

Kim

, et al. A stiffness-adjustable hyperredundant manipulator using a variable neutral-line mechanism for minimally invasive surgery. IEEE Trans Robot 2013; 30(2): 382–395.

119.

Wang

Ahn

. Shape memory alloy-based soft gripper with variable stiffness for compliant and effective grasping. Soft Robot 2017; 4(4): 379–389.

120.

Liang

Cao

Chen

, et al. Bio-inspired rigid-soft coexisting robotic fish. Sci Sin Technol 2018; 48: 1295–1301.

121.

Guofeng

. Design and experiment of variable stiffness gripper. North China University of Technology, 2021.

122.

Youquan

. Theoretical analysis and experimental study on variable stiffness pneumatic networks soft robotic manipulator. Hainan University, 2018.

123.

Kim

Cheng

Kim

, et al. A novel layer jamming mechanism with tunable stiffness capability for minimally invasive surgery. IEEE Trans Robot 2013; 29(4): 1031–1042.

124.

Rodrigue

Wang

Han

, et al. An overview of shape memory alloy-coupled actuators and robots. Soft Robot 2017; 4(1): 3–15.

125.

Liu

. Research on the software robot for fruits and vegetables picking in greenhouse. Wuhan: Hubei University of Technology, 2019.

126.

Yan

Shi

Zhang

, et al. Review of biomimetic mechanism, actuation, modeling and control in soft manipulators. J Mech Eng 2018; 54(15): 1–14.

127.

Wang

Hou

. Development of machine vision system for fruit harvesting robot. Hubei Agric Sci 2012; 51(21): 4705–4708.

128.

Zhao

. Research on technologies of target recognition, localization and control for tomato harvesting robot. Shanghai Jiao Tong University, 2018.

129.

Liu

. Research on image recognition methods for multifunctional fruit and vegetable harvesting robot. Jiangshu University, 2020.

130.

Patel

Jain

Joshi

. Automatic segmentation and yield measurement of fruit using shape analysis. Int J Comput Appl Technol 2012; 45(7): 19–24.

131.

Thendral

Suhasini

Senthil

. A comparative analysis of edge and color based segmentation for orange fruit recognition. In: Third international conference on communication and signal processing, ICCSP 2014, Melmaruvathur, Tamil Nadu, India, 3–5 April 2014, pp. 463–466. IEEE. DOI: 10.1109/ICCSP.2014.6949884.

132.

Dean

Xiaoyang

, et al. Image recognition at night for apple picking robot. Trans Chin Soc Agric Mach 2015; 46(3): 15–22.

133.

Jidong

Wei

. Fast tracing recognition method of target fruit for apple harvesting robot. Trans Chin Soc Agric Mach 2014; 45(1): 65–72.

134.

Yang

Chang

Bao

, et al. Recognition and localization system of the robot for harvesting Hangzhou White Chrysanthemums. Int J Agric Biol Eng 2018; 11(1): 88–95.

135.

. Recognition method for apple fruit based on SUSAN and PCNN. Multimed Tools Appl 2018; 77(6): 7205–7219.

136.

Wang

Song

, et al. An improved contour symmetry axes extraction algorithm and its application in the location of picking points of apples. Spanish J Agric Res 2015; 13(1): 205.

137.

Lyu

Zhao

, et al. Recognition of apple fruit in natural environment. Optik 2016; 127(3): 1354–1362.

138.

Wei

X Q

Jia

Lan J

, et al. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot. Optik 2014; 125(19): 5684–5689.

139.

Peng

Zou

Chen

, et al. Fast recognition of multiple color targets of litchi image infield environment based on double Otsu algorithm. Trans Chin Soc Agric Mach 2014; 45(4): 61–68.

140.

Luo

Tang

Zou

, et al. Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components. Sensors 2016; 16(12): 2098.

141.

Xiong

Lin

Liu

, et al. The recognition of litchi clusters and the calculation of picking point in a nocturnal natural environment. Biosyst Eng 2018; 166: 44–57.

142.

Tola

Al-Mallahi

, et al. A novel image processing algorithm to separate linearly clustered kiwifruits. Biosyst Eng 2019; 183: 184–195.

143.

Liu

Gong

Zhang

. Manipulating complex robot behavior for autonomous and continuous operations. Pisa: Service Robotics, 2020.

144.

Zheng

Jiang

Feng

. Vision based target recognition and location for picking robot: a review. Chinese J Sci Instrum 2021; 42(9): 28–51.

145.

Liu

. A multi-sensory end-effector for spherical fruit harvesting robot. In: 2007 IEEE international conference on automation and logistics, Jinan, China, 18–21 August 2007, pp. 258–262. IEEE. DOI: 10.1109/ICAL.2007.4338567.

146.

Xie

Guo

, et al. An object detection method for quasi-circular fruits based on improved Hough transform. Trans Chin Soc Agric Eng 2010; 26(7): 157–162.

147.

Meng

Wang

Tian

, et al. Research on navigation of agricultural UAV based on single chip microcomputer control. J Agric Mech Res 2020; 42(3): 245–248.

148.

Bao

Cai

, et al. Multi-template matching algorithm for cucumber recognition in natural environment. Comput Electron Agric 2016; 127: 754–762.

149.

Hannan

Burks

Bulanon

. A machine vision algorithm combining adaptive segmentation and shape analysis for orange fruit detection. Agric Eng Int: The CIGR J, 2009; XI: 1–7.

150.

Kelman

Linker

. Vision-based localisation of mature apples in tree images using convexity. Biosyst Eng 2014; 118: 174–185.

151.

Zhao

Tow

Katupitiya

On-tree fruit recognition using texture properties and color data. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, Edmonton, AB, Canada, 2–6 August 2005, pp. 263–268. IEEE. DOI: 10.1109/IROS.2005.1545592.

152.

Rakun

Stajnko

Zazula

. Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry. Comput Electron Agric 2011; 76(1): 80–88.

153.

Kurtulmus

Lee

Vardar

. Green citrus detection using ‘eigenfruit’, color and circular Gabor texture features under natural outdoor conditions. Comput Electron Agric 2011; 78(2): 140–149.

154.

Chaivivatrakul

Dailey

. Texture-based fruit detection. Precis Agric 2014; 15(6): 662–683.

155.

Lee

Gan

, et al. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis. Biosyst Eng 2018; 171: 78–90.

156.

Duan

Zou

, et al. Banana detection based on color and texture features in the natural environment. Comput Electron Agric 2019; 167: 105057.

157.

Kapach

Barnea

Mairon

, et al. Computer vision for fruit harvesting robots—state of the art and challenges ahead. Int J Comput Vis Robot 2012; 3(1–2): 4–34.

158.

Hannan

Burks

Bulanon

. A real-time machine vision algorithm for robotic citrus harvesting. Paper presented at the ASAE Annual International Meeting, 2007, Minneapolis, USA, ID: 073125.

159.

Yin

Chai

Yang

S X

, et al. Ripe tomato recognition and localization for a tomato harvesting robotic system. In: 2009 international conference of soft computing and pattern recognition, Malacca, Malaysia, 4–7 December 2009, pp. 557–562. IEEE. DOI: 10.1109/SoCPaR.2009.111.

160.

Patel

Jain

Joshi

. Fruit detection using improved multiple features based algorithm. Int J Comput Appl 2011; 13(2): 1–5.

161.

Linker

Cohen

Naor

. Determination of the number of green apples in RGB images recorded in orchards. Comput Electron Agric 2012; 81: 45–57.

162.

Nuske

Wilshusen

Achar

, et al. Automated visual yield estimation in vineyards. J Field Robot 2014; 31(5): 837–860.

163.

Liu

Whitty

. Automatic grape bunch detection in vineyards with an SVM classifier. J Appl Logic 2015; 13(4): 643–653.

164.

Zhao

Lee

. Immature green citrus detection based on colour feature and sum of absolute transformed difference (SATD) using colour images in the citrus grove. Comput Electron Agric 2016; 124: 243–253.

165.

Lee

Wang

. Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor colour images. Precis Agric 2016; 17(6): 678–697.

166.

Jana

Basak

Parekh

Automatic fruit recognition from natural images using color and texture features. In: 2017 devices for integrated circuit (DevIC), Kalyani, West Bengal, India, 23–24 March 2017, pp. 620–624. IEEE. DOI: 10.1109/DEVIC.2017.8074025.

167.

Inkyu

Zongyuan

Feras

, et al. Deep fruits: a fruit detection system using deep neural networks. Sensors 2016; 16(8): 1222–1228.

168.

Zhou

Zheng

, et al. Classification and recognition approaches of tomato main organs based on DCNN. Trans Chin Soc Agric Eng 2017; 33(15): 219–226.

169.

Feng

Elkamil

, et al. Image recognition method of multi-cluster kiwifruit in field based on convolutional neural networks. Trans Chin Soc Agric Eng 2018; 34(2): 205–211.

170.

Xiong

Liu

Tang

, et al. Visual detection technology of green citrus under natural environment. Trans Chin Soc Agric Mach 2018; 49(4): 45–52.

171.

Xue

Huang

, et al. Immature mango detection based on improved YOLOv2. Trans Chin Soc Agric Eng 2018; 34(7): 173–179.

172.

Liu

Yang

Ling

, et al. A visual system of citrus picking robot using convolutional neural networks. In: 2018 Fifth international conference on systems and informatics (ICSAI), Nanjing, China, 10–12 November 2018, pp. 344–349. IEEE. DOI: 10.1109/ICSAI.2018.8599325.

173.

Tang

. A deep learning method for recognizing elevated mature strawberries. In: 2018 33rd youth academic annual conference of Chinese Association of Automation (YAC), Nanjing, China, 18–20 May 2018, pp. 1072–1077. IEEE. DOI: 10.1109/YAC.2018.8406530.

174.

Tang

Chen

Wang

, et al. Recognition and localization methods for vision-based fruit picking robots: a review. Front Plant Sci 2020; 11: 510.

175.

Rocchini

Cignoni

Montani

, et al. A low cost 3D scanner based on structured light. Comput Graph Forum 2001; 20(3): 299–308.

176.

Kraus

Pfeifer

. Determination of terrain models in wooded areas with airborne laser scanner data. ISPRS J Photogramm Remote Sens 1998; 53(4): 193–203.

177.

Göbel

Kampa

Helmchen

. Imaging cellular network dynamics in three dimensions using fast3D laser scanning. Nature Methods 2007; 4(1): 73–79.

178.

Park

Kim

Tai

, et al. High quality depth map upsampling for 3d-tof cameras. In: 2011 international conference on computer vision, Barcelona, Spain, 6–13 November 2011, pp. 1623–1630. IEEE. DOI: 10.1109/ICCV.2011. 6126423.

179.

Schwarz

. Mapping the world in 3D. Nat Photonics 2010; 4(7): 429–430.

180.

Scharstein

Szeliski

. High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society conference on computer vision and pattern recognition, Madison, WI, USA, 18–20 June 2003, Vol. 1, pp. I–I. IEEE. DOI: 10.1109/CVPR.2003.1211354.

181.

Zhang

Huang

Wang

, et al. Computer vision recognition of stem and calyx in apples using near-infrared linear-array structured light and 3D reconstruction. Biosyst Eng 2015; 139: 25–34.

182.

Feng

Chen

Yang

. Identification and localization of overlapping tomatoes based on linear structured light vision system. J China Agric Univ 2015; 20(4): 100–106.

183.

Dong

Tong

Yibin

. Measurement of optical property parameters of fruit skin and flesh using structured illumination reflectance imaging. Tran Chin Soc Agric Eng (Trans CSAE) 2020; 36(7): 284–291.

184.

Jiménez

Ceres

Pons

. A vision system based on a laser range-finder applied to robotic fruit harvesting. Mach Vision Appl 2000; 11(6): 321–329.

185.

Tanigaki

Fujiura

Akase

, et al. Cherry-harvesting robot. Comput Electron Agric 2008; 63(1): 65–72.

186.

Feng

Liu

, et al. Apple fruit recognition algorithm based on laser scanning 3-D image. Trans Chin Soc Agric Mach 2013; 44(4): 217–222.

187.

Zhang

, et al. Design for apple-picking robot of intelligent recognition based on laser vision. J Agric Mech Res 2016; 38(7): 60–64.

188.

Liu

Feng

. 3D reconstruction of agriculture and forestry crops. Nongye Jixie Xuebao/Trans Chin Soc Agric Mach 2014; 45(6): 38–46.

189.

Wei

Wang

, et al. Structure optimization design of large field of view optical lens for the space TOF camera. Chinese J Sci Instrum 2020; 41(10): 121–128.

190.

Karkee

Adhikari

Amatya

, et al. Identification of pruning branches in tall spindle apple trees for automated pruning. Comput Electron Agric 2014; 103: 127–135.

191.

Nakarmi

Tang

. Automatic inter-plant spacing sensing at early growth stages using a 3D vision sensor. Comput Electron Agric 2012; 82: 23–31.

192.

Gongal

Silwal

Amatya

, et al. Apple crop-load estimation with over-the-row machine vision system. Comput Electron Agric 2016; 120: 26–35.

193.

Singh

Burks

Lee

. Autonomous robotic vehicle development for greenhouse spraying. Trans ASAE 2005; 48(6): 2355–2361.

194.

Yuhan

Han

Man

, et al. Navigation system for inspection robot based on LiDAR. Trans Chin Soc Agric Mach 2018; 49(2): 14–21.

195.

Shuai

Xiaogang

Chengtao

, et al. Overview on vision-based 3D reconstruction. Appl Res Comput 2011; 28(7): 2411–2417.

196.

Mehta

Burks

. Vision-based control of robotic manipulator for citrus harvesting. Comput Electron Agric 2014; 102: 146–158.

197.

YSH

Liu

Feng

, et al. Location of apples in trees using stereoscopic vision. Comput Electron Agric 2015; 112: 68–74.

198.

Tang

, et al. 3D imaging of greenhouse plants with an inexpensive binocular stereo vision system. Remote Sens 2017; 9(5): 508.

199.

Helveston

Boudreault

. Binocular vision and ocular motility: theory and management of strabismus. Am J Ophthalmol 1986; 101(1): 135.

200.

Yaguchi

Nagahama

Hasegawa

, et al. Development of an autonomous tomato harvesting robot with rotational plucking gripper. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), Daejeon, South Korea, 9–14 October 2016, pp. 652–657. IEEE. DOI: 10.1109/IROS.2016.7759122.

201.

Malekabadi

Khojastehpour

Emadi

. Disparity map computation of tree using stereo vision system and effects of canopy shapes and foliage density. Comput Electron Agric 2019; 156: 627–644.

202.

Liu

Yang

, et al. Three-dimensional spatial localization of overlapping citrus based on binocular stereo vision. J Agric Sci Technol 2020; 22(9): 104–112.

203.

Lei

Huirong

, et al. Design of binocular vision system for fruit and vegetable picking based on embedded arm. J Optoelectron Laser 2020; 31(1): 71–80.

204.

Baillard

Zisserman

. A plane-sweep strategy for the 3D reconstruction of buildings from multiple images. In: Proceedings of the 2000 international archives of photogrammetry and remote sensing. Amsterdam, Netherlands: ISPRS, 2000, pp. 56–62.

205.

Zheng

Huang

, et al. Key techniques for vision based 3D reconstruction: a review. Acta Autom Sin 2020; 46(4): 631–652.

206.

Hernández

Vogiatzis

Cipolla

. Overcoming shadows in 3-source photometric stereo. IEEE Trans Pattern Anal Mach Intell 2010; 33(2): 419–426.

207.

Park

Lee

Sull

. Efficient viewer-centric depth adjustment based on virtual fronto-parallel planar projection in stereo 3D images. IEEE Trans Multimed 2013; 16(2): 326–336.

208.

Bai

Rao

Wang

Shape vocabulary: a robust and efficient shape representation for shape matching. IEEE Trans Image Process 2014; 23(9): 3935–3949.

A review on structural development and recognition–localization methods for end-effector of fruit–vegetable picking robots

Abstract

Keywords

Introduction

Characteristics of end-effectors considering different grasping and separating ways

Characteristics of applied instances with different grasping methods

End-effectors with a sucking function

End-effectors with a clamping function

End-effectors with the sucking-clamping function

Separating methods and their instances

Rigid separation type

Flexible separation type

Summary of traditional structures

Developments of underactuated and soft manipulators

Characteristics of underactuated manipulators and their applied instances

Characteristics of soft manipulators and their applied instances

Materials for soft manipulators

Different driving modes

Variable stiffness technology

Target recognition methods

Single-feature vision method

Multi-feature fusion method

Deep learning method

3D reconstruction technologies for positioning fruits

Active vision-based reconstruction technologies

Passive vision-based 3D reconstruction technologies

Conclusion and future prospective

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References