Sage Journals: Discover world-class research

Abstract

Soft materials are driving the development of a new generation of robots that are intelligent, versatile, and adept at overcoming uncertainties in their everyday operation. The resulting soft robots are compliant and deform readily to change shape. In contrast to rigid-bodied robots, the shape of soft robots cannot be described easily. A numerical description is needed to enable the understanding of key features of shape and how they change as the soft body deforms. It can also quantify similarity between shapes. In this article, we use a method based on elliptic Fourier descriptors to describe soft deformable morphologies. We perform eigenshape analysis on the descriptors to extract key features that change during the motion of soft robots, showing the first analysis of this type on dynamic systems. We apply the method to both biological and soft robotic systems, which include the movement of a passive tentacle, the crawling movement of two species of caterpillar (Manduca sexta and Sphacelodes sp.), the motion of body segments in the M. sexta, and a comparison of the motion of a soft robot with that of a microorganism (euglenoid, Eutreptiella sp.). In the case of the tentacle, we show that the method captures differences in movement in varied media. In the caterpillars, the method illuminates a prominent feature of crawling, the extension of the terminal proleg. In the comparison between the robot and euglenoids, our method quantifies the similarity in shape to ∼85%. Furthermore, we present a possible method of extending the analysis to three-dimensional shapes.

Introduction

Soft materials are inspiring a new generation of intelligent, versatile, and adaptive robots capable of handling unexpected interactions in uncertain environments.¹ The nature of these materials is such that they exhibit large deformations under the loads that they typically encounter.² As a result, the robots fabricated out of these materials are compliant and readily change shape. The pursuit of such soft robots is driven by scientific endeavors to mimic a biological organism,³ to operate safely for human assistance,⁴ to harvest energy,⁵ or to impart intelligence to a robot's body.⁶ In these applications, elastic and nonlinear properties of soft materials are exploited to implement mechanisms that vary by body shape and stiffness.⁷

Unlike in the case of rigid bodies, it is not trivial to describe the shape of a soft deformable body. A quantitative measure to describe body shape is therefore extremely useful in the design and analysis of soft robots. Abstraction is a key step in bioinspired design⁸ and identifying the key features of a shape could influence design choices such as the material of construction or actuation technology. A numerical representation of the shape could be used to build a model of the compliant body. This description could be used to optimize locomotion of a soft robot,⁹ actively deform a compliant object through visual inspection,¹⁰ or in the case of morphological computation, be used to represent the intelligence inherent to the body of a robot.¹¹

Several approaches have been proposed to quantitatively describe shape. The difference between approaches is characterized by the features used to describe a form, such as the measures of distances, position of landmarks, description of the boundary, and details of texture. Landmark-based approaches consider relative pose of an identified set of points while excluding information about curvature between them. It has been argued that landmark based metrics are incomplete representations of the form and that much of the information of potential significance is not captured, particularly in the case of biological systems.¹² The choice of landmarks is also subjective. In contrast, methods that rely on description of boundaries focus on the outline of the form. In this article, we consider one such method that uses elliptic Fourier descriptors.¹³ This descriptor has been used to describe biological shapes such as those of nuclei,¹⁴ shells,¹⁵ leaflets,¹⁶ and roots.¹⁷ We have used this method and performed eigenshape analysis to characterize the shapes of model organisms and a soft robot based on them. We believe that this model-free method is appropriate to describe the shape of nonlinear, dynamic soft robots where the complexity of high dimensions restricts the use of model based approaches. It should be noted at the outset that since this approach uses information of the boundary alone and does not capture motion of landmarks, it comes with associated drawbacks such as its inability to capture mechanical interactions. Possible extensions to overcome this limitation are discussed at the end of the article. While the studies mentioned above used the descriptors to study static shapes, the novelty of the present work is in applying this method to describe dynamic shapes of soft robots.

In addition to presenting shape descriptors as a means of understanding changes in shape, we also compare the shape of a soft robot to that of its biological inspiration. A boundary based approach is advantageous in this case as there is no need to define equivalent landmarks between the systems, which may not be possible in all cases. We start with the analysis of a passive and compliant soft tentacle. We then look at the crawling locomotion of two different species of caterpillar, the Manduca sexta and the Sphacelodes sp. followed by a study on the shapes of individual segments in the M. sexta, whose movements have been used to inspire several nonpneumatic soft robots.¹⁸ Finally, we use the method to compare the performance of a soft robot³ to that of the unicellular organism, Eutreptiella sp., that has a characteristic form of locomotion featuring large shape change.

Materials and Methods

Shape estimation using elliptic Fourier descriptors

The elliptic Fourier descriptor of shape for closed contours¹³ belongs to a class of descriptors that use a Fourier series to approximate the boundary of a shape. It is a procedure that fits a closed curve to a set of two-dimensional (2D) points with arbitrary precision. The advantage of these descriptors over other boundary methods is that they preserve information of the contour such that it can be reconstructed in the absence of the original specimen.¹² In addition, both global and localized aspects of the contour are amenable to analysis. The descriptor can also be made invariant to translation, rotation, and scale. In this work, we implemented the descriptor in 2D (although line contours can be processed in three dimensional [3D^19,20]).

To capture shapes of the robots and organisms in all the experiments presented in this study, their movement was first recorded on video. Movement of the specimen can be described as a progression of shapes, each of which is captured in a single frame of the video. Individual frames were extracted as images and processed to extract the elliptic Fourier descriptors of shape (Fig. 1). Wherever required, these images were processed to highlight the regions of interest. In cases where the image had poor contrast, such as in the recording of organisms in their natural environments with no artificial light, desired regions were marked by hand. This was not required for the case of the robots since experiments were carried out in a well lit area against a contrasting background. An automated computer vision method was used to extract the desired region in such cases.

FIG. 1.

From the input image (a), the region of interest (b) is extracted. Next, the contour (c) is identified. From this, the Fourier coefficients are determined. The shape can then be reconstructed (d) from the coefficients.

A median filter with a 3 × 3 kernel was used to remove noise. The high resolution of the camera used for capture resulted in a jagged contour (Fig. 1b), which is likely to produce erroneous high frequency coefficients when performing Fourier analysis.¹² To avoid this, the image was dilated using a disk-shaped structuring element with a radius of three pixels. Next, the contour of this smoothed region was determined as a discrete representation of the boundary. The Freeman chain of integers²¹ was then used to obtain a piece-wise linear approximation of the contour while preserving information about the local orientation of the curve. The x and y projections of the chain segments can be represented as a Fourier series.¹³ The approximation up to N harmonics (X_n and Y_N) is given as $\begin{matrix} X_{N} (l) = a_{0} + \sum_{n = 1}^{N} [a_{n} cos \frac{2 n π l}{L} + b_{n} sin \frac{2 n π l}{L}], \\ Y_{N} (l) = c_{0} + \sum_{n = 1}^{N} [c_{n} cos \frac{2 n π l}{L} + d_{n} sin \frac{2 n π l}{L}] . \end{matrix}$ (1)

Here, l is one step along the contour (periodic with period L), a₀ and c₀ are the bias components of the Fourier series corresponding to a frequency of 0, and a_n, b_n, c_n, and d_n are coefficients of the nth harmonic. These coefficients are given by the following expressions and constitute the description of the shape. Here, K is the number of points on the contour. $\begin{matrix} a_{n} = \frac{L}{2 n^{2} π^{2}} \sum_{p = 1}^{K} \frac{Δ x_{p}}{Δ l_{p}} [cos \frac{2 n π l_{p}}{L} - cos \frac{2 n π l_{p - 1}}{L}], \\ b_{n} \frac{L}{2 n^{2} π^{2}} \sum_{p = 1}^{K} \frac{Δ x_{p}}{Δ l_{p}} [sin \frac{2 n π l_{p}}{L} - sin \frac{2 n π l_{p - 1}}{L}] . \end{matrix}$ (2)

\begin{matrix} c_{n} = \frac{L}{2 n^{2} π^{2}} \sum_{p = 1}^{K} \frac{Δ y_{p}}{Δ l_{p}} [cos \frac{2 n π l_{p}}{L} - cos \frac{2 n π l_{p - 1}}{L}], \\ d_{n} = \frac{L}{2 n^{2} π^{2}} \sum_{p = 1}^{K} \frac{Δ y_{p}}{Δ l_{p}} [sin \frac{2 n π l_{p}}{L} - sin \frac{2 n π l_{p - 1}}{L}] . \end{matrix}

(3)

Once the set of Fourier coefficients has been computed, the curve was reconstructed using the same equations as those for the approximation. The number of harmonics dictates the accuracy of the approximation. This is shown visually in Figure 2a. To quantify the closeness of fit, an error function in terms of the mean difference between points on the reconstructed contour and the original contour was used.

FIG. 2.

(a) Estimate of the shape of the tentacle as the number of harmonics increases. (b) Mean error in the estimate of 98 distinct shapes of the tentacle during its movement. The shaded region indicates one standard deviation away from the mean.

e_{f} = \frac{1}{K} \sum_{i = 1}^{K} \sqrt{{(x_{i} - r_{i})}^{2} + {(y_{i} - s_{i})}^{2}} .

(4)

Here, x_i and y_i are points on the original contour, and r_i and s_i are points on the reconstructed contour that are closest in Euclidean distance from x_i and y_i, respectively. The average e_f across 98 distinct shapes of the tentacle during its movement was computed for increasing the number of harmonics (Fig. 2b). As can be observed, the residual drops significantly with increasing harmonics up to a certain point. In this case, at 20 harmonics, the mean error across shapes is less than 0.5 mm (for a robot of length 400 mm and mean diameter of 25 mm) and can be considered to be a sufficiently detailed approximation.

Eigenshape analysis

As described above, the representation of a shape in terms of elliptic Fourier features produces four coefficients per harmonic, which constitute its mathematical description. An approximation using N harmonics results in a set of 4N − 3 normalized coefficients. For a video containing a sequence of M frames with one shape in each frame, the complete description of all shapes is a matrix of coefficients $D \in ℝ^{M \times (4 N - 3)}$ (Fig. 3). These coefficients have little physical meaning in their raw form. To extract key features that describe the shape in a physical sense, principal component analysis was used.²² The aim is to describe independent trends in shape, for which the principal components are an ideal choice by virtue of being mutually orthogonal. In addition, we seek to reduce the dimensionality of data by describing it in terms of a smaller number of components, N_e << (4N − 3).

FIG. 3.

The matrix $D \in ℝ^{M \times (4 N - 3)}$ consists of normalized Fourier coefficients (ⁱa_j, superscript indicating the frame number and subscript indicating the harmonic that it corresponds to). Using principal component analysis followed by dimensionality reduction, a reduced matrix of coefficients $D E \in ℝ^{M \times N_{e}}$ is obtained, where $E \in ℝ^{(4 N - 3) \times N_{e}}$ is the matrix of N_e eigenvectors. Elements of DE, that is, ⁱs_j represent component scores with superscripts indicating the frame number and subscripts indicating the principal component that the coefficient corresponds to.

Each row of D represents one shape. First, the covariance matrix of D, $C_{D} \in ℝ^{(4 N - 3) \times (4 N - 3)}$ is computed. Next, the eigenvectors of the covariance matrix are evaluated. Let $E \in ℝ^{(4 N - 3) \times N_{e}}$ be the matrix whole columns that contain the first N_e eigenvectors of the covariance matrix, C_D. The rows in the product matrix $D E \in I ℝ^{M \times N_{e}}$ represent the coefficients (also referred to as component scores) in terms of the eigenvectors and convey a more meaningful description of the shape. Variance of columns in DE is the corresponding eigenvalues. These represent the amount of variance in shape captured by each eigenvector.

The mean shape can be constructed using the mean of the coefficients. The effect of each eigenvector can be independently studied to understand which features of the shape change during deformation. To compare shapes of two entities such as a robot and an organism, variances of columns in the product matrices D_rE_o and D_oE_r (subscripts indicate robot or organism) were considered. The resulting eigenvalues, computed between two sets of data, are thus a quantitative measure of similarity.

Case Studies

Soft tentacle

In the first case study, we analyzed the movement of a soft tentacle.¹¹ The body is entirely passive with no sensors or actuators present along its length. In the presence of an oscillatory actuation at one extremity, the shape of the body is purely a result of its interaction with the environment. The material of construction (Ecoflex 30, Smooth-On) is nonlinear.²³ The deformation of the structure and shape of the body are difficult to predict. This makes it a suitable candidate for the current study where a model-free approach is used to describe the shape.

The tentacle was suspended from a movable platform and enclosed in a tank filled with either water or air. The platform was oscillated along a horizontal rail at a specific frequency and amplitude. Two different input oscillations were provided, one with an amplitude of 12.5 mm and frequency of 3 Hz and the other with an amplitude of 40 mm and frequency of 1.5 Hz. The experiment was performed twice, once with the tentacle in water and the next with it in air. The deformation of the tentacle in the plane of motion was recorded at 30 fps using a camera (Fujifilm S2100HD). Descriptors of shape were extracted from each frame using an approximation of up to 20 harmonics.

The data from the shapes were then reduced to three principal components. The first two components are shown for each frame of the swinging tentacle (f = 3Hz, A = 12.5mm) in Figures 4 and 5a and b. Horizontal lines in the plot indicate one standard deviation (σ) from the mean (μ). To illustrate the effect of principal components on the shape representation of the robot tentacle, weights on the components were independently varied up to two standard deviations on either side of the mean, and the reconstructed shapes have been plotted. Figure 4c and d corresponds to the case when the tentacle is moving in water, and Figure 5c and d corresponds to that in air. The components for which the weights were varied are indicated as $μ_{i} \pm σ_{i}, i \in \{1, 2\}$ , whereas those that were maintained constant at the mean are indicated as $μ_{j}, j \in \{1, 2\}$ . The third principal component is not shown here and was fixed at its mean in all the reconstructed shapes. The corresponding figures for the second actuation signal (f = 1.5Hz, A = 40mm) are presented in the Supplementary Data (Supplementary Figs. S1 and S2).

FIG. 4.

Estimated shapes of the tentacle when moving in water. Change in scores on the (a) first and (b) second principal components. (c, d) show the effect on shape due to changes in weights on these principal components. The subscripts 1 and 2 indicate the component over which the mean and standard deviation were computed.

FIG. 5.

Estimated shapes of the tentacle when moving in air. Change in scores on the (a) first and (b) second principal components. (c, d) The effect on shape due to changes in weights on these principal components. The subscripts 1 and 2 indicate the component over which the mean and standard deviation were computed.

These results demonstrate the ability of the elliptic Fourier descriptors to describe the shape of an unpredictable, deformable soft robot to an arbitrary degree of precision. The principal components describe different aspects of the shape. In case of the experiment in water, the first principal component captures the curvature of a larger portion of the structure (Fig. 4c), while the second component has a more local influence and describes subtle changes in shape near the tip (Fig. 4d). This implies that there is a more prominent global change in shape as opposed to the curving of the tip.

As expected, the shapes exhibited by the tentacle are different in various media. The shape has a higher order of curvature when the medium is water (S shaped, Fig. 4) compared to that in air (C shape, Fig. 5). Curving of the tip is not predominant when the medium is air. This is immediately evident when the shapes are plotted as points in the space of the components (Fig. 6). The points have less span along the second principal component in the case of air. In terms of modes of vibration, movement in air can be described as being composed of fewer prominent modes than that in water, as shown by the principal components.

FIG. 6.

Shapes plotted as points in the space of the components. The motion of the tentacle is a closed loop in the space of the first three principal components. The curve in violet represents the movement of the tentacle through water. The green curve is for the experiment in air. Color images are available online.

Crawling motion of caterpillars

The crawling motion of two different species of caterpillar was studied next. The first organism, the M. sexta (tobacco hornworm), has been described as an ideal organism to study soft-bodied locomotion.²⁴ This caterpillar shifts the segments of its body as a succession of steps, delayed in phase, from the posterior to the anterior. In the second species, Sphacelodes sp., there is a more pronounced change in the body shape termed “inching” locomotion, with the organism curling up into a shape similar to the Greek letter Ω (Fig. 8c).

Movement of both the caterpillars was recorded in a natural setting. Individual frames from the video (recorded at 30 fps) were processed manually to highlight the body of the caterpillar. This was necessary because the creatures were camouflaged and it was not possible to isolate the creature from its background using an automated computer vision technique. In both cases, six harmonics were sufficient to capture the details of shape.

Deformations of the body during locomotion of the caterpillars, M. sexta and Sphacelodes sp., are shown in Figure 7. Again, scores on the first two principal components were varied to illustrate the effect of their change on specific features of the shape. The difference in gaits of the two species is immediately evident. The first principal component in the case of the M. sexta captures the convexity in the shape of the body. In other words, it gives information on whether the body is raised or lying flat on the ground. The location along the length of the body where the body is raised from the ground is captured by the second component. In the case of the Sphacelodes sp., the first principal component again describes the convexity of the body. It is more pronounced in this case, capturing the curling up of the body. The second component represents the leaning of the body in the curled up configuration, either toward the anterior or the posterior.

FIG. 7.

Effect of changes in the first two principal components (a, b) on the estimated shapes of Manduca sexta and Sphacelodes sp. The subscripts 1 and 2 indicate the component over which the mean and standard deviation were computed. Color images are available online.

In addition to looking at the deformation of the whole body of the caterpillar, changes in shapes of individual segments of the body in the case of the M. sexta were also analyzed. A potential application is in identifying parts of the body that perform specific functions during locomotion such as initiating movement. Individual segments of the body were identified (Fig. 8) and marked by hand on each frame of a high resolution video showing the creature in motion. In particular, abdominal segments A3 to A6 and the terminal segment, TS, were analyzed. Movement of the thoracic segments was not considered as it has been found that they are not necessary to achieve locomotion.²⁴

FIG. 8.

(a) Segments on the body of the Manduca sexta as identified in Ref.,²⁴ marked using dotted lines: TS, TP, (A3–A6) abdominal segments, (T) thoracic segments, and (H) head. (b) Close-up views showing the extension and contraction of the TP. (c) Two frames from the motion of the Sphacelodes sp. showing the neutral from and characteristic curled up Ω form. TP, terminal proleg; TS, terminal segment.

Changes in shape of individual segments of the M. sexta are shown in Figure 9. The first principal component was varied while keeping the second component fixed at the mean. To understand the phase difference between movement of segments, the scores on these components are plotted as function of time (Fig. 10).

FIG. 9.

Effect of changes in the first principal component on the estimated shapes of segments in the Manduca sexta. TS, (A3 to A6) abdominal segments (defined in Fig. 8). The subscripts 1 and 2 indicate the component over which the mean and standard deviation were computed. Color images are available online.

FIG. 10.

Scores on the first principal component for various segments in the Manduca sexta as a function of time. Color images are available online.

Since the principal components capture the most significant changes in shape, analysis of individual body segments revealed interesting insights regarding the nature of movement of each segment. The most prominent change is seen in case of the terminal segment, where the shape changes are due to the extension of the terminal proleg (TP). The extension of TP (Fig. 8b) precedes the swing phase of the segment at the start of each crawl cycle and has been described as being a distinctive feature of caterpillar locomotion.²⁴ This extension represents the drag force component of the stance phase, which is a characteristic of the tension-based crawling strategy used by Manduca.²⁵ This key feature is captured by the first principal component (Fig. 9). In addition, the kinematic analysis of movement of the abdominal segments revealed that segments A3 to A6 see little extension of the prolegs.²⁴ This is seen in Figure 9 where there is no significant change in the shape toward the bottom of the segments A3 to A6. In contrast, the segments bulge out toward the top and curve inwards near the middle. More analysis is required to understand what these changes represent and if they are purely a result of the rotation of segments or if they are caused due to movement of internal organs which has been observed in the M. sexta.²⁶

The principal component scores plotted against time for the various segments (Fig. 10) show the progression of movement from the posterior to the anterior through the abdominal segments. There is about 2.5 s of phase delay between the motion of the segments, which agrees with the results from the kinematic analysis. This demonstrates potential for the use of the shape descriptors in combination with eigenanalysis to discern dynamic properties of locomotion. Other features of locomotion such as steering and grasping that are commonly observed in these caterpillars can be studied in a similar manner using images from multiple viewing directions.

Euglenoid movement and comparison with a robot

In the last case, the locomotion of a soft robot²⁷ was analyzed and compared to its biological inspiration, a unicellular protist called a euglenoid. The specimen in our study is the Eutreptiella spyrogyra.²⁸ The purpose of this case study is to present this method of shape analysis as a quantitative measure of comparison between shapes, which, in this case, allows us to compare a soft robot and a biological organism.

The euglenoids display a characteristic type of locomotion called euglenoid movement in which the body undergoes a giant change in shape.²⁹ The body undergoes a transition in shape from a spherical ball-like form to an elongated rod-like form with a wide range of intermediate shapes (Fig. 11b). Measurements of the cell body of Euglena fusca have shown that it nearly doubles in radius while contracting by about 37% in the longitudinal direction.³⁰ Change in shape is crucial to the locomotion of this organism because of the low Reynolds number regime in which it moves.³¹ A detailed analysis is presented in Reference.³² Taking inspiration from the euglenoid, we designed a completely autonomous soft robot, using hyperelastic bellow actuators,³ that moves by replicating euglenoid movement (Fig. 11c). The robot was made to swim autonomously in a tank of viscous fluid to replicate the natural hydrodynamics. Change in shape was achieved by moving fluid between the chambers of the robot. Videos of the robot were recorded on camera and processed as described in Shape Estimation Using Elliptic Fourier Descriptors section. The shapes of the robot were compared to that of the euglenoid.

FIG. 11.

(a) Euglenoid (Eutreptiella spyrogyra²⁸) and a soft robot that replicates euglenoid movement. (b) Various shapes of the euglenoid ranging from an elongated slender form to a rounded up form. (c) Various configurations of the soft robot demonstrating shapes similar to that of the euglenoid.

The comparison of shapes is presented in Figure 12. The dashed line shows the true shape of the euglenoid (Fig. 12a) and the soft robot (Fig. 12b). Following the comparison process described in Eigenshape Analysis section, shapes of the robot were reconstructed (solid outlines) using components from the organism and vice versa.

FIG. 12.

Comparison of shapes between euglenoid and the soft robot at three different instances during one cycle of locomotion. (a) The dashed line indicates the true shape of the euglenoid. The solid outline is the shape estimated using the scores on the first principal component from the shape of the robot. (b) The dashed line indicates the true shape of the soft robot. The solid outline is the shape estimated using the scores on the second principal component from the shape of the euglenoid.

In addition to demonstrating visual likeness, the principal components can be used to quantify similarity in shapes. Care must be taken to choose the components that describe the same trend in shape from both the systems for a meaningful comparison. The first principal component that describes the anterior–posterior mass transfer captures 78.78% of the shape of the euglenoid when described using eigenvectors from the robot. Similarly, 84.95% of robot shapes were described by the eigenvectors from the euglenoid. A key highlight in this study is that it enabled the direct qualitative and quantitative comparison of a microscopic organism and a centimeter scale robot. Thus, the method of shape comparison can be applied to studies across a diversity of scales.

Discussion

Generality of shapes

The method of describing contours as a set of descriptors is applicable to a wide range of shapes. The case studies showed the approximation applied to simple shapes. A few specific examples are presented here to demonstrate the generality of shapes to which the method may be applied and the limitations involved.

First, consider the case of hollow shapes such as the deformable silicone ring presented in Figure 13a and b. Shapes such as this are characterized by more than one contour; one for the outer boundary and one for each hollow form within it. The shape descriptors can then be computed for each contour separately. Change in shape can also be tracked independently. Note that the number of contours is an upper bound for the number of sets of descriptors for each shape. In special cases such as a toroid where the deformation of one contour may be deduced from knowing the deformation of the other, or in the case of a meta-material where the deformation of hollow forms is similar across the bulk of the material, fewer sets of descriptors may be required.

FIG. 13.

(a) Hollow object before deformation. (b) Hollow object after deformation. (c) Koch snowflake of 1 fractal iteration approximated using elliptic Fourier descriptors with 20 harmonics. (d) Koch snowflake of 5 fractal iterations approximated using descriptors with 1000 harmonics. (e) Koch snowflake of 10 fractal iterations approximated using descriptors with 100,000 harmonics. (f) Dragon fractal with six iterations. (g) Approximation using descriptors with 100 harmonics. (h) T shaped curve with arrows indicating direction of contour. (i) Approximation using descriptors with five harmonics. (j) Approximation using descriptors with 25 harmonics. (k) Two rectangles overlapping each other. (l–o) Approximations of contours using different traversal rules. Color images are available online.

Shapes with more complex contours such as fractals of increasing complexity are considered next. Take the Koch snowflake shown in Figure 13c–e, with 1, 5, and 10 iterations, respectively. This fractal generates closed curves in the plane. The number of harmonics used to reproduce the curves was 20, 1000, and 100,000, respectively. Although the approximation captures the general shape of the contour, a very large number of harmonics was required in the case of the complex fractal (10 iterations) to capture the more intricate undulations. In such a case, the shape is perhaps better described quantitatively in terms of fractal iterations rather than in terms of the elliptic Fourier descriptors.

In theory, the shape descriptors presented are strictly defined for closed contours. However, open curves in the 2D plane may also be approximated. The contour in this case is a directed curve formed by traversing the open curve from one end to the other and back again in the opposite direction. A simple example is that of a straight line which the descriptors approximate as a flattened ellipse. A more complex open curve with multiple changes in direction and no self-intersection is shown in Figure 13f and g, which is the sixth iteration of the dragon fractal. The approximated curve with 100 harmonics resembles the fractal to a close degree but fails to capture all the details.

Curves with intersecting lines or branches are also amenable to approximation. Consider the T-shaped curve in Figure 13h. The contour in this case may be defined as a directed curve that traces around the top line and shaft forming a closed loop (see arrows). Approximations using 5 and 25 harmonics are shown in Figure 13i and j, respectively. Self-intersecting curves may also be treated in a similar manner by carefully defining the contour as a directed closed curve. In these cases, it is also possible to calculate an upper bound for the error of approximation.¹³

The final case that is presented, which may be more relevant to image based generation of contours, is that of two rectangles overlapping each other (Fig. 13k). Assume, for the sake of discussion, that this picture is the result of some preprocessing applied to the image of a system. Many variations of directed closed curves exist for this pair of rectangles, each distinguished by the direction that the contour follows at the intersection. For instance, when traversing the curve in an anticlockwise direction, turn right at intersection rule results in the approximation of Figure 13l, turn left at intersection rule results in the approximation of Figure 13m, go straight at intersection rule results in that of Figure 13n, and an alternating turn rule results in a single curve as in that of Figure 13o. Resolving the direction of the curve at the intersection is a key step in arriving at the correct representation of the contour. The location of the starting point of the contour also affects the result. Additional information about the system is necessary to determine the correct rule for traversing the contour. For example, occlusion analysis might reveal that the two rectangles have different depth values, and hence, the correct resolution is that from Figure 13n. This would be useful in the case of the tentacle arm from Soft Tentacle section when it overlaps itself. Other information such as that from 3D capture, knowledge of geometry, and multiple viewpoints could potentially assist in defining the contour correctly.

The case studies of Case Studies section looked at motions that were periodic for the most part. This resulted in shapes that repeat periodically, and hence, the weights on principal components were also periodic (e.g., Figs. 4 and 5). However, the analysis may also be applied to nonperiodic movements. In this case, the observations would vary depending on the duration over which they are made. An example of a nonperiodic motion is shown in the Supplementary Data (Supplementary Fig. S3). The system analyzed was the passive tentacle from Soft Tentacle section, but the actuation was nonperiodic. Conclusions regarding the trends in shape captured by the principal components can still be drawn and are distinguishable in the figure even when the change in component weights is not periodic.

Extension to three dimensions

The analysis presented above was on 2D shapes of robots and organisms. The method of using elliptic Fourier descriptors can be extended to 3D movement. One possible extension would be to consider a third coordinate z in addition to the x and y coordinates and represent it as a Fourier series on its own. This method was followed to study the shape of the skull in rabbits.¹⁹ It is to be noted that this method helps in describing 3D curves but not surfaces or volumes. An example would be the curve traced by the end effector of a soft manipulator.

A different approach is to treat the 3D object as a collection of 2D shapes. For instance, a study on the anatomy of the nasal cavity used 3D geometry obtained from CT scans and sliced it into 2D curves, each of which was represented using Fourier descriptors.³³ A method of reconstructing the 3D geometry from sectional slices using radial basis functions is also proposed. In a similar manner, we visualize the 3D shape from two different viewing planes, essentially reducing the problem to a pair of simultaneous 2D representations. Since the silhouette is recorded in both the viewing planes, this approach captures surface projections of a 3D object. We applied this approach to the motion of an actively controlled soft pneumatic tentacle. Unlike the passive tentacle considered in the first case study, the continuum robot considered here consists of three segments that can be actuated independently and bend in orthogonal directions. The tentacle was suspended from a rigid support and its motion was recorded from two perpendicular viewing directions. Two of the segments (S1 and S3 in Fig. 14) were actuated to move in directions perpendicular to each other. The corresponding shapes and principal components are shown in Figure 14. The first principal component in each case clearly captures the deformation of the tentacle at different positions on the body as seen from the two viewpoints.

FIG. 14.

Estimated shapes of the active tentacle when captured from two perpendicular viewpoints. Bending of segment S1 is in the plane of the camera recording from view 2 and perpendicular to that from view 1 and is therefore not visible in the first camera. The effect on shape due to change in the first principal component is shown.

Limitations

In this section we point out some of the limitations of using elliptic Fourier descriptors. The descriptors only contain information regarding the perimeter (silhouette) of the organism or robot under investigation. Information concerning position and orientation of an entity is neglected. This limits their use in kinematic analysis of motion. For example, the analysis of body segments of the caterpillar cannot determine the degree of rotation or the lateral displacement of each segment during a cycle of locomotion. Using our shape descriptors in combination with other additional shape information such as a set of landmarks would help overcome this problem. In addition, some details of the shape are not captured. For example, points and features on the surface of the 3D object are not projected. There is therefore a partial loss in information in terms of the evolving location in space of these markers. Hence this approach may not be suitable for studies of mechanical interactions between objects and their surroundings where the interaction is defined by the position and velocity of markers on the surface. Because of this, our approach cannot say anything about the frictional interaction between the caterpillar's feet and the substrate, which is a key feature of caterpillar locomotion and its control. Similarly, in the case of hydrodynamic studies, this approach is unable to provide information regarding the flow of the surface. This problem has been addressed in the case of euglenoids³⁴ and in the change in shape and the emerging flow around fluid membranes.³⁵ A third example is one that dealt with evolving shapes and flows of liquid droplets.³⁶ The study on nasal cavities mentioned earlier³³ proposed that the slicing of geometry could be chosen to pass through landmark points, providing yet another way to keep track of them. The methods described in these studies could provide inspiration to overcome some of the limitations of the proposed approach.

Using multiple viewpoints of the 3D object has been proposed as a possible solution to capture more information about the shape of the deformable object. This approach may not be feasible in certain applications such as the study of microscopic organisms where it is difficult to obtain images from different directions with sufficiently high magnification. Additional information about the system can help overcome this limitation. An interesting study is the flagellar swimming in the case of a unicellular organism (Euglena gracilis) where simultaneous knowledge of 2D shapes and the corresponding orientation of the object were used to reconstruct the 3D shape of a revolving object,³⁷ thus recovering the lost information.

The descriptors are susceptible to artifacts caused during recording. For example, in earlier trials with the passive tentacle, the camera that was used to record motion was not sufficiently fast to capture the swinging movement of the robot. This resulted in some image frames containing a blur at the tip where the body was swinging at the fastest velocity. If not accounted for during preprocessing, the principal components capture this as a bulging of the tip. This might result in the under representation of some components, skewing the analysis.

Conclusion and Future Work

In this work, we have shown how shapes of soft robots and soft organisms can be described using elliptic Fourier descriptors. The eigenshape analysis identified key features that change during the movement of these entities. Different modes of oscillation were identified in various media in the case of the passive tentacle. Differences in locomotion between two species of caterpillar were captured by the shape descriptors. In the case of the locomotion of the M. sexta, the analysis correctly identified extension of the terminal segment as the key feature of crawling. Dynamics such as cyclic shape change and phase delay between movements of segments were also captured. Furthermore, we used the descriptors to quantitatively measure similarity of shapes and showed an 85% similarity in shape between a soft robot and the euglenoid. We also showed that the method works across diverse scales.

Extending the use of elliptic Fourier descriptors to dynamic tasks such as the control of a deformable robot will be considered in the future. A possible application is in using visual feedback to deform objects into a desired shape.³⁸ Applications to morphological computation will also be studied. An interesting use of the measures would be to quantify the amount of computation happening in the body using a low dimensional representation of observed behavior to distinguish between morphologies that are useful and harmful to a desired functionality.³⁹ Quantifying shape change also enables the use of artificial intelligence approaches (e.g., neural networks) to analyze or control soft robots, since the component scores can be used to train the neural network.

Footnotes

Acknowledgments

This work was supported by the EPSRC Centre for Doctoral Training in Future Autonomous and Robotic Systems (FARSCOPE, grant EP/L015293/1) at the Bristol Robotics Laboratory where K.M.D. is a PhD student. A.T.C. was supported by EPSRC grant EP/P025846/1. J.R. was supported by EPSRC grants EP/M020460/1 and EP/M026388/1 and was also funded by the Royal Academy of Engineering as a Chair in Emerging Technologies. Work on caterpillar locomotion was supported by National Science Foundation (USA) grant IOS:145671 awarded to BAT. The authors thank Gabor Soter and Yufan Wu for designing the soft robotic tentacles used in this study.

Data Disclosure Statement

All underlying data to support the conclusions are provided within this article.

Author Disclosure Statement

No competing financial interests exist.

Supplementary Material

References

Kim

, Laschi

, Trimmer

. Soft robotics: a bioinspired evolution in robotics. Trends Biotechnol, 2013; 31:287–294.

Trimmer

, Lin

. Bone-free: soft mechanics for adaptive locomotion. Integr Comp Biol, 2014; 54:1122–1135, .

Digumarti

, Conn

, Rossiter

. Euglenoid-inspired giant shape change for highly deformable soft robots. IEEE Robot Autom Lett, 2017; 2:2302–2307.

Diteesawat

, Helps

, Taghavi

, et al. High strength bubble artificial muscles for walking assistance. In IEEE International Conference on Soft Robotics. Livorno, Italy: IEEE, 2018.

Cao

, Conn

. Elastic actuation for legged locomotion. In Bar-Cohen

(ed.). Electroactive Polymer Actuators and Devices (EAPAD), 2017, vol. 10163. Bellingham, WA: International Society for Optics and Photonics, p. 101632W, 2017.

Pfeifer

, Iida

, Bongard

. New robotics: design principles for intelligent systems. Artif Life, 2005; 11:99–120.

Calisti

, Picardi

, Laschi

. Fundamentals of soft robot locomotion. J R Soc Interf, 2017; 14:20170101.

Kovač

The bioinspiration design paradigm: a perspective for soft robotics. Soft Robot, 2014; 1:28–37.

Vikas

, Grover

, Trimmer

. Model-free control framework for multi-limb soft robots. In IEEE International Conference on Intelligent Robots and Systems, vol. 2015. Hamburg, Germany: IEEE, pp. 1111–1116, 2015.

10.

Navarro-Alarcón

, Liu

Y-H

, Romero

, et al. Model-free visually servoed deformation control of elastic objects by robot manipulators. IEEE Trans Robot, 2013; 29:1457–1468.

11.

Soter

, Conn

, Hauser

, et al. Bodily aware soft robots: integration of proprioceptive and exteroceptive sensors. In IEEE International Conference on Robotics and Automation. Brisbane, Australia: IEEE, 2018.

12.

Lestrel

PE.

Fourier Descriptors and Their Applications in Biology. Cambridge, United. Kingdom: Cambridge University Press, 2008.

13.

Kuhl

, Giardina

. Elliptic Fourier features of a closed contour. Comput Graph Image Process, 1982; 18:236–258.

14.

Diaz

, Zuccarelli

, Pelligra

, et al. Elliptic Fourier analysis of cell and nuclear shapes. Comput Biomed Res, 1989; 22:405–414.

15.

Ferson

, Rohlf

, Koehn

. Measuring shape variation of two-dimensional outlines. Syst Biol, 1985; 34:59–68.

16.

Furuta

, Ninomiya

, Takahashi

, et al. Quantitative evaluation of soybean (Glycine max L. Merr.) leaflet shape by principal component scores based on elliptic Fourier descriptor. Japan J Breed, 1995; 45:315–320.

17.

Iwata

, Niikura

, Matsuura

, et al. Evaluation of variation of root shape of Japanese radish (Raphanus sativus L.) based on image analysis using elliptic Fourier descriptors. Euphytica, 1998; 102:143–149.

18.

Umedachi

, Vikas

, Trimmer

. Softworms: the design and control of non-pneumatic, 3d-printed, deformable robots. Bioinspir Biomimet, 2016; 11:025001.

19.

Lestrel

, Read

, Wolfe

Size and shape of the rabbit orbit: 3-D Fourier descriptors. In Lestrel

(ed.). Fourier Descriptors and Their Applications in Biology. Cambridge, United Kingdom: Cambridge University Press; 1997, pp. 359–378.

20.

Godefroy

, Bornert

, Gros

, et al. Elliptical Fourier descriptors for contours in three dimensions: a new tool for morphometrical analysis in biology. C R Biol, 2012; 335:205–213.

21.

Freeman

Computer processing of line-drawing images. In Organick

(ed.). ACM Computing Surveys (CSUR), vol. 6. New York: ACM, pp. 57–97, 1974.

22.

Adebowale

, Nicholas

, Lamb

, et al. Elliptic Fourier analysis of leaf shape in southern African Strychnos section Densiflorae (Loganiaceae). Bot J Linn Soc, 2012; 170:542–553.

23.

Elsayed

, Vincensi

, Lekakou

, et al. Finite element analysis and design optimization of a pneumatically actuating silicone module for robotic surgery applications. Soft Robot, 2014; 1:255–262.

24.

Trimmer

, Issberner

. Kinematics of soft-bodied, legged locomotion in Manduca sexta larvae. Biol Bull, 2007; 212:130–142.

25.

Lin

, Trimmer

. The substrate as a skeleton: ground reaction forces from a soft-bodied legged animal. J Exp Biol, 2010; 213:1133–1142.

26.

Simon

, Woods

Jr , Serebrenik

, et al. Visceral-locomotory pistoning in crawling caterpillars. Curr Biol, 2010; 20:1458–1463.

27.

Digumarti

, Conn

, Rossiter

. Eumobot: replicating euglenoid movement in a soft robot. J R Soc Interf, 2018; 15:20180301.

28.

The euglenoid project. http://euglena.msu.edu/index.shtml (accessed February 5, 2018).

29.

Suzaki

, Williamson

. Euglenoid movement in Euglena fusca: evidence for sliding between pellicular strips. Protoplasma, 1985; 124:137–146.

30.

Suzaki

, Williamson

. Cell surface displacement during euglenoid movement and its computer simulation. Cytoskeleton, 1986; 6:186–192.

31.

Purcell

EM.

Life at low Reynolds number. In Huang

(ed.). Physics and Our World: Reissue of the Proceedings of a Symposium in Honor of Victor F Weisskopf. Singapore: World Scientific; 2014, pp. 47–67.

32.

Arroyo

, Heltai

, Millán

, et al. Reverse engineering the euglenoid movement. Proc Natl Acad Sci, 2012; 109:17874–17879.

33.

Gambaruto

, Taylor

, Doorly

. Decomposition and description of the nasal cavity form. Ann Biomed Eng, 2012; 40:1142–1159.

34.

Arroyo

, DeSimone

Shape control of active surfaces inspired by the movement of euglenids. J Mech Phys Solids, 2014; 62:99–112.

35.

Arroyo

, DeSimone

. Relaxation dynamics of fluid membranes. Phys Rev E, 2009; 79:031915.

36.

Bouillant

, Mouterde

, Bourrianne

, et al. Leidenfrost wheels. In APS Meeting Abstracts. College Park, MD, 2017.

37.

Rossi

, Cicconofri

, Beran

, et al. Kinematics of flagellar swimming in Euglena gracilis: helical trajectories and flagellar shapes. Proc Natl Acad Sci, 2017; 114:13085–13090.

38.

Navarro-Alarcon

, Liu

Y-H

. Fourier-based shape servoing: a new feedback method to actively deform soft objects into desired 2-D image contours. IEEE Trans Robot, 2018; 34:272–279.

39.

Ghazi-Zahedi

, Deimel

, Montúfar

, et al. Morphological computation: the good, the bad, and the ugly. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada, pp. 24–28, 2017.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.14 MB

0.13 MB

0.14 MB

0.00 MB

Quantifying Dynamic Shapes in Soft Morphologies

Abstract

Introduction

Materials and Methods

Shape estimation using elliptic Fourier descriptors

Eigenshape analysis

Case Studies

Soft tentacle

Crawling motion of caterpillars

Euglenoid movement and comparison with a robot

Discussion

Generality of shapes

Extension to three dimensions

Limitations

Conclusion and Future Work

Footnotes

Acknowledgments

Data Disclosure Statement

Author Disclosure Statement

Supplementary Material

References

Supplementary Material