Sage Journals: Discover world-class research

Abstract

Humans use active touch to gain behaviourally relevant information from their environment, including information about contained objects. Although most common, the perceptual basis of interacting with containers remains largely unexplored. The first aim of this study was to determine how accurately people can sense, by touch only, the location of a contained rolling object. Experiment 1 used tubes containing physical balls and demonstrated a considerable degree of accuracy in estimating the rolled distance. The second aim was to identify the relative effectiveness of the various available physical cues. Experiment 2 employed virtual reality technology to present, in isolation and in various combinations, the constituent haptic cues produced by a rolling ball, which are, the mechanical noise during rolling, the jolts from an impact with an internal wall, and the intensity and timing of the jolts resulting from elastic bounces. The rolling noise was of primary importance to the perceptual estimation task suggesting that the implementation of the laws of motion is based on an analysis of the ball’s movement velocity. Although estimates became more accurate when the rolling and impact cues were combined, they were not necessarily more precise. The presence of elastic bounces did not affect performance.

Keywords

Haptics active touch tactile cognition tactile invariants

Humans are expert at gaining behaviourally relevant information from their environment through the sense of touch or haptic perception (see Grunwald, 2008). To do so, humans possess a large repertoire of actions, such as the exploratory procedures documented by Lederman and Klatzky (1987). We lift and wield objects to obtain knowledge about their weight; we compress and bend objects to obtain knowledge about their material properties; and we enclose objects in our hands to obtain knowledge about their shape and volume (Carello & Turvey, 2017; Lederman & Klatzky, 1987). Such active engagements with objects have been the topic of scholarly discussion going back to at least the late 18th century (Wagner, 2016). They are typically referred to as active touch (Gibson, 1966) or dynamic touch (Turvey, 1996) and involve an instrumental contribution of motor effort and its consequent sensory signals. When wielding an object, various stimuli impinge on the receptors in the skin, tendons, and muscles; the stimulation, in turn, is a function of our movements and the inertial properties of the object. Active touch is capable of many perceptual feats (Gibson, 1966; Katz, 1989), including estimating the length of a rod only from wielding it (Solomon & Turvey, 1988; Turvey, 1996) or estimating the location of impact on a handheld tool (Miller et al., 2017; Okazaki & Kajimoto, 2014).

Sensing content by touch

Here, we extend our interest from solid objects to contained objects. The pertinent feature of containers is their ability to provide sensory signals related to their content. This is evident in common daily experiences such as when shaking a juice can or rattling a box of chocolate sprinkles to estimate whether there is enough left to drink or to eat (Hirota & Sekiguchi, 2009; Plaisier & Smeets, 2017; Tanaka & Hirota, 2012). These types of everyday experiences have inspired engineers to develop devices that mimic containers with virtual contents, such as balls in a box (Linjama et al., 2005; Sekiguchi et al., 2005; Williamson et al., 2007), balls in a cup (Minamizawa et al., 2012), liquids in a bottle (e.g., Koshiyama et al., 2015), or balls inside a tube (Yao & Hayward, 2006).

Balls in a box

Previous studies investigated people’s ability to obtain information about content in natural settings. Plaisier and Smeets (2017) described an experiment to test the hypothesis that people can estimate an exact number of objects (i.e., wooden balls) inside a box, provided that the number of objects is small enough. The number of balls placed inside a box could be surmised from either hearing or feeling the consequences of shaking the box. Their numerosity was estimated from the collisions of the balls with the container walls, from the collisions between the balls themselves, and from the rolling movement of the balls. Participants in the experiment verbally estimated the number of balls inside a cardboard box after having manipulated it for 5 s. They performed this estimation under two conditions, one in which they received auditory and haptic information from handling the box and one in which auditory recordings from that handling were passively played back. The results showed that participants could perform the perceptual task accurately when the number of balls was between one and three but tended to underestimate their number when it was larger. The main difference between the two conditions was that performance was considerably more variable in the auditory-only condition. The authors attribute this difference to the involvement of a multisensory cue integration process, which produced more precise sensory estimates when participants are provided with congruent sensory information across different senses (Ernst & Bülthoff, 2004).

Our ability to accurately estimate numerosity when the number of balls is small is also reported by Sekiguchi et al. (2005). These investigators employed augmented reality techniques to present participants with a consecutive pair of real boxes each containing several virtual objects (between one and five). The participants’ task was to indicate which of the two boxes contained the most objects. Participants had a nearly perfect ability to report the difference between a box with one single object and a box with more than one object (100% correct), and a near-perfect ability to report the difference between a box with two objects and a box with more than two objects (better than 96%). Performance dropped when discriminating between three objects and four or five objects (92% and 80%, respectively). When four objects were compared with five objects, performance was at chance level (56%).

The container studies of Plaisier and Smeets (2017) and Sekiguchi et al. (2005) demonstrated that people can achieve a considerable level of accuracy in estimating the number of objects inside a container. There were, however, key differences between having only one ball in the container and having more than one ball. With only one ball, there can only be one impact with a wall at any given instant and the cues arising from rolling may arise from a single ball. The addition of a second ball, or more, introduces ambiguous information given the possibility of concurrent impacts, mutual collisions between balls, and superposed rolling vibrations.

One ball in a tube

The studies summarised above clearly reveal that wielding a container produces appropriate haptic feedback that allows observers to make inferences about the container’s content. These studies, however, did not intend to reveal strategies that their participants employed in making their perceptual inferences. A virtual ball apparatus introduced by Yao and Hayward (2006) enabled the investigation of perceptual strategies as the cues relevant to the task of sensing the location of a freely moving object in a container were generated artificially, thus simplifying the complicated stimulus arrangement of previous box studies. The apparatus comprised a single rigid tube housing a vibrotactile transducer that could be programmed to reproduce key components of the mechanics of a ball rolling inside a tube. These components were (1) the pattern of vibration caused by a ball rolling down the rough inner surface of a tube, (2) the intensity of the impact felt when a ball encountered an internal wall, and (3) the timing of the elastic rebound following impact against an internal wall. Importantly the apparatus could render these cues virtually in any desired combination and properties (See Figure 1a). What the apparatus did not reproduce was (4) the variation of torque as a function of the distance of the ball from the fulcrum.

Figure 1.

Apparatuses used in the experiments. (a) A handle contained a recoil actuator and an accelerometer. (b) In Experiment 1, the handle was connected to interchangeable tubes with internal walls and containing a metal ball, with the actuator disengaged. In Experiment 2, the handle could also be connected to a tube without a ball. An external computer (in panel a) simulated and synthesised vibration signals for rolling, impact, and bounce that were transduced by the recoil actuator.

Yao and Hayward (2006) found preliminary evidence that human participants could, through touch, spontaneously perceive the location of a ball rolling inside the cavity of the handheld tube. Without any preliminary training, participants were informed that there were three tubes of equal length but with inner cavities of three different lengths: 18, 24, or 60 cm. In separate conditions, the apparatus rendered either the rolling vibration or the impact of the ball against inner walls. Participants were divided equally among the two conditions and had the task to estimate the length of the cavity by tilting the tube in a controlled two-phase motion: first tipping downwards, then lifting upwards. Following a three-alternative forced-choice paradigm, participants reported their answer by pointing to one of three markings on the tube. Overall, participants could solve the perceptual problem. They gave the correct answer on most trials, although there were two peculiarities in the results. In the rolling vibration condition, the 24-cm virtual length tended to be underestimated to 18 cm (63% underestimations compared with 28% correct identifications). In the impact condition, the 18-cm virtual length was overestimated to 24 cm (63% compared with 20% correct identifications). Interestingly, according to self-reports, participants performed the task by imagining the movement of the virtual ball with their mind’s eye and, indeed, seemed to spontaneously “track” the virtual ball with their gaze during trials.

The virtual simulations provided sufficient information for the participants to perform the task with reasonable success. The between-subject experiment design, however, provided an incomplete picture of the way the participants utilised the sensory information available to them to sense the location of the ball. The experiment did not have a condition where the rolling vibration and the intensity of impact were available together. The elastic rebound following impact was also absent from the testing conditions.

Present study

This study has two aims. The first is to determine how accurately people can sense by touch only the location of an unseen moving rolling object. The second is to identify the relative effectiveness of the various cues available in the natural world by controlling the access to the different sources of information (Cabe, 2010). The experiments reported here were conducted according to the guidelines set out in the Declaration of Helsinki and were approved by the McGill University research ethics board.

Experiment 1: accuracy of locating unseen rolling balls

Untrained participants were asked to move a tube containing a real ball using either small or large angles. The proximal source of sensory stimulation was vibrations impinging on the hand holding the tube.

The two angles were included to preclude a trivial interpretation of the results, namely that participants performed the task using a simple timing heuristic. There is a simple relationship between distance and travel time: longer distances require more time than shorter distances. If participants relied on this heuristic, then estimated distances for large angles should be smaller than estimated distances for small angles because the ball rolls faster with large angles and therefore rolls for a shorter period. If, however, participants were genuinely able to perceive the motion of the ball, the movement angle should be inconsequential to the distance estimates.

Method

Apparatus

There was a total of four black 60 cm opaque fibreglass tubes. The tubes contained a physical ball and had an internal wall set at 20, 30, 40, or 50 cm from the base. The tubes were hidden from the participants’ view behind a partition. An accelerometer and an actuator were concealed inside a detachable handle (see Figure 1), which was attached, on every trial, and in plain view of the participant, to whatever tube was handed to them next. Participants wore headphones playing white noise that effectively masked the sound of the rolling ball.

Procedure

Nine participants (age range: 19–29 years; five females) were handed one of the four tubes with the handle attached and asked to move the tube in a prescribed manner: Up to five movements were allowed in the fronto-parallel plane. The tube was always handed to the participant’s dominant hand, and in an upward angle so that the ball was at the base, near the hand. Thus, the typical movement sequence was down-up-down-up-down, although participants were allowed to provide their answer as soon as they felt ready.

Participants indicated the final resting location of the ball by sliding a rubber hair band along the surface of the tube from its initial position at the tube’s base, near the participant’s hand. The participant then handed the tube back to the experimenter who measured the position of the hair band against a measuring tape affixed to the desk in front of them. The position of the band was read off to the nearest 5 mm after which the band was reset to its initial position. The answer was entered into a custom computer program that then gave the next randomised condition.

The task was performed under two conditions. Participants made either large or small angles to create different rolling velocities of the ball. They were instructed that any movement angle was fine as long as there was a noticeable difference between the two conditions. The actual amount of the tilt was calculated after the fact from recordings from the accelerometer in the handle. Prearranged hand signals were used to tell the participant what size angle was required on any given trial. Each combination of target distance and movement angle was tested four times for a total of 32 completely randomised trials. Short breaks between trials were allowed whenever the participants requested them.

Before actual testing started, participants were allowed a brief familiarisation—without explicit instruction about the angle of movement—of the feel of the ball rolling through the tube. During this familiarisation, participants were shown the actual distance the ball had rolled (by means of a marker on the outside of the tube, in this case, at 30 cm). This lasted only 30 s or 10 movements, whichever came first.

Results and discussion

The results demonstrated that participants were remarkably apt at spontaneously differentiating and estimating the various rolling distances (Figure 2a) albeit with a tendency to underestimate distance as rolling duration increased. All participants produced significantly different angles. Paired t-tests, one for each participant, showed that all t-values were >10.0 and all p-values <.001 (Figure 2b). Despite the significantly different angles produced, there was no difference in the distance estimates between the small and large angle conditions—repeated-measures analysis of variance (ANOVA): F(1, 8) ~ 0, MSE = 0.347, p = .993, $η_{p}^{2} ~ 0$ . There was a significant main effect of target distance—F(3, 24) = 34.99, MSE = 93,658.6, p < .001, $η_{p}^{2} = . 814$ , but no interaction between angle and target distance—F(3, 24) = 2.12, MSE = 2,208.2, p = .124, $η_{p}^{2} = . 210$ . The absence of an effect of angle was also apparent at an individual level (Figure 2c).

Figure 2.

Results of Experiment 1. (a) Average response of all participants: estimated distance versus actual distance. Error bars represent 95% empirical bootstrap confidence intervals based on 10,000 bootstrap samples. (b) Mean angles produced by the participants. Error bars represent 2 SDs. (c) Individual length estimates made using large angles versus those made using small angles. Each symbol represents the average across the four replications of the corresponding condition for one of the participants.

A second experiment was designed to uncover the different types of tactile information potentially contributing to the perceptual estimation of the location of rolling objects. To provide a basis for this design, we next discuss an analysis of the tactile signals available from this kind of object. It is instructive to see sensory cues as being associated with invariant relationships between sensory signals or to invariant relationships between motor command and sensory signals. Like for other sensory modalities, tactile invariants can arise from the organisation of the sensory system, from inherent mathematical properties, or from the natural behaviour of objects (Hayward, 2008). The macroscopic mechanical invariants identified in the analysis arise from the law of energy conservation.

Movement duration

Galileo discovered that rolling objects on planes inclined by an angle, $α$ , had movement durations that were independent from their mass and geometry in what can be said, along with Archimedes’ principle, to be one of the first historical examples of the description of a physical invariant (Settle, 1961). With modern notation, we find the distance travelled by spherical and homogeneous objects from a stationary initial condition to be $x_{f i n a l} \approx 3.5 \sin (α) T^{2}$ where $T$ is the total movement duration. The coefficient 3.5 follows from the intensity of the gravity field on earth and the assumption of uniform mass distribution in a sphere. (It can be noted that other reasonable axisymmetric geometries give coefficients that do not differ greatly from this case. A cylinder, for example, would give a coefficient equal to 2.8.) Thus, the length of the cavity can be computed from the movement duration of the ball, $T$ , under the assumptions of a reasonably shaped object, of a known inclination, and of being on earth.

Movement velocity

The rolling vibrations provide information to estimate movement velocity, even if no assumption is made about the action of gravity. These vibrations $p (t)$ may be modelled by a generating function, $g (x)$ , which represents the micro-displacement of the surface displaced by the action of the ball composed with the ball trajectory, $x (t)$ , that is, $p (t) = g (x (t))$ . On the grounds of the law of conservation of momentum, the derivative of $g (x)$ provides a representation of an invariant property of rolling vibrations exposed by applying the chain rule, $p ̇ (t) = g^{'} v (t)$ , where $v (t)$ is the ball’s velocity. Thus, given vibration signals from two independent trials, $p ̇_{1} (t) = g^{'} v_{1} (t)$ and $p ̇_{2} (t) = g^{'} v_{2} (t)$ , it is possible to compute $g (x)$ as the solution of a linear inverse problem, giving access to $v (t)$ . We do not suggest that the brain follows these steps to solve the perceptual problem at hand. These steps only show that such computation is possible. The discussion of more biologically plausible approaches relying, for instance, on the properties of chirping signals, or on the analysis of the time-series of signal extrema, or other computational methods is beyond the scope of this study.

Speed at the instant of impact

Collisions are mechanical events in which kinetic energy is lost during a brief instant. In 1656, some 30 years before Newton’s Principia, Christiaan Huygens used collisions to formulate the aforementioned law of conservation of momentum, which applies even when kinetic energy is lost during impact. A concept that follows from the conservation of momentum in collisions is the coefficient of restitution, $0 \leq ε < 1$ , defined by the ratio of velocities of an object just after and before impact against a surface, $ε = v_{n + 1} / v_{n} = T_{n} / T_{n - 1}$ . When there is no bouncing, the coefficient of restitution is zero. If there is bouncing, the impact velocities decrease at a constant rate, $ε$ , and exhibit a “Zeno behaviour” (Bostock, 1972), which means that the object ends up coming to rest (strictly speaking, it ends up bouncing at infinite frequency with zero magnitude), $v_{\infty} = 0$ , in finite time, $T_{\infty} = (1 + ε) / (1 - ε) T$ , where $T$ is the duration of the initial roll (Falcon et al., 1998). The recursion, $v_{1} = ε v_{0}, v_{2} = ε v_{1}, \dots, v_{\infty} = 0$ corresponds to $t_{1} = ε T, t_{2} = ε t_{1}, \dots, t_{\infty} = 0$ , which means that the timing of the bounces is informative of the sequence of velocities although it is ambiguous as to their magnitudes. Conservation of momentum furnishes an additional cue as for any impact, $i$ , of a ball of mass, m, with a container of mass, M, $Δ v_{i} = (m / M)$ . In other words, the jolt is as great as the ball velocity. This cue is informative albeit ambiguous if the ratio of the masses is unknown.

In summary, solving the perceptual problem requires some or all the available signals as well as some prior knowledge. The analysis above identified the available signals to be the inclination, $α$ , of the tube with respect to gravity; the duration of the rolling movement, $T$ ; the pattern $p (t)$ , of mechanical noise during rolling; the intensity and timing of the jolts resulting from bounces, if any. One additional signal is the varying torque, $τ$ , at the fulcrum (i.e., the wrist) needed to hold the tube at a given inclination as the ball rolls down the tube. The required prior knowledge pertains to the shape of the rolling object, the intensity of the gravity field, and the mass of ball relative to the mass of the container.

Experiment 2: relative effectiveness of available cues

Method

Apparatus

The same four tubes and handle from Experiment 1 were used. A fifth tube without any ball was added for a virtual ball condition, which consisted of virtual haptics technology very similar to the one described by Yao and Hayward (2006) and illustrated in Figure 1.

To good approximation, the accelerometer returned a measurement that was directly proportional to the acceleration of a virtual ball. A computer simulated the key aspects of the physics described earlier and in Figure 1. In a real-time loop, the computer read the virtual acceleration, $\bar{x} = 7.0 \sin α$ , and numerically integrated it twice to obtain an estimate of $x (t)$ . The numerical accuracy of the estimate is of the order of 0.2 mm and the gains were calibrated by matching the simulation with the movements of the real ball in bench experiments. The actuator was of a recoil type and designed to provide vibrotactile feedback between 50 and 500 Hz, covering the majority of bandwidth of tactile sensing (Yao & Hayward, 2010).

To synthesise the rolling noise, a source waveform made of a rectified sine wave provided a generating function, $g (x)$ , (i.e., small-scale variations in the height of our virtual surface) in which the lowest spatial frequency was 30 mm with a rich spectrum. One arch of the sine wave was sampled into 30 samples that were stored in a table. To synthesise the vibration due to the rolling noise, the table was read using the index function $i = x m o d 30$ and the amplified result sent to the actuator. The ball’s impact was simulated by sending a pulse to the actuator with amplitude proportional to the incoming velocity. For the bounce effect, the virtual ball had impact velocities as described earlier. With a sampling rate of 250 Hz, the system did not cause any significant delays and the simulation faithfully reproduced most aspects of the physics of a ball rolling inside a tube, except that there was no transfer of mass.

Procedure

Eight new participants (age range: 19–27 years; five females) were tested using the same task as in Experiment 1 except for two small procedural differences. First, no more explicit instruction was given about movement angles. Second the familiarisation phase now consisted of feeling two real rolling balls (a short and a long distance) and one virtual one, this time without any corrective feedback. As before, participants wore headphones playing white noise that effectively masked the sound of the rolling ball.

The task was performed under six conditions, five of which involved a virtual ball that rendered (1) ball impacts only; (2) the rolling only; (3) the combination of impacts and rolling; (4) the combination of impacts and bounces; and (5) the combination of impact, rolling, and bounces. A sixth condition, using real balls served as a replication of Experiment 1 as well as a reference for assessing performance in the virtual ball conditions. Each combination of ball condition and the four target distances was tested four times for a total of 96 completely randomised trials.

At no point before or during the experiment were the participants informed about any simulation or virtual balls. In other words, as far as the participants were concerned, the entire experiment was conducted with physical balls. In fact, in informal debriefs after the experiment, once they were informed that they had experienced a simulation, participants typically expressed surprise and, occasionally, disbelief. They were genuinely under the impression of having manipulated a real ball and they frequently asked for the permission to inspect the inside of the tube (for similar participant responses, see the study by Yao & Hayward, 2006).

Results and discussion

Inspection of Figure 3a reveals that performance with real balls was in close agreement with that of Experiment 1. Participants were remarkably good at estimating the rolled distance, and again there was a tendency to underestimate the distance with increasing pre-set length.

Figure 3.

Results of Experiment 2. For all panels, filled markers indicate conditions that include bouncing. (a) Average response of all participants: estimated distance versus actual distance. Error bars were omitted for legibility. (Legend also applies for panel b.) (b) From left to right, the mean of estimates normalised relative to performance with the real ball, the overall mean across the four target distances, the individual overall means. Error bars represent 95% empirical bootstrap confidence intervals based on 10,000 bootstrap samples.

Data were reduced to address the experiment’s main question—which tactile information was utilised in the estimations. First, for each participant, estimates were normalised with respect to their performance with the real ball. This step effectively detrended the estimates with respect to rolling distance (see Figure 3b, leftmost panel). Second, the normalised estimates were collapsed across rolling distance to obtain an overall measure of task performance (see Figure 3b, middle panel).

The virtual balls conditions revealed that the impacts alone are inadequate as rolling distance was grossly underestimated, and for most participants, this was the worst performing condition (see the green and red markers in the panel for individual performance of Figure 3b for exceptions). It also had the largest inter-individual variance. On the contrary, participants performed well with only rolling information, and even better when rolling and impacts were both available. The effect of bounce was small (a 5% improvement) but non-significant—F(1, 7) = 4.13, MSE = 0.006, p = .08, $η_{p}^{2} = . 371$ —and we therefore collapsed the data for subsequent statistical analyses.

A repeated-measures ANOVA showed a significant effect of the type of cue—F(2, 14) = 8.39, MSE = 0.360, p = .019, $η_{p}^{2} = . 545$ . Within-subjects contrasts revealed that the difference between the impact-only and rolling-only condition was not significant—F(1, 7) = 3.74, MSE = 0.308, p = .094, $η_{p}^{2} = . 348$ . However, this could be traced back to the disproportionate effect of one participant on the variance (see green marker in Figure 3b); when excluding this participant’s data, the effect was significant—F(1, 6) = 12.38, MSE = 0.519, p = .013, $η_{p}^{2} = . 673$ . The addition of the impact cue to the rolling cue increased performance significantly—F(1, 7) = 16.57, MSE = 0.107, p = .005, $η_{p}^{2} = . 703$ .

It seems that participants primarily used the rolling information, suggesting that the implementation of the laws of motion is based on an analysis of the ball’s movement velocity. As the impact cue was largely ineffective on its own, we propose that its contribution was by adding energy to the sensory signal that contributed to the perception of the roll velocity. Next, the addition of bounces did not substantially affect performance, which suggests that we can rule out the use of the principle of conservation of energy as effective stimulus information. Finally, even in the condition with all information available, there is an underestimation of length with respect to the real ball condition. This could be construed as indirect evidence for the effectiveness of the Archimedean torque, which was necessarily missing from the virtual ball conditions.

General discussion

This study set out to determine how accurately humans can perceive the motion of a ball rolling inside a tube and to tease out the effective stimulus information they employed. The work extends the results of the earlier rolling ball study by Yao and Hayward (2006) in that its more ecologically valid experimental approach yielded a better quantitative estimate of perceptual accuracy as well as a better notion of the relative efficacy of the various kinds of stimulation. The results demonstrate a reliable ability to perceive the motion of a real ball inside a tube. On average, participants demonstrated a considerable degree of accuracy in estimating the rolled distance, although there was a tendency to underestimate as distance increased. The results appear to be in line with the level of accuracy observed in the balls-in-a-box experiments (Plaisier & Smeets, 2017; Sekiguchi et al., 2005) as well as with the tendency to underestimate (Plaisier & Smeets, 2017).

The study provides several hints as to the perceptual strategies employed by the participants in solving the perceptual task. It appears that the most important information is obtained from the rolling noise. The next most important information comes from impact. The bounce, on the other hand, had effectively no role to play. In terms of the potential perceptual strategy underlying the observed performance, the results are consistent with a perceptual system that uses a combination of two strategies: roll duration and indirect access to object velocity, with the latter carrying the most weight. Although there is little evidence for a strategy that relies on the conservation of momentum, there remains the logical possibility that a strategy based on the laws of levers is at play.

This study also underscores the dominant role of vibratory signals in the perception of objects concealed in containers. It has been known for a long time that several types of mechanoreceptors could subserve the detection of the vibrations of handheld objects. Earlier studies highlight the likely role of the Pacinian corpuscles that have been repeatedly found to be exquisitely responsive to vibrations transmitted by objects in contact with the hand (Cauna & Mannan, 1958; Goodwin et al., 1981; Johansson & Vallbo, 1979). Mechanoreceptor populations possibly also include far-flung muscle spindles receptors (Brisben et al., 1999; Libouton et al., 2012) as well as receptors normally associated with static contact (Gottschaldt & Vahle-Hinz, 1981). Recently, the mechanical waves propagating in the tissues of the upper extremity have been implicated in the transmission of tactile signals (Delhaye et al., 2012; Manfredi et al., 2012; Shao et al., 2016). The central brain processes are largely unknown for the task at hand but are likely to involve a network of sub-cortical structures that include at least the cuneate nucleus (Jörntell et al., 2014), the cerebellum (Blakemore et al., 1999), basal ganglia, and thalamic nuclei, resulting in the activation of a neo-cortical network that presumably includes posterior parietal cortex areas (Ehrsson et al., 2003; Miller et al., 2019; Van Boven et al., 2005), in addition to the primary somatosensory and motor areas (Kaas, 2012). The perceptual task, per se, is likely to involve a complex network, including visual (Ricciardi et al., 2007) and pre-frontal areas (Pleger et al., 2006).

Having demonstrated the ability to accurately perceive the motion of a real ball inside a tube opens several avenues for future research on the haptic perception of container content. First, it is noteworthy that the ability did not require dedicated task training because participants were given no more than a very brief familiarisation period. Apparently, there is enough information in the stimulus to accomplish this uncommon task and participants came to it with all the necessary perceptual tools already in place. Still, people did possess a lifetime of experience with handling container objects, which raises questions around the development of how people learn to obtain information from containers.

Second, behaviour is conducted in relation to a specific goal, which regularly requires adapting to changing circumstances by means of switching to different anatomical components and their related coordination patterns (e.g., switching hands) (Wagman & Hajnal, 2014). As a way of linking the various sensory components (receptors in the skin, joints, muscles, and connective tissues) with the body’s compression and tension elements (forces due to muscular contraction and from resistance to external loads), Turvey and Fonseca (2014) conceptualised the medium for the haptic perceptual system as a kind of tensegrity system. Such a system would be sensitive to invariants in the world, which, in turn, would support the perception of the objects from which those invariants originate. Empirical support for this comes from a study by Wagman and Hajnal (2014) who investigated the anatomical independence of the ability to perceive whether an inclined surface could be stood on. Across a series of experiments participants explored an inclined surface with an object that could be held in various ways, including with the preferred or nonpreferred hand, with both hands, and with the feet. The results confirmed that perception was virtually unaffected by whatever configurations of anatomical components were used. For our ball-in-a-tube scenario, a testable implication is that participants should be able to accurately perceive the ball’s motion even if they were to wield a tube with another part of the body part, say with their feet (Hajnal et al., 2007).

Third, the virtual ball display affords the investigation of how the perception of the ball changes if there are conflicting (or contradictory) cues. Specifically, what perception would manifest from a stimulus array in which the rolling and the impact components specify different balls. We could suspect several perceptual outcomes. First, we perceive two concurrent balls. Second, we perceived a single ball that is informed by one of the components and ignorant of the other. Third, we perceived a single ball that constitutes a compromise of the two components. Several of these outcomes can come to light depending on the characteristics of the discrepancies. Such work could be a complement to haptics research from the perspective of cue integration, which takes sensory information consisting of a constellation of cues. For instance, when moving a finger over a bumpy surface, there are position cues from following the surface geometry and force cues from running into the slopes of the surface (Robles-de-la-Torre & Hayward, 2001). One of the central questions in cue integration pertains to the relative weighting of the available cues, which is typically addressed by injecting sensory noise into the cues so as to manipulate their reliability and impose conflicts between the cues (Drewing & Ernst, 2006; Ernst & Bülthoff, 2004).

Finally, while the present results suggest that participants primarily use the movement velocity strategy, which does not necessarily require assumptions about gravity, they do not exclude the possibility that participants used knowledge about gravity. The possibility is reasonable given research suggesting that people have internal models of the laws of motion that they employ in perceiving and acting in the world (Lacquaniti et al., 2015). For instance, Ceccarelli et al. (2018) employed virtual reality to test the hypothesis that visual judgements of the naturalness of a ball rolling down an incline are consistent with Newtonian physics. Participants viewed a sphere rolling down an incline, while either the slope of the incline or the acceleration of the sphere were manipulated to be either consistent or inconsistent with terrestrial physics. The results demonstrated that participants were accurate at adjusting either the slope or acceleration so as to be consistent with Newtonian mechanics. Similarly, the virtual ball display allows for manipulations of terrestrial physics, specifically the gravitational constant, thus allowing for direct experimental tests of the role of gravity in the perceptual estimation of rolling ball motion and location.

Conclusion

This study is one of only very few container perception studies and shows that people can and do accurately track motion and position of a rolling ball simply from handling its container. This appears to be the first formal demonstration of the perception of the motion of a ball in a container and as such extends the human haptic system’s impressive repertoire of capabilities.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: All authors were supported by grants from the National Sciences and Engineering Research Council of Canada (NSERC). Vincent Hayward received additional funding from the Sorbonne Université, Paris.

ORCID iD

Ilja Frissen

Data available statement

Materials and analysis code for this study are available by emailing the corresponding author.

References

Blakemore

S. J.

Wolpert

D. M.

Frith

C. D.

(1999). The cerebellum contributes to somatosensory cortical activity during self-produced tactile stimulation. NeuroImage, 10(4), 448–459. https://doi.org/10.1006/nimg.1999.0478

Bostock

(1972). Aristotle, Zeno, and the potential Infinite. Proceedings of the Aristotelian Society, 73, 37–51.

Brisben

A. J.

Hsiao

S. S.

Johnson

K. O.

(1999). Detection of vibration transmitted through an object grasped in the hand. Journal of Neurophysiology, 81(4), 1548–1558. https://doi.org/10.1152/jn.1999.81.4.1548

Cabe

P. A.

(2010). Sufficiency of longitudinal moment of inertia for haptic cylinder length judgments. Journal of Experimental Psychology: Human Perception and Performance, 36(2), 373–394. https://doi.org/10.1037/a0018395

Carello

Turvey

M. T.

(2017). Useful dimensions of haptic perception: 50 years after the senses considered as perceptual systems. Ecological Psychology, 29(2), 95–121. https://doi.org/10.1080/10407413.2017.1297188

Cauna

Mannan

(1958). The structure of human digital Pacinian corpuscles (corpuscula lamellosa) and its functional significance. Journal of Anatomy, 92(1), 1–20.

Ceccarelli

La Scaleia

Russo

Cesqui

Gravano

Mezzetti

Moscatelli

d’Avella

Lacquaniti

Zago

(2018). Rolling motion along an incline: Visual sensitivity to the relation between acceleration and slope. Frontiers in Neuroscience, 12, Article 406. https://doi.org/10.3389/fnins.2018.00406

Delhaye

Hayward

Lefèvre

Thonnard

J. L.

(2012). Texture-induced vibrations in the forearm during tactile exploration. Frontiers in Behavioral Neuroscience, 6, Article 37. https://doi.org/10.3389/fnbeh.2012.00037

Drewing

Ernst

M. O.

(2006). Integration of force and position cues for shape perception through active touch. Brain Research, 1078, 92–100. https://doi.org/10.1016/j.brainres.2005.12.026

10.

Ehrsson

H. H.

Fagergren

Johansson

R. S.

Forssberg

(2003). Evidence for the involvement of the posterior parietal cortex in coordination of fingertip forces for grasp stability in manipulation. Journal of Neurophysiology, 90(5), 2978–2986. https://doi.org/10.1152/jn.00958.2002

11.

Ernst

M. O.

Bülthoff

H. H.

(2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169. https://doi.org/10.1016/j.tics.2004.02.002

12.

Falcon

Laroche

Fauve

Coste

(1998). Behavior of one inelastic ball bouncing repeatedly off the ground. The European Physical Journal B: Condensed Matter and Complex Systems, 3(1), 45–57. https://doi.org/10.1007/s100510050283

13.

Gibson

J. J.

(1966). The senses considered as perceptual systems. Houghton Mifflin.

14.

Goodwin

A. W.

Youl

B. D.

Zimmerman

N. P.

(1981). Single quickly adapting mechanoreceptive afferents innervating monkey glabrous skin: Response to two vibrating probes. Journal of Neurophysiology, 45(2), 227–242.

15.

Gottschaldt

K. M.

Vahle-Hinz

(1981). Merkel cell receptors: Structure and transducer function. Science, 214(4517), 183–186. https://doi.org/10.1126/science.7280690

16.

Grunwald

(Ed.) (2008). Human haptic perception: Basics and applications. Birkhauser. https://doi.org/10.1007/978-3-7643-7612-3

17.

Hajnal

Fonseca

Harrison

Kinsella-Shaw

Carello

(2007). Comparison of dynamic (effortful) touch by hand and foot. Journal of Motor Behavior, 39(2), 82–88. https://doi.org/10.3200/JMBR.39.2.82-88

18.

Hayward

(2008). Haptic shape cues, invariants, priors and interface design. In Grunwald

(Ed.), Human haptic perception: Basics and applications (pp. 381–392). Birkhauser.

19.

Hirota

Sekiguchi

(2009). Transmission of information through haptic interaction. In Shumaker

(Ed.), Virtual and Mixed Reality (VMR 2009) (Lecture Notes in Computer Science, Vol. 5622, pp. 313–317). Springer. https://doi.org/10.1007/978-3-642-02771-0_35

20.

Johansson

R. S.

Vallbo

A. B.

(1979). Tactile sensibility in the human hand: Relative and absolute densities of four types of mechanoreceptive units in glabrous skin. Journal of Physiology, 286(1), 283–300. https://doi.org/10.1113/jphysiol.1979.sp012619

21.

Jörntell

Bengtsson

Geborek

Spanne

Terekhov

A. V.

Hayward

(2014). Segregation of tactile input features in neurons of the cuneate nucleus. Neuron, 83(6), 1444–1452. https://doi.org/10.1016/j.neuron.2014.07.038

22.

Kaas

J. H.

(2012). Somatosensory system. In Mai

Paxinos

(Eds.), The human nervous system (3rd ed., pp. 1074–1109). Elsevier. https://doi.org/10.1016/B978-0-12-374236-0.10030-6

23.

Katz

(1989). The world of touch. Lawrence Erlbaum.

24.

Koshiyama

Kikuchi

Morita

Sugimoto

(2015). VolRec: Haptic display of virtual inner volume in consideration of angular moment. In Cheok

A. D.

(Ed.), Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology (Article 32, pp. 1–4). Association for Computing Machinery. https://doi.org/10.1145/2832932.2832970

25.

Lacquaniti

Bosco

Gravano

Indovina

La Scaleia

Maffei

Zago

(2015). Gravity in the brain as a reference for space and time perception. Multisensory Research, 28(5–6), 397–426. https://doi.org/10.1163/22134808-00002471

26.

Lederman

S. J.

Klatzky

R. L.

(1987). Hand movements: A window into haptic object recognition. Cognitive Psychology, 19(3), 342–368. https://doi.org/10.1016/0010-0285(87)90008-9

27.

Libouton

Barbier

Berger

Plaghki

Thonnard

J. L.

(2012). Tactile roughness discrimination of the finger pad relies primarily on vibration sensitive afferents not necessarily located in the hand. Behavioural Brain Research, 229(1), 273–279. https://doi.org/10.1016/j.bbr.2012.01.018

28.

Linjama

Häkkilä

Ronkainen

(2005, April 5–7). Gesture interfaces for mobile devices-minimalist approach for haptic interaction [Paper presentation]. CHI Workshop: Hands on Haptics: Exploring Non-Visual Visualisation Using the Sense of Touch, Portland, OR. http://www.chi2005.org/index.html

29.

Manfredi

L. R.

Baker

A. T.

Elias

D. O.

Dammann

J. F.

III Zielinski

M. C.

Polashock

V. S.

Bensmaia

S. J.

(2012). The effect of surface wave propagation on neural responses to vibration in primate glabrous skin. PLOS ONE, 7(2), Article e31203. https://doi.org/10.1371/journal.pone.0031203

30.

Miller

L. E.

Cawley-Bennett

Longo

M. R.

Saygin

A. P.

(2017). The recalibration of tactile perception during tool use is body-part specific. Experimental Brain Research, 235(10), 2917–2926. https://doi.org/10.1007/s00221-017-5028-y

31.

Miller

L. E.

Fabio

Ravenda

Bahmad

Koun

Salemme

Luauté

Bolognini

Hayward

Farnè

(2019). Somatosensory cortex efficiently processes touch located beyond the body. Current Biology, 29(24), 4276–4283. https://doi.org/10.1016/j.cub.2019.10.043

32.

Minamizawa

Kakehi

Nakatani

Mihara

Tachi

(2012). TECHTILE toolkit: A prototyping tool for design and education of haptic media. In S. Richir (Ed.), Proceedings of the 2012 Virtual Reality International Conference (Article 26). ACM (Association for Computing Machinery). https://doi.org/10.1145/2331714.2331745

33.

Okazaki

Kajimoto

(2014). Altering distance perception from hitting with a stick by superimposing vibration to holding hand. In Auvray

Duriez

(Eds.), Haptics: Neuroscience, devices, modeling, and applications. EuroHaptics 2014 (Lecture Notes in Computer Science, Vol. 8619, pp. 112–119). Springer. https://doi.org/10.1007/978-3-662-44196-1_15

34.

Plaisier

Smeets

J. B. J.

(2017). How many objects are inside this box? In M.A. Otaduy, J.-H. Ryu, & G. Gerling (Eds.), Proceedings of IEEE World Haptics Conference (pp. 240–244). IEEE. https://doi.org/10.1109/WHC.2017.7989908

35.

Pleger

Ruff

C. C.

Blankenburg

Bestmann

Wiech

Stephan

K. E.

Capilla

Friston

K. J.

Dolan

R. J.

(2006). Neural coding of tactile decisions in the human prefrontal cortex. Journal of Neuroscience, 26(48), 12596–12601. https://doi.org/10.1523/JNEUROSCI.4275-06.2006

36.

Ricciardi

Vanello

Sani

Gentili

Scilingo

E. P.

Landini

Guazzelli

Bicchi

Haxby

J. V.

Pietrini

(2007). The effect of visual experience on the development of functional architecture in hMT+. Cerebral Cortex, 17(12), 2933–2939. https://doi.org/10.1093/cercor/bhm018

37.

Robles-De-La-Torre

Hayward

(2001). Force can overcome object geometry in the perception of shape through active touch. Nature, 412(6845), 445–448. https://doi.org/10.1038/35086588

38.

Sekiguchi

Hirota

Hirose

(2005). The design and implementation of ubiquitous haptic device. In Proceedings of the IEEE World Haptics Conference (pp. 527–528). https://doi.org/10.1109/WHC.2005.128

39.

Settle

T. B.

(1961). An experiment in the history of science: With a simple but ingenious device Galileo could obtain relatively precise time measurements. Science, 133(3445), 19–23. https://doi.org/10.1126/science.133.3445.19

40.

Shao

Hayward

Visell

(2016). Spatial patterns of cutaneous vibration during whole-hand haptic interactions. Proceedings of the National Academy of Sciences, 113(15), 4188–4193. https://doi.org/10.1073/pnas.1520866113

41.

Solomon

H. Y.

Turvey

M. T.

(1988). Haptically perceiving the distances reachable with hand-held objects. Journal of Experimental Psychology: Human Perception and Performance, 14(3), 404–427. https://doi.org/10.1037/0096-1523.14.3.404

42.

Tanaka

Hirota

(2012). Shaking a box to estimate the property of content. In Isokoski

Springare

(Eds.), Proceedings of EuroHaptics 2012 (Lecture Notes in Computer Science, Vol. 7282, pp. 564–576). Springer. https://doi.org/10.1007/978-3-642-31401-8_50

43.

Turvey

M. T.

(1996). Dynamic touch. American Psychologist, 51(11), 1134–1152. https://doi.org/10.1037/0003-066X.51.11.1134

44.

Turvey

M. T.

Fonseca

S. T.

(2014). The medium of haptic perception: A tensegrity hypothesis. Journal of Motor Behavior, 46(3), 143–187. https://doi.org/10.1080/00222895.2013.798252

45.

Van Boven

R. W.

Ingeholm

J. E.

Beauchamp

M. S.

Bikle

P. C.

Ungerleider

L. G

. (2005). Tactile form and location processing in the human brain. Proceedings of the National Academy of Sciences, 102(35), 12601–12605. https://doi.org/10.1073/pnas.0505907102

46.

Wagman

J. B.

Hajnal

(2014). Task specificity and anatomical independence in perception of properties by means of a wielded object. Journal of Experimental Psychology: Human Perception and Performance, 40(6), 2372–2391. https://doi.org/10.1037/xhp0000014

47.

Wagner

(2016). Pre-Gibsonian observations on active touch. History of Psychology, 19(2), 93–104. https://doi.org/10.1037/hop0000028

48.

Williamson

Murray-Smith

Hughes

(2007). Devices as interactive physical containers: The shoogle system. In Begole

Payne

Churchill

St. Amant

Gilmore

Rosson

M. B.

(Eds.), CHI ‘07 Extended Abstracts on Human Factors in Computing Systems (pp. 2013–2018). ACM. https://doi.org/10.1145/1240866.1240941

49.

Yao

H.-Y.

Hayward

(2006). An experiment on length perception with a rolling stone. In Proceedings of the 1st EuroHaptics Conference (pp. 325–330). IEEE.

50.

Yao

H.-Y.

Hayward

(2010). Design and analysis of a recoil-type vibrotactile transducer. Journal of the Acoustical Society of America, 128(2), 619–627. https://doi.org/10.1121/1.3458852

Humans sense by touch the location of objects that roll in handheld containers

Abstract

Keywords

Sensing content by touch

Balls in a box

One ball in a tube

Present study

Experiment 1: accuracy of locating unseen rolling balls

Method

Apparatus

Procedure

Results and discussion

Movement duration

Movement velocity

Speed at the instant of impact

Experiment 2: relative effectiveness of available cues

Method

Apparatus

Procedure

Results and discussion

General discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

Data available statement

References