Abstract
This work investigates new physical–digital interaction paradigms during VR-based model manipulation, addressing the growing need for accessible, intuitive interaction approaches in immersive environments. It advances understanding of how graspable, user-defined gestures and physical affordances can support spatial reasoning and interaction confidence during design review tasks. Drawing on a synthesis of prior literature and a user elicitation study, the research produces new empirical insights into tangible gesture preferences and the role of familiar physical metaphors in VR interactions. A structured review of VR model manipulation systems was conducted, examining physical controllers, in-air hand gestures, direct and indirect manipulation techniques, and different haptic feedback approaches. This review is consolidated through the development of a new taxonomic classification of tangible interaction modalities for VR model manipulation, organised by feedback type and functional specificity. Building on this foundation, a requirement elicitation survey engaged participants in mapping everyday objects and gestures to manipulation tasks and evaluating physical interaction concepts informed by earlier work on accessible VR tools. Findings highlight consistent preferences for interaction metaphors grounded in everyday experiences, emphasising the importance of spatial alignment, distinct physical affordances, and viewpoint coherence. These insights informed the design and implementation of a VR physical interface prototype, a novel tabletop device that integrates multiple manipulation functions within a single, grounded interaction space. User-based evaluation demonstrates promising outcomes in terms of perceived intuitiveness, spatial understanding, and confidence during model manipulation tasks. The findings establish a foundation for developing further user-elicited, physically grounded gestural interaction paradigms and outline pathways for future research and practice in the design of accessible, human-centred physical–digital interfaces. Collectively, these directions have the potential to support human understanding, agency, and participation within increasingly complex immersive environments.
Keywords
Introduction
Virtual reality (VR) technologies are increasingly adopted across engineering and product design as powerful tools to support collaboration, communication, and decision-making during product development. In these situations, VR offers significant opportunities for rapid visualisation and evaluation of design proposals during review meetings where designers, clients, and stakeholders evaluate and refine digital models. Traditional design reviews rely on physical prototypes, static computer-aided design (CAD) renderings, or screen-based presentations, which often limit the ability to understand complex geometries, spatial relationships, and aesthetic qualities of products. VR environments can overcome these limitations by enabling immersive, rapid visualisation and manipulation of three-dimensional (3D) models for the purpose of evaluating design alternatives (Horvat et al., 2022; Wolfartsberger, 2019). Moreover, recent work highlights trends in VR research that encompass efficient immersive visualisation and interaction, including considerations relevant to design and evaluation during complex tasks (Korkut & Surer, 2023). In parallel, such immersive systems are also framed through the lens of spatial computing, emphasising multimodal and embodied interaction as central to how users perceive, understand, and act within VR environments (Wang et al., 2025).
While VR has demonstrated promise for improving spatial comprehension and engagement during design reviews (Bassanino et al., 2010; Bruno & Muzzupappa, 2010), it remains challenging to create interaction paradigms that are both efficient and intuitive for non-expert users. Standard VR controllers often rely on abstract button mappings, complex menu structures, and unfamiliar metaphors that impose a steep learning curve and cognitive burden, particularly for users without prior exposure to VR tools or 3D modelling software (Jerald & Marks, 2016). Similarly, direct manipulation through hand-tracking can lack precision and generate user fatigue due to inconsistent gesture recognition (Caggianese et al., 2018; Navarro & Sundstedt, 2019). These usability barriers may diminish the potential benefits of VR by increasing disorientation, reducing confidence, and fragmenting collaborative discussions between the participants (Jerald & Marks, 2016; Cummings & Bailenson, 2016).
Relevant research has highlighted the importance of grounding VR interactions in physical analogies to reduce cognitive load and improve training and performance (Huang et al., 2020; Rettinger & Rigoll, 2023). The introduction of tangible props and haptic interfaces can bridge the gap between the virtual and physical worlds, supporting more natural and embodied forms of interaction (McClelland et al., 2017). Yet, despite emerging examples, there is not conclusive understanding of how familiar affordances can inform the design of intuitive physical–digital interfaces. Thus, this work aims to investigate how users perceive the congruence between physical actions and the resulting virtual manipulation, and how such mappings influence their engagement, comprehension, and sense of control in VR. Building upon relevant research in tangible VR interactions, this work investigates how users leverage embodied knowledge from everyday props to intuitively perform spatial transformations such as rotation, scaling, and translation of 3D models. The derived insights have informed the configuration of a functional interface prototype intended to support intuitive manipulation of 3D models in design review contexts.
This paper is structured as follows: Chapter 2 presents a review of VR-based model manipulation research and introduces a novel taxonomy of interaction modalities, establishing the conceptual framework that underpins this study. Chapter 3 outlines the methodological design of the gesture elicitation study and reports empirical derived insights into users’ preferred physical metaphors and manipulation mappings. Chapter 4 describes the development of the VR physical interface prototype and reports findings from user-based evaluation. Finally, Chapter 5 discusses the implications of these findings, examining their capacity to influence future research trajectories in the physical–digital realm and articulating their contribution to accessible, human-centred immersive systems via transferable interaction frameworks.
Relevant Work and Development of Taxonomic Classification
Model manipulation in virtual reality can be understood as the set of actions enabling users to transform, explore, and understand digital models in three-dimensional space. Vinayak et al. (2013) distinguish between CAD manipulation, which involves translating, rotating, or scaling an object without altering its intrinsic geometry, and CAD design, which modifies the shape itself through creation or deformation operations. In the context of this work which focuses on design reviews, the emphasis is put on digital model manipulation, that is, moving objects to inspect their form, revealing sectional views, and scaling them to better apprehend spatial proportions; all critical operations for achieving shared understanding among stakeholders and identifying design issues at early stages.
The following review is structured to progressively build an understanding of the different modes through which users manipulate 3D models in virtual environments. A baseline comparison between physical and abstract interaction, discussing existing controller-based and in-air gesture systems, is initially established. Then, the spectrum of direct and indirect manipulation techniques is introduced, outlining how control mappings range from literal, embodied actions to mediated, metaphorical ones. The next sections examine main types of haptic feedback mechanisms and how these shape user perception, precision, and realism, as well as the functionality of physical proxies, distinguishing between their specificity. The chapter culminates in a taxonomic classification diagram that synthesises these dimensions into a coherent analytical framework for understanding and comparing interaction paradigms in VR-based model manipulation.
Controller Types and Hand Gestures in VR
A substantial body of research has compared the effectiveness of various controller-based, physical input devices and in-air hand gesture recognition systems for VR model manipulation tasks. One recurring theme is the trade-off between naturalness and precision. While both hand-tracking and gesture recognition promise intuitive, controller-free interaction, studies often show that conventional controllers outperform them in accuracy and efficiency. In detail, Kangas et al. (2022) found that hand-only manipulation was significantly slower and less reliable than controller-based methods, largely due to hand pose recognition issues. Similar results were reported by Masurovsky et al. (2020), who compared a modified Leap Motion with enhanced user feedback and Oculus Touch controllers for grab and place tasks. Across all measured parameters, including ease of use, precision, comfort, and satisfaction, the Oculus Touch controller consistently outperformed hand-tracking approaches. Caggianese et al. (2018) further demonstrated that HTC Vive controllers produced faster completion times and lower perceived difficulty in manipulating virtual boxes compared to Leap Motion, emphasising the importance of stable tracking and distinctly established affordances for the benefit of users. Comparable findings are reported by Navarro and Sundstedt (2019), who compared HTC Vive controllers and Leap Motion in VR gaming contexts, reporting decreased player performance and gesture reliability when using Leap Motion, attributing these issues to limited tracking range and inconsistent hand pose recognition.
Beyond technological limitations, several studies highlight the cognitive dissonance and the experiential consequences of in-air gesture interaction. Kangas et al. (2022) noted that recognition issues led to user fatigue and reduced confidence during manipulation tasks. In collaborative VR scenarios, Gusai et al. (2017) showed that controller-based manipulation resulted in higher task accuracy and usability than hand-tracking, while Galais et al. (2019) found that controller input imposed significantly less cognitive load during cube manipulation tasks. Beattie et al. (2015) also observed that although in-air gestures can feel accessible and intuitive for inexperienced users, they are often hindered by accuracy limitations and ambiguous feedback. Relatedly, Zhang et al. (2025a) showed that visible self-hands and visually congruent co-embodied hands can strengthen ownership and agency in collaborative VR tasks. Beyond these studies, alternative devices such as data gloves and pen controllers have been explored, indicating that glove-based input offered higher perceived accuracy and responsiveness compared to controllers (Lee et al., 2017), while Pham and Stuerzlinger (2019) showed that a pen device outperformed controllers in speed and comfort, particularly for users familiar with 2D input tools. More recently, Han et al. (2023) presented VR-HandNet, a controller-based system that synthesises plausible grasping motions, enabling dexterous object manipulation without external hand-tracking. These findings suggest that modified physical inputs inspired by familiar metaphors can enhance ease of use and precision beyond conventional controller-based systems.
Direct and Indirect Model Manipulation Techniques
Beyond purely controller-based or hand-tracked systems, relevant research has also investigated how manipulation tasks are mapped to interaction paradigms, distinguishing direct manipulation techniques from indirect ones which blend metaphors and tangible affordances for object handling. In general, direct manipulation approaches allow users to grab, translate, rotate, or scale objects with gestures that closely mimic real-world actions, enabling an accessible and intuitive form of HCI for inexperienced users (Vinayak et al., 2013). Beattie et al. (2015) also showed that low-cost hand-tracking setups enable users to intuitively inspect and section mechanical models without requiring explicit instructions. Similarly, Xiao and Peng (2017) found that Leap Motion-based interfaces can support natural interaction for translation, rotation, scaling, and model switching. Kang et al. (2013) presented a system for 3D conceptual design using simple hand gestures and real-time recognition, while Song et al. (2014) proposed GaFinC, which combines gesture tracking with gaze pointing to improve intuitiveness and reduce fatigue. Moreover, Noor and Aras (2015) introduced the zSpace system, blending virtual holography with physical interaction to enable natural 3D object manipulation. Still, a recurring finding across all these studies suggests that even though direct manipulation techniques are accessible and low in learning effort, they frequently suffer from hand pose recognition issues and inconsistent feedback; limitations that often require additional instructions or calibration, consequently reducing user confidence during complex tasks.
To address these challenges, researchers have proposed several indirect manipulation techniques that introduce intermediary metaphors or controls to improve accuracy and reduce fatigue (Figure 1). For example, Song et al. (2012) developed the HandleBar metaphor, where users hold a virtual rod with both hands to translate and rotate objects in space. Caputo et al. (2018) created the Smart Pin, enabling one-handed selection, translation, rotation, and scaling through a pin-like handle, freeing the other hand for parallel tasks. Bossavit et al. (2014) presented the Crank Handle, which separates translation and rotation across three virtual handles, while Mendes et al. (2017) introduced MAiOR, a technique that decouples degrees of freedom and provides personalised axes to improve placement accuracy. According to the aforementioned paradigms, indirect strategies tend to provide higher precision and faster performance than direct gestures, especially for tasks requiring fine control. However, they typically impose a steeper learning curve and may increase cognitive load due to requiring the user to understand abstract mappings between physical movements and virtual effects, a fact that confirms the need for establishing clear gesture affordances. Examples of Indirect Manipulation Paradigms: Crank Handle (Bossavit et al., 2014) (Left); and Smart Pin (Caputo et al., 2018) (Right)
Haptic Feedback Mechanisms in VR
While gesture types and manipulation metaphors strongly influence perceived intuitiveness, haptic feedback mechanisms play a complementary role by reinforcing user familiarity, accuracy, and perceived realism during manipulation tasks. Nilsson et al. (2021) introduce a taxonomy that organises VR haptic proxy techniques into eight categories, defined by their deployment timing (offline or real time), the reality being manipulated (physical or virtual), and their functional purpose, either shaping object perception or mediating real–virtual mappings. Although this framework provides a comprehensive classification of haptic strategies within VR system design, the discussion in this subsection focuses specifically on how haptic feedback is communicated to the user. In this context, haptic mechanisms can be categorised into active haptic feedback (AHF) and passive haptic feedback (PHF), with more recent hybrid approaches blending aspects of both, as shown in Figure 2. AHF involves the use of actuators that actively exert force on the user. These systems are often grounded mechanical devices capable of delivering precise kinaesthetic sensations, such as the HapticMaster (van der Linde et al., 2002). More portable ungrounded systems, including exoskeleton gloves (Bouzit et al., 2002) or gyro-based torque devices (Badshah et al., 2012), have been proposed to provide force feedback in handheld configurations. The principal advantage of AHF lies in its generality; a single device can simulate a wide range of textures, forces, and interactions. However, these systems tend to be mechanically complex, computationally intensive, and potentially safety-critical if failures occur during the rendering of active forces (Zenner & Krüger, 2017). Haptic Devices: Rutgers Master II (Bouzit et al., 2002) (Left); Haptic Retargeting (Azmandian et al., 2016) (Middle); and Shifty (Zenner & Krüger, 2017) (Right)
In contrast, PHF refers to the use of props or mechanisms that provide tactile and kinaesthetic sensations which are grounded on their physical features (design, shape, weight, and texture) without actuators applying active forces. The most common approach involves static physical proxies, that is, real objects that are spatially registered with virtual models. For example, Azmandian et al. (2016) demonstrated that inexpensive props can significantly improve user presence during VR interactions, even when they do not perfectly match the shape of their corresponding virtual objects, while Bouzbib et al. (2023) showed that pseudo-haptic deformation can extend a rigid prop’s perceived compliance, with believable stiffness emerging from visual cues alone. Similarly, Feick, Zenner, et al. (2023) noticed that control display gain can be used to render multiple levels of perceived rotational resistance through a static knob. Static passive haptic feedback is valued for its simplicity, low cost, and natural feel. However, its inherent limitation is a lack of generality, as opposed to AHP: each prop only represents a narrow range of virtual objects and cannot dynamically adapt to different scenarios (Insko, 2001; Zenner & Krüger, 2017). To address this constraint, Zenner and Krüger (2017) introduced the concept of dynamic passive haptic feedback (DPHF) where actuators are integrated into passive feedback props to change their physical properties over time, without exerting direct forces onto the user, and therefore extend their expressive range. In particular, Shifty shifts an internal weight to alter its moment of inertia, allowing it to simulate variations in virtual object size, thickness, or weight distribution. Notably, emerging haptic systems have moved beyond force and vibration, with wearable multimodal interfaces combining mechanical, electrotactile, and thermal stimulation to render richer tactile qualities such as roughness, slipperiness, and temperature via skin-integrated interfaces (Huang et al., 2023). Thus, DPHF has been shown to enhance perceived realism and user enjoyment when augmented and synchronised with visual, auditory, and sensing cues.
Single-Purpose and Multi-Purpose and Reconfigurable Physical Proxies
Beyond differentiating feedback mechanisms according to their dynamic properties, haptic proxies can also be characterised by their functional specificity, that is, whether they are designed to represent a single type of virtual object (single-purpose proxies) or can be reconfigured to simulate a broader range of objects (multi-purpose or reconfigurable proxies). Existing research has examined how single-purpose passive props can enhance user immersion, task performance, and satisfaction within VR environments. In detail, Seo et al. (2024) presented that proxy-based object interaction becomes faster, more accurate, and less demanding when virtual representations preserve major affordances and support grasping through adaptive blending, while Medeiros et al. (2023) further showed that nearby passive surfaces improve performance, comfort, and agency under spatially constrained VR conditions. Yang et al. (2018) developed the VR Grabbers, chopsticks-like handheld controllers, designed specifically for precise selection using ungrounded haptic retargeting to enhance the illusion of tactile presence despite the props’ simplicity. Likewise, localised tactile cues on tangible handles such as line and ring forms can significantly strengthen VR embodiment, especially in rehabilitation contexts (Zhang et al., 2024). Visuo-haptic systems have also combined grasping and force cues for automotive model reviews (Ortega & Coquillart, 2005), while weighted props and gloves have been used to simulate effort, inertia, and grip constraints in space operations (Dufresne et al., 2024). In cultural heritage contexts, passive tangible proxies such as desks, chairs, and buttons in a virtual museum and tracked 3D-printed artefact replicas have been used to support immersive navigation, haptic inspection, and effective object assessment (Hulusic et al., 2023; Krumpen et al., 2021). Further than domain-specific setups, Muender et al. (2019) investigated tangible props including uniform-shaped objects, LEGO constructs, and 3D-printed models, showing that while 3D-printed props provided the highest fidelity, LEGO-based proxies were faster to assemble and enabled high performance; Fang et al. (2023) further showed that everyday household objects can function as effective passive proxies when matched through functional affordances rather than precise geometric similarity. Finally, Chang et al. (2017) introduced TASC (Tangibles for Augmented Spatial Cognition), combining movement tracking and tangible objects to foster a strong sense of embodiment during spatial puzzle-solving, demonstrating enhanced engagement and problem-solving for inexperienced users.
Research has also explored multi-purpose passive proxies and reconfigurable objects that adapt to diverse tasks without requiring a dedicated physical item for each virtual element. VirtualBricks (Arora et al., 2019) introduced a toolkit that allows users to construct physical controllers tailored to different VR applications, enabling highly versatile passive haptics. TurkDeck (Cheng et al., 2015) and iTurk (Cheng et al., 2018) extend this idea by dynamically repositioning or replacing props around the user to represent multiple virtual objects. While TurkDeck relies on human operators to rearrange props in real time, iTurk automates this process with robotic actuation. HaptoBend (McClelland et al., 2017) presents a noteworthy approach, using a deformable device that can bend into various 2D or multi-surface 3D shapes to emulate a wide range of interactions as Figure 3 shows. Several systems have also focused on dynamic properties and configurable shape or force feedback. PuPoP (Teng et al., 2018) uses lightweight pneumatic airbags on the palm to simulate grasping different primitive shapes, enhancing realism and presence compared to controllers alone. Drag:on (Zenner & Krüger, 2019) and Transcalibur (Shigeyama et al., 2019) both explore dynamic passive haptic feedback through weight-shifting mechanisms that alter inertia to match the virtual object’s mass properties. HapTwist (Zhu et al., 2019) demonstrates a low-cost toolkit inspired by Rubik’s Twist, allowing users to create diverse graspable shapes in VR. Other concepts, like Sparse (Cheng et al., 2017) and Lightweight Tangible 3D (Jackson et al., 2013), explore reconfigurable or repositionable props combined with perceptual illusions to increase versatility, whereas VoxelHap (Feick, Biyikli, et al., 2023) uses modular functional voxels and attachable plates to assemble handheld proxies with configurable shape, texture, weight, vibration, tracking, and rotational resistance. Devices such as DEXMO (Gu et al., 2016) and the Elastic Arm (Achibet et al., 2015) combine passive haptics with exoskeletons or elastic bands to provide adaptable resistance and tactile cues. ExoInterfaces (Tsetserukou et al., 2010) leverage wearable structures to simulate force and deformation by reproducing human muscle properties, while Urquhart et al. (2023) discuss how additive manufacturing and LEGO-based techniques can be employed for bespoke arm controllers for VR rehabilitation, demonstrating yet another avenue for scalable, reconfigurable haptic feedback. Reconfigurable, Dynamic Objects: PuPoP (Teng et al., 2018) (Left); and HaptoBend (McClelland et al., 2017) (Right)
A Taxonomic Classification of Interaction Modalities in VR-based Manipulation
The reviewed literature is synthesised along four key dimensions to categorise interaction modalities for VR-based model manipulation in a systematic taxonomic framework, as illustrated in Figure 4. Emphasis is placed on haptic interaction, reflecting this study’s focus on the design of a tangible interface. At the highest level, user feedback modality differentiates between haptic and non-haptic forms of interaction, the latter relying primarily on visual, proprioceptive, or symbolic cues. The second dimension, manipulation mapping strategy, distinguishes between direct and indirect techniques, depending on whether user movements correspond directly to virtual transformations or are mediated through abstract metaphors or control mechanisms. The third dimension classifies feedback mechanisms, while the fourth dimension addresses functional specificity, capturing the extent to which a physical proxy is dedicated to a single task or adaptable across multiple functions. A Taxonomic Classification of Interaction Modalities for VR-Based Model Manipulation
These dimensions establish a coherent structure for examining how interaction strategies, sensory feedback, and tangible affordances intersect to define the design space of VR-based 3D model manipulation. This novel taxonomy provides a structured lens for analysis and comparing existing interaction approaches, while also supporting the systematic identification of design opportunities and gaps, towards the development of new interaction paradigms. Beyond classification, it enables both designers and researchers to reason explicitly about the relationships between manipulation intent, physical mediation, and feedback modality, making trade-offs between expressiveness, precision, and accessibility across different manipulation tasks and contexts. In this work, it directly informs the focus and structure of the empirical investigation by guiding the selection, design, and evaluation of physical manipulation strategies.
Gesture Elicitation Study
Motivation and Survey Design
The study was primarily motivated by the need to identify gestures and prop metaphors that feel instinctive to users when manipulating 3D models via VR-based interfaces. Building on the taxonomy presented in Figure 4, the study examines how different forms of physical manipulation are perceived and enacted by users. While prior research has broadly explored user-defined, physical gestures for manipulation tasks (Karam, 2006; Moran-Ledesma et al., 2021; Wobbrock et al., 2009), the present study extends this body of evidence through empirical data that captures how interaction strategies are interpreted, selected, and enacted. Moran-Ledesma et al. (2021) investigated a wide range of everyday props and gestures across both CAD operations and open-ended interaction scenarios, with the aim of deriving a generalised gesture vocabulary and agreement metrics for prop-based interaction. The present work narrows the scope to design review and evaluation contexts, prioritising the implementation and validation of selected metaphors within a functional VR interface prototype. This shift enables a focus on reducing cognitive overhead and supporting spatial comprehension for non-technical stakeholders engaged in model manipulation.
The study composed of an online survey which was designed and distributed through Qualtrics software. It employed a combination of GIF (Graphics Interchange Format) animated images alongside text descriptions to illustrate various handling techniques and aimed to develop an empirically grounded vocabulary of gesture types and physical props to inform 3D model manipulation. The user survey was structured as follows: • The survey’s • • The Five Ring-Based Animated (GIF) Options in Section 1 of the User Survey
Survey Results and Discussion
The study was distributed to one hundred participants. The findings are reported as percentages of responses, with the valid response count (n) provided for each question in the corresponding figures. In Section 1, n ranged from 81 to 91, indicating an item completion rate of 89.0% from the first to the final question. In Section 2, n ranged from 47 to 59, indicating an item completion rate of 79.7%.
The respondents were aged 18 to 34 (n = 56), 35 to 54 (n = 25), and 55+ (n = 8), with a high proportion of them holding professional or postgraduate degrees (55%). They represented a diverse range of domains including product and mechanical design, engineering, architecture, and computer-related fields, reflecting a strong design and technical orientation. Experience profiles indicated a largely non-expert cohort for both tools: for CAD/3D modelling, 54% were not knowledgeable at all or only slightly knowledgeable with smaller groups at moderate (26%), very (15%), and extreme (5%) knowledge levels. For VR, the non-expert tilt is stronger: 76% were not or slightly knowledgeable, 21% moderately knowledgeable, and only ∼1% very knowledgeable. This combination of educational and professional backgrounds suggests a sample that, while largely design-aware, still includes many non-expert or infrequent VR users, aligning with the study’s goal of investigating gesture intuitiveness and tangible metaphors accessible to broader user groups beyond VR specialists. Also, respondents’ lack of strong VR familiarity suggests that the elicited gestures and prop associations reflect natural, transferable intuitions rather than learned behaviour patterns linked to specific VR tools and workflows.
In Overview of Gesture Elicitation Study Section 1 Results
In
Users also consistently preferred props drawn from everyday interactions, such as the Controller joystick (#5), Gear shift knob (#3), and Mouse scroll (#16), reflecting the influence of prior experience in gaming or operating physical devices. Interestingly, Scaling metaphors were split between 2-finger phone pinch (#10) and the Elastic band stretch (#6), showing that users apply both compression and extension metaphors to resizing. When props lacked clear affordances or when their shape or movement could be interpreted in numerous ways, participants sometimes did not associate them to any manipulation tasks (None) highlighting that not all familiar objects translate well to virtual model manipulation. Throughout the results, the movement plane and gesture direction were again critical; even similar props were perceived differently depending on whether the interaction was depicted as horizontal, vertical, or axial, reinforcing the importance of spatial alignment and perspective when designing tangible interfaces. The results are overviewed in Figure 7. Overview of Gesture Elicitation Study Section 2 Results
Beyond the discussed patterns, more nuanced or unexpected observations also emerged. For instance, props were sometimes mapped to manipulation types based on symbolic associations rather than purely spatial affordances; Finger rotary spinner (#20), Rotary Dial (#4), and Gear Shift Knob (#3) were occasionally selected for Scaling tasks, even though neither device physically stretches nor expands; a finding that may reflect familiarity with using dials or buttons to increase/decrease values in other contexts, such as volume control. Similarly, Ball squeeze (#9) received high ratings for Scaling, suggesting that compressive gestures can be perceived as reducing an object’s size.
Props with mixed affordances, like the Gear shift knob (#3), were associated with different tasks, illustrating how real-world objects that combine movement types can increase ambiguity in VR interface design. The Controller joystick (#5) and the Pump handle pull (#18) were also interpreted as appropriate for multiple transformations, underlining the need to clarify constraints when adapting familiar props into VR systems. Several props that are highly familiar in daily life, for example, the Foot pedal press (#11) and the Light switch toggle (#12), still attracted a substantial proportion of None responses, highlighting that familiarity alone is insufficient if the movement does not feel directly relevant to 3D model manipulation and the prop maps a spatial transformation. Another notable finding concerned the Tennis Ball Squeeze (#9); although it was primarily selected for Scaling, it also received a sizable proportion of responses for Changing Appearance. This likely emerged because the compressive action of squeezing closely mimics the pressing of a spring-return button, as in prop #13, leading some participants to interpret it as a discrete trigger rather than a continuous transformation. The Gear Shift Knob (#3) also presented an interesting combination of responses as it was associated almost exclusively with Translations while receiving very few selections for Rotation. This is probably due to the constrained plane of motion in the real-world device, which primarily moves along linear paths. However, the presence of votes for Scaling and a relatively high number of None responses suggest that participants found the metaphor ambiguous overall, with no clear consensus about its intended purpose.
With regard to the Controller Joystick (#5), it showed a very balanced distribution across Horizontal Translation, Vertical Translation, Horizontal Rotation, and Vertical Rotation. While this spread reflects the joystick’s versatility and familiarity as a general-purpose control, it also indicates potential confusion about its specific mapping to any single type of manipulation when context is lacking. The Linear Lever (#2) similarly attracted a very even spread of selections across multiple manipulation types, perhaps due to the fact that the lever moves only in a single forward–backward direction, which does not strongly evoke any particular 3D transformation. As a result, participants may have felt unsure whether it should be interpreted as a slider, a toggle, or a control for translation or rotation, leading to an ambiguous prop profile. Finally, comparisons between the Multi-Arm Lathe Handle (#1) and the Single-Arm Rotary Handle (#17) highlighted subtle differences in perception that may relate to both the number of grasping points and the orientation depicted in the images. While both props have a broadly similar rotary function, the Lathe Handle appears more clearly designed for continuous turning with multiple contact points, whereas the Single-Arm Handle more closely resembles a valve or steering wheel. This likely influenced whether participants saw them as suited to Rotation or other operations.
An age-based comparison indicated that preferred mappings remained broadly consistent in both sections, suggesting that age did not substantially affect preference for particular metaphors. Clearer variation appeared only for a small number of ambiguous props in Section 2, including Light switch toggle (#12), which the 18 to 34 group more often associated with screenshots and changing appearance, and Colour toggle buttons (#14), for which the 35 to 54 group more often favoured vertical translation. The 55+ group did not display a systematic divergence from the other groups, although its small size limits stronger interpretation.
Prototype Development and User Evaluation
Configuration and Design Features of the Physical–Digital VR Interface
The results of the gesture elicitation study strongly informed the development of a working prototype for tangible VR interaction by Loud1Design and the University of Strathclyde. The primary goal of this setup was to consolidate the most instinctive and widely agreed-upon metaphors into a single, cohesive interface that could support multiple manipulation modes. This approach reflects previous insights from the literature on the enforcement of haptic feedback and the application of single- and multi-purpose/reconfigurable physical props, to reduce cognitive load and physical complexity by enabling one device to serve multiple functions depending on alignment, plane of motion, and orientation. Since the effectiveness of a circular table setup for collaborative design reviews has already been demonstrated in previous work, the present study extends this concept by embedding graspable mechanisms into the same familiar round form factor. The design configuration of the prototype was developed by merging users’ preferable gestures into a unified setup, as illustrated in Figure 8, which shows how the movement axes and directional principles for each of the four primary manipulation types were combined. This diagram directly informed the way features were designed, attached, and positioned onto the novel VR physical interface prototype. Diagram Illustrating Four Manipulation Techniques Were Synthesised to Formulate a Single Setup
Each manipulation type is represented through distinct movements and alignments of the same round surfaces. The larger horizontally placed ring acts as a table for displaying and exploring the model, and as a handle for Horizontal Rotation when grasped and rotated in-plane. Vertical Translation is performed by lifting or lowering this same ring along the vertical axis, facilitated by a mechanism attached between its lower part and the system’s stand. To accommodate Vertical Rotation, smaller half-ring-shaped handles are attached onto a central handlebar and can be simultaneously rotated around their axis in a way that aligns with users’ perspective and point of view. Since the components’ rigid materials do not allow lateral stretching, Scaling is also facilitated by the half-ring handles, which can move closer together or farther apart to simulate the ‘stretching’ motion identified as most intuitive and the ‘stretched ribbon’ metaphor preferred by 67% of respondents. The arc-shaped, half-ring handles were preferred to complete rings to signal a bounded motion rather than continuous full rotation, to clearly indicate hand placement and to maintain a stronger formal association with the inward–outward movement used for scaling. Horizontal Translation was not included as the interface follows a tabletop logic, in which the model remains centrally anchored on the physical/virtual table. Within this configuration, Vertical Translation supports adjustment for model scale and user height, while alternative sides are accessed through Horizontal Rotation rather than lateral displacement.
The overall physical configuration was designed around a minimum number of handles that dynamically switch between functions depending on their orientation and the plane in which they operate, as Figure 9 shows. The task-specific features are highlighted in light green, while the red cube indicates the potential movement of the reviewed model. By avoiding a separate controller for each manipulation task, this approach reduces mechanical complexity while preserving an intuitive mapping between movement and manipulation, ensuring that interactions remain closely aligned with the model’s underlying spatial transformations. The 4 Manipulation Tasks Incorporated Into the VR Physical Interface Prototype
Several detailed features further reinforce intuitive interaction. The flat round surface serves both as a recognisable table metaphor and a comfortable grasping area, echoing the positive associations with turntable-like controls. Clearly visible slots provide distinct grabbing points, helping users understand where to place their hands and how to initiate rotational movement, thereby increasing both accuracy and perceived satisfaction. Figure 10 illustrates the developed VR Physical Interface along with its key components and design features. The VR Physical Interface With Key Design Components and Features Highlighted
User Evaluation Trials and Observations
Preliminary evaluation of the system was conducted with four participants, two recruited from academia and two from industry with variant levels of VR experience. Each one engaged in a design review scenario using the system, and their interactions were filmed for later analysis. Participants were not given a separate warm up phase; instead, they received an initial explanation of the system before proceeding directly into the trial, during which a ‘think aloud’ exercise with concurrent dialogue was used to clarify operation where necessary and to capture their immediate feelings, reactions, usability issues, and feedback. The VR setup consisted of an HTC Vive headset paired with trackers attached to the tabletop interface to align the physical handles with model manipulation tasks. Intriguing instances of user interaction behaviours are overviewed in Figure 11, which depicts the different ways in which participants navigated the physical elements of the interface during the evaluation sessions. Instances of Users Interacting With the VR Physical Interface, Showing Diverse Ways of Navigating the Handles
Examples of Captured User Quotes During Prototype Evaluation, Grouped by Identified Themes
Clear affordances and spatial cues were also highlighted, with users commenting positively on the slot features, which effectively guided grasping and turning gestures. One user noted that ‘the overall wheel analogy is very intuitive’, reinforcing the value of maintaining the round table metaphor as an anchoring element for orientation and familiarity. Despite occasional difficulties locating the handles in VR, most participants were still able to complete most manipulation tasks without extensive instruction, signifying that the configuration and alignment of the controls were generally understandable. The large, clearly shaped handles were also praised, as they made rotation and scaling feel approachable even for first-time users. Yet, some struggled to locate the handles while wearing the VR headset, underscoring the need to ‘calibrate’ their perception of where virtual and physical controls aligned, along with the potential addition of hand-tracking technology to improve visibility and orientation.
Most notably, the trials revealed that users often experimented with diverse ways of interacting with the handles, sometimes performing actions differently than intended. For example, one user tended to grab the VR tracker, misinterpreting it for a handle, while another grabbed the two vertical handles from the middle bar instead of the intended endpoints. In some cases, the Vertical Rotation handles were also used to rotate the entire interface horizontally. These observations suggest that while the multi-purpose design was flexible, clearer cues may be needed to reduce confusion between functions. Furthermore, some users were interacting with the whole setup at a different alignment once the main Horizontal Rotation ring had been turned, which sometimes resulted in grasping vertical handles with one hand. This points to a limitation of the current prototype and indicates that in refined versions, the main horizontal ring should be able to rotate independently from the vertical handles to maintain consistent orientation relative to user perspective. Likewise, participants who were unfamiliar with the setup spent more time circling the interface to find comfortable positions. Additional usability issues emerged during Scaling tasks, where one participant noted that resizing ‘was not very obvious’, requiring explicit guidance to discover the gesture.
Feedback highlighted several design refinements for future iterations to further enhance user feedback, satisfaction, and accuracy during the required tasks, including reconfiguring the handle shapes to reduce wrist extension and make the scaling function more apparent, adding control panel buttons to reset model orientation or enable fixed 90-degree rotations, and introducing ratcheting rotational notches between the handles and supporting bars to provide discrete auditory–haptic cues that improve precision and confirm incremental input. Further refinements could also explore variable resistance or soft-end stop cues to communicate transformation limits, as well as the separation of movements so that users do not need to reposition themselves physically around the device.
Regarding comparable research paradigms, a few superficial commonalities are shared with the Haptic-go-round platform (Huang et al., 2020) which uses a motorised circular arrangement to reposition encounter-type props around the user, as opposed to this work which allows direct control and manual reorientation over automated reconfiguration. This distinction tends to strongly emphasise the importance of continuous manual input, reflecting the project’s emphasis on preserving fine-grained control and tactile congruence throughout the interaction. Thus, this work contributes to the VR interaction field by embedding domain-specific gestures and metaphors, derived directly from empirical user data, into a unified control surface. Through the lens of Nilsson et al. (2021) taxonomy, the developed prototype can be defined as ‘a real-time deployment technique that manipulates the physical reality of tangible props to support object perception, primarily through spatial collocation and congruent movement between the graspable handles and the virtual model’.
Discussion
Physical–Digital Interaction for Accessible User Experiences
Grounded Physical–Digital Interaction in VR
The present work advances physical–digital capabilities in VR interaction by embedding empirically elicited gestures and haptic metaphors within a novel physical interface, supporting users in maintaining spatial orientation and sustained focus, while diminishing the feeling of overwhelmedness by complex controls commonly associated with immersive environments. The configuration of multi-purpose physical handles enables users to manipulate models through spatially congruent, graspable interactions while preserving the key qualities of presence, contextualisation, and engagement. The developed system relies on the user’s proprioception and spatial memory to align real handles with virtual transformations; thus, it composes of a deliberately indirect and diegetic design of an anchored platform that reinforces presence and ease of use.
Viewed in relation to recent advances in tangible and embodied VR interaction, this contribution engages with a broader shift towards physically grounded, human-centred approaches that prioritise accessibility, confidence, and understanding over abstract or controller-centric interaction paradigms. Research in immersive interaction design emphasises the importance of haptic engagement in supporting confidence, sustained attention, and shared understanding in virtual environments; the present work extends this trajectory by demonstrating how spatially anchored, graspable controls can function as stable reference structures during complex 3D manipulation tasks. This is particularly relevant in contexts such as design reviews, where precise spatial alignment and physical–digital correspondence are critical, and where tangible interaction has been shown to reduce cognitive load and foster trust in virtual environments, enabling users to draw on embodied knowledge (Zhang et al., 2025b). Similarly, when shared artefacts meaningfully span the real–virtual boundary, tangible interaction can enhance user presence by reinforcing continuity between physical action and virtual effect (Bozgeyikli, 2024).
The value of physical grounding has also been associated with improved task focus and experiential realism, particularly when compared to free-hand interaction alone (Rettinger & Rigoll, 2023). Tangible augmentation in particular has been identified not merely as an input modality but as a cognitive and emotional scaffold capable of sustaining attention and engagement among users with reduced capacity or heightened vulnerability (Feng et al., 2025). In this context, the present work extends these insights into VR-based design reviews by systematically configuring physical–digital interaction to support shared decision-making, spatial comprehension, and inclusive participation.
Furthermore, this research contributes to ongoing debates around what constitutes ‘natural’ interaction in VR. While recent studies highlight the rise of multimodal, controller-free techniques such as in-air gestures, gaze and speech (Chen et al., 2024; Wang et al., 2025), the findings presented here indicate that naturalness may also emerge from clearly recognisable physical affordances, as participants consistently favoured interactions grounded in familiar bodily actions when coupled with a stable tangible reference frame. Thus, the tabletop device illustrates how indirect, graspable interaction can be perceived as more intuitive than direct hand-tracking, as it reduces ambiguity and aligns virtual manipulation with prior embodied experience.
The Role of Physical–Digital in Supporting Human Agency and Engagement
With regard to human capability and user engagement, this work seeks to extend users’ existing embodied skills without presupposing prior technical expertise. This is achieved by structuring interaction around stable and familiar physical reference points, prompting users to engage in model transformation in ways that align with everyday encounters. Such an approach can be particularly relevant in design-related contexts and user requirement elicitation scenarios, where participants may demonstrate diverse levels of design proficiency and familiarity with emerging technologies. By reducing cognitive barriers and clarifying interaction mappings, the system supports confidence collaboration and meaningful engagement across varied user groups.
Building on this emphasis on bodily familiarity and the physical–digital continuum, this work contributes to a broader research agenda that frames physical–digital interaction as a mechanism for augmenting human capabilities in immersive environments. The device’s physical elements support orientation, confidence, and sustained engagement, shaping how users interpret and act within the virtual space. Engagement is therefore conceptualised as a progressive and negotiable process, emerging through users’ movements, experience, and intent, rather than being dictated by complex or attention-intensive interaction techniques.
In foregrounding the role of familiar physical references in maintaining cognitive clarity, social awareness, and agency during immersive tasks, this research preserves and extends embodied skills such as spatial reasoning, bodily coordination, and shared orientation. In doing so, it advances design approaches that prioritise accessibility, confidence, and participation as central attributes of enhanced human experience, while identifying directions for further research into supportive, human-centred physical–digital interaction paradigms within extended-reality environments.
Conclusions and Future Work
This work contributes a design-led and user-centred exploration of tangible interaction for VR design reviews, developed through a coherent methodological process that synthesises insights from literature review, gesture elicitation, prototyping, and user-based evaluation. The resulting prototype demonstrates how empirically grounded interaction mappings and a multi-functional physical interface can support spatial understanding and manipulative interactions within VR environments, advancing current approaches to 3D model manipulation.
While the evaluation focused on the overall user experience, perceived intuitiveness, and observed patterns of use, the findings indicate promising effects in terms of user confidence, engagement, and ease of model manipulation. A limitation of the present work lies in the absence of a systematic assessment against predefined performance metrics, such as task efficiency, accuracy, temporal or physical demand, and perceived competence. Addressing this through larger-scale studies, targeted manipulation tasks, and comparative evaluations constitutes a clear direction for future research, supporting further validation and refinement of the prototype and its underlying interaction paradigms. More detailed participant profiling, including prior exposure to comparable systems, will also form part of future work to support precise interpretation of usability behaviours.
From a practical perspective, the outcomes of this research have the capacity to inform the design of VR interaction systems in domains where accurate spatial manipulation and accessibility are critical, such as early stage design reviews, participatory co-design workshops, education, or training environments, while also providing a transferable foundation for developing future physical–digital gestural interaction schemes that can be adapted across diverse tasks, contexts, and user groups. This is particularly relevant as tangible elements are increasingly emerging across both commercial and research VR hybrid systems, highlighting the need for low-barrier, design-led approaches to future interaction toolkits. Modern pass-through modes may offer a useful extension to physical grounding by allowing users to remain visually connected to their surroundings and nearby collaborators, potentially reducing spatial disorientation and cybersickness. In parallel, the prototype’s core principles could transfer to Augmented/Mixed Reality by preserving the relationship between manipulation types and corresponding handles, while redesigning the overall setup as a more spatially integrated system adaptable to different physical tables and environments. In this sense, the work contributes to a broader research agenda that frames physically grounded gesture design as a pathway towards interaction languages that remain intelligible, inclusive, and supportive of human understanding, agency, and participation, in a climate of rapidly evolving immersive technologies.
Footnotes
Acknowledgements
We would like to acknowledge the contributions of Brian Loudon (Loud1Design Ltd.) and Craig Fingland in the development and evaluation of the prototype.
Ethical considerations
All participants’ identifying details have been omitted.
Author Contributions
KP and AW jointly contributed to the study conceptualisation, methodology, and analysis. AW led the funding acquisition. KP led the writing of the original draft. AW contributed to manuscript review and editing. AW provided supervision and strategic guidance. All authors approved the final manuscript.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Innovate UK, grant reference 104856.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.