Robotic systems have been developed to handle very small objects, but their use remains complex and necessitates long-duration training. Simulators, such as molecular simulators, can provide access to large amounts of raw data, but only highly trained users can interpret the results of such systems. Haptic feedback in teleoperation, which provides force feedback to an operator, appears to be a promising solution for interaction with such systems, as it allows intuitiveness and flexibility. However, several issues arise while implementing teleoperation schemes at the micro- and nanoscale, owing to complex force fields that must be transmitted to users and scaling differences between the haptic device and the manipulated objects. Major advances in such technology have been made in recent years. In this chapter, we review the main systems in this area and highlight how some fundamental issues in teleoperation for micro- and nanoscale applications have been addressed. We consider three types of teleoperation, including (a) direct (manipulation of real objects), (b) virtual (use of simulators), and (c) augmented (combining real robotic systems and simulators). Remaining issues that must be addressed for further advances in teleoperation for micro- and nanoworlds are also discussed, including (a) comprehension of phenomena that dictate very small object (<500 micrometers) behavior and (b) design of intuitive 3-D manipulation systems. Design guidelines to realize an intuitive haptic feedback teleoperation system at the micro- and nanoscale level are proposed.
AbeN.MizokamiR.KinoshitaY.HeS. (2007). Providing simulation of medical manipulation with haptic feedback. In 17th International Conference on Artificial Reality and Telexistence (pp. 143–148). New York, NY: IEEE.
2.
AmmiM.FerreiraA. (2007). Robotic assisted micromanipulation system using virtual fixtures and metaphors. In IEEE International Conference on Robotics and Automation (pp. 454–460). New York, NY: IEEE.
3.
AmmiM.LadjalH.FerreiraA. (2006). Evaluation of 3D pseudo-haptic rendering using vision for cell micromanipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2115–2120). New York, NY: IEEE.
AraiF.OgawaM.FukudaT. (2000). Indirect manipulation and bilateral control of the microbe by the laser manipulated microtools. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 665–670). New York, NY: IEEE.
6.
AshkinA.DziedzicJ. M.BjorkholmJ. E.ChuS. (1986). Observation of a single-beam gradient force optical trap for dielectric particles. Optics Letters, 11, 288–290.
7.
BargielS.RabenorosoaK.ClévyC.GoreckiC.LutzP. (2010). Towards micro-assembly of hybrid MOEMS components on reconfigurable silicon free-space micro-optical bench. Journal of Micromechanics and Microengineering, 20(4).
8.
BolopionA.CagneauB.HaliyoS.RégnierS. (2009). Analysis of stability and transparency for nanoscale force feedback in bilateral coupling. Journal of Micro-Nano Mechatronics, 4, 145–158.
9.
BolopionA.CagneauB.RedonS.RégnierS. (2010). Comparing position and force control for interactive molecular simulators with haptic feedback. Journal of Molecular Graphics and Modelling, 29, 280–289.
10.
BolopionA.CagneauB.RedonS.RégnierS. (2011). Variable gain haptic coupling for molecular simulation. In IEEE World Haptics Conference (pp. 469–474). New York, NY: IEEE.
11.
BolopionA.CagneauB.RégnierS. (2009). 2D micro teleoperation with force feedback. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 3265–3270). New York, NY: IEEE.
12.
BolopionA.DahmenC.StolleC.HaliyoS.RégnierS.FatikowS. (2012). Vision based haptic feedback for remote micromanipulation in a SEM environment. International Journal of Optomechatronics, 6, 236–252.
13.
BolopionA.RégnierS. (2013). A review of haptic feedback teleoperation systems for micromanipulation and microassembly. IEEE Transactions on Automation Science and Engineering, 10, 496–502.
14.
BolopionA.StolleC.TunnellR.HaliyoS.RégnierS.FatikowS. (2011). Remote microscale teleoperation through virtual reality and haptic feedback. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 894–900). New York, NY: IEEE.
15.
BolopionA.XieH.HaliyoS.RégnierS. (2012). Haptic teleoperation for 3D microassembly of spherical objects. IEEE/ASME Transaction on Mechatronics, 17(1), 116–127.
16.
BoukhniferM.FerreiraA. (2006). Wave-based passive control for transparent micro-teleoperation system. Robotics and Autonomous Systems, 54, 601–615.
17.
BoukhniferM.FerreiraA. (2007). H∞ loop shaping bilateral controller for a two-fingered tele-micro-manipulation system. IEEE Transactions on Control Systems Technology, 15, 891–905.
18.
BowmanR.PreeceD.GibsonG.PadgettM. (2011). Stereoscopic particle tracking for 3D touch, vision and closed-loop control in optical tweezers. Journal of Optics, 13(4), 044003.
19.
BrooksF. P.Jr.Ouh-YoungM.BatterJ. J.KilpatrickP. J. (1990). Project GROPE: Haptic displays for scientific visualization. In Conference on Computer Graphics and Interactive Techniques (pp. 177–185). New York, NY: ACM.
20.
BukusogluI.BasdoganC.KirazA.KurtA. (2008). Haptic manipulation of microspheres using optical tweezers under the guidance of artificial force fields. Presence: Teleoperators and Virtual Environments, 17, 344–364.
21.
BustamanteC.BryantZ.SmithS. (2003). Ten years of tension: Single-molecule DNA mechanics. Nature, 421, 423–427.
22.
ChoS.-Y.ShimJ.-H. (2004). A new micro biological cell injection system. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1642–1647). New York, NY: IEEE.
23.
DaunayB.RégnierS. (2009). Stable six degrees of freedom haptic feedback for flexible ligand-protein docking. Computer Aided Design, 41, 886–895.
24.
de VriesA. H.KrennB. E.van DrielR.KangerJ. S. (2005). Micro magnetic tweezers for nanomanipulation inside live cells. Biophysical Journal, 88, 2137–2144.
25.
DixA.FinlayJ. E.AbowdG. D.BealeR. (2003). In Human–computer interaction (3rd ed., pp. 258–288). Harlow, UK: Pearson Education Limited.
26.
FahlbuschS.ShirinovA.FatikowS. (2002). AFM-based micro force sensor and haptic interface for a nanohandling robot. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1772–1777). New York, NY: IEEE.
27.
FalvoM.ClaryG.HelserA.PaulsonS.TaylorR.ChiV.. . . SuperfineR. (1999). Nanomanipulation experiments exploring frictional and mechanical properties of carbon nanotubes. Journal of Microscopy and Microanalysis, 4, 504–512.
28.
FerreiraA.MavroidisC. (2006). Virtual reality and haptics for nanorobotics. IEEE Robotics and Automation Magazine, 13(3), 78–92.
29.
FinkelsteinN. D.AdamsW. K.KellerC. J.KohlP. B.PerkinsK. K.PodolefskyN. S.. . . LeMasterR. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics–Physics Education Research, 1(1), 010103.
30.
FokL.LiuY.LiW. (2005). Modeling of nanomanipulation with an integrated teleoperated system. In IEEE International Conference on Robotics and Biomimetics (pp. 83–88). New York, NY: IEEE.
31.
GaoZ.LécuyerA. (2009). Path-planning and manipulation of nanotubes using visual and haptic guidance. In IEEE International Conference on Virtual Environments, Human–Computer Interfaces and Measurements Systems (pp. 1–5). New York, NY: IEEE.
32.
GhanbariA.AbdiH.HoranB.NahavandiS.XiaoqiC.WenhuiW. (2010). Haptic guidance for microrobotic intracellular injection. IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (pp. 162–167). New York, NY: IEEE.
33.
GosseC.CroquetteV. (2002). Magnetic tweezers: Micromanipulation and force measurement at the molecular level. Biophysical Journal, 82, 3314–3329.
34.
GrangeS.ContiF.HelmerP.RouillerP.BaurC. (2001). The delta haptic device as a nanomanipulator. In SPIE Microrobotics and Microassembly III (pp. 100–111). Bellingham, WA: SPIE.
35.
GrudininS.RedonS. (2010). Practical modeling of molecular systems with symmetries. Journal of Computational Chemistry, 31, 1799–1814.
36.
GutholdM.FalvoM.MatthewsW. G.PaulsonS.MullinJ.LordS.TaylorR. M. (1999). Investigation and modification of molecular structures with the nanoManipulator. Journal of Molecular Graphics and Modelling, 17(3–4), 187–197.
37.
GutholdM.FalvoM.MatthewsW.PaulsonS.WashburnS.ErieD.. . . TaylorR.II. (2000). Controlled manipulation of molecular samples with the nanoManipulator. IEEE/ASME Transactions on Mechatronics, 5, 189–198.
38.
HamdiM.FerreiraA.SharmaG.MavroidisC. (2008). Prototyping bio-nanorobots using molecular dynamics simulation and virtual reality. Microelectronics Journal, 39, 190–201.
39.
HamdiM.SharmaG.FerreiraA.MavroidisC. (2005). Molecular mechanics simulation of bio-nanorobotic components using force feedback. In IEEE International Conference on Robotics and Biomimetics (pp. 105–110). New York, NY: IEEE.
40.
HatamuraY.MorishitaH. (1990). Direct coupling system between nanometer world and human world. In IEEE International Conference on Micro Electro Mechanical Systems (pp. 203–208). New York, NY: IEEE.
41.
HollisR.SalcudeanS.AbrahamD. (1990). Toward a tele-nanorobotic manipulation system with atomic scale force feedback and motion resolution. In IEEE International Conference on Micro Electro Mechanical Systems (pp. 115–119). New York, NY: IEEE.
42.
IkinL.CarberryD. M.GibsonG. M.PadgettM. J.MilesM. J. (2009). Assembly and force measurement with SPM-like probes in holographic optical tweezers. New Journal of Physics, 11(2), 1–8.
43.
ISO 9241-11. (1998). Ergonomic requirements for office work with visual display terminals (VDTs): Part 11. Guidance on usability. Geneva, Switzerland: ISO.
44.
JonesM. G.AndreT.SuperfineR.TaylorR. (2003). Learning at the nanoscale: The impact of students’ use of remote microscopy on concepts of viruses, scale, and microscopy. Journal of Research in Science Teaching, 40, 303–322.
45.
JonesM. G.MinogueJ.TretterT. R.NegishiA.TaylorR. (2006). Haptic augmentation of science instruction: Does touch matter?Science Education, 90, 111–123.
46.
KimD.-H.KimB.KangH.JuB.-K. K. (2003). Development of a piezoelectric polymer-based sensorized microgripper for microassembly and micromanipulation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1864–1869). New York, NY: IEEE.
47.
KimD.-H.KimB.YunS.KwonS. (2004). Cellular force measurement for force reflected biomanipulation. In IEEE International Conference on Robotics and Automation (pp. 2412–2417). New York, NY: IEEE.
48.
KimD.-H.KimK.KimK.-Y.ChaS.-M. (2001). Dexterous teleoperation for micro parts handling based on haptic/visual interface. In International Symposium on Micromechatronics and Human Science (pp. 211–217). New York, NY: IEEE.
49.
KimJ.Janabi-SharifiF.KimJ. (2008). Haptic feedback based on physically based modeling for cellular manipulation systems. In FerreM. (Ed.), Haptics: Perception, devices and scenarios (Vol. 5024, pp. 661–667). Berlin, Germany: Springer.
50.
KimS.-G.SittiM. (2006). Task-based and stable telenanomanipulation in a nanoscale virtual environment. IEEE Transactions on Automation Science and Engineering, 3, 240–247.
51.
LadjalH.HanusJ.-L.FerreiraA. (2011). Microrobotic simulator for assisted biological cell injection. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1315–1320). New York, NY: IEEE.
52.
LadjalH.HanusJ. L.PillarisettiA.KeeferC.FerreiraA.DesaiJ. P. (2012). Reality-based real-time cell indentation simulator. IEEE/ASME Transactions on Mechatronics, 17, 239–250.
53.
Lai-YuenS.LeeY.-S. (2006). Energy-field optimization and haptic-based molecular docking and assembly search system for computer-aided molecular design (CAMD). In Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (pp. 233–240). New York, NY: IEEE.
54.
LeV. T.NahavandiS.CreightonD. (2010). A haptic training environment for the heart myoblast cell injection procedure. In IEEE International Conference on Control Automation Robotics & Vision (pp. 448–452). New York, NY: IEEE.
55.
LeeS.-W.LeeT.LeeY.-G. (2007). Stable manipulating of nanowires by line optical tweezers with haptic feedback. Optical Trapping and Optical Micromanipulation, 4, 6644.
56.
LeeY.-G.LyonsK. W. (2004). Smoothing haptic interaction using molecular force calculations. Computer-Aided Design, 36(1), 75–90.
57.
LewisJ. R. (2006). Sample sizes for usability tests: mostly math, not magic. Interactions–Waits & Measures, 13(6), 29–33.
58.
LiuL.JiaoN.TianX.DongZ.XiN.LiW.WangY. (2006). Development of a haptic user interface for surface sensing and nanomanipulation based on atomic force microscope. In IEEE International Conference on Nano/Micro Engineered and Molecular Systems (pp. 900–904). New York, NY: IEEE.
59.
LiuX.SunY.WangW.LansdorpB. (2007). Vision-based cellular force measurement using an elastic microfabricated device. Journal of Micromechanics and Microengineering, 17, 1281–1288.
60.
MarchiF.MarliereS.UrmaD.FlorensJ.-L.ChevrierJ.CadozC.LucianiA. (2005). Interactive learning of nanophysics phenomena. In International Conference on Multimedia and ICTs in Education (pp. 510–515). Badajoz, Spain: Formatex Research Center.
61.
MarchiF.UrmaD.MarliereS.FlorensJ. L.BesanconA.ChevrierJ.LucianiA. (2005). Educational tool for nanophysics using multisensory rendering. In World Haptics Conference (pp. 473–474). New York, NY: IEEE.
62.
MarliereS.UrmaD.FlorensJ.MarchiF. (2004). Multi-sensorial interaction with a nano-scale phenomenon: The force curve. In EuroHaptics (pp. 246–252). Munich, Germany: Institute of Automatic Control Engineering (LSR).
63.
MaugisD. (2000). Contact, adhesion, and rupture of elastic solids. Springer Series in Solid-State Sciences (Book 130). Berlin, Germany: Springer-Verlag.
64.
MilletG.LécuyerA.BurkhardtJ.HaliyoS.RégnierS. (2008). Improving perception and understanding of nanoscale phenomena using haptics and visual analogy. In EuroHaptics (pp. 847–856). Berlin, Germany: Springer-Verlag.
65.
MilletG.LécuyerA.BurkhardtJ.-M.HaliyoS.RégnierS. (2013). Haptics and graphic analogies for the understanding of atomic force microscopy. International Journal of Human-Computer Studies, 71(5), 608–626
66.
MinogueJ.JonesM. G.BroadwellB.OppewallT. (2006). The impact of haptic augmentation on middle school students’ conceptions of the animal cell. Virtual Reality, 10, 293–305.
67.
Mohand OusaidA.MilletG.RégnierS.HaliyoS.HaywardV. (2012). Haptic interface transparency achieved through viscous coupling. International Journal of Robotics Research, 31, 319–329.
68.
MurayamaJ.ShimizuH.NamC. S.SatohH.SatoM. (2007). An educational environment for chemical contents with haptic interaction. In International Conference on Cyberworlds (pp. 346–352). New York, NY: IEEE.
69.
NagataH.MizushimaH.TanakaH. (2002). Concept and prototype of protein-ligand docking simulator with force feedback technology. Bioinformatics, 18(1), 140–146.
70.
NeumanK. C.BlockS. M. (2004). Optical trapping. Review of Scientific Instruments, 75, 2787–2809.
71.
NiZ.BolopionA.AgnusJ.BenosmanR.RégnierS. (2012). Asynchronous event-based visual shape tracking for stable haptic feedback in microrobotics. IEEE Transactions on Robotics, 28, 1081–1089.
72.
NiZ.PacoretC.BenosmanR.IengS.RégnierS. (2011). Asynchronous event based high speed vision for microparticle tracking. Journal of Microscopy, 245, 236–244.
73.
NielsenJ. (1993). Usability engineering. San Diego, CA: Academic Press.
74.
NiguèsA.VenantN.RodriguesM.CominF.FlorensJ.MarchiF. (2012). Haptic localization and shape recognition of nano objects. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2319–2324). San Diego, CA: Academic Press.
75.
OkamuraA. M.RichardC.CutoskyM. R. (2002). Feeling is believing: Using a force-feedback joystick to teach dynamic systems. Journal of Engineering Education, 91, 345–349.
76.
OnalC. D.SittiM. (2009). A scaled bilateral control system for experimental one-dimensional teleoperated nanomanipulation. International Journal of Robotics Research, 28, 484–497.
77.
OnalC. D.SittiM. (2010). Teleoperated 3D force feedback from the nanoscale with an atomic force microscope. IEEE Transactions on Nanotechnology, 9(1), 46–54.
Ouh-YoungM.PiqueM.HughesJ.SrinivasanN.BrooksF.Jr. (1988). Using a manipulator for force display in molecular docking. In IEEE International Conference on Robotics and Automation (pp. 1824–1829). New York, NY: IEEE.
80.
PacoretC.BowmanR.GibsonG.HaliyoS.CarberryD.BerganderA.. . . PadgettM. (2009). Touching the microworld with force-feedback optical tweezers. Optics Express, 17, 10259–10264.
81.
PacoretC.RégnierS. (2013). Invited article: A review of haptic optical tweezers for an interactive microworld exploration. Review of Scientific Instruments, 84(8).
82.
PerssonP.CooperM.TibellL.AinsworthS.YnnermanA.JonssonB.-H. (2007). Designing and evaluating a haptic system for biomolecular education. In IEEE Virtual Reality Conference (pp. 171–178). New York, NY: IEEE.
83.
PillarisettiA.AnjumW.DesaiJ.FriedmanG.BrooksA. (2005). Force feedback interface for cell injection. In World Haptics Conference (pp. 391–400). New York, NY: IEEE.
84.
PillarisettiA.PekarevM.BrooksA. D.DesaiJ. P. (2007). Evaluating the effect of force feedback in cell injection. IEEE Transactions on Automation Science and Engineering, 4, 322–331.
85.
PodolefskyN. S.FinkelsteinN. D. (2006). Use of analogy in learning physics: The role of representations. Physical Review Special Topics–Physics Education Research, 2(2), 020101.
86.
RodrigoP.KelemenL.PalimaD.AlonzoC.OrmosP.GlückstadJ. (2009). Optical microassembly platform for constructing reconfigurable microenvironments for biomedical studies. Optics Express, 17, 6578–6583.
87.
RuhD. B. T.RohrbachA. (2011). Fast parallel interferometric 3D tracking of numerous optically trapped particles and their hydrodynamic interaction. Optics Express, 19, 21627–21642.
88.
SankaranarayananG.WeghorstS.SannerM.GilletA.OlsonA. (2003). Role of haptics in teaching structural molecular biology. In Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (pp. 363–366). New York, NY: IEEE.
89.
SchmidA.YechangunjaR.ThalhammerS.SrinivasanM. A. (2012). Human-operated 3D micro-manipulator with haptic feedback. In IEEE Haptics Symposium (pp. 517–522). New York, NY: IEEE.
90.
ShirinovA.KamenikJ.FatikowS. (2004). Haptic interface for a nanohandling robot. Assembly Automation, 24(1), 78–87.
91.
SittiM.HashimotoH. (1998a). Macro to nano tele-manipulation through nanoelectromechanical systems. In IEEE Industrial Electronics Society (pp. 98–103). New York, NY: IEEE.
92.
SittiM.HashimotoH. (1998b). Tele-nanorobotics using atomic force microscope. In IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1739–1746). New York, NY: IEEE.
93.
SittiM.HashimotoH. (2003). Teleoperated touch feedback from the surfaces at the nanoscale: Modeling and experiments. IEEE/ASME Transactions on Mechatronics, 8, 287-298.
94.
SourinaO.TorresJ.WangJ. (2008). Visual haptic-based biomolecular docking. In International Conference on Cyberworlds (pp. 240–247). Los Alamitos, CA: IEEE Computer Society.
95.
SteinfeldA.FongT.KaberD.LewisM.ScholtzJ.SchultzA.GoodrichM. (2006). Common metrics for human-robot interaction. In ACM SIGCHI/SIGART Conference on Human–Robot Interaction (pp. 33–40). New York, NY: ACM.
96.
SubasiE.BasdoganC. (2008). A new haptic interaction and visualization approach for rigid molecular docking in virtual environments. Presence: Teleoperators and Virtual Environments, 17(1), 73–90.
97.
SugiuraT.NakaoM.SatoT.MinatoK. (2008). Nanomanipulation and nanotechnology for future diagnostics. Studies in Health Technology and Informatics, 134, 135–142.
98.
TamadazteB.Le Fort-PiatN.MarchandE. (2011). A direct visual servoing scheme for automatic nanopositioning. IEEE/ASME Transactions on Mechatronics, 17(4), 1–10.
99.
TaylorR.IIChenJ.OkimotoS.Llopis-ArtimeN.ChiV. L.BrooksF. PJr.. . . SuperfineR. (1997). Pearls found on the way to the ideal interface for scanned-probe microscopes. In Conference on Visualization (pp. 467–470). New York, NY: IEEE.
100.
van WestE.YamamotoA.HiguchiT. (2007). The concept of “Haptic Tweezer”, a non-contact object handling system using levitation techniques and haptics. Mechatronics, 17, 345–356.
101.
VarolA.GunevI.BasdoganC. (2006). A virtual reality toolkit for path planning and manipulation at nano-scale. In Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (pp. 485–489). New York, NY: IEEE.
102.
VentureG.HaliyoD. S.MicaelliA.RégnierS. (2006). Force-feedback micromanipulation with inconditionnally stable coupling. International Journal of Micromechatronics, 3, 307–327.
103.
VijayasaiA. P.SivakumarG.MulsowM.LacoutureS.HolnessA.DallasT. E. (2010). Haptic controlled three-axis MEMS gripper system. Review of Scientific Instruments, 81, 105114.
104.
VoglW.MaB. K.-L.SittiM. (2006). Augmented reality user interface for an atomic force microscope-based nanorobotic system. IEEE Transactions on Nanotechnology, 5, 397–406.
105.
WollacottA. M.MerzK. M.Jr. (2007). Haptic applications for molecular structure manipulation. Journal of Molecular Graphics and Modelling, 25, 801–805.
106.
XieH.RégnierS. (2009). Three-dimensional automated micromanipulation using a nanotip gripper with multifeedback. Journal of Micromechanics and Microengineering, 19.
107.
XieH.RégnierS. (2011). Development of a flexible robotic system for multiscale applications of micro/nanoscale manipulation and assembly. In IEEE/ASME Transaction on Mechatronics, 16, 266–276.
108.
ZhangH.LiuK.-K. (2008). Optical tweezers for single cells. Journal of the Royal Society Interface, 5, 671–690.
109.
ZhangJ.LiG.XiN. (2005). Modeling and control of active end effector for the AFM based nano robotic manipulators. In IEEE International Conference on Robotics and Automation (pp. 163–168). New York, NY: IEEE.
