The integration of visual and inertial sensors for human motion tracking has attracted significant attention recently, due to its robust performance and wide potential application. This paper introduces a real-time hybrid solution to articulated 3D arm motion tracking for home-based rehabilitation by combining visual and inertial sensors. Data fusion is a key issue in this hybrid system and two different data fusion methods are proposed. The first is a deterministic method based on arm structure and geometry information, which is suitable for simple rehabilitation motions. The second is a probabilistic method based on an Extended Kalman Filter (EKF) in which data from two sensors is fused in a predict-correct manner in order to deal with sensor noise and model inaccuracy. Experimental results are presented and compared with commercial marker-based systems, CODA and Qualysis. They show good performance for the proposed solution.
Aggarwal, J.K. and Cai, Q. (1999). Human motion analysis: a review. Journal of Computer Vision and Image Understanding,73(3): 428—440.
2.
Alenya, G., Martinez, E., and Torras, C. (2003). Fusing visual contour tracking with inertial sensing to recover robot ego-motion. Workshop on Integration of Vision and Inertial Sensors Proceedings of International Conference on Advanced Robotics, Coimbra, Portugal.
3.
Bachmann, E.R. (1999). Orientation tracking for humans and robots using inertial sensors. Proceedings of International Symposium on Computational Intelligence in Robotics and Automation, Monterey, CA, USA.
4.
Bar-Shalom, Y. and Fortmann, T.E. (1988). Tracking and Data Association. Academic Press.
5.
Bradski, G.R. (1998). Computer vision face tracking for use in a perceptual user interface. Intel Technology Journal, Q2:15.
6.
Bussmann, H. (2000). Ambulatory monitoring of mobility-related activities in rehabilitation medicine. PhD Thesis, Erasmus UniversityRotterdam. Delft, Eburon.
7.
Chai, L., Hoff, W.A., and Vincent, T. (2002). 3D motion and structure estimation using inertial sensors and computer vision for augmented reality. Tele-operators and Virtual Environments,11(5): 474—492, MIT Press.
Chen, J. and Pinz, A. (2004). Structure and motion by fusion of inertial and vision-based tracking. Proceedings of the 28th OAGM/AAPR Conference, Vol. 179 of Schriftenreihe, 55— 62. OCG.
10.
Chen, Z. and Lee, H.J. (1992). Knowledge-guided visual perception of 3D human gait from a single image sequence. IEEE Transactions On Systems, Man, andCybernetics, 22(2): 336—342.
Deutscher, J., Blake, A., and Reid, I. (2000). Articulated body motion capture by annealed particle filtering. IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 126—133.
13.
Foxlin, E. (2002). Generalized architecture for simultaneous localization, auto-calibration, and map-building. IEEE/RSJ International Conference on Intelligent Robots & Systems, Lausanne , Switzerland.
14.
Foxlin, E. et al. (2004). FlightTracker: a novel optical/inertial tracker for cockpit enhanced vision. IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004), Washington, DC, USA.
15.
Gavrila, D.M. (1999). The visual analysis of human movement: a survey . Journal of Computer Vision and Image Understanding,73(1): 82—98.
16.
Gavrila, D.M. and Davis, L.S. (1995). 3-d model-based tracking of human upper body movement: A multi-view approach. In Int. Symposium on Computer Vision, 253— 258.
17.
GENTLE/S . URL: http://www.gentle.rdg.ac.uk/
18.
Goncalves, L. et al. (1995). Monocular tracking of the human arm in 3D. Proceedings of International Conference on Computer Vision, Cambridge, MA, USA, June, 764— 770.
19.
Huster, A. (2003). Relative Position Sensing by Fusing Monocular Vision and Inertial Rate Sensors. PhD thesis. Department of Electrical Engineering, Stanford University, USA.
20.
Huster, A. and Rock, S.M. (2001). Relative position estimation for manipulation tasks by fusing vision and inertial measurements. In Oceans 2001 Conference, Honolulu, USA, November, Vol. 2, 1025—1031.
21.
Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception and Psychophysics, 14(2): 210—211.
22.
Ju, S.X., Black, M., and Yacoob, Y. (1996). Cardboard people: a parameterised model of articulated motion. 2nd International Conference on Automatic Face and Gesture Recognition, Killington, Vermont, USA, 38—44.
23.
Lang, P., Ribo, M., and Pinz, A. (2002). A new combination of vision-based and inertial tracking for fully mobile, wearable and real-time operation. Proceedings of 26th Workshop of the Austrian Association for Pattern Recognition (GM/AAPR), Graz, Austria, Vol.160, 141—148.
24.
Lu, C.P., Hager, G.D., and Mjolsness, E. (2000). Fast and globally convergent pose estimation from video images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(6): 610—622.
25.
Moeslund, T. and Granum, E. (2000a). Multiple cues used in model-based human motion capture. The fourth International Conference on Automatic Face and Gesture Recognition, Grenoble, France.
26.
Moeslund, T. and Granum, E. (2001). A survey of computer vision-based human motion capture. Journal of Computer Vision and Image Understanding,81(3): 231—268.
27.
Moeslund, T.B. and Granum, E. (2000b). 3D human pose estimation using 2D-data and an alternative phase space representation. Workshop on Human Modelling, Analysis and Synthesis at CVPR, Hilton Head Island, South Carolina, USA.
28.
Qualisys. URL: http://www.qualisys.com/
29.
Shi, J. and Tomasi, C. (1994). Good features to track. IEEE Conference on Computer Vision and Pattern Recognition, Seattle, USA, 593—600.
30.
Sidenbladh, H., Black, M., and Fleet, D. (2000). Stochastic tracking of 3D human figures using 2D image motion. European Conference on Computer Vision, Dublin, Ireland, 702—718.
31.
Sminchisescu, C. (2002). Estimation Algorithms for Ambiguous Visual Models 3D Human Modeling & Motion Reconstruction in Monocular Video Sequences . PhD Thesis, Institute National Politechnique de Grenoble INRIA, France.
32.
Strelow, D. and Singh, S. (2003). Online motion estimation from image and inertial measurements. Online motion estimation from image and inertial measurements . Workshop on Integration of Vision and Inertial Sensors (INERVIS 2003), Coimbra, Portugal.
33.
Tao, Y. and Hu, H. (2004). Colour-based human motion tracking for home-based rehabilitation. Proceedings of IEEE International Conference on Systems, Man and Cybernetics, The Hague, The Netherlands, 773—781.
34.
Tetrud, J.W., Sabelman, E.E., and Yap, R. (2002). Accelerometer identification of freezing-of-gait in Parkinson's syndrome. Proceedings of the 7th International Congress on Parkinson's Disease and Movement Disorders, Miami Beach, FL, November.
35.
Tolani, D., Goswami, A., and Badler, N.I. (2000). Real-time inverse kinematics techniques for anthropomorphic limbs. Graphical Models,62(5): 353—388.
36.
Uncertainties and Error Propagation (2007). http://www.rit.edu/~upphysics/uncertainties/Uncertaintiespart1.html, Accessed on 18 January 2007.
37.
Veltink, P.H. et al. (1996). Detection of static and dynamic activities using uniaxial accelerometers. IEEE Trans Rehabil Eng, 4(4): 375—385.
38.
Vezhnevets, V., Sazonov, V., and Andreeva, A. (2003). A survey on pixel-based skin colour detection techniques. Graphicon-2003, Moscow, Russia.
39.
Wang, L., Hu, W., and Tan, T. (2003). Recent developments in human motion analysis. Pattern Recognition,36(3): 585— 601.
40.
Welch, G.F. (1996). SCAAT: Incremental Tracking with Incomplete Information . PhD thesis, University of North Carolina at Chapel Hill , USA.
41.
Wren, C., Azarbayejani, A., Darrel, T., and Pentland, A. (1995). Pfinder: Real-time tracking of the human body . Proceedings of SPIE, Bellingham, WA, USA.
42.
Xsens Motion Technologies. (2007). URL:http://www.xsens.com/. Accessed on 12 January 2007.
43.
You, S. and Neumann, U. (2001). Fusion of vision and gyro tracking for robust augmented reality registration. IEEE Virtual Reality, Japan
44.
You, S., Neumann, U. and Azuma, R. (1999). Hybrid inertial and vision tracking for augmented reality registration. Proceedings of IEEE VR '99 , Houston, TX, 260—267.
45.
Zhou, H. and Hu, H. (2005). Inertial motion tracking of human arm movements in home-based rehabilitation. Proceedings of IEEE International Conference on Mechatronics and Automation, Sheraton Fallsview Hotel, Niagara Falls, Ontario, Canada.
