Camera-based fall detection using real-world versus simulated data: How far are we from the solution?

Abstract

Several new algorithms for camera-based fall detection have been proposed, with the aim of reliably alerting caregivers about older persons’ falls at home. These algorithms have been evaluated almost exclusively using brief segments of video data captured in artificial environments under optimal conditions and with falls simulated by actors. By contrast, we collected real-life video data recorded over several months at seven older persons’ residences. Here, we report on our fall-detection algorithm based on the state-of-the-art, and we present an analysis of the real-life video data. The performance of our detection algorithm was compared with the performance of three previously reported algorithms that used a publicly available simulation data set. All four algorithms produced similar results when using the simulated data. However, the performance of our algorithm degraded drastically when evaluating falls in the real-life data. The false alarm rate was especially high, showing that some challenges still need to be met to make the system sufficiently robust to deploy in real-world situations. We conclude that using more realistic data sets that include longer video recordings and a broad range of activities are essential to reveal weaknesses in fall-detection algorithms.

Keywords

Fall detection video surveillance ambient assisted living evaluation

Get full access to this article

View all access options for this article.

References

[1]

Anderson,

Keller,

Skubic,

Chen and

He, Recognizing falls from silhouettes, in: 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2006, EMBS’06, 2006, pp. 6388–6391.

[2]

Auvinet,

Multon,

Saint-Arnaud,

Rousseau and

Meunier, Fall detection with multiple cameras: An occlusion-resistant method based on 3-D silhouette vertical distribution, IEEE Transactions on Information Technology in Biomedicine 15(2) (2011), 290–300.

[3]

Auvinet,

Rougier,

Meunier,

St-Arnaud and

Rousseau, Multiple cameras fall dataset, DIRO-Universite de Montreal, Tech. rep. Vol. 1350, 2010.

[4]

Charfi,

Miteran,

Dubois,

Atri and

Tourki, Definition and performance evaluation of a robust svm based fall detection solution, in: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), 2012, pp. 218–224.

[5]

J.L.

Chua,

Y.C.

Chang and

W.K.

Lim, Intelligent visual based fall detection technique for home surveillance, in: 2012 International Symposium on Computer, Consumer and Control (IS3C), 2012, pp. 183–187.

[6]

Cucchiara,

Prati and

Vezzani, An intelligent surveillance system for dangerous situation detection in home environments, Intelligenza Artificiale 1(1) (2004), 11–15.

[7]

Debard,

Karsmakers,

Deschodt,

Vlaeyen,

Dejaeger,

Milisen,

Goedemé,

Vanrumste and

Tuytelaars, Camera-based fall detection on real world data, in: Outdoor and Large-Scale Real-World Scene Analysis, Lecture Notes in Computer Science, Vol. 7474, Springer, Berlin, Heidelberg, 2012, pp. 356–375.

[8]

Dubey,

Ni and

Moulin, A depth camera based fall recognition system for the elderly, in: Image Analysis and Recognition,

Campilho and

Kamel, eds, Lecture Notes in Computer Science, Vol. 7325, Springer, Berlin, Heidelberg, 2012, pp. 106–113.

[9]

Felzenszwalb,

McAllester and

Ramanan, A discriminatively trained, multiscale, deformable part model, in: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, June 2008, 2008.

10.

[10]

Fleming and

Brayne, Inability to get up after falling, subsequent time on floor, and summoning help: Prospective cohort study in people over 90, British Medicine Journal 337(v17) (2008), 2227, 1.

11.

[11]

Foroughi,

Aski and

Pourreza, Intelligent video surveillance for monitoring fall detection of elderly in home environments, in: 11th International Conference on Computer and Information Technology, 2008, ICCIT 2008, 2008, pp. 219–224.

12.

[12]

Godbehere,

Matsukawa and

Goldberg, Visual tracking of human visitors under variable-lighting conditions for a responsive audio art installation, in: American Control Conference (ACC), 2012, 2012, pp. 4305–4312.

13.

[13]

Grest,

Frahm and

Koch, A color similarity measure for robust shadow removal in real-time, in: Vision, Modeling and Vizualization, 2003.

14.

[14]

Hartmann,

Al Machot,

Mahr and

Bobda, Camera-based system for tracking and position estimation of humans, in: 2010 Conference on Design and Architectures for Signal and Image Processing (DASIP), 2010, pp. 62–67.

15.

[15]

Htike,

Egerton and

K.Y.

Chow, A monocular view-invariant fall detection system for the elderly in assisted home environments, in: 2011 7th International Conference on Intelligent Environments (IE), 2011, pp. 40–46.

16.

[16]

D.H.

Hung and

Saito, Fall detection with two cameras based on occupied areas, in: Proc. of 18th Japan-Korea Joint Workshop on Frontier in Computer Vision, 2012, pp. 33–39.

17.

[17]

J.C.S.

Jacques and

C.R.

Jung, Background subtraction and shadow detection in grayscale video sequences, in: The XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI’05), 2005.

18.

[18]

Krekovic,

Ceric,

Dominko,

Ilijas,

Ivancic,

Skolan and

Sarlija, A method for real-time detection of human fall from video, in: 2012 Proc. of the 35th International Convention, MIPRO, 2012, pp. 1709–1712.

19.

[19]

Large,

Gan,

Basic and

Jennings, Using the timed up and go test to stratify elderly inpatients at risk of falls, Clinical Rehabilitation 20(5) (2006), 421–428.

20.

[20]

Y.-S.

Lee and

W.-Y.

Chung, Automated abnormal behavior detection for ubiquitous healthcare application in daytime and nighttime, in: 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), 2012, pp. 204–207.

21.

[21]

Y.-S.

Lee and

Lee, Multiple object tracking for fall detection in real-time surveillance system, in: 11th International Conference on Advanced Communication Technology, 2009, ICACT 2009, Vol. 03, 2009, pp. 2308–2312.

22.

[22]

Lee and

Mihailidis, An intelligent emergency response system: Preliminary development and testing of automated fall detection, Journal of Telemedicine and Telecare 11(4) (2005), 194–198.

23.

[23]

Liu and

Zuo, An improved algorithm of automatic fall detection, AASRI Procedia 1) (2012), 353–358, AASRI Conference on Computational Intelligence and Bioinformatics.

24.

[24]

Makantasis,

Protopapadakis,

Doulamis,

Grammatikopoulos and

Stentoumis, Monocular camera fall detection system exploiting 3d measures: A semi-supervised learning approach, in: Computer Vision – ECCV 2012, Workshops and Demonstrations, Lecture Notes in Computer Science, Vol. 7585, Springer, Berlin, Heidelberg, 2012, pp. 81–90.

25.

[25]

Mastorakis and

Makris, Fall detection system using kinect’s infrared sensor, Journal of Real-Time Image Processing (2012), 1–12.

26.

[26]

N.J.B.

McFarlane and

C.P.

Schofield, Segmentation and tracking of piglets in images, Machine Vision and Applications 8(3) (1995), 187–193.

27.

[27]

Miao,

Naqvi and

Chambers, Fall detection in the elderly by head tracking, in: IEEE/SP 15th Workshop on Statistical Signal Processing, 2009, SSP’09, 2009, pp. 357–360.

28.

[28]

Miao,

Rhuma,

Naqvi and

Chambers, Fall detection for the elderly in a smart room by using an enhanced one class support vector machine, in: 2011 17th International Conference on Digital Signal Processing (DSP), 2011, pp. 1–6.

29.

[29]

Miao,

Rhuma,

Naqvi,

Wang and

Chambers, A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment, IEEE Transactions on Information Technology in Biomedicine 16(6) (2012), 1274–1286.

30.

[30]

S.G.

Miaou,

Sung and

Huang, A customized human fall detection system using omni-camera images and personal information, Distributed Diagnosis and Home Healthcare (2006), 39–42.

31.

[31]

Milisen,

Detroch,

Bellens,

Braes,

Dierickx,

Smeulders,

Teughels,

Dejaeger,

Boonen and

Pelemans, Falls among community-dwelling elderly: A pilot study of prevalence, circumstances and consequences in Flanders, Tijdschr. Gerontol. Geriatr. 35(1) (2004), 15–20.

32.

[32]

Milisen,

Geeraerts and

Dejaeger, Use of a fall prevention practice guideline for community-dwelling older persons at risk for falling: A feasibility study, Gerontology 55(2) (2008), 169–178.

33.

[33]

Mubashir,

Shao and

Seed, A survey on fall detection: Principles and approaches, Neurocomputing 100 (2013), 144–152, Special issue: Behaviours in video.

34.

[34]

N.M.

Nachreiner,

M.J.

Findorff,

J.F.

Wyman and

T.C.

McCarthy, Circumstances and consequences of falls in community-dwelling older women, Journal of Women’s Health 16(10) (2007), 1437–1446.

35.

[35]

Nait-Charif and

S.J.

McKenna, Activity summarisation and fall detection in a supportive home environment, in: ICPR’04: Proc. of the Pattern Recognition, 17th International Conference on (ICPR’04), Vol. 4, IEEE Computer Society, Washington, DC, USA, 2004, pp. 323–326.

36.

[36]

Nater,

Grabner and

Van Gool, Visual abnormal event detection for prolonged independent living, in: International Mobile Health (mHealth) Workshop, 2010.

37.

[37]

Neubeck and

Van Gool, Efficient non-maximum suppression, in: 18th International Conference on Pattern Recognition, 2006, ICPR 2006, Vol. 3, IEEE, 2006, pp. 850–855.

38.

[38]

Noury,

Rumeau,

Bourke,

ÓLaighin and

Lundy, A proposal for the classification and evaluation of fall detectors, IRBM 29(6) (2008), 340–349.

39.

[39]

Ozcan,

A.K.

Mahabalagiri,

Casares and

Velipasalar, Automatic fall detection and activity classification by a wearable embedded smart camera, IEEE Journal on Emerging and Selected Topics in Circuits and Systems 3 (2013), 125–136, IEEE.

40.

[40]

S.N.

Robinovitch,

Feldman,

Yang,

Schonnop,

P.M.

Leung,

Sarraf,

Sims-Gould and

Loughin, Video capture of the circumstances of falls in elderly people residing in long-term care: An observational study, The Lancet 381(9860) (2013), 47–54.

41.

[41]

Rougier,

Meunier,

St-Arnaud and

Rousseau, Monocular 3d head tracking to detect falls of elderly people, in: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2006.

42.

[42]

Rougier,

Meunier,

St-Arnaud and

Rousseau, Fall detection from human shape and motion history using video surveillance, in: 21st International Conference on Advanced Information Networking and Applications Workshops, 2007, AINAW’07, Vol. 2, 2007, pp. 875–880.

43.

[43]

Rougier,

Meunier,

St-Arnaud and

Rousseau, Robust video surveillance for fall detection based on human shape deformation, IEEE Transactions on Circuits and Systems for Video Technology 21(5) (2011), 611–622.

44.

[44]

Rougier,

Meunier,

St-Arnaud and

Rousseau, 3D head tracking for fall detection using a single calibrated camera, Image and Vision Computing 31(3) (2013), 246–254.

45.

[45]

Samet and

Tamminen, Efficient component labeling of images of arbitrary dimension represented by linear bintrees, IEEE Transactions on Pattern Analysis and Machine Intelligence 10 (Jul. 1988), 579–586.

46.

[46]SeniorWatch, Fall detector: Case study of European IST SeniorWatch Project, Tech. rep., SeniorWatch, 2001.

47.

[47]

W.-Y.

Shieh and

J.-C.

Huang, Falling-incident detection and throughput enhancement in a multi-camera video-surveillance system, Medical Engineering & Physics 34(7) (2012), 954–963.

48.

[48]

Shoaib,

Dragon and

Ostermann, View-invariant fall detection for elderly in real home environment, in: 2010 Fourth Pacific-Rim Symposium on Image and Video Technology (PSIVT), 2010, pp. 52–57.

49.

[49]

Syngelakis and

Collomosse, A bag of features approach to ambient fall detection for domestic elder-care, in: Proc. Intl. Symp. on Ambient Technologies (AMBIENT 2011), 2011.

50.

[50]

Thome,

Miguet and

Ambellouis, A real-time, multiview fall detection system: A LHMM-based approach, IEEE Transactions on Circuits and Systems for Video Technology 18(11) (2008), 1522–1532.

51.

[51]

M.E.

Tinetti, Preventing falls in elderly persons, New England Journal of Medicine 348(1) (2003), 42–49.

52.

[52]

M.E.

Tinetti,

Lui and

Claus, Predictors and prognosis of inability to get up after falls among elderly persons, JAMA 269 (1993), 65–70.

53.

[53]

B.U.

Toreyin,

Dedeoglu and

A.E.

Cetin, HMM based falling person detection using both audio and video, in: Proc. of Computer Vision in Human-Computer Interaction, Lecture Notes in Computer Science, Vol. 3766, Springer Verlag, Berlin, 2005, pp. 211–220.

54.

[54]

Vapnik, Statistical Learning Theory, Wiley, New York, 1998.

55.

[55]

Vlaeyen,

Deschodt,

Debard,

Dejaeger,

Boonen,

Goedemé,

Vanrumste and

Milisen, Fall incidents unraveled: A series of 26 video-based real-life fall events in three frail older persons, BMC Geriatrics 13(1) (2013), 103.

56.

[56]

Willems,

Debard,

Vanrumste and

Goedemé, A video-based algorithm for elderly fall detection, in: Medical Physics and Biomedical Engineering World Congress, WC2009, 2009.

57.

[57]

Yang and

Ramanan, Articulated pose estimation with flexible mixtures-of-parts, in: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 1385–1392.

58.

[58]

Yongli,

O.S.

Yin and

P.Y.

Han, State of the art: A study on fall detection, Proc. of World Academy of Science, Engineering and Technology, no. 62, 2012.

59.

[59]

Zivkovic, Improved adaptive Gaussian mixture model for background subtraction, in: Proc. of the 17th International Conference on Pattern Recognition, ICPR 2004, IEEE, 2004, pp. 28–31.