Methods for volume inference of non-medical objects from images: A short review

Abstract

Nowadays, the object’s volume is essential for monitoring any scene. Technological equipment is evolving, and mobile devices and other devices embed high-resolution cameras. The high-resolution cameras open a window for different research studies, where the volume measurement is vital for different areas. This study aims to identify image processing techniques for measuring the object’s volume. Thus, a systematic review was performed with a Natural Language Processing (NLP)-based framework for identifying studies between 2010 and 2023 related to the measurement of object volume. As a result of this search, this paper reviewed and analyzed 25 studies, verifying that different computer vision methods accurately handle object recognition. Additionally, an evaluation of the databases presented by the studies above is performed to consider further the design of a new approach to infer the volume of objects from an image.

Keywords

Volume measurement image processing objects systematic review

Get full access to this article

View all access options for this article.

References

Adnan and

Akbar, An analytical study of information extraction from unstructured and multidimensional big data, Journal of Big Data 6(1) (2019), 1–38.

Alcantarilla,

Nuevo and

Bartoli, Fast explicit diffusion for accelerated features in nonlinear scale spaces, in: Procedings of the British Machine Vision Conference 2013, British Machine Vision Association, Bristol, 2013, pp. 13.1–13.11. doi:10.5244/C.27.13.

Bándi,

R.-B.

Tunyogi,

Szabó,

Farkas and

Sulyok, Image-based volume estimation using stereo vision, in: 2020 IEEE 18th International Symposium on Intelligent Systems and Informatics (SISY), 2020, pp. 000055–000060. doi:10.1109/SISY50555.2020.9217089.

Bellazzi

et al., Virtual reality for assessing visual quality and lighting perception: A systematic review, Building and Environment (2021), 108674.

Börold,

Teucke,

Rust and

Freitag, Recognition of car parts in automotive supply chains by combining synthetically generated training data with classical and deep learning based image processing, Procedia CIRP 93 (2020), 377–382. doi:10.1016/j.procir.2020.03.142.

Calli,

Singh,

Walsman,

Srinivasa,

Abbeel and

A.M.

Dollar, The YCB object and model set: Towards common benchmarks for manipulation research, in: 2015 International Conference on Advanced Robotics (ICAR), IEEE, Istanbul, Turkey, 2015, pp. 510–517. doi:10.1109/ICAR.2015.7251504.

Calli,

Walsman,

Singh,

Srinivasa,

Abbeel and

A.M.

Dollar, Benchmarking in manipulation research: Using the Yale–CMU–Berkeley object and model set, IEEE Robot. Automat. Mag. 22(3) (2015), 36–52. doi:10.1109/MRA.2015.2448951.

D.B.

Camarillo,

T.M.

Krummel and

J.K.

Salisbury Jr, Robotic technology in surgery: Past, present, and future, The American Journal of Surgery 188(4) (2004), 2–15. doi:10.1016/j.amjsurg.2004.08.025.

N.D.F.

Campbell,

Vogiatzis,

Hernandez and

Cipolla, Automatic 3D object segmentation in multiple views using volumetric graph-cuts, Image and Vision Computing 28(1) (2010), 14–25. Elsevier, Radarweg 29, 1043 NX Amsterdam, Netherlands. doi:10.1016/j.imavis.2008.09.005.

10.

H.-C.

Chen,

Jia,

Li,

Y.-N.

Sun and

Sun, 3D/2D model-to-image registration for quantitative dietary assessment, in: 2012 38th Annual Northeast Bioengineering Conference (NEBEC), IEEE, 345 E 47th st, New York, NY 10017 USA, 2012, pp. 95+.

11.

Ciocca,

Napoletano and

Schettini, Food recognition: A new dataset, experiments, and results, IEEE Journal of Biomedical and Health Informatics 21(3) (2017), 588–598. IEEE-Inst Electrical Electronics Engineers Inc, 445 Hoes Lane, Piscataway, NJ 08855-4141 USA. doi:10.1109/JBHI.2016.2636441.

12.

Cordts

et al., The cityscapes dataset for semantic urban scene understanding, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 345 E 47th st, New York, NY 10017 USA. 2016, pp. 3213–3223. doi:10.1109/CVPR.2016.350.

13.

Dalai,

Dalai and

K.K.

Senapati, An accurate volume estimation on single view object images by deep learning based depth map analysis and 3D reconstruction, Multimed Tools Appl 82(18) (2023), 28235–28258. doi:10.1007/s11042-023-14615-7.

14.

Dalai and

K.K.

Senapati, A heuristic grid area based segmentation approach for weight estimation of an object from image, in: 2018 4th International Conference for Convergence in Technology (I2CT), IEEE, 345 E 47th st, New York, NY 10017 USA, 2018.

15.

B.L.

Daugherty

et al., Novel technologies for assessing dietary intake: Evaluating the usability of a mobile telephone food record among adults and adolescents, Journal of Medical Internet Research 14(2) (2012), Jmir Publications, inc, 130 Queens Quay E, Ste 1102, Toronto, On M5A 0P6, Canada. doi:10.2196/jmir.1967.

16.

Fang,

Liu,

Zhu,

E.J.

Delp and

C.J.

Boushey, Single-view food portion estimation based on geometric models, in: IEEE International Symposium on Multimedia-ISM, IEEE, 345 E 47th st, New York, NY 10017 USA, 2015, pp. 385–390. doi:10.1109/ISM.2015.67.

17.

Ferreira

et al., A systematic investigation of models for color image processing in wound size estimation, Computers 10(4) (2021), 43. doi:10.3390/computers10040043.

18.

Ferreira,

I.M.

Pires,

Ponciano,

Costa and

N.M.

Garcia, Approach for the wound area measurement with mobile devices, in: 2021 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), IEEE, Toronto, ON, Canada, 2021, pp. 1–4. doi:10.1109/IEMTRONICS52119.2021.9422661.

19.

Gao,

F.P.-W.

Lo and

Lo, Food volume estimation for quantifying dietary intake with a wearable camera, in: 2018 IEEE 15th International Conference on Biomedical and Health Informatics (BHI) and the Wearable and Implantable Body Sensor Networks (BSN), International Conference on Wearable and Implantable Body Sensor Networks, IEEE, 345 E 47th st, New York, NY 10017 USA, 2018, pp. 110–113.

20.

Gao,

Tan,

Ma,

Wang and

Tang, MUSEFood: Multi-sensor-based food volume estimation on smartphones, in: 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), 2019, pp. 899–906. doi:10.1109/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00182.

21.

N.M.

Garcia and

J.J.P.C.

Rodrigues (eds), Ambient Assisted Living, CRC Press, 2015. doi:10.1201/b18520.

22.

Geiger,

Lenz and

Urtasun, Are we ready for autonomous driving? The KITTI vision benchmark suite, in: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Providence, RI, 2012, pp. 3354–3361. doi:10.1109/CVPR.2012.6248074.

23.

Gomes and

L.A.

da Silva Cruz, Pseudo-sequence light field image scalable encoding with improved random access, in: 2019 8th European Workshop on Visual Information Processing (EUVIP), IEEE, 2019, pp. 16–21. doi:10.1109/EUVIP47703.2019.8946268.

24.

Grum and

A.G.

Bors, 3D modeling of multiple-object scenes from sets of images, Pattern Recognition 47(1) (2014), 326–343. Si. Elsevier Sci Ltd, The Boulevard, Langford Lane, Kidlington, Oxford Ox5 1GB, Oxon, England. doi:10.1016/j.patcog.2013.04.020.

25.

Grum and

A.G.

Bors, Pattern Recognition 47(1) (2014), 326–343. Si. Elsevier Sci Ltd, The Boulevard, Langford Lane, Kidlington, Oxford Ox5 1GB, Oxon, England. doi:10.1016/j.patcog.2013.04.020.

26.

Hadi,

Phon-Amnuaisuk and

S.-J.

Tan, Semantic instance segmentation in a 3D traffic scene reconstruction task, in: 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), 2020, pp. 186–191. doi:10.23919/SICE48898.2020.9240300.

27.

Hotfilter,

Kempf,

Becker,

Reinhardt and I.

Baili

, Embedded image processing the European way: A new platform for the future automotive market, in: 2020 IEEE 6th World Forum on Internet of Things (WF-IoT), IEEE, 2020, pp. 1–6.

28.

Ischuk,

Dolgov and

Tyapkin, Verification of a mathematical model for constructing thermal tomograms based on the reduction of a set of different-time visible and infrared images, in: 2020 International Conference on Information Technology and Nanotechnology (ITNT), IEEE, Samara, Russia, 2020, pp. 1–5. doi:10.1109/ITNT49337.2020.9253218.

29.

Jeon and

Y.S.

Heo, Efficient multi-scale stereo-matching network using adaptive cost volume filtering, Sensors 22(15) (2022), 5500. doi:10.3390/s22155500.

30.

R.E.

Jerome,

S.K.

Singh and

Dwivedi, Process analytical technology for bakery industry: A review, Journal of Food Process Engineering 42(5) (2019), e13143. doi:10.1111/jfpe.13143.

31.

Jia,

Yue, John ,

Fernstrom,

Zhang,

Yang and

Sun, 3D localization of circular feature in 2D image and application to food volume estimation, in: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE Engineering in Medicine and Biology Society Conference Proceedings, IEEE, 345 E 47th st, New York, NY 10017 USA, 2012, pp. 4545–4548.

32.

Kalpitha and

Murali, 3D recognition & automated access control reconstruction from a single image, in: Materials Today: Proceedings, 2021. doi:10.1016/j.matpr.2021.04.012.

33.

Kawano and

Yanai, Automatic expansion of a food image dataset leveraging existing categories with domain adaptation, in: Computer Vision – ECCV 2014 Workshops, Pt III, Lecture Notes in Computer Science, Vol. 8927, Springer-Verlag Berlin, Heidelberger Platz 3, D-14197 Berlin, Germany, 2015, pp. 3–17. doi:10.1007/978-3-319-16199-0_1.

34.

Küçükyildiz, Image processing based package volume measurement system using kinect sensor, Sigma J Eng Nat Sci – Sigma Müh Fen Bil Derg (2022). doi:10.14744/sigma.2022.00003.

35.

Litjens

et al., A survey on deep learning in medical image analysis, Medical Image Analysis 42 (2017), 60–88. doi:10.1016/j.media.2017.07.005.

36.

F.P.-W.

Lo,

Sun,

Qiu and

Lo, Food volume estimation based on deep learning view synthesis from a single depth map, Nutrients 10(12) (2018). MDPI, St Alban-Anlage 66, Ch-4052 Basel, Switzerland. doi:10.3390/nu10122005.

37.

F.P.W.

Lo,

Sun,

Qiu and

Lo, Image-based food classification and volume estimation for dietary assessment: A review, IEEE J. Biomed. Health Inform. 24(7) (2020), 1926–1939. doi:10.1109/JBHI.2020.2987943.

38.

Luo,

Ma,

Tang,

Wang and

Xiong, 3D-SSD: Learning hierarchical features from RGB-D images for amodal 3D object detection, in: Neurocomputing, Vol. 378, Elsevier, Radarweg 29, 1043 NX Amsterdam, Netherlands, 2020, pp. 364–374. doi:10.1016/j.neucom.2019.10.025.

39.

Mayer

et al., A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4040–4048. doi:10.1109/CVPR.2016.438.

40.

Menze and

Geiger, Object scene flow for autonomous vehicles, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Boston, MA, USA, 2015, pp. 3061–3070. doi:10.1109/CVPR.2015.7298925.

41.

A.S.

Mian,

Bennamoun and

Owens, Three-dimensional model-based object recognition and segmentation in cluttered scenes, IEEE Transactions on Pattern Analysis and Machine Intelligence 28(10) (2006), 1584–1601. doi:10.1109/TPAMI.2006.213.

42.

A.S.

Mian,

Bennamoun and

R.A.

Owens, A novel representation and feature matching algorithm for automatic pairwise registration of range images, International Journal of Computer Vision 66(1) (2006), 19–40. doi:10.1007/s11263-005-3221-0.

43.

Mittal,

Srivastava and

J.P.

Jayanth, A survey of deep learning techniques for underwater image classification, IEEE Transactions on Neural Networks and Learning Systems (2022).

44.

P.S.

Mittapalli and

G.B.

Kande, Segmentation of optic disk and optic cup from digital fundus images for the assessment of glaucoma, Biomedical Signal Processing and Control 24 (2016), 34–46. doi:10.1016/j.bspc.2015.09.003.

45.

O.Y.

Morozova, Analysis of typical electric power facilities requiring remote monitoring, in: 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), IEEE, St. Petersburg and Moscow, Russia, 2020, pp. 1255–1257. doi:10.1109/EIConRus49466.2020.9039389.

46.

N. C. for H. Statistics (US), Plan and operation of the third National Health and Nutrition Examination Survey, 1988–94, US Department of Health and Human Services, Public Health Service, Centers, …, 1994.

47.

P.A.

Neves

et al., Thought on food: A systematic review of current approaches and challenges for food intake detection, Sensors 22(17) (2022), 6443. doi:10.3390/s22176443.

48.

Paetz and

Preusser, Segmentation of stochastic images with a stochastic random Walker method, IEEE Transactions on Image Processing 21(5) (2012), 2424–2433. IEEE-Inst Electrical Electronics Engineers Inc, 445 Hoes Lane, Piscataway, NJ 08855-4141 USA. doi:10.1109/TIP.2012.2187531.

49.

A.S.

Parihar,

Gupta,

Sikka and

Kaur, Dimensional analysis of objects in a 2d image, in: 2017 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), International Conference on Computing Communication and Network Technologies, IEEE, 345 E 47th st, New York, NY 10017 USA, 2017.

50.

Parker

et al., National Center for Health Statistics data presentation standards for proportions, 2017.

51.

Pintus,

T.G.

Dulecha,

Ciortan,

Gobbetti and

Giachetti, State-of-the-art in multi-light image collections for surface visualization and analysis, in: Computer Graphics Forum, Wiley Online Library, 2019, pp. 909–934.

52.

I.M.

Pires

et al., Pattern recognition techniques for the identification of activities of daily living using a mobile device accelerometer, Electronics 9(3) (2020), 509. doi:10.3390/electronics9030509.

53.

I.M.

Pires and

N.M.

Garcia, Wound area assessment using mobile application, Biodevices (2015), 271–282.

54.

Poply and

J.A.A.

Jothi, Refined image segmentation for calorie estimation of multiple-dish food items, in: 2021 IEEE International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), IEEE, 345 E 47th st, New York, NY 10017 USA, 2021, pp. 682–687.

55.

et al., Review of multi-view 3D object recognition methods based on deep learning, Displays 69 (2021), 102053. doi:10.1016/j.displa.2021.102053.

56.

Ravikumar and

Arulmozhi, Digital image processing – a quick review, International Journal of Intelligent Computing and Technology (IJICT) 2(2) (2019), 11–19.

57.

Reinhard,

Heidrich,

Debevec,

Pattanaik,

Ward and

Myszkowski, High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting, Morgan Kaufmann, 2010.

58.

C.M.

Sabliov,

Boldor,

K.M.

Keener and

B.E.

Farkas, Image processing method to determine surface area and volume of axi-symmetric agricultural products, Int. J. of Food Properties 5(3) (2002), 641–653. doi:10.1081/JFP-120015498.

59.

Shabairou,

Cohen,

Wagner,

Malka and

Zalevsky, Color image identification and reconstruction using artificial neural networks on multimode fiber images: Towards an all-optical design, Optics Letters 43(22) (2018), 5603–5606. doi:10.1364/OL.43.005603.

60.

Shah and

Du, Spatial-aware collaboration – competition preserving graph embedding for hyperspectral image classification, IEEE Geoscience and Remote Sensing Letters 19 (2022), 1–5. doi:10.1109/LGRS.2021.3074328.

61.

Shi,

Huang,

Liu,

Li and

Wang, Spatial-spectral multiple manifold discriminant analysis for dimensionality reduction of hyperspectral imagery, Remote Sensing 11(20) (2019), 20. doi:10.3390/rs11202414.

62.

J.P.

Shim,

Warkentin,

J.F.

Courtney,

D.J.

Power,

Sharda and

Carlsson, Past, present, and future of decision support technology, Decision support systems 33(2) (2002), 111–126.

63.

Silberman,

Hoiem,

Kohli and

Fergus, Indoor segmentation and support inference from RGBD images, in: Computer Vision – ECCV 2012,

Fitzgibbon,

Lazebnik,

Perona,

Sato and

Schmid, eds, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2012, pp. 746–760. doi:10.1007/978-3-642-33715-4_54.

64.

R.D.

Singh,

Mittal and

R.K.

Bhatia, 3D convolutional neural network for object recognition: A review, Multimedia Tools and Applications 78(12) (2019), 15951–15995. doi:10.1007/s11042-018-6912-6.

65.

Song,

S.P.

Lichtenberg and

Xiao, SUN RGB-D: A RGB-D scene understanding benchmark suite, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 567–576. doi:10.1109/CVPR.2015.7298655.

66.

Suzuki,

Futatsuishi,

Yokoyama and

Amaki, Point cloud processing method for food volume estimation based on dish space, in: 42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society: Enabling Innovative Technologies for Global Healthcare EMBC’20, IEEE Engineering in Medicine and Biology Society Conference Proceedings, IEEE, 345 E 47th st, New York, NY 10017 USA, 2020, pp. 5665–5668.

67.

Tombari,

Salti and

Di Stefano, Performance evaluation of 3D keypoint detectors, International Journal of Computer Vision 102(1) (2013), 198–220. doi:10.1007/s11263-012-0545-4.

68.

V.-I.

Tomescu, FoRConvD: An approach for food recognition on mobile devices using convolutional neural networks and depth maps, in: 2020 IEEE 14th International Symposium on Applied Computational Intelligence and Informatics (SACI 2020), IEEE, 345 E 47th st, New York, NY 10017 USA, 2020, pp. 129–134.

69.

Udendhran,

Balamurugan,

Suresh and

Varatharajan, Enhancing image processing architecture using deep learning for embedded vision systems, Microprocessors and Microsystems 76 (2020), 103094. doi:10.1016/j.micpro.2020.103094.

70.

T.L.

van den Heuvel,

de Bruijn,

C.L.

de Korte and

van Ginneken, Automated measurement of fetal head circumference using 2D ultrasound images, PloS one 13(8) (2018), e0200412.

71.

Varma,

Subramanian,

Namboodiri,

Chandraker and

C.V.

Jawahar, IDD: A dataset for exploring problems of autonomous navigation in unconstrained environments, in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE Winter Conference on Applications of Computer Vision, IEEE, 345 E 47th st, New York, NY 10017 USA, 2019, pp. 1743–1751. doi:10.1109/WACV.2019.00190.

72.

Wang,

Mo,

Zhang and

Miao, A deep learning method for bearing fault diagnosis based on time-frequency image, IEEE Access 7 (2019), 42373–42383. doi:10.1109/ACCESS.2019.2907131.

73.

Wang,

Pang and

Lin, Monocular 3D object detection with depth from motion, arXiv, 1 de março de 2023. Acedido: 24 de outubro de 2023. [Em linha]. Disponível em, http://arxiv.org/abs/2207.12988.

74.

Wittstruck,

Kuehling,

Trautz,

Kohlbrecher and

Jarmer, UAV-based RGB imagery for Hokkaido pumpkin (Cucurbita max.) detection and yield estimation, Sensors 21(1) (2021). MDPI, St Alban-Anlage 66, CH-4052 Basel, Switzerland. doi:10.3390/s21010118.

75.

et al., 3D ShapeNets: A deep representation for volumetric shapes, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Boston, MA, USA, 2015, pp. 1912–1920. doi:10.1109/CVPR.2015.7298801.

76.

Yang,

Zhang,

Xiao and

Cao, TOLDI: An effective and robust approach for 3D local shape description, in: Pattern Recognition, Vol. 65, Elsevier Sci Ltd, The Boulevard, Langford Lane, Kidlington, Oxford, GB, Oxon, England, 2017, pp. 175–187. doi:10.1016/j.patcog.2016.11.019.

77.

Yang

et al., Human-mimetic estimation of food volume from a single-view RGB image using an AI system, Electronics 10(13) (2021), 1556. doi:10.3390/electronics10131556.

78.

R.D.

Yogaswara,

E.M.

Yuniarno and

A.D.

Wibawa, Instance-aware semantic segmentation for food calorie estimation using mask R-CNN, in: 2019 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2019, pp. 416–421. doi:10.1109/ISITIA.2019.8937129.

79.

Zdravevski

et al., Automation in systematic, scoping and rapid reviews by an NLP toolkit: A case study in enhanced living environments, in: Enhanced Living Environments,

Ganchev,

N.M.

Garcia,

Dobre,

C.X.

Mavromoustakis and

Goleva, eds, Lecture Notes in Computer Science, Vol. 11369, Springer International Publishing, Cham, 2019, pp. 1–18. doi:10.1007/978-3-030-10752-9_1.

80.

Zendel,

Honauer,

Murschitz,

Steininger and

G.F.

Dominguez, WildDash – creating hazard-aware benchmarks, in: Computer Vision – ECCV 2018, PT VI, Lecture Notes in Computer Science, Vol. 11210, Springer International Publishing AG, Gewerbestrasse 11, Cham, CH-6330, Switzerland, 2018, pp. 407–421. doi:10.1007/978-3-030-01231-1_25.