Artificial intelligence (AI) offers the potential for the development of e-textiles that give wearers a smart and intuitive experience. An emerging challenge in intelligent materials design is hand gesture recognition textiles. Most current research focuses on number gesture recognition via smart gloves, so there is a gap in research that studies contact-less number gesture recognition textiles via computer vision. Meanwhile, there is lack of exploration on the integration of illuminating function and number gesture recognition textiles to improve interactivity by real-time visualizing detection results. In this research, a novel interactive illuminating textile with a touch-less number gesture recognition function has been designed and fabricated by using an open-source AI model. It is used in sync with a polymeric optical fiber textile with illuminative features. The textile is color-changing, controlled by the system's mid-air interactive number gesture recognition capability and has a woven stripe pattern and a double-layer weave structure with open pockets to facilitate integration of the system's components. Also described here is a novel design process that permits textile design and intelligent technology to integrate seamlessly and in synchronization, so that design in effect mediates continuously between the physical textile and the intangible technology. Moreover, this design method serves as a reference for the integration of open-source intelligent hardware and software into e-textiles for enhancement of the intuitive function and value via economy of labor.
Fernández-Caramés TM and Fraga-Lamas P. Towards the Internet of smart clothing: A review on IoT wearables and garments for creating intelligent connected e-textiles. Electronics 2018; 7: 405.
2.
Tan J, Bai Z, Ge L, et al. Design and fabrication of touch-sensitive polymeric optical fibre (POF) fabric. The Journal of the Textile Institute 2019.
3.
Hayward J. E-Textiles and Smart Clothing 2020–2030: Technologies, Markets and Players. 2020. Cambridge, UK: IDTechEx Research.
4.
MarketsAndMarkets. Gesture Recognition and Touchless Sensing Market by Technology (Touch-Based and Touchless), Product (Sanitary Equipment, Touchless Biometric), Type (Online, Offline) Industry and Geography - Global Forecast to 2025. April 2020. Northbrook, USA: MarketsandMarkets Research Private Ltd.
5.
Bhat G, Deb R and Ogras UY. OpenHealth: open-source platform for wearable health monitoring. IEEE Design & Test 2019; 36: 27–34.
6.
TsaiT-HHuangC-CZhangK-L.Design of hand gesture recognition system for human-computer interaction. Multimed Tool Appl2020;
79: 5989–6007.
7.
Mohamed ASA, Wahab MA, Suhaily S, et al. Smart mirror design powered by raspberry pi. In: Alazab M, (ed.). AICCC ‘18: Proceedings of the 2018 Artificial Intelligence and Cloud Computing Conference. Tokyo Japan Association for Computing Machinery, 21–23 December 2018, paper no. 3299819.3299840, p. 166–173, New York, NY, United States.
8.
Amir A, Taba B, Berg D, et al. A Low Power, Fully Event-Based Gesture Recognition System. In: Yanxi Liu JMR, Camillo J. Taylor, Ying Wu, (ed.). 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), 21–26 July 2017, paper no. CVPR.2017.781, p. 7243–7252.
9.
Rautaray SS. Real time hand gesture recognition system for dynamic applications. International Journal of UbiComp (IJU) 2012; 3.
10.
Gupta S, Bagga S and Sharma DK. Hand Gesture Recognition for Human Computer Interaction and Its Applications in Virtual Reality. In: Gupta D, Hassanien AE and Khanna A (eds) Advanced Computational Intelligence Techniques for Virtual Reality in Healthcare. Springer, 2020, pp.85–105.
11.
Xu Y and Dai Y. Review of hand gesture recognition study and application. Contemporary Engineering Sciences 2017; 10: 375–384.
12.
Rautaray SS and Agrawal A. Interaction with virtual game through hand gesture recognition. In: Section IoEaEEUP, (ed.). 2011 International Conference on Multimedia, Signal Processing and Communication Technologies. Aligarh, India: IEEE, 17–19 Dec 2011, p. 244–247.
13.
Zhu Y and Yuan B. Real-time hand gesture recognition with Kinect for playing racing video games. In: 2014 International Joint Conference on Neural Networks (IJCNN) (ed Networks IJCoN), Beijing, China, 6–11 July 2014, pp.3240–3246. IEEE.
14.
Cohen MW, Voldman I, Regazzoni D, et al. Hand rehabilitation via gesture recognition using leap motion controller. In: 2018 11th international conference on human system interaction (HSI) (ed Interactions CoHS), Gdansk, Poland, 4–6 July 2018, pp.404–410. Piscataway, N.J. : IEEE.
15.
Han S-H, Ahn E-J, Ryu M-H, et al. Natural hand gesture recognition with an electronic textile goniometer. Sensors Mater 2019; 31: 1387–1395.
16.
Li W-J, Hsieh C-Y, Lin L-F, et al. Hand gesture recognition for post-stroke rehabilitation using leap motion. In: Meen T-H, Lam ADK-T and SD P, (eds.). 2017 international conference on applied system innovation (ICASI). Sapporo, Japan: IEEE, 13–17 May 2017, ICASI.2017.7988433, p. 386–388.
17.
Mahmoud NM, Fouad H and Soliman AM. Smart healthcare solutions using the internet of medical things for hand gesture recognition system. Complex & intelligent systems 2021; 7: 1253–1264. DOI: 10.1007/s40747-020-00194-9.
18.
Moschetti A, Fiorini L, Esposito D, et al. Toward an Unsupervised Approach for Daily Gesture Recognition in Assisted Living Applications. IEEE sensors journal 2017; 17: 8395–8403. DOI: 10.1109/JSEN.2017.2764323.
19.
Veeralingam S, Sahatiya P, Kadu A, et al. Direct, onestep growth of NiSe2 on cellulose paper: a low-cost, flexible, and wearable with smartphone enabled multifunctional sensing platform for customized noninvasive personal healthcare monitoring. 2019; 1: 558–568.
20.
Yang D, Lim J-K and Choi Y. Early childhood education by hand gesture recognition using a smartphone based robot. In: Communication IISoRaHI, (ed.). Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on Robot and Human Interactive Communication. Edinburgh, UK: IEEE, 25–29 Aug. 2014, ROMAN.2014.6926381, p. 987–992.
21.
He S, Han M, Patel N, et al. Converting Handwritten Text to Editable Format via Gesture Recognition for Education. In: Education tsATSoCS, (ed.). SIGCSE ‘20: the 51st ACM Technical Symposium on Computer Science Education. Portland, OR, USA: Association for Computing Machinery, 11–14 March 2020, 3328778.3372678, p. 1369–1369.
22.
uerra-Casanova J, Sánchez-Ávila C, Bailador G, et al. Authentication in mobile devices through hand gesture recognition. International journal of information security 2012; 11: 65–83. DOI: 10.1007/s10207-012-0154-9.
23.
Lee U and Tanaka J. Finger identification and hand gesture recognition techniques for natural user interface. In APCHI 2013 / India HCI 2013 - Joint Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction and the 5th Indian Conference on Human Computer Interaction. Bangalore, India: Association for Computing Machinery, 24–27 September 2013, 2525194.2525296, p. 274–279.
24.
Huang Xa, WangQZangS, et al.
Tracing the motion of finger joints for gesture recognition via sewing RGO-coated fibers onto a textile glove. IEEE Sensor J2019;
19: 9504–9511.
25.
WuHGuoHSuZ, et al.
Fabric-based self-powered noncontact smart gloves for gesture recognition. J Mater Chemy A2018;
6: 20277–20288.
26.
Wen F, Sun Z, He T, et al. Machine Learning Glove Using Self – Powered Conductive Superhydrophobic Triboelectric Textile for Gesture Recognition in VR/AR Applications. Advanced science 2020; 7: 2000261(1–15). DOI: 10.1002/advs.202000261.
27.
ShukorAZMiskonMFJamaluddinMH, et al.
A new data glove approach for Malaysian sign language detection. Proc Comput Sci2015;
76: 60–67.
28.
Lu D, Yu Y and Liu H. Gesture recognition using data glove: An extreme learning machine method. In: Biomimetics ICoRa, (ed.). 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO). Qingdao, China IEEE, 3–7 Dec 2016, ROBIO.2016.7866514, p. 1349–1354.
29.
Luzhnica G, Simon J, Lex E, et al. A sliding window approach to natural hand gesture recognition using a custom data glove. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI) 2016, pp.81–90. IEEE.
30.
Rosero-Montalvo PD, Godoy-Trujillo P, Flores-Bosmediano E, et al. Sign language recognition based on intelligent glove using machine learning techniques. In: Meeting ITETC, (ed.). 2018 IEEE Third Ecuador Technical Chapters Meeting (ETCM). Cuenca, Ecuador: IEEE, 15-19 Oct 2018, ETCM.2018.8580268, p. 1–5, Piscataway, New Jersey.
31.
Kiliboz NC and Gudukbay U. A hand gesture recognition technique for human -computer interaction. Journal of visual communication and image representation 2015; 28: 97–104. DOI: 10.1016/j.jvcir.2015.01.015.
32.
Huang Xa, Wang Q, Zang S, et al. Tracing the Motion of Finger Joints for Gesture Recognition via Sewing RGO-Coated Fibers Onto a Textile Glove. IEEE sensors journal 2019; 19: 9504–9511. DOI: 10.1109/JSEN.2019.2924797.
33.
Bai VZ, Tan J, Chen A, et al. Enhancing the wearablity and accessibility of illuminated POF garment. International journal of clothing science and technology 2019; 32: 218–230. DOI: 10.1108/IJCST-03-2019-0028.
34.
Bai Z-q, Tan J, Johnston C, et al. Enhancing The Functionality of Traditional Interior Textiles with Integration of Optical Fibers. Research Journal of Textile and Apparel 2012; 16: 31–38. DOI: 10.1108/RJTA-16-04-2012-B004.
35.
Bai Z-Q, Tan J, Johnston CF, et al. Connexion: Development of interactive soft furnishings with polymeric optical fibre (POF) textiles. International journal of clothing science and technology 2015; 27: 870–894. DOI: 10.1108/IJCST-05-2014-0058.
36.
Chen A, Tan J, Henry P, et al. The design and development of an illuminated polymeric optical fibre (POF) knitted garment. The Journal of The Textile Institute 2020; 111: 745–755. DOI: 10.1080/00405000.2019.1661937.
37.
Ge L and Tan J. Development of three-dimensional effects and stretch for polymeric optical fiber (POF) textiles with double weave structure containing spandex. The Journal of The Textile Institute 2021; 112: 398–405. DOI: 10.1080/00405000.2020.1761679.
38.
Tan J. Photonic Fabrics for Fashion and Interior. Singapore: Singapore: Springer Singapore, 2015, p. 1005–1033.
39.
Hillesund T. Digital reading spaces: How expert readers handle books, the Web and electronic paper abstract HTML. First Monday. Bridgman MI USA: University of Illinois at Chicago Library, 2010.
40.
Kurtenbach, G. and Hulteen, E.A. (1990) Gestures in Human-Computer Communication. In: Laurel, B., Ed., The Art of Human-Computer Interface Design, Addison-Wesley Publishing Company, Inc., New York.
41.
Kendon A. How gestures can become like words. In: Poyatos F (ed) Cross-cultural perspectives in nonverbal communication. Chicago, Dec 1983, Hogrefe & Huber Publishers, pp.131–14, 1988.
Orth M, Post R and Cooper E. Fabric computing interfaces. In: Systems CHFiC, (ed.). CHI 98 conference summary on Human factors in computing systems. Los Angeles, California: Association for Computing Machinery, 18–23 April 1998, paper no. 286498.286800, p. 331–332. New York, NY, United States.
44.
Wu T, Fukuhara S, Gillian N, et al. ZebraSense: A Double-sided Textile Touch Sensor for Smart Clothing. In: Machinery AfC, (ed.). the 33rd Annual ACM Symposium on User Interface Software and Technology. Virtual Event USA: Association for Computing Machinery, 20–23 Octorber 2020, paper no. 3379337.3415886, p. 662–674, New York, NY, United States.
45.
Karrer T, Wittenhagen M, Lichtschlag L, et al. Pinstripe: eyes-free continuous input on interactive clothing. In: Systems CoHFiC, (ed.). CHI ‘11: CHI Conference on Human Factors in Computing Systems. Vancouver BC Canada: Association for Computing Machinery, 7–12 May 2011, paper no. 1978942.1979137, p. 1313–1322, New York, NY, United States.
46.
Parzer P, Sharma A, Vogl A, et al. SmartSleeve: real-time sensing of surface and deformation gestures on flexible, interactive textiles, using a hybrid gesture detection pipeline. In: Technology ttAASoUISa, (ed.). UIST '17: The 30th Annual ACM Symposium on User Interface Software and Technology Québec City QC Canada: Association for Computing Machinery, 22–25 Octorber 2017, paper no. 3379337.3415886, p. 565–577, New York, NY, United States.
47.
Schneegass S and Voit A. GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches. In: Computing TAIJCoPaU, (ed.). UbiComp ‘16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. Heidelberg Germany: Association for Computing Machinery, 12–16 September 2016, paper no. 2971763.2971797, p. 108–115, New York, NY, United States.
48.
Olwal A, Moeller J, Priest-Dorman G, et al. I/O Braid: Scalable Touch-Sensitive Lighted Cords Using Spiraling, Repeating Sensing Textiles and Fiber Optics. In: Technology SoUISa, (ed.). UIST ‘18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. Berlin Germany Association for Computing Machinery, 14–17 Octorber 2018, paper no. 3242587.3242638, p. 485–497, New York, NY, United States.
49.
Wu H, Guo H, Su Z, et al. Fabric-based self-powered noncontact smart gloves for gesture recognition. Journal of materials chemistry A, Materials for energy and sustainability 2018; 6: 20277–20288. DOI: 10.1039/c8ta08276f.
50.
Wen F, Sun Z, He T, et al. Machine Learning Glove Using Self – Powered Conductive Superhydrophobic Triboelectric Textile for Gesture Recognition in VR/AR Applications. Advanced science 2020; 7: 2000261-(1-15). DOI: 10.1002/advs.202000261.
51.
Khan RZ and Ibraheem NA. Survey on gesture recognition for hand image postures. Computer and information science 2012; 5: 110.
52.
RautaraySSAgrawalA.Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev2015;
43: 1–54.
53.
Zhou Q, Xing J, Li J, et al. A device-free number gesture recognition approach based on deep learning. In: (CIS) tICoCIaS, (ed.). 2016 12th International Conference on Computational Intelligence and Security (CIS) Wuxi, China: IEEE, 16–19 Dec 2016, paper no. CIS.2016.0022, p. 57–63.
54.
DinhD-LLeeSKimT-S.Hand number gesture recognition using recognized hand parts in depth images. Multimed Tool Appl2016;
75: 1333–1348.
55.
Islam MM, Siddiqua S and Afnan J. Real time hand gesture recognition using different algorithms based on American sign language. In: IEEE international conference on imaging vpri, (ed.). 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR). Dhaka, Bangladesh: IEEE, 13–14 Feb 2017, paper no. ICIVPR.2017.7890854, p. 1–6.
56.
GongZXiangZOuYangX, et al.
Wearable fiber optic technology based on smart textile: a review. Materials2019;
12: 3311.
57.
Lin M-T. The chameleonic garment integrated POF fabric with a tricolour LED and wearable device technology. International journal of clothing science and technology 2020; ahead-of-print. DOI: 10.1108/IJCST-06-2020-0084.
58.
YangJ-HChoH-SKwakH-K, et al.
Influence of detailed structure and curvature of woven fabric on the luminescence effect of wearable optical fiber fabric.Sci Emot Sensibil2018;
21: 55–62.
59.
Kim HY, Tan J and Toomey A. Co-designing Interactive Textile for Multisensory Environments. In: Textiles AIoFa, (ed.). AITA 2018: International Conference on Artificial Intelligence on Textile and Apparel. Hong Kong, China: Cham: Springer International Publishing, 27–29 June 2018, paper no. 978-3-319-99695-0_25, p. 205–211, Nature Switzerland AG 2019.
60.
Kim H, Tan J and Toomey A. User experience and interactive textiles: A textile designer’s perspective. International Journal of Design Management and Professional Practice 2019; 13: 1–10.
61.
Kim J-E, Kim Y-H, Oh J-H, et al. Interactive Smart Fashion Using User-Oriented Visible Light Communication: The Case of Modular Strapped Cuffs and Zipper Slider Types. Wireless communications and mobile computing 2017; 2017: 1–13. DOI: 10.1155/2017/5203053.
62.
Kim J-E, Kim Y-H, Oh J-H, et al. Interactive Smart Fashion Using User-Oriented Visible Light Communication: The Case of Modular Strapped Cuffs and Zipper Slider Types. Wireless communications and mobile computing 2017; 2017: 1–13. DOI: 10.1155/2017/5203053.
63.
GargPAggarwalNSofatS.Vision based hand gesture recognition.World Acad Sci Eng Technol2009;
49: 972–977.
64.
Ullah A, Ahmad J, Muhammad K, et al. Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features. IEEE access 2018; 6: 1155–1166. DOI: 10.1109/ACCESS.2017.2778011.
