Abstract
In this article, we adapted a sensor-based smart insole to monitor the workload of the construction material carrying work frequently occurring at the construction site. Generally, the tasks of the construction material carrying work by the construction site workers proceed through walk. Therefore, we designed and implemented an application and server to receive and calculate data from the Internet of Things sensors to automatically estimate the weight of the construction material being carried and time of these works based on the characteristic of walking. As a result of the experimental tests with 15 people using the proposed method, it was confirmed that there was a correlation between the signal change at the foot plantar pressure during walking and the weight change of the construction material carried by the workers. It was confirmed that the foot pressure value during walking can be used to estimate the weight of the construction material that the worker currently possesses. Based on this, we were able to estimate the actual weight of the object with an accuracy of 91% from the 20 new test workers, and we were able to measure the work time with an accuracy of 97%.
Introduction
In general, the construction workers’ job at the construction site includes carrying construction materials, pouring, drilling, construction, and electrical excretion. Therefore, in order to effectively manage the construction work, it is most important to monitor these tasks of construction workers accurately. However, it is very difficult to precisely measure the amount and time of work of individual construction workers that takes place on a typical construction site.
In particular, the workload is difficult to measure because it is necessary to measure only the working time among the various activities occurring in a construction site. There is also difficulty in measuring the workload of performing simple construction object movements because there is no precise standard for this measurement. This research can contribute to an efficient construction site management system when sufficient research is conducted and appropriate standards are established to fairly measure the workload of construction workers.
Human walking is the most basic exercise for human actions in everyday life. Analyzing this walking information can be an important clue to find out what kind of activities construction workers are doing. Foot plantar pressure data have been analyzed over a long period of time in the field of exercise science research, 1 which allows us to observe changes in the pressure applied to feet during simple activities and many specific activities. In this article, the correlation between the workload and the walking characteristics of construction workers was analyzed using the foot plantar pressure values of smart insoles.
In general, special sensors or devices have been applied to detect various human movements, such as walking or sleeping. Previously, various studies have been conducted to accurately monitor such human movements in daily life. Lee and Seo 2 used human faces and movements to monitor the power saving mode of smart TVs. Cui et al. 3 proposed a preprocessing algorithm to achieve more accurate image processing results using human motion detection. In this article, since movement data are required for the working detection of a construction worker, we applied previously developed our smart insoles, 4 which can be worn when walking, in order to collect more accurate foot plantar pressure data.
At present, the construction worker’s workload at the construction site is manually assessed by the construction manager through eye observation, but cannot be accurately measured because the manager cannot fairly monitor every worker’s workload at every moment. Therefore, the workload of construction workers is often determined by the subjective judgment of the manager. To solve these problems, this article proposes a system that automatically measures the workload of construction workers such as the construction material moving and the movement of the worker using smart insoles to precisely estimate the workload of work and construction material carrying.
We developed a construction management system to measure the foot plantar pressure of the construction worker in order to find out the characteristics during walking. In order to utilize this system, we also developed a foot plantar pressure measurement and analysis program, and thus we were able to derive the characteristics of walking.
Finally, we analyzed the change of the gait data related to the workload of construction material carrying, which was obtained from our analyses of real-time characteristic data according to the weight of the object when carrying the construction material. As a result of this, our proposed In-Work Weight Estimation (IWE) algorithm provides 91% estimation accuracy for the workload of construction material carrying.
The rest of this article is organized as follows: A brief overview of human gait and related works on smart insoles and workload estimation is presented in section “Related work.” Section “Experimental system design” presents the information on the proposed experimental system for construction material transport. In section “Experimental results and analysis,” we provide the experimental results and analysis to estimate the weight of the construction material. Section “Validation” presents the verification of the proposed system with experimental tests and the conclusion of the proposed method is presented in section “Conclusion.”
Related work
Human gait
Gait is one of the most basic movement behaviors of humans. The behavior of a person carrying a simple object and moving is very common in everyday life. In addition, when people are working, they move with carrying objects or move for any construction works, which are very common behaviors at construction worksites. The most commonly used method for analyzing such gait is to measure various behaviors using foot plantar pressure. Foot plantar pressure has been used in many areas, especially in clinical and exercise science research. 5
General gait is basically performed by repeating eight steps: foot strike, opposite toe-off, reversal of fore-aft shear, opposite foot strike, toe-off, foot clearance, tibia vertical, and foot strike, and each step is divided into the action of leg position and force as shown in Figure 1. Therefore, each step can be used as an appropriate indicator to accurately estimate the walking process of construction workers. 6
Eight general steps for a gait cycle. 6
Smart insole
Recently, research and development of smart shoes, which measure and analyze the human gait and body temperature by placing pressure and various sensors on shoes, have been carried out.7,8 Most of the researches have been conducted to measure the foot plantar pressure using smart insoles and to extract information about gait and human body movements. 9 In addition, a lot of research has been carried out in the field of health care using this information. For example, Wang et al. 4 developed a system for health management using their proposed smart insole.
Xu et al. 10 developed a smart insole system and algorithms applied to a mobile device for improving health care environments using gait information. Lin et al. 11 developed a smart insole using flexible fiber and gyro sensors and also developed a system that allows users to monitor and graph them in real time. Ho and Min 12 developed a fiber proximity sensor that is applied to analyze the correlation between drug use and gait in order to evaluate the drug use, and their system achieved high accuracy.
The smart insole system for real-time health management in sports and rehabilitation has also been studied. 13 In addition to these health management systems, systems that detect actual activities and accidents, and those that can help the sports technology by understanding the balance and characteristics of the body movements were also studied. Manupibul et al. 14 have developed a system that can measure and monitor human body balance and pressure. This technique can be used not only for improving the performance of sports technology but also in the rehabilitation-related field.
There is a phenomenon in which a foot flexes flexibly while walking, and it is effective to use a flexible pressure sensor to measure the foot plantar pressure with high accuracy even in such a phenomenon. Lin et al. 15 showed that their analysis results obtained from their proposed device using such a flexible sensor were accurate. In order to measure the foot plantar pressure according to the movement of a flexible foot, a smart insole suitable for working with a flexible sensor was manufactured. Recently, sensors such as smart insoles have been used to collect foot pressure data. Moreover, similar studies have been conducted on the effects of human balance or load on the human body. Tan et al. 16 used a smart insole to measure weight with Pedar-X under six activities, namely, slow, medium, and fast walk, medium and fast run, and limping. Recently, since construction works require heavy physical workloads, Yu et al. 17 proposed the automatic workload assessment using smart insoles and a computer vision algorithm with image data using inverse dynamics, and the research reported in this paper has focused on the prevention of ergonomic risks. Their research estimates the joint level of workload based on video images from an RGB camera and plantar sensors to prevent the workers from moving or lifting heavy construction materials.
Although smart insoles are used in many areas for detecting various movements of the human body, the automatic estimation of construction workload such as moving or lifting heavy construction materials and movements for construction-related works has not yet been studied with smart insoles. Therefore, there is a need for further studies to estimate the load while holding construction objects from the foot pressure data. In this article, we measured the workload using the smart insole incorporating a flexible pressure sensor and performed a simplified measurement of the complex workload.
Workload estimation
Workload measurement has been studied thoroughly and there are two main types of workloads: mental and physical workloads. 18 Physical workload is the measurable portion of physical resources expended when performing a given task (manual lifting and carrying, repetitive work, and other physical strain) and is affected by a range of factors including the nature of work, training, motivation, and environmental factors. An objective measurement of physical workload can be classified into energy capacity from evaluating the energy cost of performing an activity and biomechanical approach from evaluating stress on the musculoskeletal system. Mental workload depends on attention and is inferred from changes in work performance. When mental workload is too high or too low, the work performance declines accordingly.
Guhe et al. 19 developed a real-time and non-intrusive measurement system that does not interfere with the worker because the workload is divided into two parts: task load and cognitive workload, and the measurement is complex. However, in this study, it was difficult to apply the method to the majority of the population by focusing on three cameras and conducting the research. Liu et al. 20 developed a device composed of an electromyography (EMG) transducer, a goniometer, and an accelerometer to measure the workload using various sensors. However, these devices are relatively large in size, making them difficult to use in many active or hazardous work environments.
Ellegast et al. 21 determined the worker’s handled load weight by measuring the plantar pressure and applying a load in static and dynamic postures. However, the total experiment duration was as short as 40 s, and it was inconvenient to carry out because it used an expensive commercial gait measurement device. Therefore, there is a disadvantage that it is difficult to apply to workers in actual construction sites. Garet et al. 22 measured the physical workload using heart rate monitoring. Although this study was helpful in preventing the risk of work, it cannot be applied to measure the amount of work actually done by the worker.
Studies related to measuring the mental and physical workloads have been conducted to prevent accidents that may occur due to stress. In addition, the estimation of the workload of each construction worker has not been deeply studied yet. In our previous work, we developed a construction site monitoring system to prevent work-related injuries. The purpose of this study was to develop a wearable textile pressure insole sensor and examine its effectiveness in managing the real-time safety of construction workers, but not for workload estimation in construction workers. 23
In this article, since the workload of construction workers is measured manually and incorrectly by a construction manager at current construction worksites, we have focused on the workload to automatically and accurately measure the workload of each construction worker for the construction process management system based on Internet of Things (IoT) sensors. The proposed method utilizes the foot plantar pressure values which can provide diverse information on human body movements, which were measured using smart insoles to estimate the workload. For this, we applied our previously proposed smart insole sensors and this system is experimented with some people to achieve meaningful results for workload estimation without any disturbance and inconvenience for construction works.
Experimental system design
Plantar pressure insole
In this article, to measure the foot plantar pressure value, we adapted a gait measurement system developed in our previous research for health management. 4 However, we designed a new insole sensor shown in Figure 2 to fit the worker’s shoes. Refer to our previous research for the configuration of the detailed gait measurement system and the structure of the sensor (Table 1). 4
The smart insole used for implementing the proposed system.
Configuration of the device (board).
Communication program: plantar pressure sensor insole, PC, and application
Communication between the PC or the application and the insole with a built-in sensor is conducted using a seven-channel communication program via Bluetooth. The developed program shown in Figure 3 was implemented for C# using Visual Studio 2015 as a program for PC. In addition, it is developed to download the available application through debugging on Android.
Communication program.
In the case of the PC program, if the PC and the insole pressure sensor are connected using Bluetooth, the program selects the port where the insole is connected in the program. It provides a real-time pressure graph function of each foot and contains a save function to save the data as a text or excel file at a desired point in time. The program then reads the current time and stores it as a result with the packet. The program also reads the current time and stores it as a result of our proposed packet structure. Figure 4 shows the packet structure for communication.
Communication program packet.
In this article, the program flow of the workload measurement system based on the IoT sensor designed and implemented is shown in Figure 5. The PC or smartphone application stores 20 Hz sensor data received from the pressure measurement insole in real time into a database (DB) built in the server. Since the raw data are stored in the DB, these data can be shown to the worker or the administrator using the graph. At the same time, the proposed algorithm infers the weight of the objects being carried and informs the user. After the measurement is finished, the program on the server fetches all the raw data entered in the DB and analyzes them.
Communication program flow.
Experimental results and analysis
We analyzed the behavior of various workers based on the raw data stored in the DB. In particular, we measured the quantitative values of the workload, such as the weight of the material being carried and the time during which the subject (or user) carried the construction material with a specific weight. We designed the IWE algorithm in order to measure the weight of the material when the construction worker carries the construction material. To develop the IWE algorithm, we analyzed data changes of the insole sensor using a constantly increasing number of objects for a total of 15 adult males. They were healthy and did not have any walking-related diseases.
The weight of the object was increased from 0 to 30 kg in 5-kg increments. Through this experimental test, we derived the linear equations from this experimental test and estimated the weight of the current object through the initial insole sensor data. In addition, to design the formula of the IWE algorithm, we performed experimental tests in both standing (static situation) and moving (dynamic situation) cases.
Analysis of weight change in static situations
The experimental test for the static situation was carried out with the shoes with the smart insole mounted in a standing position for 1 min. First, we measured the pressure data in the plantar pressure sensor from a subject without holding any object at a normal state. We then measured the pressure data using the sensor from the subject holding 5-, 10-, 15-, 20-, 25-, and 30-kg objects, respectively. In order to confirm the change of the total pressure due to the weight change of the object, the average value of 14 pressure data measured at two feet was obtained and compared. Figure 6 shows the average data change in five subjects who performed an experimental test with the object held in place for 1 min. In the graph, the
Detected pressure data change with load (average value of 15 subjects during the static situation).
As a result of the experimental test, even if the weight of the object increased by 5 kg, the change in the average plantar pressure value did not increase linearly. In particular, in the case of a low-weight object of less than 10 kg, the change in the plantar pressure value was measured to be so small that it was difficult to accurately distinguish it from noise or any other inaccurate measurement. It is thought that this case occurred because the body adapts to the weight added naturally and disperses the weight to both feet. The human feet adapt to the weight state while maintaining the medial longitudinal arch.
Increasing the weight activates a compensation mechanism that maintains the longitudinal arch and moves the load to the medial and medial forelimbs. 24 Therefore, as the weight is added, the added pressure is not large because the human body is tilted forward. In addition, we observed that the amount of increase in the pressure data became smaller because the force added to the person was divided into both feet according to the added load. For these reasons, it is difficult to estimate the weight of an object when using only foot plantar pressure data in a static situation.
Analysis of weight change in dynamic situations
When estimating the weight of objects under static conditions, the change of the plantar pressure value was not obvious, especially at low weights. Therefore, in this article, we analyzed the weight change based on the experimental results through the gait pattern, which is inevitably used when carrying construction materials in construction worksites. At this time, the weights of the objects provided to the subjects were 5, 10, 15, 20, 25, and 30 kg similar to the experimental test in the static situation. In the dynamic situation, the subjects were asked to walk 80 steps instead of just holding each object and standing in place. Gait during the experimental test was not limited and proceeded freely in various floor situations by the subject. In the graph shown in Figure 7, the
Measurement of the pressure value during walking with carrying the object (averaging around peak data).
Figure 7 shows an example of the measured foot plantar pressure during walking. As shown in the red squares in the graph, the maximum pressure value is measured when one foot is in the ground and the other is in the air. At this time, since the pressure due to all the weight is concentrated on one foot that is in contact with the ground, the ground and sole of the foot are in close contact with each other. In this situation, we were able to obtain a significantly accurate plantar pressure value with peaks prominent compared to standing on both feet.
Therefore, the proposed algorithm finds out the maximum peak point in the plantar pressure value for each walking step, and the weight of the object was measured by the correlation analysis of the peak values of the foot pressure data in various walking situations.
Experimental tests of gait were conducted without lifting any objects and after rest for a while and then adding a specific weight. In all experimental test cases of gait, the left and right feet touched the ground, extracting a situation with a high pressure value. We then found the highest measured value (peak data) for each step in the foot plantar pressure data of both feet obtained from each experimental test. Then, the pressure value in the current step was averaged by collecting five pressure values before and after the maximum peak value. Since we need to compensate for the situation where the peak data were measured at a somewhat high level due to the error of the sensor or the type of walking of the subject, the five data were used together. The value obtained was used as the foot plantar pressure value representing one step of walking in each situation.
In order to obtain reliable walking data from the subjects, an average of the 60 steps were considered excluding the 10 steps in the front part and 10 steps in the rear part (total: 20 steps) which can readily have noisy data among the measured 80 steps of data. Thus, the average values of the peak data obtained when carrying the objects for 10 subjects are shown in Figure 8. Here, the
Detected pressure value change with load (average value of 10 subjects during walking with carrying objects).
When subjects are carrying 0-, 5-, and 10-kg objects, the pressure data showed a relatively small difference, so the measured foot plantar pressure value increased by about 3% at 5 kg weight and increased by about 6% at 10 kg. From the comparison of values between movement without carrying any object and movement with carrying a 20-kg object, it was confirmed that the average foot pressure value increased by about 15% when moving with a 20-kg object. In addition, the average foot pressure value when carrying a 30-kg object increased by 20%. Unlike static situations, the foot pressure value is increased in proportion to the weight of the objects being carried in dynamic situations. Therefore, the proposed method estimates the weight of an object carried by a construction worker based on the results obtained from experiments of dynamic situations of 10 subjects.
The IWE algorithm
In this experimental test, we used the change of foot plantar pressure data when the subject was walking, which is a common process during construction works, to estimate the weight of the construction materials carried by a worker. Since construction workers usually carry construction materials within 30 kg at the construction worksite, the object weight was increased from 0 to 30 kg in 5-kg increments to carry up and walk. Based on the measured data, the formula for the weight estimation while carrying the object was derived as follows.
As a result of the experimental tests, it was confirmed that the measured value of the pressure sensor increased linearly with the change of weight. We have derived an algorithm that uses these characteristics to estimate the current weight from the initial state. First, we derived the liner equation (1) through the change of two points using the measured data (pressure value) when the weights of the object are 0 and 30 kg
Then, by summarizing the equations, we can derive the formula of the final interpolation equation that can be used to estimate the weight of the object currently held by the construction worker
However, the proposed equation is not perfectly linear, and the weight prediction slope of less than 10 kg is relatively small. Therefore, there is a problem that an error may occur when the construction worker carries an object of less than 10 kg.
Validation
To verify the weight estimation algorithm designed in this article, 20 new subjects were recruited for the validation of experimental tests and their heights, weights, and age information are described in Table 2. In addition, Figure 9 shows the whole process of the proposed method to estimate the work time and workload of subjects.
Subject information.
Flowchart of the proposed method for the estimation of workload and work time of workers.
To test whether the proposed method accurately estimates the weight of the construction material being carried, the subjects of the experimental test carried objects weighing 5, 15, 17, 20, 22, 25, 27, and 33 kg, respectively, in the order shown in Figure 9. Also, to test whether the proposed method precisely measures the work time of the worker, each subject walked for 30 min while wearing the pressure sensor insole. The process of experimental tests is shown in Figure 10, and the subject rested in the middle of the experimental tests.
Configuration of the experimental test.
The experimental test was designed to last for about 50 min, and the subject walked for 3 min as the initial common state to measure the initial foot pressure then took a 3-min rest. During this rest time, we changed the weight of the object being carried. Then the subject walked for 3 min carrying the changed construction object. Thus, we measure the initial foot pressure by walking without any object at the time of measuring the common state. After that, the subject was going to carry out eight measurements in progress. The subject walked with a specific weight every 3 min. When the work condition was measured, they took a 3-min rest and added weight to continue measuring the next work condition. The addition of weight was carried out in the form of adding an object previously weighed to a backpack or girdle carrying it on its back.
Figure 11 shows one of the snapshots taken at the time of the experimental test. To test accurately under the working conditions of construction workers, the subjects walked on various grounds such as stairs, flat ground, unpaved ground, and slope ground which are common ground conditions on construction sites. Figure 12 presents a graph showing the average error rate between the estimated weight of the object from the proposed method and the actual weight of the object in each case when the subject carries the object during the experimental test.
Snapshot of the experimental test.
Result of the error rate from the IWE algorithm.
From the test results of 20 subjects, the error for the weight of each object was measured at a maximum of 4 kg and a minimum of −3 kg, and the average error was about 1.5 kg. Especially, it was frequent and bigger compared to in the case of carrying an object less than 10 kg. In the case of carrying an object more than 10 kg, there was a relatively low average error. We used the following equation to calculate the error rate of the estimated weight of the carrying object
Table 3 shows the results of the estimated weight using the proposed method at 5 and 15 kg and the error rates for 20 subjects, respectively. MAX indicates the highest error and MIN indicates the lowest error among all subjects.
Weight estimates and error rates for the 5- and 15-kg objects.
In the case of the 5-kg experimental test, the average weight of the estimated object was 5.52 kg, so the error rate was 24.40%. The average of the 15-kg experimental test was 15.63 kg and the error rate was 6.87%. An average error rate of less than 10% was also observed for the cases higher than 15 kg. We could confirm that the error of the estimated weight is larger when the weight of the object being carried is less than 10 kg. The overall error rate was calculated by analyzing all the data measured through this experimental test, and the average error rate was 8.76%. Therefore, the proposed method can automatically estimate the weight of the object being carried with about 91% reliability using smart insoles.
In addition to the workload estimation of the construction worker, the verification of work time for each construction worker was estimated using smart sensors. The work time was estimated as the time during which the subject is moving. On the other hand, when the movement of the subject is smaller than a predetermined threshold, the period is considered as the rest time. Figure 13 shows the time difference in seconds between the actual work time and the estimated work time derived from the proposed method.
Result of the error rate for the estimated work time.
In this experimental test, the work time was measured for 3 min per each state as in the previous experiment based on these results. The work time measured by the proposed method applied in this experimental test showed an error from a maximum of 14 s to a minimum of 3 s compared to the actual work time, and an average error of about 7 s was shown. After 540 min of work time for 20 subjects, the measured time of the system was 544 min and 20 s. As a result, the work time error was about 4 min and 20 s, and the proposed method was able to automatically estimate the work time with about 97% accuracy. Accuracy was calculated using the following equation
The experimental test process and the results of the proposed system in this article are summarized as follows:
To measure the largest load when the subject walks with carrying the object (0–30 kg), the pressure value when only one foot touches the ground is measured for 10 subjects.
Based on these results, the interpolation formula was developed to predict the weight of an object carried by the construction worker.
The work time of the worker is estimated based on the time of carrying the object and the worker’s moving time.
The proposed method estimates the work time with 97% accuracy and the weight of the object being carried with 91% accuracy.
When a construction worker is walking, the proposed method can automatically estimate the work being performed using smart sensors. However, such an error is determined as an error generated in the process of estimating the walking continuity, which determines whether the walking state has correctly ended. Unlike the results of the previous weight estimation method in the static situation, the proposed work time measurement method did not show a large difference even for objects less than 10 kg, because it was not significantly affected by the weight of the objects being carried since it was based on the presence of walking conditions.
Conclusion
In this article, the work time and the weight of the object being carried were estimated using the smart insole based on the IoT sensor developed, and a change in the pressure value during gait was observed by adding a certain amount of weight during walking. The proposed method can automatically estimate the weight of the construction materials currently being carried by the worker through the foot pressure value during walking. The experimental test was conducted on 15 subjects (5 subjects for the static situation and 10 subjects for the dynamic situation), and the experimental test was conducted by applying various weights of the objects being carried by the subject and thus the IWE algorithm was derived. In addition, for verification purposes, data from 20 subjects excluding subjects used in the static and dynamic experimental tests were additionally collected.
As a result of the experimental test, when the weight of the construction material is 10 kg or more, the average error rate is 6.15%. However, when the object’s weight is less than 10 kg, the error rate is 25.32%, and the reliability is relatively low. Also, the proposed algorithm was implemented using a simple interpolation approach and implemented as an application in android smartphones and PC. From these results, the proposed method is capable of estimating the weight of the object while carrying construction materials and of measuring the work time with or without carrying objects under various walking conditions.
We believe that the proposed method can help measure the amount of work on a construction site that is difficult to measure, and when combined with other information, it is expected to be used in more fields. Therefore, our proposed method and these experimental tests are expected to be used to measure the workload of the construction object movement work in a construction site where the actual construction object carrying work is frequent. After that, we will collect more data and perform experimental tests on the actual construction site to improve the accuracy and reliability of data and to apply them to the construction process management system.
Footnotes
Handling Editor: Pascal Lorenz
Authors’ note
All authors have contributed equally to this work.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Bio and Medical Technology Development Program of the National Research Foundation (NRF) funded by the Ministry of Science, ICT and Future Planning (NRF-2015M3A9D7067388) and was supported through the Soonchunhyang University Research Fund.
