Forensic Footwear: A Retrospective of the Development of the MANTIS Shoe Scanning System

Abstract

There currently are no shoe-scanning devices developed in the United States that can operate in a real-world, variable-weather environment in real-time. Forensics-focused groups, including the NIJ, expressed the need for such a system. To accomplish this, there was first a need to collect a large amount of sample data to train a first-of-its-kind machine-learning model. The work aimed to develop a system to collect shoe scans of a populace in real-time as they walk without committing privacy violations. This system would be able to identify specific traits found on the soles of the shoes, enabling insight into the origins of the shoes, where and how they are worn, as well as the people who wear them. The primary question of this research is if we could develop a prototype of such a system and what challenges would have to be overcome to make this possible.

Introduction

Shoe scanning devices and technologies have a number of important applications within the field of forensic investigation. These devices can operate in a laboratory setting to analyze and compare to evidence found at a crime scene. They also can be deployed in the open world, like security cameras, to collect evidence that could be used to identify specific details about shoes that would otherwise be lost in an investigation, such as wear patterns and particulate types or even manufacturing point of origin. To the end of footwear identification, different shoes have manufacturing origins and identifying details are exceedingly difficult to replicate. This information can be used to help localize where a suspect may be from or work or exonerate a suspect due to small details that prove their innocence and would be lost otherwise. This type of evidential evaluation is normally regulated to the domain of experts, but systems that can automatically identify shoes have the potential to stand in for or further support people in this type of role (Venkatasubramanian et al., 2021).

Devices like this clearly can solve many issues that are forefront in the minds of forensic scientists and law enforcement officers. Forensics-focused groups, including the NIJ, have expressed the need for systems to do exactly these tasks via individual characteristics and identifiable features. To leverage the power of such a system, a large amount of sample data would have to be available to train a machine-learning model. This type of data collection has been done before, as exemplified by Butdee and Tangchaidee (2008) when they created a new standard size model of the Thai population based on 3D scans. Additionally, devices have been shown to be capable of building such a database with paired images, as seen in Wu et al. (2016) when they did so to verify shoe manufacturing with robotic systems. Any such device would also have to be capable of operating safely in a wide variety of weather conditions to be able to serve outdoors across the United States.

There currently are no shoe-scanning devices developed in the United States that can operate in a real-world environment in real-time. A comparable system developed by Everspry in Liaoning, China, the Everspry Outsole Scanner, has been used by several teams over the years. Lin et al. (2022) assembled a small database of shoes in a lab-based setting using this system to get a start for recognition.

Surprisingly, there are also no complete databases of footwear treads and upper soles. This type of system was used by Cui et al. (2019) to enable the identification of partial prints thanks to their database created with an Everspry Outsole Scanner system. Therefore, our team set out to develop a deployable system capable of collecting shoe scans of a populace in real-time as they walk over the scanner without committing any privacy violations. This system would be able to identify specific traits found on the soles of the shoes, enabling insight into careers, habits, and frequented locations of the subject in question. It would also be capable of identifying traits about the shoes themselves, such as their proportional prevalence in communities and the likelihood of a suspect having committed a crime based on the shoe type, for example, if it was used as a primary point of evidence but it is worn by thousands in the area that would dramatically weaken the strength of any such claim. Our primary question was if we could develop a prototype of such a system and what challenges would have to be overcome to make this possible.

Methods

Initial Design and Prototyping

To begin work on this project, we considered a number of different methods, including the Analytical Network Process we had done previously in Schnieders et al. (2024). This project followed the standard iterative design method of the ATHENA Lab: design requirement determination by a panel of human factors experts, initial design refinement, physical development, testing of the physical system in a realistic and high-stress environment, assessment of goal accomplishment of the system, redesign to assess weaknesses or improve functionality. This project started with a prototype made from wood and acrylic with cameras at the top level and underneath for proof-of-concept testing. This system is operated with an ultrasonic distance sensor via a limited spring actuation with a stopper to provide a degree of freedom and maintain regularity while walking.

Initial Design

Once that design was proven functional, in that it could be activated when stepped on to have both cameras (one going sideways via a mirror and the other capturing the bottom of the shoe) take pictures, the system was scaled up to be more robust and weatherproof. These cameras were initially GoPro models synchronously controlled with a wireless connection from the central module, as shown below.

This new version was 18 inches high by 24 inches wide and 36 inches long, made of mild steel, and thus over 180 pounds (over 80 kilograms). To view the bottom and side sole of the shoes, the optics were manipulated via a combination mirror and field-of-view limiter systems to ensure the privacy of walkers overtop by enabling the photos to only go a maximum of 10 inches up the leg regardless of where on the system someone would walk. This system also utilized a spring-based actuation system, but instead of an ultrasonic distance sensor, this one utilized a magnetic trigger to fire the cameras to enable accurate capture regardless of direction of approach. An internal lighting system was installed to enable functionality in high and low light conditions without losing clarity of the tread pattern. Due to the intensity of Iowa winters, the team installed an internal heater to moderate the temperature at a minimum of 40 Fahrenheit regardless of the external temperature. A set of rechargeable power banks provided all power.

Testing and Redesigns

Following the initial design, as shown in Figure 1, the team conducted extensive testing over several years, with multiple redesigns along the way. It was deployed at least once a week over the course of two and a half years throughout the different redesigns. These deployments would be met with varied success, as some would get dozens, occasionally over a 100 images, while other sessions would get as few as zero if no one elected to walk on the system.

Figure 1.

An image of the initial design for the internal camera mechanism.

The first component redesigned was an anti-glare modification to reduce the reflective shine onto the plexiglass from the internal components, as light from either the internal system or external sources would cause a loss of fidelity with regard to capturing the shoe treads. This implementation had multiple forms and iterations, but in the end a lightweight, foldable, matte black sheet was what produced the best results.

A privacy screen was also investigated. This was intended to make it so the shoe treads could be seen but nothing past them. This was attempted with various methods, including different colored plastic screens and gel barriers that would move out of the way when exposed to weight. These methods had several failings, though. The screens would make it so the sole and shoe side could not be seen with the required detail. The gel barrier had issues including self-aeration, membrane ruptures, and issues with compression under the full weight of a person. These issues resulted in the team abandoning the modification from field-of-view limiters and holding with the restrictions of the current system.

Another modification to the original system in this testing period was a complete overhaul of the camera and CPU systems. Our team modified from using an Arduino Uno system to control and interact with the sensors and central controller to an all-on-board Raspberry Pi system using Pi-branded cameras. This change was made due to unreliability within the GoPro connectivity process, as cameras would fail to connect or disconnect seemingly at random throughout deployments. Converting to an all-in-one system for the cameras drastically improved functionality overall.

When the camera style was modified, that also mandated a change to the angles utilized, as the new cameras had a different orientation and zoom. This was enabled easily by instituting a new field-of-view limiter on the camera designed to capture the side of the shoe while moving the camera that captured the sole to utilize the long angle of the system rather than the short angle.

Conclusions

This project resulted in the initial MANTIS system structurally sound enough to deploy in the real world. The system did require significant maintenance at times, with damage to the system during interstate transit being extensive and costing significant time and money to repair and restore functionality. This project has collected over 7,000 images in just under three complete years of operation to be contributed to the initial database compiled by collaborators of the CSAFE group.

The MANTIS system proved capable of operation across all seasons, weather conditions, and temperatures that can be expected in day-to-day life in the continental United States. Furthermore, the system was able to capture usable upper sole and tread patterns regardless of external lighting conditions, having operated both before dawn and after dusk. This allowed the scanner not only to log shoe types but also to provide insight into the manufacturer, model of shoe, specific factory, potential point of purchase within the United States, and style of shoe worn in various seasons for a regional population.

Following the capture of shoe images, the data was securely transferred to an encrypted database. This database feeds an advanced algorithm to identify patterns within the individual characteristi-cs of shoes. These findings produced a system that could not only identify the details of the make of the shoe captured, but also the specific shoe due to its usage patterns.

We can take the lessons learned during this process to inform future advancements and create smaller, easier-to-use systems. A lesson that is less easily applied is that of database collection. The voluntary public walkership decreased significantly over time, with spikes in August each year as new students first saw and interacted with the device. This resulted in, at the end of the term for this project, a near-zero walkership rate for the voluntary public experimentation of the device. This likely can be attributed to the novelty of the device simply wearing off.

Advice for future work would be to install the device in-ground to decrease effort for system management or variability. This style of implementation would also decrease the effort of people walking over the device, enabling more theoreticcally deployable locations and higher walkership rates in the long term. The broader impact of this project is to lead to a world that can be safer for the innocent by increasing empirical evidence of shoe representation within an area to reduce false convictions and enable faster recovery of victims of kidnappings and human trafficking.