In the context of internet of things (IOT), it has become ever demanding for groundbreaking technologies that can better connect human to human, human to machine and machine to machine. The immediate benefits of such connected technologies will enable human healthcare personalized and stay connected anytime anywhere, as well as offer immersive real-virtual interactions in the upcoming metaverse world. To date, state-of-the-art technologies are able to connect electronic devices to global internet virtually anytime anywhere. However, the connection between electronic devices and human health conditions or human activity is far from “seamless”.

The fundamental gap lies at that electronic and biological worlds are incompatible, where the former is mainly made of rigid matters functioning under dehydrated states but the latter comprises of mainly soft matter working under wet conditions. Wearable skin-like electronic devices (referred as ‘E-skin’ devices) may be able to overcome this challenge to better connect biological system to electronic world in that they are soft and elastic, matching human skins in conformable and imperceptible manners. Thin yet soft skin-like sensors have the potential to attach and deform freely on your skin, monitor your health vitals, and interact you with machine interface.

Recently, researchers have demonstrated smart on-body wearable skin systems that can recognize gesture and sign languages. This is typically accomplished through artificial intelligence to learn and classify between multiple sensor outputs. In particular, soft resistive sensors called resistive skins can be a simple yet scalable strategy to detect various biometric biosignals. However, the information from a single resistive sensor is usually non-specific to biometric information leading to largely entangled biosignals, which cannot be used for recognising specific activities combos. On the other hand, human skin is a complex soft tissue that may contain multiple biometric signals within a small area. For instance, Human throat skin may contain several biometric including speaking, heartrate, breath rate, neck movement, and tactile touch. Such abundant information cannot be easily collected by a single resistive E-skin.

    To address this challenge, we developed a hierarchically resistive skin (HR skin) offering the ability of “one resistive signal - multiple biometrics”. This HR skin was devised with three resistive skin layers composed of top Platinum (Pt) layer, middle vertically aligned gold nanowires (v-AuNWs) layer, and bottom ultrathin gold nanowires (u-AuNWs) layer with respective distinct response to different biometrics (Figure. 1a), Such fabricated HR skin exhibited distinct staircase-shaped electrical responses and linearity in three different strain ranges, leading to identification of five physical/psychological activities (speak, heartrate, breath rate, touch, and neck movement, Figure 1b) when worn on throat.

Figure 1 Bio-inspired HR skin-neuron network sensory processing framework. a, Schematic of the wearable HR skin as a single biosensor to report multiple biometrics including physical (speak, neck motion, and tactile) and physiological (breath rate and heart rate) information. b, Framework of our HR skin sensory system to disentangle a single resistive signal into multiple classified biometrics via our data process and DeepHS. 

Our design concept involves in hierarchical HR skins with specific sensing layers, namely high conductivity/high sensitivity for the top layer; medium conductivity/medium sensitivity for the middle layer, and low conductivity/low sensitivity required for the bottom layer. All the three layers are connected in parallel. With this design, the HR skin conductor displayed a staircases-shaped resistance changes to tensile strains, with high linearity in the specific strain regimes (Figure 2a, b). As a result, our HR skin maintained ultrahigh sensitivity with respect to a full strain range to pick up biometric signals from weak throat vibration (ε<0.01%), carotid pulsation ( ε<1%), breath (ε<5%) to neck motion (ε>30%) (Figure 2c), The working principle of the HR skin is illustrated in Figure 2d. When a constant voltage is applied to the sensing circuit, majority of redox current would pass through the low-resistivity top Pt layer when a small tensile strain (ε=0-3%) is applied. With the strain increased in the medium range (3%-35%), the top Pt layer will be electrically non-conductive, thereby activating the medium resistivity v-AuNW layer. Further increase of the strain to >35% will block electron transport in both top and middle layers, in which the high resistivity u-AuNW layer dominate the overall current.

Figure 2 Characterization and working principles of the HR skin. a, The resistance changes of each individual layers in the strain range of 0-50%. b, The resistance changes of HR skin comprised of Pt film/u-AuNWs bilayers and Pt film/v-AuNWs/u-AuNWs tri-layers in the strain range of 0-50%. c, Comparison of dynamic gauge factors between hierarchical resistive skin and other resistive E-skins in the literature. d, The sensing mechanism of the tri-layers hierarchical resistive skin.

Figure 3. Development of DeepHS for classification and recognition of throat activities. a, Visualization of 11 throat activities.  b, Visualizing the 1094 samples of all throat activities by using UMAP dimension reduction technique. c, Testing accuracies of motion, speak, combination and all.

Furthermore, we have developed a deep learning-based signal processing method called Deep Hybrid-Spectro (DeepHS), which allows for classification of entangled biometric signals into 11 classes including speaking, neck movement, pressing, and their combinations (Figure 3a). The program uses a combination of signal processing and machine learning to identify these different actions by analyzing the characteristics of the sound produced by the throat. To filter out any irrelevant noise, a bandpass filter was used. By looking at the sound's amplitude and a special image called a spectrogram, the program can distinguish between low-frequency motions and high-frequency speaking signals. We have achieved a classification accuracy of up to 92.73% for one person and an average accuracy of 84.21% for a group of five participants (Figure 3b, c).

        This work titled “Hierarchically Resistive Skins as Specific and Multimetric On-Throat Wearable Biosensors” was published in the latest volume of Nature Nanotechnology. (https://www.nature.com/articles/s41565-023-01383-6).