It has long been our nature to explore the curiosities of vast systems, such as the earth and the universe. But how can we possibly fully understand how these incredibly large systems work? A simple approach would be to capture the status of every component of the system through extracting the various types of signals they provide, such as radio waves or electricity. However, it would be impractical to place signal-catchers at every part of the system due to the high cost and inefficiency.

Advanced technologies, like the global seismographic network and the radio telescope system, provide an efficient approach by collecting the signals by placing highly sensitive detectors at positions where signals converge. This allows for decoupling and integration of the information that is entangled in the converged signals into one frame of knowledge for an accurate observation of the entire system.

 We thought that this main idea can be also applicable for tracking the motions of our own human body. In general, soft sensors are attached in every point of our joints connected with thousands of wirings for motion tracking. Furthermore, attempts to measure the muscle activity by electromyography (EMG) signals are affected by inconsistencies due to coupling between neighboring muscles which leads to labor-intensive data pre-processing.

 Brining the knowledge from the advanced technologies, our colleagues suggested that we could provide a new sensing paradigm for motion tracking by attaching a highly sensitive sensor at a point where signals converge. We also planned to use machine intelligence for unveiling the motions deep inside the coupled signals.    Based on the laser patterning of silver nanoparticles on flexible substrates in our recent paper (Nature Communications volume 11, 2149, 2020), we captured that the laser power changes the grain boundary of the nanoparticles, and at a certain point, the particle turns into a conductive and also stable cracking layer. Combined with ultra-violet laser cutting, we designed a serpentine structure which can be unobtrusively attached above the skin.

We attached a single sensor above the wrist to capture the finger motions. As the same way detecting the earthquake, the sensor is placed at an epicentral distance (above the wrist) and decodes the hidden movements beneath the converged signal. By the aid of the neural network, a single sensor on the wrist successfully decoded the five finger movements. Surprisingly, the same concept can be expandable to other body parts such as identifying gait motions on pelvis and keyboard typing identification with only a single sensor! (check out the supplementary data in our paper) We anticipate that this sensory system can track the motion of our entire body with minimum amount of sensory network. The new device would facilitate the in-direct remote measurement of human motions and also applicable to soft robotics.

The paper:    

K.K. Kim, I. Ha, M. Kim, J. Choi, P. Won, S. Jo*, S.H. Ko*, “A deep-learned skin sensor decoding the epicentral human motions”, Nature Communications, 11, 2149, 2020.      

Lab website: http://ants.snu.ac.kr/

Twitter handle: @SeungHwanKo1