The advancements in tactile sensor technology have found wide-ranging applications in robotic fields, resulting in remarkable achievements in object manipulation and overall human-machine interactions. However, the widespread availability of high-resolution tactile skins remains limited, due to the challenges of incorporating large-sized, robust sensing units and increased wiring complexity. One approach to achieve high-resolution and robust tactile skins is to integrate a limited number of sensor units (taxels) into a flexible surface material and leverage signal processing techniques to achieve super-resolution sensing. Here, we present a magnetic skin consisting of multi-direction magnetized flexible films and a contactless Hall sensor array. The key features of the proposed sensor include the specific magnetization arrangement, K-Nearest Neighbors (KNN) clustering algorithm and convolutional neural network (CNN) model for signal processing. Using only an array of 4*4 taxels, our magnetic skin is capable of achieving super-resolution perception over an area of 44100 mm2, with an average localization error of 1.2 mm. By employing neural network algorithms to decouple the multi-dimensional signals, the skin can achieve multi-point and multi-scale perception. We also demonstrate the promising potentials of the proposed sensor in intelligent control, by simultaneously controlling two vehicles with trajectory mapping on the magnetic skin.

In summary, we developed a magnetic skin that can acquire super-resolution X and Y coordinates, as well as magnitude information, for pressing actions. The 48400 mm² continuous multi-point and multi-scale sensing areas and 45.8 SR factor can be achieved by 48-dimensional magnetic signal data from sixteen magnetic Hall sensors combing with machine learning methods.

By integrating a magnetization and arrangement method, expanding the datasets using a linear universal Kriging equation, employing the k-nearest neighbors (k-NN) algorithm to interpret three-dimensional magnetic flux density signals, and machine learning algorithm, our sensor surpasses current technologies. This combined approach enables us to achieve superior performance in determining the super-resolution magnitude and position of contact force. The average error of super-resolution sensing is reduced to 1.2 mm by utilizing higher-dimensional magnetic field signals.

The recognition of multi-scale objects can be achieved by magnetic skin through the combination of magnetic field signals and neural networks. By employing a neural network that utilizes low-dimensional magnetic field signals from neighboring sensors, we can recognize the pressing signals of small-scale objects. Additionally, by combining the overall 48-dimensional magnetic signal data with the neural network, we can identify the pressing of large-scale objects.

With its super-resolution recognition capability and a full range sensing area of 48,400 mm², magnetic skin can enable real-time handwriting applications. By continuously rolling a small ball (15 mm in diameter) on the magnetic skin, the corresponding rolling trajectory of “Z,” “J,” and “U” can be obtained by connecting the corresponding points of pressure perception.

Few discussions about magnetic tactile sensors have addressed the ability to perceive multi-point and multi-scale pressing information due to the integrity of magnetic materials and the low-dimensional information of magnetic sensors. Our magnetic skin overcomes these limitations by employing a combination arrangement of multiple small magnetic pieces and utilizing the multi-dimensional magnetic field variations obtained from sixteen magnetic sensors. As a result, our magnetic skin enables the perception of multi-point and multi-scale pressing. Here, we demonstrate this feature using an intelligent car control system. Unlike traditional control methods that rely on four directional buttons (forward, backward, left, and right) to control the movement of the car, our car control demonstration introduces a mapping technique. This technique involves accurately mapping the positions of two balls on the magnetic skin, allowing us to precisely control the movement of two cars and achieve accurate trajectory mapping.

Moreover, the magnetic field has the ability to penetrate most materials, allowing for the separation of the magnetic films (perception module) and the Hall sensor array (signal-receiving module). This is unlike traditional wire-connected soft electronic sensors. This characteristic helps to address issues such as the difficulty of repairing damaged sensor arrays and the complexity of wiring.

By altering the ratio of the top and bottom silicone layers, the sensitivity of the sensor can be altered. For example, adding silicone oil during the molding of the bottom flexible silicone layer can adjust the elastic modulus of the substrate, thereby making the magnetic signal changes generated by pressing more sensitive. Due to the low-frequency magnetic field’s transparency through the human body and its non-invasive nature, our sensor has the potential to be applied to human-robot interaction applications. The only rigid components of magnetic skin are the Hall sensors which can be replaced by flexible printed circuits, enabling the development of continuous magnetic skin. This advancement would enhance the safety and reliability of human-machine interactions, as the flexible nature of the sensors would minimize the risk of injury or damage.

In summary, we have developed a large-area super-resolution magnetic skin with multi-point and multi-scale perception ability. The proposed design approach can be further applied to robot large-area tactile sensing skins, providing robots with the ability to perceive tactile information over a large surface area and allow for reliable and safe human-machine interactions.