N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning

N-Omniglot can be considered an effective, robust, and challenging neuromorphic dataset for developing future spiking neural networks.
Published in Research Data
N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning
Like

The success of deep learning is attributed mainly to the common-used datasets for the benchmark like ImageNet and COCO. DVS simulates the human visual nervous system from a specific perspective and level of abstraction which allows spiking neural networks (SNNs) to take full advantage of the temporal information provided by such sensors. To facilitate the development of brain-inspired artificial intelligence, especially SNNs, several neuromorphic datasets has been produced using dynamic vision sensor (DVS). For example, N-MNIST, N-caltech101, and DVS-CIFAR10 are obtained by capturing images or camera movements from traditional classification datasets, avoiding the corruption of the original image information during encoding. Besides, more researchers acquire event data from natural environments, such as DVS-Gesture and N-Cars. However, the existing datasets have very low temporal correlation and could not fully reflect and exploit the temporal information processing capability of SNNs. Their characteristics are shown in Table 1. In addition to sparse coding to reduce energy consumption, rapid learning of new concepts from a few samples is one of the essential capabilities of the human brain, but this remains an open problem in spiking neural network-based machine learning.

Table 1: Characteristics of Neuromorphic datasets

To address these issues and bridge this gap, we propose the first neuromorphic dataset for few-shot learning: the N-Omniglot. The original Omniglot dataset is widely used in few-shot learning. The N-Omniglot consists of 1,623 handwritten characters from 50 different languages. Each character has only 20 different samples. It is usually considered a static text image and ignores the rich temporal information in the writing process.

Fig 1: The complete process of N-Omniglot data generation

Fig 2: Examples of N-Omniglot

Fig 3: Demonstration of using Davis346 to record the character writing process

We note that SNNs is still in its infancy in neuromorphic few-shot learning. Although we have constructed N-Omniglot, very few existing algorithms are suitable  for this task. All experiments have been performed on both N-Omniglot and encoded Omniglot, to demonstrate the difference between them. As shown in the Figure below, we treat static images with poisson and constant coding. Since the DVS camera for acquiring neuromorphic data has a high temporal resolution, the excessively long time axis is a huge burden for the current clock-driven SNN algorithms. Therefore, we process the event data with OR operations and firing rate. To demonstrate the effectiveness of N-Omniglot and the potential to provide new challenges for training SNN algorithms, we conducted experiments on four SNN algorithms, including adaptations or SNN versions of two general classical pattern classification methods and two few-shot learning algorithms.

Fig 4: Encoding of static images and pre-processing of event data

Fig 5: Four Few-shot Learning Benchmarks

N-Omniglot dataset is more sparse in the spatial dimension, and the data is less similar in temporal dimension than a static image or poisson coding-based inputs, which brings new challenges to spiking neural network learning. It is crucial to improve the ability of spiking neural network to extract more important spatio-temporal features. Therefore, N-Omniglot can be considered an effective, robust, and challenging neuromorphic dataset for developing future spiking neural networks.

Prof. Yi Zeng is the corresponding author of this article, Yang Li, Yiting Dong and Dr. Dongcheng Zhao are the co-first authors of this article.

 

Homepage: http://www.brain-cog.network/dataset/N-Omniglot/

For more details: https://doi.org/10.1038/s41597-022-01851-z

Citation:

Yang Li, Yiting Dong, Dongcheng Zhao, Yi Zeng. N-Omniglot, a large-scale neuromorphic dataset for spatio-temporal sparse few-shot learning. Scientific Data, 9(746), 2022.  

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Subscribe to the Topic

Research Data
Research Communities > Community > Research Data

Related Collections

With collections, you can get published faster and increase your visibility.

Medical imaging data for digital diagnostics

This Collection presents a series of articles describing annotated datasets of medical images and video. All medical specialities are considered and data can be derived from study participants, tissue samples, electronic health records (EHRs) or other sources.

Publishing Model: Open Access

Deadline: Dec 20, 2023

Ecological data for tracking biological diversity and environmental change

This collection presents data contributions addressing topics in biodiversity and ecology.

Publishing Model: Open Access

Deadline: Jan 31, 2024