Helformer: an attention-based deep learning model for cryptocurrency price forecasting
Published in Mathematical & Computational Engineering Applications, Statistics, and Business & Management
🔍 Behind the Scenes: Methodology & Innovation
1. The Helformer Architecture
Helformer integrates three key components to outperform existing models:
-
Series Decomposition: Using Holt-Winters smoothing, we break price data into level, trend, and seasonality components (Fig. 1). This step isolates patterns that traditional Transformers might miss.
-
Multi-Head Attention: Unlike sequential models (e.g., LSTM), Helformer processes all time steps simultaneously, capturing long-range dependencies efficiently.
-
LSTM-Enhanced Encoder: Replacing the standard Feed-Forward Network with an LSTM layer improves temporal feature extraction.
Fig. 1: Helformer architecture.
2. Data & Hyperparameter Tuning
We trained Helformer on Bitcoin (BTC) daily closing prices (2017–2024) and tested its generalization on 15 other cryptocurrencies (e.g., ETH, SOL). To optimize performance, we used Bayesian optimization via Optuna, automating hyperparameter selection (e.g., learning rate, dropout) and pruning underperforming trials early.
3. Evaluation Metrics
Helformer was benchmarked against RNN, LSTM, GRU, and vanilla Transformer models using:
-
Similarity metrics: R², Kling-Gupta Efficiency (KGE), EVS
-
Error metrics: RMSE, MAPE, MAE
-
Trading metrics: Sharpe Ratio, Maximum Drawdown, Volatility, Cumulative returns
💡 Key Findings & Practical Impact
1. Superior Predictive Accuracy
Helformer achieved near-perfect R² (1.0) and MAPE (0.0148%) on BTC test data, outperforming all baseline models (Table 1). Its decomposition step reduced errors by 98% compared to vanilla Transformers.
Table 1: Model Performance Comparison
|
Model |
RMSE |
MAPE |
MAE |
R² |
EVS |
KGE |
|
RNN |
1153.1877 |
1.9122% |
765.7482 |
0.9950 |
0.9951 |
0.9905 |
|
LSTM |
1171.6701 |
1.7681% |
737.1088 |
0.9948 |
0.9949 |
0.9815 |
|
BiLSTM |
1140.4627 |
1.9514% |
766.7234 |
0.9951 |
0.9952 |
0.9901 |
|
GRU |
1151.1653 |
1.7500% |
724.5279 |
0.9950 |
0.9950 |
0.9878 |
|
Transformer |
1218.5600 |
1.9631% |
799.6003 |
0.9944 |
0.9946 |
0.9902 |
|
Helformer |
7.7534 |
0.0148% |
5.9252 |
1 |
1 |
0.9998 |
2. Profitable Trading Strategies
In backtests, a Helformer-based trading strategy yielded 925% excess returns for BTC—tripling the Buy & Hold strategy’s returns (277%)—with lower volatility (Sharpe Ratio: 18.06 vs. 1.85), as shown in Fig. 2.
Fig. 2: Trading results.
3. Cross-Currency Generalization
Helformer’s pre-trained BTC weights transferred seamlessly to other cryptocurrencies, achieving R² > 0.99 for XRP and TRX. This suggests broad applicability without retraining—a boon for investors managing diverse portfolios.
🌍 Relevance to the Community
-
For Researchers: Helformer’s architecture opens avenues for hybrid time-series models in finance, healthcare, and climate forecasting.
-
For Practitioners: The model’s interpretable components (decomposition + attention) make it adaptable to volatile markets beyond crypto.
-
For Policymakers: Reliable price forecasts could inform regulations to stabilize crypto markets and protect investors.
🤝 Acknowledgments & Open Questions
This work wouldn’t have been possible without my brilliant co-authors Oluyinka Adedokun, Joseph Akpan, Morenikeji Kareem, Hammed Akano, and Oludolapo Olanrewaju, or the support of The Hong Kong Polytechnic University.
We’d love to hear your thoughts!
-
How might Helformer adapt to non-financial time-series data?
-
Could integrating sentiment analysis further improve accuracy?
-
What ethical considerations arise with AI-driven trading?
🔗 Access the full paper: SpringerLink | ReadCube
I am Dr. Temitope Kehinde, a dedicated researcher, educator, and consultant with a solid background in Industrial & Systems Engineering. Currently, I serve as a Postdoctoral Fellow at The Hong Kong Polytechnic University, where I am working under the Hong Kong Government ITF project grant titled "Increasing First-dye Accuracy of Dyeing Processing by Artificial Intelligence" (Project account code K-45-35-ZWEF). This multimillion-dollar grant, fully funded by the Innovation and Technology Fund (ITF), is a testament to my expertise and contribution to the field. I am fortunate to work under the supervision of Prof. Sai-Ho Chung, conducting cutting-edge research aimed at improving dyeing processing accuracy using artificial intelligence.
My academic journey began at the University of Ibadan, Nigeria, where I earned a Bachelor’s degree in Industrial & Production Engineering with First Class Honours. This academic excellence was further complemented by my Master’s degree in Industrial & Production Engineering (Engineering Management option), which I completed with Distinction. My educational background laid a strong foundation for my current research, which integrates advanced technologies such as machine learning, data envelopment analysis (DEA), and multi-criteria decision-making (MCDM) techniques for real-world applications like smart production systems, smart manufacturing, financial modeling, and portfolio optimization.
In 2021, I began my Ph.D. in Industrial & Systems Engineering at The Hong Kong Polytechnic University, where I developed my research expertise in areas such as smart manufacturing, data-driven scheduling, and forecasting. My role as a Research and Teaching Assistant enabled me to contribute to both academia and industry, helping shape the next generation of engineers. In recognition of my dedication to teaching, I was honored with the Best Teaching Assistant Award in 2022.
In addition to my academic pursuits, I have gained valuable professional experience. I served as a Graduate Assistant at Zenith Bank Plc in Nigeria, gaining hands-on experience in financial services, including treasury management and asset management. I also work as a consultant for Pathway Education Holdings Limited, where I assist in recruiting African students for international study programs, bridging educational services and global opportunities.
I have presented my research at several international conferences, including the Computing Conference in London, the International Conference on Smart Mobility and Vehicle Electrification in Detroit, and the PolyU RS Conference in Hong Kong. My work focuses on leveraging machine learning and deep learning to enhance forecasting accuracy, particularly in volatile markets like cryptocurrency and stock trading.
I have been fortunate to receive several accolades, including the Hong Kong PhD Fellowship Scheme and my involvement in community-focused initiatives such as Smile4Me Initiative, where I serve as the founder and chairman. My work in both academia and the industry is driven by a passion for using data-driven solutions to solve complex, real-world problems.
As I continue to work on the Hong Kong Government ITF project and collaborate with industry partners, my goal is to further develop smart manufacturing solutions and contribute to the academic and professional communities. My future aspirations include advancing research in the intersection of sustainability, technology, and economic resilience, ultimately benefiting both business operations and societal well-being.
Follow the Topic
Related Collections
With Collections, you can get published faster and increase your visibility.
2025 Australasian Data Science and Machine Learning (AusDM25) Special Issue
The Australasian Data Science and Machine Learning (AusDM) Conference has firmly established itself as the premier Australasian meeting point for both practitioners and researchers in Data Science, Data Analytics, Data Mining, and Machine Learning—including cutting-edge areas such as Deep Learning and Generative AI. Since its inception in 2002, the AusDM conference series has provided a leading forum for presenting, disseminating, and discussing the latest research advances spanning algorithms, software, systems, and practical applications across diverse industries.
Continuing this tradition, AusDM 2025 aims to foster cross-disciplinary knowledge exchange, highlight emerging research directions, and promote collaboration across academia, government, and industry.
This Special Issue cordially invites authors of accepted AusDM 2025 conference papers to submit extended versions of their work. Topics of interest include, but are not limited to, the following areas of Data Science and Machine Learning:
Foundational Techniques in Machine Learning and AI
Supervised, unsupervised, semi-supervised, and self-supervised learning Deep learning and representation learning Reinforcement learning and federated learning Transfer learning, meta-learning, few-shot and continual learning Multitask and multimodal learning Generative models (GANs, diffusion models) Large Language Models (LLMs) and Large Multimodal Models (LMMs) Zero-shot learning and prompt-based learning
Learning from Diverse and Complex Data
Analytics over structured, semi-structured, and unstructured data Text, time-series, graph, spatial, spatio-temporal, and network data Web, social media, multimedia, IoT, and sensor data Sequential, temporal, and dynamic data modelling
Data-Centric AI and Data Engineering
Data preprocessing, cleaning, integration, matching, and linkage Privacy-preserving and secure data mining Data-centric AI pipelines and dataset curation Computational aspects of data mining and large-scale data management
Scalable and Real-Time Data Analytics
Big data analytics and scalable Machine Learning Parallel and distributed learning algorithms Data stream mining and real-time analytics Edge, cloud, and IoT-enabled Machine Learning systems
Interactive and Visual Analytics
Visual analytics and explainability through visualisation Human-in-the-loop machine learning Interactive data exploration and decision support
Responsible, Causal, and Explainable AI
Explainable and interpretable Machine Learning Fairness, accountability, transparency, and ethics in AI Causal inference and causal machine learning Robustness, generalization, and uncertainty quantification
Applied Data Science and Machine Learning Across Domains
Applications in business, finance, education, agriculture, urban planning, healthcare, sports, social sciences, cybersecurity, arts, and humanities Domain-specific AI systems in biomedical informatics, environmental science, astronomy, engineering, and beyond Industrial case studies and data-driven product innovation
Submission Requirements
Submitted journal manuscripts must include at least 30% new content beyond the conference version. All submissions will undergo a rigorous peer-review process in accordance with the journal’s standards.Publishing Model: Open Access
Deadline: Apr 30, 2026
Big Data and Data-driven in Sports
Journal of Big Data is calling for submissions to our Collection on Big Data and Data-driven in Sports. Over the past few decades, interest in applying statistical analysis and modeling techniques to sports has been constantly growing. This trend is evident from the increasing body of scientific research and the numerous published works that provide valuable statistical insights across various sports, such as soccer, tennis, American football, baseball, and basketball, among others. This rising interest has been accompanied by a massive expansion in the availability of data, including the rise of big data, which offers a vast and detailed array of information that enhances the depth and accuracy of sports analytics.
We invite researchers to submit articles focused on advancing statistical and machine learning methods encompassing models, algorithms, and multivariate exploratory techniques in the realm of big data sports analytics. We seek contributions that introduce novel methodologies and practical techniques, showcasing significant advancements, extensions, and applications within this context. Submissions should emphasize innovative, complex, or scientifically compelling aspects of statistical analysis across a broad spectrum of sports, including both professional and amateur disciplines.
Topics:
The goal of this special issue is to collect articles that explore a variety of topics, including (but not limited to):
• Models and algorithms for forecasting game outcomes
• Methods for assessing and measuring teams’, players’ and athletes’ performance
• Implementation of optimal game strategies
• Methods to deal with players’ tracking data
• Analysis of the impact of vital parameters to athletes’ performance
• Exploitation of training data collected with sensors, wearables or other technologic equipment
• Data analysis problems associated with massive, complex datasets in sports
• Novel statistical approaches and data mining methods in sports
• Comparing and contrasting techniques for solving research questions in sports
• The role of social media and public sentiment analysis in sports analytics
• Economic impact analysis of sports events using big data
This Collection welcomes submission of original Research Articles. Should you wish to submit a different article type, please read the submission guidelines to confirm that type is accepted by the journal you are submitting to. Articles for this Collection should be submitted via our submission system, Snapp. During the submission process you will be asked whether you are submitting to a Collection, please select "Big Data and Data-driven in Sports" from the dropdown menu.
Articles will undergo the standard peer-review process of the journal they are considered in the Journal of Big Data and are subject to all of the journal’s standard policies. Articles will be added to the Collection as they are published.
The Editors have no competing interests with the submissions which they handle through the peer review process. The peer review of any submissions for which the Editors have competing interests is handled by another Editorial Board Member who has no competing interests.
Publishing Model: Open Access
Deadline: Dec 31, 2025
