The Problem That Sparked Our Research

Every day, railway systems around the world transport billions of passengers and countless tons of freight. It's one of the safest forms of transportation we have — but when accidents happen, they can be catastrophic. Derailments, collisions with obstacles, and track obstructions continue to pose significant threats to railway operations globally.

Traditional methods for detecting track obstructions rely heavily on two things: manual track inspections and the vigilance of locomotive operators. But here's the challenge — human attention has limits. Fatigue sets in during long shifts. Visibility drops in fog, rain, or at night. A split-second distraction can mean the difference between spotting an obstacle in time or not.

We asked ourselves: What if we could give train operators an AI co-pilot that never gets tired and never looks away?

Our Approach: Teaching Computers to "See" Like Train Drivers

Our research, published in Discover Computing, introduces a visual intelligence framework that uses deep learning and computer vision to segment railway tracks in real-time and assess potential hazards based on their location.

Breaking Down the Track into Safety Zones

One of our key innovations was dividing the track area visible from the locomotive into four distinct regions, each representing a different threat level:

![Track Segmentation Zones Diagram] Figure: The four safety zones our system identifies in every frame

This classification scheme allows our system to not just detect that something is on or near the tracks, but to immediately assess how dangerous it is based on where it's located. An object in the "Major" zone requires immediate attention, while something in the "Left Minor" zone might be monitored but isn't an immediate threat.

Building Our Own Dataset: When No One Has Done This Before

One of the biggest challenges we faced was the lack of existing datasets specifically designed for railway track segmentation. While there are plenty of datasets for autonomous vehicles on roads, railway environments present unique challenges — different track types, platform configurations, curves, and infrastructure elements.

So we built our own.

We extracted frames from publicly available videos filmed from locomotive cab views, capturing diverse scenarios:

• Straight tracks and curved tracks (both left and right curves)
• Stations with platforms on one side, both sides, or no platforms
• Various lighting conditions and infrastructure configurations

Each of the 1,638 images in our dataset was meticulously annotated using polygon shapes to define the exact boundaries of each safety zone. We then used data augmentation techniques — including mosaic composition, horizontal flipping, and multi-scale scaling — to make our models more robust to real-world variations.

The YOLO Family: Our Detection Champions

For our detection engine, we turned to the YOLO (You Only Look Once) family of algorithms — renowned for their speed and accuracy in real-time object detection. We evaluated six different YOLO variants across three generations:

• YOLOv8: YOLO-n8 (nano) and YOLO-m8 (medium)
• YOLOv9: YOLO-c9 (compact) and YOLO-e9 (enhanced)
• YOLOv11: YOLO-n11 (nano) and YOLO-m11 (medium)

Why so many? Because different deployment scenarios have different requirements. A system running on a powerful cloud server can handle larger, more accurate models. But a system deployed directly on a locomotive's embedded computer needs to be lightweight while still reliable.

The Results: High Accuracy Across All Safety Zones

After training our models for 70 epochs on an NVIDIA A100 GPU, we achieved impressive results:

Best Performers: YOLO-m8 and YOLO-c9

• Mean Average Precision (mAP@0.5): 94%
• Precision: 97%
• Recall: 96%

Class-wise Accuracy (YOLO-c9):

• Major zone: 89.3%
• Front Minor: 87.9%
• Left Minor: 84.7%
• Right Minor: 83.5%

What's particularly encouraging is that the "Major" zone — representing the highest-threat area directly in the locomotive's path — achieved the highest accuracy. This is exactly where we need the system to perform best.

Watching the AI Learn

One of the fascinating aspects of our research was observing how the models improved over time. Early in training (around epoch 10), the models struggled with the "Major" class, achieving confidence scores as low as 0.76. But by epoch 70, this same class reached confidence scores of 0.97-0.98.

The learning curves told an interesting story: most models achieved about 90% of their final performance within just 20-30 epochs, with incremental refinements thereafter. This finding has practical implications — for rapid prototyping or resource-constrained deployments, shorter training periods might be acceptable.

Connecting to India's Railway Modernization

Our work aligns with real-world railway safety initiatives, particularly India's Gati Shakti Project and the deployment of Kavach — an Automatic Train Protection (ATP) system designed to prevent collisions.

Current ATP systems like Kavach use radio frequency-based communication for collision avoidance, but they haven't yet fully leveraged computer vision for enhanced situational awareness. Our framework provides a pathway for integrating visual intelligence into these existing systems, adding another layer of safety without replacing proven technologies.

Limitations and the Road Ahead

We're proud of our results, but we're also honest about our limitations:

1. Geographic Scope: Our dataset primarily covers specific regions and track types. Expanding to include diverse international railway systems would improve generalizability.
2. Weather Conditions: Further validation under adverse weather (heavy rain, fog, snow) and extreme lighting is needed.
3. Complete System Integration: Our current framework focuses on track segmentation. A complete safety system would need integration with obstacle detection and classification modules.
4. Field Testing: Extended trials under continuous operational conditions are required before real-world deployment.

What This Means for the Future of Railway Safety

Our research demonstrates that AI-powered visual intelligence can achieve the accuracy and speed needed for real-time railway safety applications. The ability to segment tracks into threat-level zones in milliseconds opens possibilities for:

• Real-time driver alerts when obstacles enter high-threat zones
• Reduced cognitive burden on locomotive operators during long shifts
• Complementary safety layer to existing ATP systems
• Data collection for improved track monitoring and maintenance

As railways continue to modernize and speeds increase, the need for intelligent, automated safety systems will only grow. We hope our framework provides a foundation for future developments in this critical area.

The Team Behind the Research

This research was conducted at the Center of Excellence in Signal and Image Processing at the Electronics and Telecommunication Department of COEP Technological University, Pune, India.

Authors:

• Yogesh Madhukar Gorane (Corresponding Author)
• Radhika D. Joshi

Published in Discover Computing (2025) 28:328

DOI: 10.1007/s10791-025-09872-z

Journal Badge: Discover Computing

Channel: Behind the Paper

Topics: #ComputerVision #MachineLearning #RailwaySafety #DeepLearning #YOLO #InstanceSegmentation #TransportationSafety #ArtificialIntelligence #IndianRailways #AutonomousSystems

The datasets generated during this study are available from the corresponding author upon reasonable request.

This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.