A Deep Learning Framework for Automated Railway Inspection of Track Environments

Zhang, Zelin

A Deep Learning Framework for Automated Railway Inspection of Track Environments

Files

zhang_zelin_mesc_2026_thesis.pdf (4.38 MB)

Date

2026-04-15

Authors

Zhang, Zelin

Abstract

Automated inspection of railway track defects is essential for improving safety and reducing maintenance costs in large-scale rail networks. This thesis proposes a practical two-stage vision-based inspection framework designed for real-world railway environments. In the first stage, object detection models are systematically evaluated for Fastener localization and region-of-interest (ROI) generation. Comparative experiments demonstrate that YOLO-based detectors outperform CNN-FPN and transformer-based approaches in both detection accuracy and computational efficiency. Consequently, YOLO v11m is selected to provide reliable and real-time ROI extraction. In the second stage, a multi-task learning network operates on the extracted ROIs to jointly predict slot-wise Fastener presence and tightness. Conditional supervision is introduced to align tightness prediction with physical Fastener existence. Experimental results show stable and consistent performance across slot-level metrics, validating the effectiveness of decoupling coarse localization from fine-grained structural inspection. The proposed framework offers a robust and deployable solution for automated railway Fastener inspection.

Summary for Lay Audience

Railways are a critical part of Canada’s transportation system, carrying millions of passengers and tons of freight every day. To keep trains running safely, railway tracks must be inspected regularly. One especially important component is the fastener - a small metal part that holds the rail firmly to the wooden or concrete ties underneath. If a fastener is missing or becomes loose, the track can shift, leading to rough rides or even serious accidents. Traditionally, inspecting fasteners is done by workers walking along the tracks and looking at each fastener by eye. This process is slow, tiring, expensive, and can miss problems - especially when lighting is poor or the worker is fatigued. This thesis explores how to automate fastener inspection using cameras and artificial intelligence (AI). A special railway vehicle equipped with downward facing cameras travels along the track and takes many images. Then, a computer program automatically analyzes those images to find fasteners and check whether they are present and properly tightened. The inspection is split into two simple steps. First, the program quickly finds where each fastener is located in the image. This step uses a type of AI that is very fast and accurate. Second, the program zooms in on each fastener and examines it more carefully. It decides, for each spike on the fastener, whether the spike exists and whether it is tight or loose. To make the second step more reliable, the program only checks tightness on spikes that actually exist - which matches how a human inspector would think. Through extensive testing, this two step method proved to be both accurate and fast enough to work in real time. It can reliably detect missing or loose fasteners under varying lighting and track conditions. By reducing the need for manual track inspection, this technology can help make Canada’s railways safer, lower maintenance costs, and allow more frequent and consistent inspections. In the long term, it supports smoother train rides, fewer unexpected track failures, and a more modern, data driven approach to keeping our railway infrastructure in good condition.