Abstract: Rib fractures are a prevalent consequence of thoracic trauma and typically identified through chest CT imaging. However, their accurate analysis is hindered by challenges such as small size, morphological variability, and low contrast with surrounding tissues. These factors complicate both the classification of fracture types and determination of healing stages in clinical and computational settings. To address these issues, we have proposed a novel multi-task learning framework designed to simultaneously perform fracture type classification and healing stage assessment. The multi-task learning approach is strategically chosen to leverage the intrinsic relatedness between the two tasks. By learning them jointly, the model develops more robust and generalizable feature representations, which is particularly advantageous given the common constraints of medical imaging data, such as limited sample sizes and class imbalance. Our methodology consists of two core components, each engineered to overcome specific challenges. For the task of fracture type classification (e.g., simple, displaced, comminuted), we designed a three-branch hierarchical multi-scale fusion model. This architecture explicitly extracts and integrates feature information from different hierarchical levels and spatial scales within the CT data. One branch captures fine-grained, high-resolution details essential for identifying subtle fracture lines. The second branch processes intermediate features for contextual information, whereas the third branch focuses on high-level semantic features. Then, a dedicated fusion module combines these multi-scale representations, significantly enhancing the model’s ability to discern subtle morphological differences between fracture types. For the more complex task of healing stage determination, we confronted obstacles such as limited longitudinal data, class imbalance across stages, and the need to model temporal dependencies. Our solution was a novel depthwise separable convolution-enhanced long short-term memory (DSC–xLSTM) network. This hybrid module first utilizes depthwise separable convolutions for efficient and powerful spatial-feature extraction from individual scans, reducing parameter count and overfitting risk. Then, these spatial features are processed by an LSTM network, which excels at modeling long-range sequential dependencies. This integration allows the model to perform a robust spatio-temporal analysis, learning the progression of healing from acute injury to bony union. The proposed model was rigorously evaluated using a substantial dataset of clinical rib fracture CT scans, annotated by expert radiologists for fracture type and healing stage. The results demonstrate the framework’s high efficacy. For fracture type classification, the model achieved exceptional accuracy and recall rates, both exceeding 0.95. Importantly, in the challenging task of healing stage determination, it also exhibited remarkable stability and accuracy, with both metrics consistently surpassing 0.91 across different healing phases. These results underscore the model’s capability to overcome data limitations and effectively learn the complex features required for accurate spatio-temporal analysis. In conclusion, this study introduces an effective multi-task learning framework that addresses key challenges in automated rib-fracture analysis. The proposed three-branch model for multi-scale feature fusion and innovative DSC–xLSTM network for spatio-temporal integration together enable state-of-the-art performance in classifying fracture types and determining healing stages. The model’s high accuracy and robustness highlight its potential as a valuable tool to support clinical decision-making and enhance diagnostic precision in managing rib fractures. Future studies need to focus on further validation with expanded datasets and steps toward clinical integration.