Attention-Driven CNN–LSTM Framework for Multi-Behavior Recognition in Underwater Aquaculture Systems

Abstract

Automated recognition of fish behavior is essential for advancing sustainable aquaculture by enabling continuous monitoring of fish welfare, feeding patterns, and stress indicators. Conventional approaches, reliant on manual observation or static vision models, lack temporal modeling capabilities and fail under complex underwater conditions. This study introduces an attention-driven deep learning framework that integrates spatial and temporal cues for real-time multi-behavior recognition from underwater video streams. The proposed architecture employs a dual-stream approach comprising a ResNet-18-based spatial encoder and an optical flow-guided motion stream, whose outputs are fused and processed by a bi-directional Long Short-Term Memory (Bi-LSTM) network enhanced with attention mechanisms to capture sequential dependencies. A custom-labeled dataset encompassing eight fish behavior classes was developed, incorporating environmental diversity such as turbidity, lighting variations, and overlapping fish. The model was optimized for edge deployment through quantization and pruning, achieving a compact footprint suitable for the NVIDIA Jetson Nano platform. Experimental evaluation demonstrates a classification accuracy of 92.4% and a macro-F1 score of 91.1%, with a sustained inference rate of 28 FPS and power consumption limited to 4.9 W. The system outperforms baseline models including YOLOv7-tiny and 3D-ResNet18, affirming its suitability for real-time, resource-efficient aquaculture monitoring. These findings highlight the potential of attention-driven spatial–temporal models in enabling intelligent, scalable, and low-power behavior analysis in real-world aquaculture environments.