Sea cucumber object detection is the premise of realizing automatic fishing of sea cucumber. To solve the problem of missed object detection caused by occlusion and the color similarity between object and background in the complex seabed environment, Swin RCNN object detection algorithm was proposed under the framework of Faster R-CNN. The backbone network of the algorithm adopted the Swin Transformer, and the multi-dimensional feature extraction layer was integrated into the structure, which improved the adaptive feature fusion ability of the algorithm and improved the object recognition ability of the model for the different sizes of objects under occlusion in complex environments. The actual experimental results showed that the mean average precision achieved 94.47% for the detection of sea cucumbers by the proposed approach, which was increased by 4.49 percentage points, 4.56 percentage points, 4.46 percentage points, 11.78 percentage points, and 22.07 percentage points compared with Faster R-CNN, SSD, YOLO v5, YOLO v4, and YOLO v3, respectively. The research result had certain reference significance for object detection in other complex environments. Therefore, the study of sea cucumber object detection algorithm in complex seabed environment had important theoretical and application value, and also had guiding significance for intelligent identification of other marine products.