Using machine vision technology to detect the morphological position and quantity distribution information of underwater freely live crabs in the pond is a key step to realize the precise variable feeding for automatic feeding boats. However, it is very challenging to detect crabs quickly and reliably in the underwater images because of the large differences in posture, similarity joints of crabs, and the uneven color characteristics of the underwater images. For this reason, a real-time lightweight underwater live crab detector based on compound scaling convolutional neural networks was proposed. Firstly, aiming to the characteristics of underwater image blur and color imbalance, K-SVD denoising and Retinex color-correction enhancement were performed respectively according to the information composition characteristics with different frequencies after sparse decomposition. Secondly, a lightweight EfficientNet that perfectly coordinated the accuracy and efficiency by compound scaling the network width, depth and resolution was adopted as the backbone network. After that, a compound scaling factor was introduced to perform global overall compound scaling of the efficiently integrated feature network, which stacked two-layer weighted bi-directional feature pyramid structures, and the class/boundary-box prediction network that stacked three-layer convolution modules, to build a small-sized efficient detector for limited resources. In the class/boundary-box prediction network, an orthogonal Softmax layer was also adopted to replace the fully-connected classification layer to ensure that the detector can learn more distinguishing features from the small-sample dataset, which effectively alleviated the over-fitting problem of small-sample detection and improved the generalization ability of detectors. The model was trained and tested with self-built 20625 data samples. Experiments showed that images after denoising and correction were color-balanced and high-definition, and the detection Iou was increased by nearly 8 percentage points. The training detection model EfficientNet-Det0 can achieve 9621% precision and 9486% recall rate where only 15MB storage memory was needed;the detection delay of a single image was only 10.6ms (GPU) and 35.0ms (CPU). Compared with the YOLOv3 algorithm, FLOPs was reduced to 1/15, and operating speed of CPU was increased by 2 times, yet the accuracy was comparable to or even better than YOLOv3. The EfficientNet-Det0 proposed was suitable for applying on resource-restricted automatic feeding bait to quickly and accurately detect underwater live crabs, which could realize the statistics of the distribution for freely live crabs in ponds, and provide reliable decision information for establishing scientific feeding mechanism.