To solve the problem of weak generalization caused by diversity of source of shrimp samples, a novel shrimp features difference model based on shannon information theory was proposed. The model was actually a recognition framework, calculating hyper-parameters based on deep convolutional neural network (DCNN) using entropy reduction rule with multi-source datasets. This rule can clear up the special information entropy from the random input to regular output, breaking the data types changing from three dimensional input to one-dimensional output, realizing dimensionality reduction of shrimp image reducing from high dimension space to low dimensional space. Thus, the DCNN adaptive optimization strategies can be acquired to improve the generlization effectiveness of recognizing diseased shrimp from multiple sources. The experimental results showed that the proposed method in a single dataset can achieve highest accuracy of 97.96%. The generalization experiment was also tested through other four shrimp image datasets, and the generalization precision falling scope was no more than 5 percentage points.