Crop identification and classification, as a fundamental part of modern agriculture, is an important prerequisite for ensuring regional food security and sustainable agricultural development. With the development of remote sensing technology, high-resolution visible light remote sensing imagery has become a convenient and reliable source of remote sensing data. At present, the use of these remote sensing data to carry out detailed classification research of typical crops has extremely important practical significance. However, these remote sensing images lack enough spectral information and therefore are unable to give high-accuracy recognition of the crops, it is necessary to apply deep learning techniques on the crops classification research to solve these problems. Based on composite wing unmanned aerial vehicle which equipped a remote sensing imaging equipment to obtain remote sensing data, including cotton, corn and squash, covering an area of over 867hm2. According to the characteristics of the data, the convolutional neural network (CNN) was designed to extract the crop classification information. By adjusting the network parameters and the sample spectral combination, the parameter optimization of the training process was divided into two levels: at the level of network parameters, the adjustment included the learning rate and the batch, the scale of the convolution kernel and the depth of the network;at the spectral feature level of the sample, three types of samples included single-band, dual-band and triple-band were constructed as inputs, and the model was trained in turn. The experimental results showed that the CNN can effectively extract the crop information in the image and realize the fine classification of crops. Most sensing areas had qualified classification result except the edge places planted with sparse and mixed crops. The overall classification accuracy achieved 97.75% by using the optimized CNN. Compared with the support vector machine based on radial basis kernel function and the back propagation neural network, the optimized CNN had the best effect and the highest classification accuracy. It was worthy of noting that the adjustment of network parameters would affect the final training effect. A CNN model with large learning rate (0.1), small convolutional kernel (7×7) and appropriate depth (7) was advised on the basis that the typical crops in the remote sensing images were with high density, small features and rich textures. This promised the small features in the sample would not be missed when the convergence of the training accuracy was increased. Samples with different spectral features also had an impact on the training of the model. The blue band in the visible light was more appropriate than the green and red ones on training the CNN for crops recognition. A combination of the three bands would improve the recognition accuracy and stability, but more training time was required since more input information was given. The experiments demonstrated the effectiveness and reliability of the proposed CNN on crops fine classification. This method can be regarded as a reference for the application of deep learning in agricultural remote sensing.