In the automatic assembly line of agricultural machinery, the on-chip resources of its embedded control platform are extremely limited, and the parameter amount of the convolutional neural network-based deep learning detection system is too large, which is difficult to be directly transplanted to the embedded platform. Therefore, a deep learning real-time detection method based on improved ResNet18-SSD (single shot multi-box detector) and field programmable gate array (FPGA) acceleration engine was proposed. In order to improve the accuracy of the detection model while reducing the number of parameters, a deep learning fast detection model based on ResNet18-SSD was proposed, which utilized the optimized and improved ResNet18 network to replace the VGG16 predecessor network of the SSD model, and introduced a multi-branch isomorphic structure and an asymmetric parallel residual structure, so as to adapt to the complex scenes such as occlusion, dim light; and in the case of meeting the detection accuracy requirements, a dynamic fixed-variance network was used to meet the detection accuracy requirements. Under the condition of meeting the requirements of detection accuracy, the dynamic fixed-point quantization was adopted to reduce the model data volume to improve the execution efficiency of the detection model. Aiming at improving the convolutional layer in the ResNet18-SSD model, which consumed serious resources, an FPGA acceleration engine based on the Winograd algorithm was proposed to improve the real-time performance of the model detection, and through the software-hardware co-design, joint optimization was carried out from the perspectives of the hardware gas pedal and the lightweighting of the software network, so as to achieve a balance between the lightweighting, acceleration performance, and accuracy in the complex scene. Experimental results on the Xilinx FPGA embedded platform showed that the detection accuracy of the proposed method reached 93.5%, and the detection time of a single image under the operating frequency of 100MHz was 80.232ms, which met the real-time demand.