Objective: An automatic identification method of jujube defects based on improved convolution neural network was established. Methods: Using the dual branch convolution neural network structure, branch 1 combined with the transfer learning strategy for pre training, analysis 2 extracted the feature information from the jujube image based on the lightweight network fusion feature map. The superiority of this method was verified by comparative experiments. Results: Compared with the improvement before, the improved defect recognition method optimized the structure of the convolutional neural network, and the detection accuracy was further improved, from 96.02% to 99.50%. Conclusion: This method improved the network learning speed and convergence speed, and had good classification and recognition effect.