In this paper, a method based on a multi-scale convolutional neural network for detecting defects in jujube is proposed. Parallel multi-scale convolution modules were added to the AlexNet convolutional neural network to increase the depth and width of the network and reduce the parameters in the network; Added batch normalization processing to the convolutional layer to reduce changes in data distribution during training and improve the generalization ability of the network. Taking the yellow-skinned jujube, moldy jujube, broken-head jujube and normal jujube in Xinjiang dried jujube as the research objects, these dried jujubes were trained and verified. The results showed that the recognition rates of this model for yellow-skinned jujubes, moldy jujubes, broken-head jujubes and normal jujubes were 96.67%, 96.25%, 98.57%, and 97.14% respectively, and the comprehensive recognition rate could reach 97.14%. Compared with other algorithms, this algorithm was more robust and had higher accuracy in identifying defective red jujubes.