Objective: This study focuses on solving the problems of large memory consumption, low recognition accuracy and slow recognition speed in the classification process of peanut quality. Methods: A method for classification of peanut quality based on deep learning and image processing was proposed. The Coordinate Attention module was firstly introduced to encode the obtained feature graph into a pair of direction-aware and position-sensitive attention graph, which improved the ability to obtain the information of the region of interest of the graph. Then, Gradient Centralization was used to improve the optimizer. By modifying the parameters of the last fire layer and the convolution layer. An improved model, CG-SqueezeNet, was applied to peanut pod quality grading. Results: The classical convolutional network models VGG16, AlexNet, DenseNet121, ResNet50, Squeezenet were improved, and five different base classifier models were trained by transfer learning. By comparing with the classic model, it was found that the CG-SqueezeNet model could better learn the features of the region of interest in the image. The detection accuracy of the actual peanut pod image database was 97.83%, and the parameter memory was only 2.52 MB. Conclusion: The method is suitable for deployment on embedded resource-limited devices such as mobile terminals, which helps to realize real-time and accurate identification of peanut pod quality.