Objective: In order to expand the scope of cherry sales and achieve rapid grading of cherries under industrial conditions. Methods: Firstly, the YOLOX network was used to detect the defective fruit, in order to solve some problems where the defect was not obvious. The detection accuracy of the inconspicuous defect was improved by setting the appropriate fusion factor for the feature pyramid network, and in order to solve the problem of imbalance between various types of real samples, Focal Loss was integrated into the loss function. Then, the intact fruit was graded using the YOLOX network, and the attention mechanism CBAM was introduced to enhance the network feature extraction. Results: Experimental results showed that 97.59% of the mAP detected for cherry surface defects and 95.92% of the mAP of size and color grading. Conclusion: The accuracy of cherry defects and grading has been significantly improved by the improved YOLOX network.