[Objective] To meet the practical requirements for comprehensive grading based on the appearance quality and size of apples, and to address issues such as low efficiency of manual sorting, complex structure, and high cost of sorting equipment for Chinese apples. [Methods] A YOLOv5s-apple model was proposed. The transformer module and CBAM attention module were introduced into the backbone network, and the weighted Bidirectional feature pyramid network (Bi-FPN) was added to improve the neck network. Then, combined with HALCON software, a self-designed intelligent apple damage detection system was used to carry out damage sorting and size classification. [Results] The experimental results showed that compared with the original YOLOv5s model, the mAP of the YOLOv5s-Apple model was improved by 6.2%, and the accuracy of apple sorting system could reach 97.5%, the processing speed of the system was 5 s/apple. [Conclusion] The system can effectively carry out apple grading and sorting, and provide a reference for the intellectualization and low cost of Apple detection equipment.