Objective: In order to improve the accuracy of machine vision technology in grading the appearance quality of red Fuji apples, a red Fuji apple appearance grading method based on improved whale optimization algorithm （WOA） and CNN is proposed. Methods: A red Fuji apple image database with different appearance quality levels was established, and the database images were preprocessed so as to improve the training effect and generalization ability of the model. The improved CNN-LSTM was designed as the weighted grey correlation method was used to compress the CNN convolution scale, in order to reduce redundant interference between features and improve the computational speed of the model. The improved whale optimization algorithm was used to optimize the hyperparameters configuration of CNN-LSTM, effectively reducing the impact of improper hyperparameter configuration on model classification results. Results: The simulation results showed that the proposed classification method had a higher accuracy, with classification accuracy and sensitivity improved by about 2.05% and 2.46%. Conclusion: The proposed method can effectively achieve the appearance grading of red Fuji apples.