Objective: Improve the identification accuracy of banana ripeness. Methods: A novel method was established to identify banana ripeness based on CNN and XgBoost. Firstly, convolutional neural network was used to extract banana image features, and full-connected layer network and linear discriminant analysis were used to simplify banana image features. Then, the hyperparameters of the limit gradient lifting algorithm were optimized by Bayesian optimization algorithm. Finally, the simplified banana image features were input into the limit gradient lifting algorithm, and the banana ripeness was judged by the limit gradient lifting algorithm. Results: The identification accuracy of the method for banana ripeness was 91.25%. Compared with the existing methods, the proposed method was more accurate to distinguish the ripeness of bananas with small data volume. Conclusion: The proposed method can realize the accurate identification of banana ripeness, which is helpful for warehouse managers and exporters to monitor banana ripeness in real time.