Objective To propose a deep learning-based method for the freshness determination of winter jujube by dividing the fruit into five freshness stages, aiming to improve determination accuracy and reduce the influence of light reflection.Methods In this study, a freshness determination method is proposed for winter jujube by combining an efficient ResNet, an attention mechanism, and Faster R-CNN. First, ResNet is used for convolutional processing on the image to extract the global feature map. Next, key features are enhanced through a channel attention module, and multi-scale features are extracted using a feature pyramid network (FPN). Then, Faster R-CNN selects candidate regions from the features, followed by region of interest (ROI) pooling before inputting to fully connected layers. Therefore, the model performance is optimized through a multi-angle loss function. The model’s effectiveness is validated using physicochemical indicators such as hardness, conductivity, as well as vitamin C (VC) and polyphenol content.Results In freshness determination, the improved Faster R-CNN model achieves an accuracy of 98.60%.Conclusion The improved Faster R-CNN model outperforms existing methods in small-scale samples.