Objective: Improve the accuracy and robustness of papaya ripeness detection. Methods: A method of papaya ripeness detection based on multi-target sampling and improved Mask R-CNN was proposed. In the process of data expansion, the method introduced multi-object sampling technology to generate enhanced images from small data sets taken under controlled conditions, which was conducive to extending the proposed method to data sets with complex features of actual papaya images. The effectiveness and robustness of the proposed method were verified by means of average accuracy, accuracy, accuracy-recall curve and calculation time, and the results of papaya ripeness detection effect were compared with those of Faster R-CNN, RetinaNet and CenterMask. Results: The values of mean awerage precision, 50% mean awerage precision and 75% mean awerage precision for the papaya ripeness detection were 98.43%, 98.67% and 98.68%, respectively. The average accuracies for the ripeness detection of immature, semi-mature and mature papayas were 99.38%, 98.81% and 99.37%, respectively. Conclusion: This method can be used to develop an electronic system for papaya ripeness detection and improve the performance of papaya ripeness detection and grading.