Objective: To identify mouldy peanuts in a fast and non-destructive way and improve the identification efficiency. Methods: Collected hyperspectral peanut data using a spectrometer, identify moldy peanuts using deep learning technology, and established a Hypernet PRMF model, which was compared with Deeplab v3+, Segnet, Unet, and Hypernet as control models. Integrated the proposed peanut recognition index into hyperspectral images as data feature pre extraction. Simultaneously integrating the constructed multi feature fusion blocks into the control model to improve the recognition efficiency of moldy peanuts. Results: The average pixel accuracy of all models exceeded 87%. the Hypernet-PRMF model had the highest detection accuracy of 90.35%, while for the whole peanut dataset, Hypernet-PRMF had a low false recognition rate and could effectively identify all mouldy peanuts in the figure. Conclusion: The Hypernet-PRMF model built based on deep learning has high pixel accuracy and detection precision, which can effectively identify mouldy peanuts and provide a reference basis for the identification and detection of other mouldy food and other hyperspectral objects.