The classification probability output based on machine vision and near infrared spectroscopy technology was established, and the classification decision model suitable for heterogeneous data was built by using DS evidence fusion rules. The directional gradient histogram and principal component extraction method were used to extract spectral features, and support vector machine and AdaBoost classifier were used for recognition. On this basis, the potato classification model based on feature layer fusion was constructed. By using multi-source information fusion technology, a multi-source information fusion agricultural product quality identification model was established, which integrated the classification decision of nondestructive testing and feature level fusion. The simulation results showed that, compared with the single identification model, the recognition rate of multi-source information fusion identification model was improved by 12.7%~30.2%, and reached more over 95.7%.