In order to adapt to the request of apple grading technology, a classification method for apple grading was proposed based on hidden Markov model (HMM). Three different colors and shapes of apples were studied. The hexagonal pyramid model (HSV) was extracted as the color features of the apple, and the Hu invariant moment was extracted as shape features of the apple. These features of the apple were coded by Lloyd algorithm, which was used as the inputting of HMM. According to the HMM pattern recognition method, the different colors and shapes of apples were identified and classified, and then the apple grading was completed. The tests showed that the apple grading results were correct by the proposed method.