Aiming at the tracking problem of elastic deformable objects, a control method based on RGB-D sensor of pizza chef robot was proposed in this study. An in-depth study on the difficult problem of dough tracking and recognition in pizza making. A method to track 3D textured objects with elastic deformation based on real-time point cloud data provided by RGB-D sensor was also proposed. Firstly, the Grabcut in the visual segmentation was described in detail, and the temporal consistency and real time of the visual segmentation were emphatically analyzed. Secondly, the FEM model with contour weighting was established by finite element analysis. Finally, the method of registration point cloud data was discussed in detail to realize the tracking and recognition of deformable objects. In order to evaluate the proposed method, the validity of the proposed algorithm was verified by using the simulated open frame architecture (SOFA) simulator. The research results showed a positive significance for the quick response of pizza chef robot in food cooking process.