Objective: To solve the problems of complex structure and large calculation amount of the visual servo control system of the current food sorting robot, which unable to meet the flexibility and adaptability of the sorting robot to the visual servo control system. Methods: Based on the structure of robot visual servo control system, a visual servo control method of food sorting robot based on improved particle swarm optimization algorithm and BP neural network was proposed. The particle swarm algorithm used crossover and mutation in the iterative process to maintain population diversity, the initial weight and threshold of BP neural network were optimized. Results: Compared with conventional control methods, the control method could bring the food production line robot to the predetermined position in a short time, the relative error of position approximation was 0.38%. Conclusion: When dealing with more complicated tasks, with strong adaptability, it has certain practical value.