Recently, the interaction between humans and robots has become an important issue for extending the use of robots. The purpose of this project is to show how a real-time human-robot interaction with hand gesture recognition which can be constructed using color recognition System. Hence, it is required that the following two major questions be resolved in this study. Firstly, either staying in indoor or outdoor lighting conditions, the robot must be able to detect and spot hand gestures with a specific color from data of robot vision system. Secondly, it is necessary that the robot also can interpret hand gestures performed by humans.