Application of Robotic Arm Control using Computer Vision and Object Detection Techniques

Sarah Vyvyan

Engineering

Eli Buckner

The increased technological advancements in and utilization of robotic systems have greatly improved efficiency and safety in many areas of society, from industrial and manufacturing environments to medicine and healthcare. These robotic systems are often used to recognize or search for certain objects and perform actions on them, such as finding an object on a conveyor belt and moving it to a new location. Embedded cameras and computer vision systems are often implemented in this object detection and recognition process. This project demonstrates this concept by having a 4-jointed robotic arm with an embedded RGB D camera to recognize an object, specifically a human hand in front of a white backdrop, and determine its 3D location in space. The process of determining the object's location implements a simple threshold-based image segmentation method as well as uses the depth image provided by the embedded camera. Once a 3D location of the hand is determined, the arm will move the end effector to that location and follow the hand. This is done by controlling the angles of rotation of the arm’s 4 joints through numerical inverse kinematics. The image segmentation method was tested on 113 different images of hands with solid white backgrounds. This method achieved a 98.2),, accuracy in detecting a hand and a 95.5% accuracy in locating the center of mass within the actual boundaries of the hand. The inverse kinematics successfully moved the end effector of the robot within range of grabbing the hand 100% of the time and successfully moved it within 5 centimeters of the hand 70% of the time.

Jan-9-2024