Object Recognition Using the D435i
We are currently working on a visual recognition project, using the realsense D435i camera. During the recognition, we found that the hand-eye calibration was inaccurate. Currently we believe that the reason is that the angle of the camera is too biased, causing the center position of the pixel of the object in the image to differ from the three-dimensional shape of the object in the real world. If the center deviation is too large, the three-dimensional coordinates obtained by the camera will not be accurate, so the calibration results will also be inaccurate.
The previous zed camera library function itself integrated a method of realizing three-dimensional target detection by combining the target detection algorithm, including a method of obtaining the three-dimensional bounding box of the target from the scene point cloud. We can directly read the three-dimensional coordinates of the centroid of the target, so The three-dimensional coordinates and calibration results in the robotic arm will be more accurate. Can the realsense camera provide corresponding library functions for identification?
-
Hi, When you say that the angle of your camera is biased, do you mean that it is tilted strongly up or down? It is fine to tilt RealSense 400 Series cameras up to 30 degrees in the up or down direction, but beyond that angle (such as 40 degrees or more), problems may manifest in the depth sensing. The camera can be pointed 90 degrees up or down without problems though.
The RealSense SDK does not have built-in object detection. A third-party tool called YOLO is commonly used with RealSense cameras for that application.
-
Sorry, I may not have described it clearly.
At present, we cannot accurately identify the centroid of the object through the yolo algorithm. It may be because of the perspective problem that the recognized centroid of the bounding box is not the real centroid of the object. Can you provide a method to calculate the centroid of the object through the three-dimensional bounding box. -
Typically when a 3D bounding box needs to be generated, I recommend that YOLO is used in conjunction with darknet_ros_3d.
darknet_ros
https://github.com/leggedrobotics/darknet_ros
darknet_ros_3d
https://github.com/IntelligentRoboticsLabs/gb_visual_detection_3d
If you already have a 3D bounding box and believe that inaccurate hand-eye calibration may be the issue then you could explore the RealSense-compatible hand-eye calibration methods highlighted in the link below.
https://github.com/IntelRealSense/librealsense/issues/11736#issuecomment-1525124566
Please sign in to leave a comment.
Comments
3 comments