D455 RGBD data to Camera World Coordinate Conversion
I read the related articles but couldn't quite find the exact answer to my problem of getting (X, Y, Z) camera world coordinate for the corresponding (u,v) pixel value using the D455 camera. I looked through various solutions online mostly when they used to visualize as a point cloud but they directly assign depth as a Z-coordinate value.
I then tried using:
The code is using Python binding for SDK and color-depth alignment.
height, width, _ = color_image.shape
pixel_coordinates = np.indices((height, width)).reshape(2, -1).T
depth_values = depth_image[pixel_coordinates[:, 0], pixel_coordinates[:, 1]]
# Calculate the normalized coordinates
normalized_x = (pixel_coordinates[:, 1] - color_intrinsics.ppx) / color_intrinsics.fx
normalized_y = (pixel_coordinates[:, 0] - color_intrinsics.ppy) / color_intrinsics.fy
X = normalized_x*depth_values
Y = normalized_y * depth_values
Z = (depth_values**2 - X**2 - Y**2)**0.5
# Compute the (x, y, z) coordinates using matrix multiplication
xyz_coordinates = np.column_stack((X,Y,Z))
# Reshape the coordinates to match the color image dimensions
xyz_coordinates = xyz_coordinates.reshape(height, width, 3)
When I tried testing this approach, The Z-coordinate is not stable and values varies even when I test its value along the XY plane where it is supposed to be constant.
Can I get help on getting exact (x,y,z) camera coordinates of the given pixel(u,v) using the D455 camera and Pyrealsense SDK 2.0?
-
Hi Pandeymanoj As you have already developed a script but the main problem with it is instability of depth values, you could try stabilizing their fluctuation by defining a post-processing Temporal Filter and giving its Smooth Alpha setting a very low value of '0' or '0.1'. A Python script for doing so can be found at the link below.
-
Thank you @MartyG for the possible solution. After reading the real sense white paper for the D400 series, I found out that I misunderstood the depth value firsthand. The depth value we get is Z-coordinate from the imager plane to the 3D point of the corresponding pixel. Again, Thank you for the suggestions for smoothing.
Please sign in to leave a comment.
Comments
3 comments