In the program, I was trying to merge point clouds taken from two D435 cameras.
But there is miss alignment in the merged point cloud.
In the code,
the program uses the Charuco board to find a transfer matrix for each camera to the world coordinate.
cv::aruco::detectMarkers() is called first to detect corners and markers.
cv::aruco::interpolateCornersCharuco() is called to interpolate corners and markers.
cv::aruco::estimatePoseCharucoBoard() is called to get rotation vector and translation vector, which are all 3x1 vectors.
cv::Rodrigues() is used to find rotation matrix 3x3. Then combined with the translation vector to get the transfer matrix 4x4.
RS2_STREAM_DEPTH is aligned to RS2_STREAM_COLOR for each camera using the align.process() function.
First, I thought it's the viewport issue since RGB image and depth image are taken from different viewports.
So I used the function get_extrinsics_to() to get extrinsic from RS2_STREAM_DEPTH to RS2_STREAM_COLOR or STREAM_COLOR to STREAM_DEPTH.
And convert extrinsics.rotation and extrinsics.translation to a 4x4 matrix.
I multiply the transfer matrix with the extrinsic matrix together to test.
But miss alignment still occurs,
Can someone help me, please?
Please sign in to leave a comment.