Instrinsics and alignement
Hello,
I'd like to have the confirmation that when aligning frames, i.e:
frames = pipeline.wait_for_frames();
frames = rs2::align(RS2_STREAM_COLOR).process(frames);
then resulting individual frame intrinsics are also aligned,
auto profile = frame.get_profile();
auto stream_profile = profile.as<rs2::video_stream_profile>();
return stream_profile.get_intrinsics();
This question because in my test context, I'm trying to convert vertices calculated from the depth frame using a deprojection:
rs2_deproject_pixel_to_point(vertex, &intrinsics, pixel, depth);
Then to convert the result in another space defined by three ArUco markers (the markers give origin, x, y and z = x cross y.
But the result is not as expected: the floor on which the markers are placed on does not appear as a horizontal plane. Things a slightly rotated and shifted.
Do you have any indication about that? Is it effectively due to the align/intrinsics? Can that be due to ArUco accuracy? Or am I doing wrong in my approach?
Thanks.
-
Hello Christian Hubert,
Yes, the align-to-color process sets depth stream intrinsics = color stream intrinsics. This can be verified with this code (modified rs-align.cpp):
int main(int argc, char * argv[]) try
{
// Create and initialize GUI related objects
window app(1280, 720, "RealSense Align Example"); // Simple window handling
ImGui_ImplGlfw_Init(app, false); // ImGui library intializition
rs2::colorizer c; // Helper to colorize depth images
texture depth_image, color_image; // Helpers for renderig images
// Create a pipeline to easily configure and start the camera
rs2::pipeline pipe;
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_DEPTH);
cfg.enable_stream(RS2_STREAM_COLOR);
pipe.start(cfg);
// Define two align objects. One will be used to align
// to depth viewport and the other to color.
// Creating align object is an expensive operation
// that should not be performed in the main loop
rs2::align align_to_depth(RS2_STREAM_DEPTH);
rs2::align align_to_color(RS2_STREAM_COLOR);
float alpha = 0.5f; // Transparancy coefficient
direction dir = direction::to_color; // Alignment direction
while (app) // Application still alive?
{
// Using the align object, we block the application until a frameset is available
rs2::frameset frameset = pipe.wait_for_frames();
if (dir == direction::to_depth)
{
// Align all frames to depth viewport
frameset = align_to_depth.process(frameset);
}
else
{
// Align all frames to color viewport
frameset = align_to_color.process(frameset);
}
// With the aligned frameset we proceed as usual
auto depth = frameset.get_depth_frame();
auto color = frameset.get_color_frame();
auto colorized_depth = c.colorize(depth);
auto depth_profile = depth.get_profile();
auto depth_stream_profile = depth_profile.as<rs2::video_stream_profile>();
auto depth_intrin = depth_stream_profile.get_intrinsics();
auto principal_point = std::make_pair(depth_intrin.ppx, depth_intrin.ppy);
auto focal_length = std::make_pair(depth_intrin.fx, depth_intrin.fy);
rs2_distortion model = depth_intrin.model;
std::cout << "Depth Principal Point : " << principal_point.first << ", " << principal_point.second << std::endl;
std::cout << "Depth Focal Length : " << focal_length.first << ", " << focal_length.second << std::endl;
std::cout << "Depth Distortion Model : " << model << std::endl;
std::cout << "Depth Distortion Coefficients : [" << depth_intrin.coeffs[0] << "," << depth_intrin.coeffs[1] << "," <<
depth_intrin.coeffs[2] << "," << depth_intrin.coeffs[3] << "," << depth_intrin.coeffs[4] << "]" << std::endl;
auto color_profile = color.get_profile();
auto color_stream_profile = color_profile.as<rs2::video_stream_profile>();
auto color_intrin = color_stream_profile.get_intrinsics();
auto c_principal_point = std::make_pair(color_intrin.ppx, color_intrin.ppy);
auto c_focal_length = std::make_pair(color_intrin.fx, color_intrin.fy);
rs2_distortion c_model = color_intrin.model;
std::cout << "Color Principal Point : " << c_principal_point.first << ", " << c_principal_point.second << std::endl;
std::cout << "Color Focal Length : " << c_focal_length.first << ", " << c_focal_length.second << std::endl;
std::cout << "Color Distortion Model : " << c_model << std::endl;
std::cout << "Color Distortion Coefficients : [" << color_intrin.coeffs[0] << "," << color_intrin.coeffs[1] << "," <<
color_intrin.coeffs[2] << "," << color_intrin.coeffs[3] << "," << color_intrin.coeffs[4] << "]" << std::endl;Send us your code and pictures or video of your observations so that we can troubleshoot further.
Regards,
Zulkifli Halim
Intel Customer Support
Please sign in to leave a comment.
Comments
2 comments