Frame drops in ROS using multiple d455 cameras
Hello there,
I'm running four d455 cameras in ROS in my project. All camera parameters are set to 1280*720 at 30 fps. But the result shows that all three cameras are fine while the last camera cannot keep up with 30 fps. Actually it's running at half of 30 fps or less.
I did some research and found out it could be related to the auto-exposure setting of the d455 cameras. Even though I turned off auto exposure and auto-exposure priority the last camera's frame rate is still way less than it's supposed to be.
All cameras are connected to NUCi7 with thunderbolt that supports 40GB. Peformancewisely it should be fine to load four d455 cameras without any bottleneck. I cannot tell what is the cause of it.
FYI, I'm using SDK version 2.42.0 with realsense ros 2.22 built from source
Thanks,
Victor
-
Hi Victor. Yes, the i7 processor that you are using should be suitable for handling 4 cameras simultaneously on the same computer.
What method are you using to launch the cameras in ROS, please. Are you launching all cameras with a single rs_multiple_devices.launch roslaunch instruction or are you launching each camera in a separate ROS terminal with rs_camera.launch as described in the multiple camera section of the RealSense ROS wrapper documentation in the link below?
https://github.com/IntelRealSense/realsense-ros#work-with-multiple-cameras
If all the D455 cameras are in the same location and observing the same scene and three of them perform at the correct FPS speed then it would reduce the probability that an auto-exposure related event is affecting the fourth, as the other cameras would likely be affected by the same environmental conditions.
You can though try to enforce a constant FPS by having auto-exposure enabled and auto-exposure priority disabled.
If auto-exposure is disabled then FPS may also reduce if manual exposure is set to a certain range, but the choice of exposure value can also enable a constant FPS to be enforced when manual exposure is being used.
These concepts are described in the link below.
https://github.com/IntelRealSense/librealsense/issues/1957#issuecomment-400715579
-
Hi MartyG I am trying to launch each camera in a separate terminal and now I can see that four cameras are all running at the desired frame rate. Clearly I cannot run all cameras in one roslaunch file because all nodes are started parallel. I'm glad that I know it now.
I found maximum acceptable frequency and minimum acceptable frequency in ros_realsense diagnostics topic. I wonder where are these two values come from. If it is possible, could I use these two values to narrow the frequency range so as to get a more accurate frame rate?
-
MartyG I also found the following Error when four cameras are running with the pointcloud filter.
An Error has ocurrred during frame callback: Error occurred during the execution of the processing block!
This Error is raised by camera_manager which can be traced back to base_realsense_node.cpp:1721
-
I was unable to find where the minimum and maximm expected frequency values in the /diagnostics topic are generated. The only diagnostics reference that I could find was the one that apparently generates the 'expected frequency' that is logged just before the launch completes with 'RealSense Node is Up!'
The librealsense SDK, and the ROS wrapper, does have an auto_exposure_limit parameter for the depth sensor that enables an upper bound value to be set for auto-exposure. This parameter is described by /camera/stereo_module/auto_exposure_limit
In regard to the frame callback error, Doronhi the RealSense ROS wrapper developer provides an explanation here:
https://github.com/IntelRealSense/realsense-ros/issues/652#issuecomment-479326583
-
MartyG Is there any way to double the image queue size? The thread above didn't mention it in the end. I found that this is a variable named frames_queue_size under rgb_camera parameters. I am not sure it is the one.
-
Another issue is that the transformed point cloud generated by multiple cameras I have got is not as good as the example shown here https://www.intelrealsense.com/how-to-multiple-camera-setup-with-ros/
There are a lot of ripples and black regions in the point cloud.
-
The only demonstrated setting of frames_queue_size in the ROS wrapper is in the link below, where it is defined as a dynamic_reconfigure parameter.
https://github.com/IntelRealSense/realsense-ros/issues/1379#issue-699601737
In most cases though it should be sufficient to leave the frame queue size as its default value.
In regard to your second question: the D455 camera model has a minimum depth sensing distance of 0.4 meters, compared to the 0.1 m of the D435 model used in the ROS multiple camera tutorial. More of the foreground detail that is closest to the camera would therefore be excluded from the depth image on the D455.
The optimal depth accuracy range (not the maximum range) of the D455 is 6 meters from the camera. Beyond that distance, 'RMS error' (drift in accuracy) may become more noticable beyond that distance.
You could try setting a camera configuration preset such as 'High Accuracy' or 'Medium Density' in your launch to help filter out bad depth values.
https://github.com/IntelRealSense/realsense-ros/issues/2055#issuecomment-908934596
-
Thank youMartyG. When I change frame_queue_size to 32 the frequency of the callback error does decrease correspondingly. But it could be hard to eliminate it completely. I found when a callback error is raised, the point cloud time stamp coming next could be drifting. My question is if there is a way in ROS to capture the error from Realsense ROS node in a try-catch way so I could do something when this error occurs.
-
If the callback error message is being generated from the librealsense file rs-processing.cpp as the link to Doronhi's comment earlier in this discussion states, then inserting some code for a recovery action at the point in this file where the message is generated and then building librealsense from source code with this modified file included may be a logical course of action.
Please sign in to leave a comment.
Comments
9 comments