Simultaneous Recording from D455 and D415 Cameras on 2018 MacBook Pro
I downloaded and compiled the macOS version on a 2018 MacBook Pro using release mode. I've been trying to use RealSense Viewer to simultaneously record the following:
- 2x 1280*720*30fps left/right images from the D455
- 2x 1280*720*30fps left/right images from the D415
- 1280*720*30fps RGB image from the D455
- 1920*1080*30fps RGB image from the D415
I estimate this generates ~380 MB of data per second. The MacBook's theoretical write speed to the internal SSD is ~2000 MB/s, but I'm finding that it's unable to keep up with the data that's being produced, both with and without ROS-bag compression. I consistently get "out of frame resources" errors when I try to record.
To circumvent the issue, I tried changing max_publish_list_size from 16 to 10000 in source.h. This seems to make RealSense Viewer use 10-12 GB of RAM buffer before running out of frame resources, but doesn't solve the problem (I was hoping that having a large buffer would give the disk time to catch up if it was just sometimes being slower).
I also found that turning on IMU data recording for the D455 consistently makes the viewer say that it's dropping frames (in the frame drop graph window), and makes the above problem worse.
Should I expect to be able to capture the data as I'm describing, or am I running into inherent limitations of the MacBook as a recording platform? Are there any settings or configurations I could change to make what I'm intending to do possible?
-
Hi David Charatan Although bag recording is the most efficient recording method in the RealSense SDK, it does of course generate very high file sizes.
I looked at possible ways to reduce the amount of data being generated. This could include:
* Reduce the D415's RGB resolution to 1280x720 to match the RGB resolution of the D455.
* Only have the left infrared channel active instead of both left and right. As both views are similar aside from the horizontal offset, there is no advantage to having both enabled unless there is some kind of processing you are going to do with the recorded bag file that requires data from both of them to be in the bag.
* Drop the RGB streams and instead just stream the left infrared channel in RGB8 format (color from left imager) from the D415 and D455. That way, you get a color image (though not like RGB color) that also has an IR dot pattern and is only using two streams in total for both cameras. On the D455, firmware 5.12.8.200 or newer should be installed to be able to access color from left infrared.
This configuration is demonstrated below.
Here is the same configuration with the Motion Module of the D455 enabled.
You could also check whether disabling the two GLSL options under the Performance section of the Settings interface of the Viewer improves performance on your Mac.
In regard to editing code, you may be able to reduce the risk of frame drops if you introduce greater latency into the streaming by altering the frame queue capacity, which is set to '1' by default. The frame management documentation suggests setting it to '2' if using two streams (e,g depth and RGB). Changing the frame queue size to a higher value other than '2' may be risky, as there is apparently a risk of accidentally breakng the streams with a custom value.
-
Hi MartyG,
Thank you for the prompt response. I am trying to do custom postprocessing with the recorded bag file, so I need both the left and the right imagers' outputs. I tried setting the left camera to RGB8 mode (and this works OK when only recording with the left camera), but when also enabling the right imager in Y8 mode, I get a very choppy recording. I've attached a screenshot of the frame drops.
Under the performance tab in the settings, I don't have any OpenGL options. Mine looks like this:
However, what I'm most concerned about is that the IMU seems to be dropping frames (see screenshot below). Using rosbag-inspector, I can see that a short recording of just the D455's IMU data gives 3460 accelerometer data points and 2798 gyroscope data points, even though the gyroscope and accelerometer are supposed to be recording at 63 and 200 fps respectively. Is this expected behavior, or does this indicate that there's some sort of problem with my setup? Ultimately, I would be fine reducing the amount of data I'm recording (whether by reducing frame rate, resolution, or the number of cameras, etc.), but would like to at least have reliable streams from the left and right imagers plus IMU data.
-
Given that SDK 2.43.0 was released yesterday, it may be worth building that SDK version on your Mac to test whether it improves performance, especially as its release notes state that it includes a fix for IMU drops on playback.
https://github.com/IntelRealSense/librealsense/releases/tag/v2.43.0
Please sign in to leave a comment.
Comments
3 comments