How to pass d455 camera data to another pc?
Hello, I recently purchased the Intel RealSense D455 camera. What I want to achieve is as follows:
- Connect the Intel RealSense D455 camera to PC1, capture color images from the left and right lenses, and then send two color images to PC2.
- Use the Intel RealSense SDK on PC2 to create a depth map using the two images received.
Is it possible to perform these tasks? I want to configure PC1 with very limited computing resources, and perform tasks such as actual depth map calculation on PC2.
Do you know of any good ways to accomplish this?
Best Regards,
Yoon
-
Hi Xpdxpddld You can capture images on one PC with the camera attached and send them to another PC for viewing and processing using the RealSense ethernet networking tool described in the guide at the link below.
However, this ethernet tool is being removed from the RealSense SDK in the next SDK version after the current one (2.53.1) as it was an experimental proof of concept rather than an official, validated Intel RealSense feature. So if you use this system then you will be permanently restricted to using SDK version 2.53.1.
There is an alternative Python-based RealSense ethernet tool that should continue beyond SDK 2.53.1.
https://dev.intelrealsense.com/docs/depth-camera-over-ethernet-whitepaper
When obtaining an RGB image from the infrared sensor, RealSense cameras can only provide RGB from the left infrared sensor though, and not both left and right. They can provide an RGB-like monochrome infrared image from both left and right.
You could stream RGB from the RGB sensor and from the left infrared sensor, though the images from these two different sensors are not exactly the same in appearance.
-
Hello, thank you for your quick response.
Q1. Based on my understanding, you are suggesting to connect the Depth camera to PC1 (Raspberry Pi), then transmit the camera frames to PC2 via Ethernet. Finally, use the SDK on PC2 to generate the Depth map.
Q2. Or, alternatively, are you suggesting to connect the Depth camera to PC1 (Raspberry Pi), then generate both the RGB and Depth map on PC1, and transmit them to PC2 via Ethernet?
Q3. Am I possibly align the RGB and Depth images obtained from the left RGB sensor and the infrared sensor at a 1:1 ratio later on?
Best regards,
Yoon -
Q1 is correct. The data is generated on PC1 and then transmitted via ethernet cable to PC2. On PC2 you can remotely access the camera on PC1 as a 'network camera' and view and control the data on PC2 as though the camera were attached to PC2. The resolution and frames per second (FPS) combinations available on a network connection are much more limited than a directly connected camera though, as described at the link below.
-
MartyG
Thank you for your kind response. Then, by any chance, would PC1 and PC2 work without any issues even if they are on different local networks? (PC1 will be installed in a farm in another region, and PC2 will be located in our laboratory) -
The company Newnex supplies fiber optic cabling that can be used at a distance of up to 300 meters. For more information, please see the bottom of Newnex's RealSense page at the link below.
https://www.newnex.com/realsense-3d-camera-connectivity.php
Ethernet network cabling can be up to 100 meters long.
If the distances between lab and farm are too great for cabling then it is possible for RealSense data to be transmitted to a web browser window.
-
MartyG
thank you for reply.
But, PC1 and PC2 are physically located about 200km apart.
The provided URL (https://support.intelrealsense.com/hc/en-us/community/posts/360039895474-Highlighting-an-Intel-RealSense-depth-camera-web-browser-demo-program) contains a code written in Node.js, which uses the RTP protocol to send depth camera frames to an HTML5 browser. However, I have never used Node.js before and find it very confusing.Is there an easier way to stream frames from PC1 with a fixed IP address and have PC2 access PC1 to receive the frames?
My ultimate goal is as follows:
- Generate the color frame using the Intel Realsense SDK and the pyrealsense2 library on PC1 (Python code).
- Send the generated color frame from PC1 to PC2 (Python code).
- Create a depth map with intel Realsense SDK and pyrealsense2 library on PC2 based on the received color frame (Python code).
-
A Python-based approach to ethernet cable communication between two computers is the EtherSense project.
-
Sorry for the confusion caused by my unclear question.
Current situation:
- PC1 (farm) and PC2 (research lab) are physically more than 200km apart, so they cannot be connected via a local network.
- Both PC1 (farm) and PC2 (research lab) have a fixed IP address.
Ultimate goal:
- Connect an Intel RealSense D455 camera to PC1 (farm) and transmit color frames to PC2 (research lab).
- Construct a depth map from the received color frames on PC2 (research lab).
Is it possible to perform the above tasks? If so, I would appreciate it if you could provide related examples implemented in Python.
-
I see, 200 kilometers, not 200 meters. :)
The options available for communication over this distance are limited, unfortunately. The could include one of the following methods.
1. A RealSense user created a Python Flask web-app called Remote-Realsense for displaying a color image and depth image from a computer that it connects to. The tool cannot be linked to from this page, so please google for the term remote-realsense to find it at the top of the search results. Presumably the data processing is done on the computer that the camera is attached to though.
2. As mentioned earlier, Intel have an ethernet networking tool for connecting a computer with a remote camera to a second computer and performing the data viewing and processing on the second computer. The tool can be used with local wi-fi, so you could explore whether its RTP protocol can be adapted for longer-range internet connection (as the tool's guide document suggests that it could).
3. Record the data to a bag file on PC1 (which is like a video recording but for camera data), download it to PC2 and use the bag file on PC2 to do further processing on the data as though it were a live camera. This option would be unsuitable if your project needs real-time operation.
4. Use a VPN tool such as Google's Chrome Remote Desktop to connect to PC1 from PC2 and control PC1 and the camera that is attached to it. This would mean doing the processing of camera data on PC1 though.
The project may be easier to implement if you can process the data on PC1 and simply view it on PC2. Whilst wanting to locally process the data on PC2 is understandable when the camera's computer is a low-spec device, if PC1 is a capable full PC then processing the data on PC1 and then transmitting it is not likely to burden PC1.
Please sign in to leave a comment.
Comments
10 comments