Realsense or LIDAR for a robot?
Hi, I'm making a robot and I want it to be able to avoid obstacles. My options are:
1- A normal camera + a LIDAR sensor like this: https://ozrobotics.com/shop/hypersen-3d-tof-lidar-8m-hps-3d160-l-pro-lan-interface/
2- A RealSense Depth Camera D455.
I want to use a Realsense because that way I save costs and programming time, instead of programming a LIDAR + a camera, I write the code just for a camera (Realsense).
But in videos like this it shows that it is buggy: https://youtu.be/mFLZkdH1yLE?t=1386
The precision when measuring distances is not perfect (I don't know if he used a D455).
Please help me.
-
Hi Raymermed The RealSense camera model in the video is D435i. The D455 model has 2x the depth measuring accuracy over distance of D435i. This means that D455 has the same accuracy at 6 meters distance that D435i has at 3 meters.
For the purposes of obstacle detection, a D435i may be a better choice though as it is able to detect objects closer to the camera than the D455. So a D455's depth sensing may not see an obstacle that is at very close range to the camera.
With RealSense 400 Series cameras, depth measuring accuracy starts at around zero at the camera lenses and increases linearly over distance as an observed object becomes further from the camera. So for close range obstacle detection for the purposes of avoidance, measurement accuracy should be good compared to if you were trying to measure depth for an object that is a couple of meters or more away.
If the robot will be used outdoors then the D435if model may be beneficial as its lenses are equipped with light-blocking filters that help to negate negative effects from light reflections.
https://www.intelrealsense.com/depth-camera-d435if/
If the robot is indoors in a well illuminated location and the objects being observed do not have highly smooth surfaces (i.e the surfaces have some texture to them) then the D405 could also be considered. It is small in size and designed to provide high quality, high accuracy images at very close range (its ideal depth sensing range is 7 cm to 50 cm / 0.5 meters).
https://store.intelrealsense.com/buy-intel-realsense-depth-camera-d405.html
-
My robot is tall so I have to make sure that there is nothing in the space through which it will pass, for example, that it can pass through a door without problems knowing that there is a wall in front of it or that there is a vehicle in front of it, that is, I don't just want to avoid objects, I want to know how far away they are.
My question is: are Realsense cameras good for that? Can you detect the distance of a wall without any objects? (the D435i for example). Does the D435i have a LIDAR or some similar sensor to measure distances in each pixel or how does it work?
-
Yes, you can detect the distance of a wall. Because walls often have little or no texture on their surface, you would need a camera model that is equipped with a projector that casts a dot pattern onto the wall so that the camera can analyze the dots to aid in the calculation of depth. The D435i has a projector and can depth-sense up to a range of 10 meters, though accuracy will start to drift noticably after around 3 meters from the camera.
RealSense 400 Series cameras such as D435i use stereo depth, calculating depth by capturing left and right images (hence the 'stereo'). RealSense L515 is a lidar depth camera, though it is no longer supported by Intel as the model has been retired.
The difference between stereo and lidar cameras is described in the Intel article at the link below.
-
Thank you very much for your help and for this article.
Just to corroborate I will put several examples, tell me if the D435i camera works for me without problems:
1. When I talk about a wall I mean a completely smooth wall, imagine that it is huge and covers the entire panorama of the camera, in the camera it will only be completely blank (example). With the D435i camera will I know that there is a wall and the closer I get will I know that I am getting closer?
2. Another scenario, imagine that there is a vehicle in front, a vehicle that is 1.5 meters high (example) and the camera is at a height of 2 meters high, with the D435i will I be able to know that there is a vehicle and not be able to pass?
3. What I am looking for is to create an imaginary box in the center of the screen and I want to know if there is an object in it or not, because my robot will pass through that box. If there is an object, it will turn left or right to avoid it, then it will go straight and it will know if it can pass or not. Am I clear? That is my goal.
-
1. All of the wall that is within the camera's field of view will be detectable if a dot pattern from the camera's projector is being cast on to the wall's surface. Otherwise, a completely plain wall will likely not be readable by the camera's depth sensing because it has no analyzable texture without the dots.
Yes, the depth value will update as the camera gets closer to the wall, until the distance from the wall is below the camera's minimum depth sensing distance. It is possible for that minimum distance to be reduced though using a setting called Disparity Shift so that the camera can get closer to the wall, at the expense of a reduced maximum depth sensing range.
The camera will not know that the observed surface is specifically a wall though, just a surface in general. Likewise, if it sees a vehicle then it will not know that it is a vehicle, just a solid surface.
2. The camera will be able to see whatever is in its field of view. So if the camera is 0.5 meters higher than the car then it may not see the ground-level bottom of the car but see the bonnet / hood and above. The camera would be able to see the ground-level parts of the car if it were tilted downwards though. A RealSense camera can be tilted up or down by up to 30 degrees without causing problems for the depth sensing.
3. The camera would not know that should not pass a car or other obstacles unless it was paired with obstacle detection / avoidance software.
-
An example of using RealSense cameras for obstacle avoidance on the ground is robotic lawnmowers.
-
When we talk about dots I don't know what you mean, to literally paint the wall with dots?
I gave the example of the wall, not because I want to know if it is a wall or not, but so that the robot does not collide with the wall, I am only interested in knowing that there is something there and not moving forward. In that case it will work?
I'm just interested in it knowing that there's an object ahead and you have to dodge it, without an AI object detector. Because if it's a flat wall it won't detect that it's an object, I just want it not to hit the wall, the car, or whatever.
From what you tell me I know the answer, I think it does work for what I want, but I'm not an expert and I prefer that you confirm me, please. This is how I make sure and educate myself before making the purchase.
Thank you very much for your help.
-
No, you do not need to paint the wall with dots. The camera's projector casts an invisible pattern of dots onto the surface that only the camera can see and then analyzes those dots, using them as a texture for the wall.
If the camera is mounted to a mobile robot then depth data can be used to inform the robot to stop moving towards the wall if the camera is paired with obstacle detection / avoidance software. Without such software, the robot would not know to stop moving towards the obstacle.
-
Wow, amazing. Are those points created by all cameras or just some versions? If so, I imagine the D435 and D455 version are better.
So that's how you know the distance to the wall, makes sense. I'm going to use python to find out the distance and prevent it from walking forward if there's a wall or some other object. That is, imagine an imaginary rectangle that covers the entire center of the screen, that will be my robot, if there is nothing enclosed in that rectangle, it will be able to move. What do you think? Makes sense?
-
Some RealSense models were retired (T265 Tracking Camera, F455 Facial Authentication and L515 Lidar Camera) but the RealSense 400 Series range of stereo camera products continues to be produced and supported.
Please sign in to leave a comment.
Comments
14 comments