Dimensional Analysis of an object using realsense D435
Hi everyone,
I am currently working on a project using an Intel RealSense D435 camera to detect objects, measure their size (height, width, and length), and identify any cavities. For this, I'm utilizing OpenCV and the pyrealsense2 library. Currently, my focus is on measuring the height and width of the object.
Setup Details
Camera Position:
- The Intel RealSense camera is mounted on a tripod.
- The camera is positioned parallel to the object's surface to get an accurate top view.
Libraries Used:
- Numpy
- OpenCV
- Pyrealsense2
Library Link: Pyrealsense2 Library : https://pypi.org/project/pyrealsense2/
Problems Faced
- Bounding Box Accuracy:
- The bounding box enclosing the contour using OpenCV does not focus on the object properly.
- Measurement Inaccuracy:
- The height and width measurements have errors ranging from 1-4 cm.
- Depth Measurements:
- Difficulty in obtaining accurate depth measurements.
- Varying Dimensions with Camera Height:
- The dimensions (length and width) vary when the camera height is adjusted because our algorithm runs on a pixel wise measurement.
- I am in need of a method to change the pixel-to-centimeter conversion ratio dynamically.
Questions
- Is it possible to measure the exact length, width, and height of the object?
- What are the prerequisites that should be followed before measuring?
- What should be the camera angle and height to measure the object accurately?
- How can I measure the accurate length, width, and height of the object? Does any specific library need to be used?
- How can I calculate the depth information?
Any guidance or suggestions on how to improve the accuracy of my measurements and the bounding box would be greatly appreciated!
Thank you!
-
Hi Darkdevelopers6710 The RealSense SDK has a Python box measuring example project called box_dimensioner_Multicam. Despite the name, it can be used with a single camera like in your setup in the images above.
The box_dimensioner_multicam has a 'sweet spot' distance of around 0.5 meters that provides best results. If the camera is nearer or closer than that then the green bounding box (shown above) that is overlaid on the RGB color image can be wrongly oriented and the measurements may be inaccurate. It uses depth information by calculating a point cloud.
Although the project requires the box to be placed on a checkerboard image, it is possible to edit the project's code to remove the need for the box if you are using a single camera.
Usually the camera is placed at the corners of the checkerboard and angled diagonally towards the box instead of being placed directly above the box pointing downwards.
-
If you are able to use a pre-existing commercial software solution then object dimensioning software by a company called MobileWorx might meet your needs. It enables objects such as boxes to be measured with a tablet that has a RealSense 400 Series camera attached to its rear, inserted into a rubber boot.
https://www.mobileworxs.com/xdim-mobile-dimensioning-software/
-
Hi MartyG Thanks for your detailed explanation and the effort. The link you shared is very helpful for two cameras. Is there any package or library to detect with a single camera? Please suggest.
-
Hi Yoges It Tech Are you a different person to Darkdevelopers6710 who created this discussion, please?
-
Me and Darkdevelopers6710 are working on the same project. MartyG
-
Thank you very much for the clarification.
box_dimensioner_multicam will work with a single camera. It is the best open-source solution available for box measuring with a RealSense camera. Because box_dimensioner_multicam and the RealSense SDK are open-source and can be freely modified, you could use them as the foundation for your own project.
In regard to commercial pre-made solutions, in addition to the Mobileworx system mentioned above, another dimensioning tool that is powered by the RealSense D455 camera model is xDIM Mobile Dimensioning
-
Hi MartyG
Thanks for the suggestion.
Please clarify the following questions:
1. Where and what parameters/code should I change to make the multicam configuration to a singlecam one?
2. While I am on the single cam config, at what degree should I place my camera when compared to the mean position (top facing)?
3. Chessboard corners are not being detected. What could be the issues? Is it just the single cam config or is there something else?
-
1. box_dimensioner_multicam can be used with a single camera without having the modify the parameters.
If you wish to use a larger chessboard image than 6x9" though then you will need to modify square size parameters in the code.
If you would prefer not to have to use a chessboard then the link below has a discussion about modifying the project to use the intrinsic parameters of a single camera instead of using a checkerboard to calculate this information.
https://github.com/IntelRealSense/librealsense/issues/10054
2. There is not a recommended tilt angle. In the two-camera image that I provided earlier though, it looks as though the cameras are angled at around 45 degrees.
If you are testing box_dimensioner_multicam, do you have the camera positioned at the corner of a 6x9" chessboard, looking diagonally towards the center of the chessboard and mounted at a distance of around 0.5 m from the board? Is the green bounding box correctly displayed around the box?
box_dimensioner_multicam calculates the chessboard corners but is not meant to display that information to the user.
-
Okay Thanks for the information MartyG
-
Hi MartyG,
I made the code work by doing some slight modifications and got great results with cuboidal objects. Moving forward, I have some queries that I couldn't resolve.
1. How can I make the cuboidal-bounding box shapeshift? For example, if there is a cylinder shape object, how can I make my bounding box trace the edges of that object (like a polygon bounding box)?
2. Is there some way I could tweak the code in order to make the program detect for cavities or holes in the object? Also how can I measure the depth of the cavity?
-
A possible approach for creating a bounding box for irregularly shaped objects is to use OpenCV contour code to form a bounding around the object from rectangles and circles.
https://docs.opencv.org/3.4/da/d0c/tutorial_bounding_rects_circles.html
In regard to detecting holes, a viable OpenCV approach may be to use a canny edge detection algorithm to find holes.
https://answers.opencv.org/question/173326/how-to-find-coordinates-of-holes/
https://github.com/IntelRealSense/librealsense/issues/7364#issuecomment-727635090
Checking the depth of holes could be difficult if the holes are small as they may not stand out enough from the surface to be recognized as separate from the overall surface.
-
Hi MartyG,
Have trying same solution, one info would like to know that, is this solution would work with any other camera instead of Intel RealSense DXXX camera with SDK 2.0 ?Thanks,
Murali.
-
Hi Murali D I can only advise about Intel RealSense cameras, not other types of camera.
Please sign in to leave a comment.
Comments
15 comments