Intel RealSense D455 object detection within defined window
Hi there,
I'm looking to use the Intel RealSense D455 to detect objects within a given spatial window (x,y,z range). Is it possible, given the RealSense software, to only look at a particular, defined window, set by myself?
This application does not require identification of specific objects, but rather of objects that are hanging or otherwise suspended/moving through the air, between a particular distance outside of which I won't look (far away from the camera), and the camera itself (or a certain distance in front of it). I intend to use Open3D's Machine Learning library, although if there are more appropriate tools for this project, that would be much appreciated as well.
Thanks!
Andrew
-
Hi Andrew John Madey If you do not require identification of specific objects, the most straightforward approach may be a C++, Python or C# language script in the RealSense SDK that applies a post-processing filter called a threshold filter. This enables you to set a minimum and maximum distance from the camera that Z-depth detail will be rendered at and exclude from the image the depth data that is outside of that defined range.
Examples of creating a bounding box in C++, Python or C# can be found at the link below.
https://github.com/IntelRealSense/librealsense/issues/5964#issuecomment-594372943
If you would also like to define an XY "region of interest" where depth data outside of that horizontal and vertical area of the image is excluded from the image then this will be more complicated. It is possible to do so with a bounding box, as described here:
https://github.com/IntelRealSense/librealsense/issues/2016
If your preference is to use Open3D then a method of cropping an RGBD image is suggested here:
-
You can also define a threshold filter in the RealSense Viewer tool without requiring any programming.
If you plan to use both RGB and depth and would like to exclude RGB detail by distance as well as depth then you could align depth to color so that the color coordinates are added or removed as the matching depth coordinates are included or excluded from the image. The rs-align-advanced C++ example program demonstrates this principle.
https://github.com/IntelRealSense/librealsense/tree/master/examples/align-advanced
-
Hi Marty,
Thanks! I guess the question from there would be, do you know of a way to tabulate the number of such objects observed, automatically?
I'm trying to figure out if there's a way to automate the counting process so that, per frame, I could figure out how many objects there are, and not have to do it by hand each time (several hundred possible).
Thanks!
Andrew
-
An OpenCV tutorial in the link below for detecting coins on an RGB image and counting them may be adaptable for your application.
https://medium.com/analytics-vidhya/detecting-and-counting-objects-with-opencv-b0f59bc1e111
Please sign in to leave a comment.
Comments
5 comments