Creating a Mesh
I am new to Real Sense. Have used Occipital Cameras in the past such as the Core. Is there sample code for creating a mesh when usng a Real Sense camera ? C++ or other ? So basically capturing depth frames and assembling them together for a larger image.
-
Hi Dan, it sounds as though the RealSense SDK's C++ / OpenCV example rs-kinfu might meet your requirements. It progressively builds up a 3D 'point cloud' image through a "frame fusion" process as you move the camera around a scene. When you are satisfied with the level of detail on the scan, you can then export it to a .ply format point cloud data file.
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/kinfu
If you need a mesh then you can import the .ply file into a free open-source 3D modelling tool called MeshLab and convert the ply data to a mesh.
https://www.andreasjakl.com/capturing-3d-point-cloud-intel-realsense-converting-mesh-meshlab/
-
The RealSense SDK has an Android project called rs-camera
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android/tools/camera
The above project is the open-source version of the RS Camera RealSense demonstration app on the Google Play store.
https://play.google.com/store/apps/details?id=com.intel.realsense.camera&hl=en&gl=US&pli=1
-
You are very welcome. Thanks very much for your kind words. :)
There are no royalties, licences or application processes involved in using RealSense technology in a commercial product and you are free to implement it into your product as you wish.
If you used the circuit boards inside a D457 (the D450 depth module board and Vision Processor D4 V5 board) to integrate them into your own product design to create a RealSense-enabled new product then you / your company would be responsible for providing technical support to your customers for that product.
Matched multipacks of these boards can be purchased if you wish to take this approach with your product instead of using Intel's pre-made D457 cameras.
Some countries may also request a particular safety certificate in order to allow RealSense-enabled commercial products to be imported. The full range of regulatory certificates for the RealSense product range can be found at the link below.
-
Hello Marty.
I subscribed to the posts and read your responses to customers questions in the last week and you are a wealth of knowledge my friend. In fact you are the reason we ordered a D455 to test next week and possibly change from the Occipital sensor we use now.
Question please: you mentioned the pre-built app for Android that can work with the sensor. RS camera. I will download the sdk today to check it out. Are the libraries modifiable in C++ ? Do I have to use Android Studio or is there a way to make modifications to a library and generate the new .apk from command prompt??
I do have a second question please: have you seen where sometimes the depth frame is not very clear, so it has a lot of black pixels (unknown areas.) From your experince, which in in the USB C (24 pins) is most likely a bad contact and contributing to this?
Thank you. Dan
-
The RealSense SDK and its tools and examples are open source and so you are free to modify the source code for your requirements.
If you will be using C++ code then you could investigate using Visual Studio as your development environment and insert code into its CMakeList.txt file and the build.gradle file to link the project to the SDK's library.
Visual Studio (Android development)
https://visualstudio.microsoft.com/vs/android/
RealSense C++ Android 'native' example
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/android/examples/native_example
The guide at the link below may be a helpful reference for generating an Android .apk from a C++ native project with Visual Studio.
Whilst a low-quality or damaged USB cable can negatively affect RealSense performance, I am not aware of a specific USB pin that would be responsible for it. Black areas on an image could have a range of different causes, such as something that makes a section of a surface more difficult for the camera to analyze for depth detail (including light reflections or surface material that is colored dark grey or black).
-
The RealSense SDK is worked on continuously by the RealSense development team, with new versions of the SDK typically being released 2 or 3 times per year. New camera firmware driver versions are released at the same time as the SDK. The most recent release at the time of writing this was version 2.55.1 two weeks ago.
https://github.com/IntelRealSense/librealsense/releases
If the above link does not work when clicked on then please copy and paste the address into your browser's address bar.
-
Thank you Marty. I feel better now. :)
My D455 arrived finally and I tested on an Android tablet running the RS Camera app. Very nice.
Do you happen to know where the recorded files are saved ?
Do you know of any way to just save a .ply file using this app ?
Where is it possible to buy additional cables for the D455 ?
Thank you again. Dan
-
The app's saved files are downloaded to the location external storage/realsense/videos
The link below discusses the possibility of modifying the app to use the internal-storage Downloads folders.
https://github.com/IntelRealSense/librealsense/issues/9840
There is not much information available about exporting a ply file from Android if you wish to add ply export to its source code, unfortunately. What there is can be found here:
https://github.com/IntelRealSense/librealsense/issues/6608
The official RealSense USB cable is not available for individual purchase, though a company called Newnex supplies high quality RealSense-validated cables. D455 uses the USB Type C / A to C cable.
-
Thank you Marty. I am plunging forward with testing now and very much considering changing platforms for my product. I also ordered a D450 module with the D4 board, so can test the components.
Question: what is difference between the D455 and the D450 with D4 ? Will the D450 with D4 work with the SDK 2.0 same as the D455 does ?
Lastly, it seems that the D450 is the best accuracy and longest distance camera in the available list today. I am only using it for Depth, not IR or RGB. Is this the camera you would recommend as the best accuracy and longest distance ?
Thanks. Dan
-
Please ensure that the D4 board is version V3 if you are aiming to connect a D450 and D4 together to make a camera that is recognized as D455.
A D455 camera is a D450 depth module board and a Vision Processor D4 V3 board, joined together with an interposer cable that can be purchased from the official RealSense store under the product name HSF Flexi Interposer. The store offers two interposer cable lengths, 50 (mm length) and 100. The two boards should perform the same as a pre-built D455 with the same feature-set.
Purchasing the D450 and D4 V3 separately can be a risk, as there are two different versions of the D450 module (an older and newer one) that may not match up with the D4 board if you do not have the right D450 version, causing an Invalid SKU error that prevents the camera from working. This is why it is preferable to purchase a matched multipack of the boards if possible to help to ensure that the Invalid SKU issue is avoided. If you bought both of the test boards from the same supplier though then they will likely match up.
D455 (what you will have when joining the D450 and D4 V3 together) is 2x more accurate over distance than the D435 class of cameras. It can also depth sense up to 20 meters whereas most other 400 Series models are limited to 10 meters, though accuracy will start to noticeably reduce once past around 6 meters.
-
TY. I purchased the D450 and D4 from same supplier. It is V3. I saw that there is a V4 available now but not sure what the difference is. After initial trials, I will purchase the 8 pack of course directly from you.
Question: do you recognize the camera in this photo? Sorry. I know its blurry. Was on a robot at a trade-show. Lastly (my last question for today, I promise:) Is there a future replacement for the D450 or something even better ? I just don't want to work with something that may have a successor in the pipeline. Thanks. Dan
-
The V4 version of the D4 is used with a D401 depth module to form a D405 when joined together. V4 is not compatible with D450. If a D450 is used with the D4 V5 then a D457 is formed.
The camera in the trade show image is likely to be the original version of StereoLabs ZED.
Intel cannot comment on possible future RealSense products. There are no plans to discontinue the D450 though. It is used in multiple products in the 400 Series range (D455, D455f, D456, D457). When a new class of products is released that has different technology, the RealSense SDK retains compatibility with previous generation models for an extended period of time.
-
Hi Marty. I wrote a question last night, but maybe forgot to press submit. Sorry, if comes through twice. Wanted to ask you recommendation on which OS to work in if I will be working with the SDK 2.0 and importing the ks-kinfu for mesh creating? Do you think MAC or Windows would be better ? The end output is a .apk to run on Android devices. Thanks.
I also noticed a strange thing in the SDK installation doc. for Windows. Please see below: Do you know what it says "Install Android Studio IDE for Linux" ???
-
The SDK has not officially supported Mac since 2018 and it has problems with modern MacOS versions such as Ventura and Sonoma. So Windows will be the preferable choice.
Originally the RealSense Android wrapper required an Android device to be in a 'rooted' state in order for the camera to work with it. The wrapper was later replaced with one that works with 'unrooted' Android devices (you do not need to modify the Android device in order for the camera to work with it). The unrooted wrapper is the one that should be used as the old original wrapper does not support modern Android OS versions.
https://github.com/IntelRealSense/librealsense/blob/master/doc/android.md
-
Trying to get a better understanding on the depth accuracy of the D455 for example. I read the specs and mostly I see <2% at 4m for example. This would mean that at 4m we can expect to be off by no more than 80mm, which is 3 inches. I probably am reading it incorrectly because my D455 which I am testing now seems much more accurate. Also the sensor which I previously worked with, the Structure Core, advertizes 0.29% depth accuracy. Can you shed some light on this? Thanks.
-
The D455 model has 2x the accuracy over distance of the D435 camera model. And the D415 model also has 2x accuracy of D435. So I use the green lower line on the diagram below as a rough visual guide for D455's depth error over distance.
The documentation link below states that on 400 Series cameras "the depth error scales as the square of the distance away".
400 Series cameras are technically capable of 1% accuracy, but 2% is a more easily achievable figure when using the camera at a few meters distance from an object / surface.
The D405 camera is the only 'sub-millimeter' detection accuracy model in the range, though it is a specialist close-range camera that can provide high quality, high accuracy images and has an optimal depth sensing range of 7 cm to 50 cm (though it can register depth for objects a couple of meters away). It can achieve <1% object detection accuracy at 7 cm.
-
Hi Marty. Thank you for the new info. We are hard working now on becoming familiar with the SDK2.0. Can I get your input please on best settings for high accuracy. Here are 2 snapshots of plys of the same wall with 2x6 studs with plywood behind and also with spray foam insulation between some of the studs. Also an RGB photo to see what we are looking at. One (green) shows the ply from the D455 camera. The other (yellow) shows the ply from the CORE camera that I used in the past. Distance of both cameras is 8ft from substrate. The d455 has a larger FOV, but I cropped to capture the same areas of the wall for comparison. In the ply you can see the face of the studs for example, which are of course a flat surface in real life. Can you give me some input on what to adjust in Real Sense Viewer or Quality Tool applications to get the stud to appear flat in the frame. It is very rough right now, which tells me accuracy is not optimized. Its possible also that the d455 software is doing too much hole filling and this is where the problem happens, so maybe there is a way to turn this off to only see the mesh points. Thanks. Dan
-
I suspect that the less than optimal RealSense depth image is mainly due to a phenomenon called repetitive pattern, where repeating horizontal or vertical arrangements of similar looking objects can confuse the camera's depth sensing. Intel provide a guide at the link below to reducing the effects of repetitive patterns.
https://dev.intelrealsense.com/docs/mitigate-repetitive-pattern-effect-stereo-depth-cameras
The first suggestion from the guide that I would recommend trying in the Viewer is to enable the HDR Enabled option to see whether the image improves. You should find this under the Stereo Module > Controls section of the side-panel.
After that, try increasing the Laser Power option - also under Controls - to its maximum value of '360'.
The third suggestion from the guide that you could try after that is increasing the value of the DS Second Peak Threshold option, which can be found under the Stereo Module > Advanced Controls > Depth Control section.
-
When the images are blinking / flickering in HDR mode, you can stop it by configuring post-processing filter settings using instructions at the link below.
https://github.com/IntelRealSense/librealsense/issues/10505#issuecomment-1126879337
By default the Temporal post-processing filter is enabled in the Viewer and it has a setting called Persistency Mode. Setting its drop-down menu to 'Disabled' disables its hole filling function.
The 'Disabled' option is not shown at first in the drop-down, so roll the mouse scroll-wheel upward to find it.
Please sign in to leave a comment.
Comments
23 comments