View my account

Incorrect depth image by D455

Comments

10 comments

  • Yzkzero

    BTW, the issue is not always "shifting with band". Sometimes the depth image is totally mess up. And in any case, a hand waving into the camera view can make the depth back to normal...

    0
    Comment actions Permalink
  • MartyG

    Hi Yzkzero When outdoors, pointing a 400 Series stereo depth camera directly at the sun can saturate the IR sensor and cause the depth image to freeze or become sparsely detailed.  When this occurs, covering the lens may cause the camera to automatically correct itself.

    Given the sun-like intensity of the ceiling light source, I wonder if a similar saturation effect is occuring here that is auto-corrected by obscuring the ceiling light with a hand wave.

    If you are using the camera in the same location all the time with the same controlled artificial lighting level, you could try disabling auto-exposure and manually setting the exposure value, experimenting to find a value where the issue no longer occurs.

    Alternatively, if auto-exposure is required then applying a physical optical filter called a Linear Polarizer over the camera lenses on the outside of the camera may significantly reduce depth image problems caused by glare.  Section 4.4 of Intel's optical filters white-paper document provides information about this subject.

    https://dev.intelrealsense.com/docs/optical-filters-for-intel-realsense-depth-cameras-d400#section-4-the-use-of-optical-filters 

    In regard to why the D455 experiences this issue but your D415 and D435 do not: whilst all 400 Series models have the potential to be affected by 'sun saturation' from being pointed directly at an intense light source, the D455 has differences in its internal hardware design that may account for a sensitivity in this particular scenario (i.e it is not a bug nor a hardware fault).

    0
    Comment actions Permalink
  • Yzkzero

    Hi MartyG,

    Thanks for the quick reply.

    I know the strong light may saturate the sensor and cause the problem. But the issue also happens in a room without string light (I never took it outdoor).

    Actually I just reproduced it in my room 5 minutes ago (no sunlight at all because it is a cloudy day).

    See the screenshot below. The band also appeared in IR images which caused the wrong depth.

     

    0
    Comment actions Permalink
  • MartyG

    Judging by the IR image, it looks as though the band is produced by a second partial image-slice being generated at the top of the screen, with a gap between the main image and the partial one.  I have seen a similar wrap-around effect in a small number of cases before, though usually it is not encountered in the RealSense Viewer.

    As you do not encounter this issue in the Viewer with D415 and D435, this suggests that it is a phenomenon specific to using the Viewer with your D455.

    Does performance improve if you perform a Hardware Reset in the Viewer after launching it, and then activating the streams after the reset?  The reset can be done by going to the More option near the top of the Viewer's side-panel and selecting the Hardware Reset menu option.

     

     

     

    0
    Comment actions Permalink
  • Yzkzero

    Hi MartyG,

    The hardware reset can only temporarily solve it and the issue will happen again later.

    Also it is not specific to using viewer, it also happened in my application using librealsense.

    Another interesting problem appeared when I reproduced the issue and used hardware reset to solve it as you suggest. The IR images became under-exposure. See screenshot1 below.

    So I unchecked "Auto exposure" and turned it on. Afterwards, a more serious issue came up. The IR images were with band and kept flickering. Depth image totally messed up. See screenshot2 below. I made a video record in both gif and bag file. If you need, I can send you via email.

    screenshot1:

     

    screenshot2:

    0
    Comment actions Permalink
  • MartyG

    How does the depth and IR behave if RGB is disabled?

    0
    Comment actions Permalink
  • Yzkzero

    When the issue happens, disabling RGB can eliminate the issue.

    But I cannot say the issue never happens if RGB is disabled because the issue is sporadic and it will need much time to test. As far as I can remember, the issue seems to always occur with RGB enabled.

    So did you get any idea for the possible root cause?

    0
    Comment actions Permalink
  • MartyG

    My advice would be that if disabling RGB is found to consistently resolve the issue on your particular computer then do that, unless RGB is vital to your project.  Having the infrared stream active would be sufficient to provide an RGB-like video representation of the scene.

    If the camera will be used in a location that is permanently well lit then you may be able to disable the infrared dot pattern, as the 400 Series cameras can use ambient light in a scene to analyse surfaces for depth detail instead of analysing the dot pattern.  Turning off the pattern will make the infrared even more like an RGB image

    0
    Comment actions Permalink
  • Yzkzero

    Thanks for the suggestion, but it does not work for my case. Because RGB is necessary to our project. Also turning off dot pattern will result in bad depth image in our scenario (ambient light is not always good enough).

    And the issue happens in both my desktop computer in the office and my person laptop in the home. Not on a particular computer or a particular scene.

    So I hope Intel can find the root case and solve or improve it.

    Thanks!

    0
    Comment actions Permalink
  • Yzkzero

    Hi MartyG,

    I found a similar issue here: https://github.com/IntelRealSense/librealsense/issues/7777

    0
    Comment actions Permalink

Please sign in to leave a comment.