Can OEM calibration do it better?
I am working with the D415 module, which is mounted in a custom aluminum frame. I have tried to calibrate multiple modules (using the same aluminum frame), using the procedure outlined in the "self-calibration white paper" (Rev 2.7):
- On-chip calibration
- Focal length calibration
- Tare calibration (including automatic calculation of the ground truth distance)
For all steps I have been using the "GaussianBlur30x100x175back-11x17-1.pdf" image printed at 100% scale. The RealSense tools (Viewer and Depth Quality) are version 2.50.0.
I get very different results when I perform the calibration, which is the reason for this post. For most devices I need to perform multiple on-chip calibrations before I get a satisfactory "health check" value. And I have tried to redo on-chip calibration when focal length calibration has been performed if "large" focal length corrections have been made.
Never the less, and despite decent looking values, for some devices I end up with a far from satisfactory result! One example could be the following, where the "health check" was very close to 0, focal length imbalance was 0.012% and estimated tilt angle 0.217 deg:
After the calibration, in the Depth Quality tool, the estimated angle was 1.32 deg, fill rage = 99.99%, Plane fit RMS error = 0.25% and subpixel RMS error = 0.11%. The calculated ground truth was 1046 mm which was well in line with the measured ground truth.
Never the less, in the above picture, the depth in the lower right hand corner is just below 1000 mm and in the upper right hand corner just below 1080 mm. In other words, a difference of almost 8 cm within what should have been a completely flat depth map with the used camera and target alignment.
The "shown" D415 module had never been used before, and came directly from a "foam-protected" box. The good question now is:
Assuming I purchase the OEM calibration target (and therefore get access to the OEM calibration software), is it likely that I would be able to end up with a better calibration? And in a bigger perspective: when calibrating multiple D415 modules, can I make them more similar from a calibration-point-of-view, if I use the OEM calibration rather than the self-calibration procedure?
Thanks.
-
Hello Het.
Thanks for your comments and details. I'll try to address your issues.
To address the main question - can OEM calibration do it better? - the answer is "maybe". It ultimately depends on what factors are limiting the performance. Based on the results you cite from the DQT, the performance is not likely to improve with OEM cal. The numbers all look to be OK (the sub-pixel of 0.11 is probably in pixels, not %, but this is a reasonable value). But, the DQT does not catch all possible calibration errors, just the most common/likely ones.
A few questions to help determine the recommended steps:
1. When you run self-cal, is the projector On or Off? In cases where you use the provided target, it's best to turn off projector since the target will provide the needed texture.
2. The apparent curvature on the right side from top to bottom is not normal. You say that the target is flat, but can you confirm this? What are you using as a target? From a distance of 1m, it would need to be a large surface or wall. This could be checked by moving to a different "target" and align to be as parallel as possible. This could be a wall, floor, or ceiling.
3. Do you see the same behavior in multiple units (how many have you tested)? If the curvature is confirmed and in only 1 or 2 units, then it could be due to uncorrected distortion. In this case, then OEM calibration would be the only practical solution. While rare, it is possible to happen. It could also be due to large extrinsic changes which self-cal does not correct. For this, I would recommend trying dynamic calibration.
4. The isolated regions of invalid pixels are also not normal for a well-calibrated camera under typical operating conditions. This could be due to saturation from projector (you could try adjusting power) or poor texture on target. But, if these areas are persistent, it's another indication there is something wrong on this particular camera at least.
5. In general, you should be able to recover acceptable performance using self-cal or dynamic calibration. OEM calibration is useful or necessary in extreme cases or those where the module is outside it's case. You say the D415 is mounted in a custom frame and this could be a reason for the poor calibration that is not fully recoverable with self-cal. If dynamic calibration is unable to sufficiently recover, then OEM would be needed.
6. If your application uses many bare D415 modules that are mounted to a frame, then OEM cal is generally recommended due to the sensitivity of the cameras to mechanical changes. However, it's possible that a combination of self- and dynamic cal may be sufficient and would save you from needing the OEM target and SW. It depends on the extent of the changes and the specific parameters that are affected.
7. If you have, or can obtain, a D415 camera peripheral (in its case), this would serve as a reference for the type of performance expected with a good calibration. Self-cal might be needed as well but any degradation should be minor.
Let me know your feedback and we'll proceed from there.
John
-
Hi John,
Thank you for your rapid and thorough reply!
Regarding your first question, the projector is off when I perform the different calibration steps.
I wanted to double check that things are as controlled as they can possibly be at my end, so I made a few changes to my setup:
The target is taped to a flat plate (all the way around the edge of the paper), and the flat plate is taped to the wall. The range finder distance is 1053 mm from the front of the aluminum frame to the target (the D415 is a couple of mm further away).
The view from the camera is:
The obvious first question is: is there something wrong with this setup? The plate which the target is attached to, has very little structure and gives a very poor depth map when the projector is turned off. Is that a problem? The plate also offsets the target approximately 5.5 mm from the surrounding wall. Is that a problem?
I tried to run the Dynamic Calibration procedure (target attached to a plate, plate being moved manually by hand), and then run the full self calibration procedure, the result now is (with projector on):
It is possible to see the plate, and the "holes" on right side are gone. But the large difference between lower right and upper right remains (about 70 mm now).
Unfortunately, currently I don't have an "original" D415 camera, so I can't compare with this.
Regarding your 3rd question, I have handled more than 10 D415 modules, some mounted in different custom aluminum frames, and some mounted in the same custom aluminum frame. What I have seen varies quite a lot, all the way from good to bad calibrations, and in no way am I able to create "similar calibration" across the devices. Not even when mounting different D415 modules in the same aluminum frame.
Based on your comments, and the fact that we are working with modules we are mounting in our own frame, it sounds like there is no way around purchasing the OEM calibration target. Unless you can tell me that I am doing something wrong with the described/illustrated setup, and that I can get proper control of the calibration across devices by updating my procedure.
Thanks,
Henrik -
Henrik,
Thanks for the additional details.
I think in general, what you are doing is correct. I suspect that with the units that are not performing as expected, their calibration may have been altered beyond what is recoverable with self- or dynamic calibration and/or aspects of the calibration (like distortion parameters) that are not modified by the user calibration methods were changed. Whether this occurred during shipping or as a result of attachment to the chassis is hard to say. The modules are inherently sensitive to any mechanical disturbances so it is quite likely the degradation occurred during mounting. This is why we generally recommended a new (OEM) calibration after installation and the camera is in its final state.
For reference, below are examples of depth map and point cloud from a D415 peripheral camera. In this case, the target is ~500-550mm from camera. I was limited in distance to the size of the target in order to fill the FOV. But, you can see very little depth variation across the image (no more than ~5-10mm and nearly random). The man artifacts are small symmetric and concentric ripples in the depth when viewing a flat scene. This is an unfortunate result of uncorrected distortion and considered "normal" on D415 cameras. But, this is not the phenomenon you see, which is not normal for a well-calibrated camera.
Your specific questions:
The obvious first question is: is there something wrong with this setup? The plate which the target is attached to, has very little structure and gives a very poor depth map when the projector is turned off. Is that a problem? The plate also offsets the target approximately 5.5 mm from the surrounding wall. Is that a problem?
> No. The setup looks ok, though I would say that you probably do not need to attach the target to a plate unless you want it to be mobile. Attaching directly to the wall is OK. That would eliminate the 5-6mm step. The step is not necessarily a problem but it depends on what you're trying to do. For these tests, it's ok.
I tried to run the Dynamic Calibration procedure (target attached to a plate, plate being moved manually by hand), and then run the full self calibration procedure, the result now is (with projector on):
It is possible to see the plate, and the "holes" on right side are gone. But the large difference between lower right and upper right remains (about 70 mm now).
> So, dynamic and self-cal improved the depth but did not resolve all issues. OEM may be necessary for cases like this.
BTW, one way to confirm the depth variation is inherent to the camera and not a measurement or target artifact would be (if possible) to rotate the camera 90 deg or better 180 deg about Z axis. If the depth distortion is part of the camera, it will move with the camera (or remain in the same relative portion of the image). If it were an artifact, then it would stay in the same place (the right side in this case when rotated 180 deg). This applies to any camera of course. I fully expect this is inherent to the camera but this type of test would prove it.
Another experiment you could try (again if feasible) would be to slightly alter the mounting of the camera module to the chassis or possibly remove it completely (e.g., loosen the screws). If the depth image changes significantly as a result, this is another indication the degradation is due to mechanical changes and likely to require a new OEM calibration.
Unfortunately, currently I don't have an "original" D415 camera, so I can't compare with this.
> See above images for examples.
Regarding your 3rd question, I have handled more than 10 D415 modules, some mounted in different custom aluminum frames, and some mounted in the same custom aluminum frame. What I have seen varies quite a lot, all the way from good to bad calibrations, and in no way am I able to create "similar calibration" across the devices. Not even when mounting different D415 modules in the same aluminum frame.
> This may be because each module has its own unique calibration and so mounting into the same frame may result in the same external state for each camera, but they each have their own internal state so require a "custom" calibration for optimal performance.
Based on your comments, and the fact that we are working with modules we are mounting in our own frame, it sounds like there is no way around purchasing the OEM calibration target. Unless you can tell me that I am doing something wrong with the described/illustrated setup, and that I can get proper control of the calibration across devices by updating my procedure.
> I think this is the situation. I was trying to explore options to avoid this, and in some cases it is possible (explaining your better results); just not guaranteed to work.
If you prefer to avoid the need for OEM calibration, I would suggest trying the 2 user calibration methods on all cameras that require some improvement after installation (probably vast majority). You could then assess whether the resulting "yield" is acceptable vs cost and added complexity of performing a new OEM calibration on each unit, which is the safer and recommended approach. Or, if acquiring the OEM cal capability is ok or preferred, just go ahead and do that.
Let me know if you have further questions.
John
Please sign in to leave a comment.
Comments
3 comments