How to resample D455 RGB images from 1280*720 to 640*480?
I have captured some RGB images using D455 in a 1280*720 resolution. I would like to resample them into 640*480, so that the finally resampled 640*480 images are identical (more or less) to the ones that would be captured by D455 if a capturing resolution of 640*480 pixels was initially selected.
I was not able to find a clear documentation of the resolution change of this camera. However, after experimentation and empirical observation of the FOV of the camera for the different resolutions, it seems like the camera initially captures in 1280*800, which then crops horizontally (several pixels on the right side for the 640*480) or vertically (several pixels at the bottom for the 1280*720) and resamples to the corresponding resolution if needed. Is that true?
So my question is: Could you provide the exact process of resampling an 1280*720 D455 RGB image to a 640*480, using the same procedure like the camera does so that the final image is as if it was originally captured by a D455 camera in a 640*480 resolution?
-
Hi Al I Papadaki In regard to translating a raw 1280x800 image to another resolution, a reference to this process is provided by an Intel RealSense team member at the link below.
"The active pixels on the D435 are 1280 x 800 (see the tuning whitepaper). When the camera returns depth, it uses rectified images. During the rectification process, the image gets downsized. In order to get the 1280x720 resolution after rectification, the extra pixels are extrapolated".
The reference to active pixels in the tuning whitepaper relates to a table in the section of Intel's camera tuning white-paper guide at the link below.
In regard to the active pixels on the D455 camera model, the 1280x800 value applies.
If you capture images at 1280x720 then you can downsample their resolution in real-time using a post-processing filter called a Decimation Filter. The RealSense SDK's RealSense Viewer tool supports the application of this filter type to RGB to divide the resolution by a specified value (the default value being '2'). So at a setting of '2', 1280x720 should downscale to 640x360 (the closest to 640x480 that can be achieved with this method).
Further information about the Decimation Filter can be found in the section of Intel's post-processing guide linked to below.
https://dev.intelrealsense.com/docs/post-processing-filters#decimation-filter
However, if you have already captured 1280x720 RGB images as image or video files and seek to resize those pre-made images to 640x480, this is not an action that can be performed directly with the RealSense SDK. I would instead recommend using OpenCV's cv2.resize() function.
https://www.tutorialkart.com/opencv/python/opencv-python-resize-image/
-
Dear MartyG,
thank you very much for the response!
What is more of my interest is what you mention in the last paragraph, since i have already captured the 1280*720 images. However, if i just apply the OpenCV functionality that you suggested (cv2.resize), it will only shrink the initial image to the resolution i will ask for.
However, what i really want is to imitate the images that would be captured by D455 if i had chosen a capturing resolution of 640*480. As i said in the initial question, i suspect that this involves some image cropping (or perspective transformation) before the resizing, because the FOV of a 640*480 D455 RGB image is much narrower than the FOV of a 1280*720 D455 RGB image. Is this relationship between the 2 different FOVs and resolutions known and available?
-
Different resolutions have different FOV values and also different fx and fy values. The fx and fy fields describe the focal length of the image, as a multiple of pixel width and height.
The FOV and fx / fy values for a particular resolution on a particular camera can be read using the RealSense SDK's 'rs-enumerate-devices' tool by launching it in calibration information mode with the command rs-enumerate-devices -c
Shown below are the FOV and fx / fy values at 1280x720 and 640x480 for my own D455 camera.
-
Exactly! But the image transformation for D455 images (from 1280*720 to 640*480) is not a simple resizing, but rather a more complex procedure.
This procedure most probably includes resizing but also an additional step before resizing, in order to reduce the FOV and also bring the images to the same width-to-height ratio as well. But this cannot be inferred by the FOV or any other information that i found on your site.
-
Raw images created at the point of capture at the camera lenses are rectified and have a distortion model applied to them. This takes place in a hardware component inside the camera called the Vision Processor D4 before the rectified images are sent through the USB cable to the computer.
The precise details of the rectification process that is carried out by the Vision Processor D4 are not available publicly, however.
Images that have had a distortion model applied to them can have the model removed in OpenCV with its undistort function though.
Please sign in to leave a comment.
Comments
9 comments