Access Color Stream from Left Imager in UYVY [16 bits] format

[adamhippoharvest]

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D455
Operating System & Version	Ubuntu 20.04
Kernel Version (Linux Only)	6.5.0-17-generic
Platform	PC
SDK Version	2.55.1
Language	python

Issue Description

Upon reading the D400 series data sheet, I see that there is access to the Color Stream from Left Imager in UYVY [16 bits] format (page 61). In the D455 model, there is no IR pass filtering, so I interpret that this image from the Left Imager is full spectra (400-950nm). However, using the pyrealsense2 SDK, I cannot find a command to access this image.

Is it possible to use the API to pull this image? Ideally, I want the full spectra from the acquisition (400-950nm), not just the overall intensity--I'm only able to access y8, y16, and z16 formats.

Thank you!

[MartyG-RealSense]

Hi @adamhippoharvest Yes, the D455 model does not have an IR filter on its left and right infrared sensors, so they can see the full visible and near-IR frequency range.

On RealSense 400 Series camera models that support the feature (which includes D455) you can access the 'RGB color from left infrared sensor' image by defining a custom 'cfg' stream configuration for the left infrared stream in your script and setting it to use UYVY, RGB8 or BGR8 format instead of the Y8 infrared format. UYVY will be monochrome RGB rather than color though.

I could not find a past case of using the UYVY format for IR RGB, but the format is supported for it. The cfg code for doing so would likely be like the one below.

cfg.enable_stream(rs.stream.infrared, 848, 480, rs.format.uyvy, 30)

Below is a UYVY monochrome color image from the left IR sensor streamed in the RealSense Viewer tool on a D455 camera.

[adamhippoharvest]

That makes sense, thanks. I just want to confirm then that this is a monochrome image, and there's no way to get a three channel image, such as doing something like this:

    cfg.enable_stream(rs.stream.infrared, 848, 480, rs.format.uyvy, 30)
    pipeline.start(config)
    frames = pipeline.wait_for_frames()
    uyvy_frame = frames....[not sure what to put here]...
    uyvy_image = np.asanyarray(uyvy_frame.get_data())
    bgr_image = cv2.cvtColor(uyvy_image, cv2.COLOR_YUV2BGR_UYVY)

[MartyG-RealSense]

You can retrieve a raw unprocessed RAW16 RGB image directly from the RGB sensor and convert it to an OpenCV format to make it displayable, as described in the Python cases at #7275 and #11451

[MartyG-RealSense]

Hi @adamhippoharvest Do you require further assistance with this case, please? Thanks!

[adamhippoharvest]

Sure, it is good to follow up. Upon trying to replicate the screen you have above, I learned that I needed to update the drivers on the Realsense camera itself (it just came out of the box). Upon updating the realsense, I was able to access BGR8, UYVY, etc from the infrared camera, which solved my question. I am also able to access it programatically via the pyrealsense2 api.

Thank you for your help!

[MartyG-RealSense]

It's excellent to hear that you achieved a solution. Thanks very much for the update. As your issue is solved, I will close this case. Thanks again!