Remove the need for a patched nvidia library for NvFBC.

[hgaiser]

Description

This PR is based on hgaiser/nvfbc-rs#5 , which in turn is based on https://github.com/keylase/nvidia-patch/blob/master/win/nvfbcwrp/nvfbcwrp_main.cpp#L23-L25 .

With this change you don't need a patched NVIDIA library to use NvFBC. My best guess is that this is what GeForce Experience uses to allow it to work on GTX / RTX cards.

Screenshot

N/A

Issues Fixed or Closed

N/A

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Dependency update (updates to dependencies)
• Documentation update (changes to documentation)
• Repository update (changes to repository files, e.g. .github/...)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

• I want maintainers to keep my branch updated

[CLAassistant]

All committers have signed the CLA.

[ReenigneArcher]

Thank you for the PR! I also didn't know we could handle it like this.

If this affects NVENC as well, I think there might need to be an additional change somewhere in https://github.com/LizardByte/Sunshine/blob/nightly/src/nvenc/nvenc_base.cpp

[hgaiser]

Yeah so I'm a bit confused by NVENC. It almost seems like enabling this patch also allows NVENC.. I never thought about it but using Moonshine I never use a patched NVIDIA library and NVENC encoding works fine there.

I feel like this might trigger something in the NVIDIA library that also allows encoding, but I'm not sure. I tried rebooting, but I still see that I can encode, even if I use nightly and not this PR.

[2024:04:27:00:38:36]: Error: Failed to create session: This hardware does not support NvFBC
[2024:04:27:00:38:37]: Info: Found H.264 encoder: h264_nvenc [nvenc]
[2024:04:27:00:38:37]: Info: Found HEVC encoder: hevc_nvenc [nvenc]

It might also mean that if you use NVENC, but not NvFBC, that you still need a patched driver 🤷 . I haven't been able to trigger this case at least.

Can you test this to see how this works for you? Whether you can capture & encode using this PR but can't capture nor encode without this PR?

[ReenigneArcher]

There might be some confusion. The ./src/nvenc code is only used on Windows (as far as I know), since nvfbc is deprecated on Windows. For reference, it was introduced here: #1427

Currently I don't have a quick way to test your PR, but I could ask for testers on our Discord server?

[hgaiser]

There might be some confusion. The ./src/nvenc code is only used on Windows (as far as I know), since nvfbc is deprecated on Windows. For reference, it was introduced here: #1427

Ah I see. I'm curious, why isn't ffmpeg used on Windows? I never looked into it much but I thought it was cross platform?

Currently I don't have a quick way to test your PR, but I could ask for testers on our Discord server?

Yeah I think that would be nice 👍

[ReenigneArcher]

We do use ffmpeg on Windows as well. I'm not up to speed on the standalone nvenc encoder, but I know the other capture methods do use ffmpeg.

[cgutman]

NVENC is not (meaningfully) limited for our purposes on consumer GPUs. Only NvFBC has this restriction of being completely unavailable on consumer GPUs, so only making this change for NvFBC seems correct to me.

If someone can validate that this still works on cards with official NvFBC support, it seems safe to merge.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 7.98%. Comparing base (17e0f1a) to head (e850162).

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #2471      +/-   ##
=========================================
- Coverage    8.02%   7.98%   -0.04%     
=========================================
  Files          88      88              
  Lines       18036   18039       +3     
  Branches     8596    8596              
=========================================
- Hits         1447    1441       -6     
+ Misses      13839   13761      -78     
- Partials     2750    2837      +87     

Flag	Coverage Δ
Linux	6.05% <0.00%> (-0.07%)	⬇️
Windows	3.76% <ø> (ø)
macOS-12	9.03% <ø> (+0.01%)	⬆️
macOS-13	8.93% <ø> (ø)
macOS-14	9.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
src/platform/linux/cuda.cpp	1.81% <0.00%> (-0.02%)	⬇️

... and 25 files with indirect coverage changes

[ReenigneArcher]

I'm not sure anyone has been able to test this. I do have an extra Quadro M4000 card (I guess it has native NvFBC, but I'm not 100% sure), but I will have to deploy a linux environment to use it in so might take me a little bit.

If anyone else already has the os/hardware combination to test this build it would be greatly appreciated.