Alert on "GPU has fallen off the bus" message

[andy108369]

Hi, we have the NVIDIA 4090 GPU has fallen off the bus on one of the nodes plugged into the Netdata, however there are no reports nor alerts in the Netdata regarding this:

root@node1:~# dmesg -T | grep NVRM
[Thu Apr  4 18:03:56 2024] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  535.161.07  Sat Feb 17 22:55:48 UTC 2024
[Fri Apr  5 21:29:39 2024] NVRM: GPU at PCI:0000:a1:00: GPU-979426f2-893a-7cbb-c4cf-81472f89a462
[Fri Apr  5 21:29:39 2024] NVRM: Xid (PCI:0000:a1:00): 79, pid='<unknown>', name=<unknown>, GPU has fallen off the bus.
[Fri Apr  5 21:29:39 2024] NVRM: GPU 0000:a1:00.0: GPU has fallen off the bus.
[Fri Apr  5 21:29:39 2024] NVRM: A GPU crash dump has been created. If possible, please run
                           NVRM: nvidia-bug-report.sh as root to collect this data before
                           NVRM: the NVIDIA kernel module is unloaded.

Can you please add this to the alerting? This is highly critical to us since we are the GPU cloud provider.

Thanks!