Weird GPU Driver crashing issue

Hey Guys I am having weird issue I am unsure how to debug or fix. So I have been using Bazzite for a good half a year and it works perfectly pretty much. All my games run great with no issue. I have uploaded a picture of my Specs. This is a GPU passthrough VM. I am passing through a RX 7800XT. I have never had any issue with any game or web browser or most apps. Now heres where the issue happens. If I try and use ANY kind of remote desktop client to RDP into a windows machine. Literally any client app it will work fine. But after a random amount of time usually within an hour it looks like the graphic driver dies. the screen freezes, it flickers, I start seeing artifacts till eventually KDE restarts or the OS dies and I have to reboot it. This doesn't just happen with RDP but also happens with chiaki (the PS5 Remote play app). So it seems any app that involves remote control or remote play causes my GPU driver to just crash. This has bene happening for months and I don't know why and ive been unable to troublshoot. Any help would be appreciated.