I sent an email off to the vendor, Creality seems to be busy with other things to reply. Hopefully they can offer me a replacement sensor, they aren't exactly a 3d print shop, but a new sensor is 40€ which is a tad much to swallow just two weeks after buying the printer.

Went through the Marlin firmware and found some lines that seem to relate to a function in bltouch called "stow" which apparently stows the probe and clears the alarm. I have managed to setup the SKR board now, hopefully that stow function is available during print, it may solve the issue temporarily. Else, I wonder if it is possible bltouch has more functionality to perhaps entirely turn off the probe during print, perhaps simply removing power from the 5V input it uses? I may just open an issue ticket with Marlin to see what they say to that.