Summary and solution:
On Bazzite, the current stable release of PCSX2, v2.4.0, crashes if a Steam Controller is connected.
Solution: for now, either do not use a Steam Controller, or use a nightly version that is v2.5.200 or newer. Alternatively, you can turn off “Enable SDL Input Source” in Settings->Controller.