TCP port external mapping keeps changing every time pod restarts.

I’m setting up my remote development environment using ssh sftp. Everything works fine. However, every time the pod restarts, the external mapping for port 22 keeps changing, which makes my local IDE unable to connect to the remote because now it’s on a different port. Is there a way to fix this? Or any workaround if it’s by design? Thanks in advance.