I recently updated my server to 1.21.11 from 1.21.8. In the process, I've updated both DH on server/client to 2.4.5-b fabric and have noticed while playing stutters will occur a lot. I noticed there's a GitLab issue which is very similar to this case, https://gitlab.com/distant-horizons-team/distant-horizons/-/issues/1200 with the video showcase perfectly replicating what I'm experiencing.
I've tested DH without it being on the server and just client and it works w/o any stuttering. It seems that it only occurs when DH is on the server.