Server loading slow with pregen chunks?

I have my own home server and am running all the mods 10.
Its just me and one other player.
I used chunky to pre generate like 20000 chunks but when flying around the chunk loading and view distance is too slow. Is this just something I gotta deal with or can I change or add something?

Specs:
OS: linux/ubuntu
CPU: I5 11600f
SSD: 2tb NVME Samsung 980
RAM: total-64gb allocated-12gb