I have a Fabric server that currently only has 10 active players. I have multiple optimization mods, I've loaded the first 15,000 blocks from spawn in every dimension and the server still gets overloaded.
If anyone else is interested in this post I can say that I have found ways to make the general server have almost no problems up to 17 players with the same hardware
I've added several optimization mods aimed towards entity and command ticks. And dimensional threads. When I get home I will give more information and do more research on CPU pinning