Server-side pregen very slow
I'm running a server on my pc to pregen chunks (I have a 7800X3D cpu) but is generating maybe 0.1 chunks out of the 1000 every second. How can I make it pregen faster?
For context, I am running a server on Oracle Cloud free tier, but it would take a while to generate on that limited hardware, so I'm going to pregen on my PC then transfer the world to the server via FTP.
70 Replies
I've changed common.threadPreset to I_PAID_FOR_THE_WHOLE_CPU but is still not running as fast as i'd like. would it be faster to pregen in singleplayer then transfer it over?
i have an oracle cloud free tier and pregenned in singleplayer and transfered it to the server. took a while, but much less time than on the server using /pregen
the servers on oracle's machine or your 7800x3d? my 5600x was able to pregen a 256 radius within a few hours, so yours will go by even faster
oracle's is some shared ampere cpu
i have a 7800x3d
can you explain how you pregenned in singleplayer? did you just open your world and let it run?
yeah. for me before i even created the world on the server i booted up a singleplayer world, turned on distant generation, internal server, cranked it to "i paid for the whole cpu", and let it sit. turned down my fps and resolution as well as shaders. once i did that, copied the world in .minecraft/saves over to the server through whatever is the default for winscp and waited even longer
singleplayer stores the sqlite in the same path as it would on the server, so no need to mess around with it. what i ended up doing was copying the sqlite to the client's server data folder so it doesn't have to load it through the server, too.
i can attest to how slow the oracle cloud free tier is, put it on balanced overnight to generate the nether and out of a 128 chunk radius it got 7 chunks deep
i actually figured out how to change the presets like "I paid for the whole cpu" and am now running that in the oracle server
running for about 10 mintues and its already 70/200 chunks around my base area
70/200 total chunks or 70/200 chunk radius?
uh chunk radius i think
what?! not even my 5600x is that fast!
idk lol
i set the cores to be utilized to the max as well maybe that changed it
what command did you use?
wait! this is really fast! eta 1 hour!
128 radius is still 4x smaller than 256, but damn!
uh /dh config threading.maxThreadCount I think??
it tab completes if you use it in-game
I set it to 4 threads so
/dh config threading.numberOfThreads actually
yea i set it to 4 cuz thats how many i have on the image
strange how its going faster than it was on my dedicated server hosted off my pc tho
i didnt set the pregen to INTERNAL_SERVER tho, it was still on FEATURES
so i ran it again with internal_server but it only lasted like 1 minute so im not sure if it really loaded the chunks
this is strange tho
i think the server is just still sending the LODs to my client, should fix itself with time
unless ARM is crazy for gen 4 threads compared to the 12 on my cpu it doesn't make sense on how it's so fast
whats weirder is comparing 2 threads to 4 the eta for the same gen went from greater than 1,000 hours to 1
/generators
- FEATURES: Complete generation of all LODs with mostly correct structures and trees. Does not save vanilla chunks, and is much faster than INTERNAL_SERVER unless C2ME is installed.
- INTERNAL_SERVER: Generates and saves both DH LODs and vanilla chunks, has 100% correct LODs, because it also saves vanilla chunks it takes the most amount of drive space. Generation speed will suck unless C2ME is installed.
- PRE-EXISTING: generates LODs for all existing vanilla chunks.
Internal server is gonna be slow unless c2me is installed
c2me is installed but speed still sucks lol
On the 7800x3d or the oracle cpu
When you run pregen on your 7800x3d, can you check task manager or whatever system monitor you use to see if your cpu is actually being used? Also check how many cores are being used.
https://docs.google.com/spreadsheets/d/1lFO4bC4OhHHCC6eqGjNbNlcj6OotxNAJ4mKBT396Wx8/edit?usp=drivesdk
You can see on this spreadsheet if you to to the "Standardized Headless Data" how fast a 7800x3d should be
Google Docs
Distant Horizons Community Benchmarks
both the arm cpu and my 5600x are pinned at the max when I was genning, yet the 4 thread, lower clock OCI cpu preformed roughly the same (or even better) than my cpu with a higher clock and tripple the thread count. seems odd
the 7800x3d
i will try it again. what are all the settings i should have set for max speed?
Set the thread preset to "i paid for the whole CPU"
Then when it is running check if its using all your cpu in task manager
in the server i run on my PC correct? i havent had any issues generating in singleplayer i dont think
Yes, the server
It should be
dh config common.threadPreset or something like thatok got it. ill delete the current DH config file and then just mess with that to make sure i didnt mess anything up
Good idea
this mod list okay?
Try only DH for now
Then we can compare to the baseline 7800x3d in the sheet
ok, no internal_server shenanagins right?
Nope
Features generator
got it. ill load a new world too
Just default config with the thread preset changed after it launched the server
yep. then
dh pregen start minecraft:overworld 0 0 256 okay?Yep
ok yup thats 100% cpu
already at 50/256 haha
That number is the radius, so it will increase slower when it gets higher
You wanna look at the cps
It should be somewhere around 253 cps
cps is steady around 255
yep
So it works correctly
ok sweet
internal server actually generates the chunks in game, right? not just LODs?
/generators
- FEATURES: Complete generation of all LODs with mostly correct structures and trees. Does not save vanilla chunks, and is much faster than INTERNAL_SERVER unless C2ME is installed.
- INTERNAL_SERVER: Generates and saves both DH LODs and vanilla chunks, has 100% correct LODs, because it also saves vanilla chunks it takes the most amount of drive space. Generation speed will suck unless C2ME is installed.
- PRE-EXISTING: generates LODs for all existing vanilla chunks.
Yes
Thats why c2me is recommended
Because it uses the vanilla chunk generator
ok, but it'll still be much slower than just features?
No
even with c2me?
oh ok
let me test that out then
Should not be very much slower
I think
Should be within variance.
ok im gonna give it a shot on internal server now that ive added c2me
new world again
stuck at around 175 cps now. CPU usage is jumping between 70-100% instead of the steady 100% it was at during the features generation
normal?
The default c2me config doesn't use all threads
oh gotcha. ill look at that rq
Solution
Set
globalExecutorParallelism to your thread count in the c2me config fileSo 16
Also redo the world then
got it
restarting it now
oh its actually made it up to 200 cps
oh yep, already better
gonna let it run to see what the cps looks like when its fully under load
hanging around 200-220 cps now
Sounds about right
ok sounds good
Its gonna have to do more things saving the chunks as well
yep
then wahts the best way to get these to my client? do i need to change any settings? the pregen just finished but things arent really filling in all that quickly
does the server need to restart first?
whats causing this stuff?
The fastest would be to copy the DB to your client
ah ok
my brother is... less technically impressive haha
Otherwise
anything i can do to increase speed?
You can change the max bandwidth
Or the ratelimit settings
There's one for synconload and generation i think
just within dh config?
Yea
You only need the command
Usually you don't need to change any of the settings that aren't in the command list
ya and the command is easier anyway