Insufficient memory during pregeneration
Hello!
I've run into a problem with pregenerating a minecraft world using Distant Horizons. Whenever i try to pregenerate a somewhat larger chunk of the world (256 chunk radius in this case), the pregeneration eats up all my RAM causing a bottleneck and essentially making the server unresponsive.
I haven't run into this issue while playing 1.21.4 and using 2.3.2b. I could generate even 512 chunk radius in the overworld + half of that in the nether and the end in one go without any issues (i7-9750H, 8 GB 2667 MHz allocated RAM, running DH [Balanced preset, Internal server mode] + Fabric API + Lithium + FerriteCore + C2ME). So i'm wondering whether this is an issue with the current version or something is wrong on my part.
Testing was done on a laptop with an i7-9750H and 16 GB 2667MHz RAM (8 GB was allocated), and also on a PC with a Ryzen 7 5700x and 32 GB 3600MHz RAM (tried with allocating 8 GB, 16 GB and 24 GB).
Steps:
1. Run Fabric installer 1.0.3 (1.21.7., loader version 0.16.14.)
2. Download the server jar from the fabric installer GUI and generate a start.bat file
3. Run the server, terminate, agree eula, run it again, terminate again, put DH 2.3.3b into the mods folder, run it again, terminate again
4. The generation mode is then set to INTERNAL_SERVER in the config file
5. Run the server again, then use dh pregen start overworld 0 0 256 (using BALANCED preset)
After the pregen starts, my Ryzen 7 5700x can generate 70-80 cps on a balanced preset. During the pregen, the RAM is slowly filling up until it hits the Xmx parameter, overshooting a bit, and after some minutes i get the following message:
"Insufficient memory detected.
This may cause stuttering or crashing.
Potential causes:
1. your allocated memory isn't high enough
2. your DH CPU preset is too high
3. your DH quality preset is too high"
After that the cps starts to tank and at this time the server is basically unresponsive to any commands, doing pregeneration until terminated externally.
12 Replies
What i've tried so far:
- Different mod combinations
- DH + Fabric API + Lithium + FerriteCore + C2ME + FasterRandom
- DH + Fabric API + FerriteCore + C2ME + FasterRandom
- DH + Fabric API + C2ME
- DH + Fabric API
- DH only
- Nothing changed
- Allocating 4 GB, 8 GB, 16 GB, 24 GB RAM
- Higher amounts of RAM will delay the error but essentially won't stop it from happening
- Setting the CPU preset
- On MINIMAL_IMPACT i was able to generate a 200 chunk radius until i terminated the pregen
- RAM was still slowly filling up to Xmx, but i did not see the error (maybe i didn't wait enough?)
- This is of course slow (30-40 cps)
- After terminating, i got a different error "Distant Horizons overloaded, too many chunks queued for LOD processing"
- Using smaller and smaller additional pregen radii (128 -> 192 -> 256 ...) and resetting the server between the pregen batches
- Need to find the right balance between the radii and the preset so i won't encounter the error
- This is the current workaround i'm using
What i haven't tried so far:
- Using different quality presets
- Correct me if i'm wrong but AFAIK this is a client-side setting and i was using only the server
- Generating in-game on a singleplayer world
- Waiting until the pregen actually finishes after getting insufficient memory
- CPS decreases to 10, and a lot of "Generation for section x has expired!" messages pop up
I've seen other posts with the basically describing the same problem, where either no solution was found, or a workaround was sufficient. For me, using smaller and smaller additional pregen radii works.
My only question is: is this behavior expected on the current version, or could this be some kind of memory leak?
Sorry for the long post, i don't have a potato, but a huge thank you for all of your hard work.
One of the logs, here i'm using the highest CPU preset to quickly demonstrate the problem (happens with other presets too except MINIMAL_IMPACT, only a matter of time and the amount of chunk radius to pregenerate)
latest.log uploaded to mclo.gsUploaded by koenigx3
i have the exact same laptop specs and it seems am getting nearly the same issue just not when generating server side, when i pregen single player the cps starts strong, 60 - 70 cps, but after couple of minutes its gets down to 7 - 10 cps which is horrible
i never got the memory overloading error except for once while in the world, but the symptoms and the slowness of the pregeneration are similar to your situation
if you find a solution please please update me, i have tried countless things as well to no avail, so please and thank you
Update: I have been generating DH chunks in-game on a Singleplayer world (Internal Server mode, highest CPU preset), it's working so far without any problems. I'm starting to wonder whether the JVM arguments are faulty, however i tried several combinations when generating on the server, even the ones that were recommended by the DH wiki, and it did not help.
what arguments were you using
First i tried with what the fabric generated .bat file gave me (which is basically just -Xmx)
Then i also tried the recommended ones on the wiki (https://gitlab.com/distant-horizons-team/distant-horizons-server-plugin/-/wikis/Recommended-setup)
GitLab
Recommended setup · Wiki · Distant-Horizons-Team / Distant Horizo...
Add support for Distant Horizons to your Bukkit server.
those are for the plugin, and also probably just a bunch of useless flags. The only thing I'd ever add is zgc and zgenerational for jvm flags, and that only above 12-16gb of allocated ram.
default jvm args are good as is
I think the default arguments from the minecraft launcher are -Xmx2G -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:G1NewSizePercent=20 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=50 -XX:G1HeapRegionSize=32M
yes
I've only changed the Xmx, and in singleplayer it works. I don't know whether it is related to any part of the problem, but at least it works.