Performance impact of DH plugin on Paper server?
Does anyone have a rough idea of the performance impact of the DH server-side plugin on servers? I have a 7950X3D machine.
66 Replies
Use the latest nightly for the best performance. It's pretty good. My 3700x manages over 900 cps for existing areas of the world, and close to Chunky's performance for new areas
When it comes to general operation; it runs fine on a Raspberry Pi 5 with only 1 gig of RAM
Thanks. I don't doubt that it runs fine but I also care about what it's impact is on mspt
Like for example if it adds 20% to mspt, that would be a problem on servers with high playercount
The plugin does almost nothing on the main thread, so it should not affect your MSPT at all unless the server is over loaded
The Pi maintained 20 TPS during testing
It does this every startup. It just sits like that for 2-4 minutes and then everything runs as normal. Seems like something is being done on the main thread in startup that takes a while.
Yes, it does a DB optimization on every launch
Although I don't want to halt the server startup for several minutes
My world is 26k radius. Could the DB optimization be done off main thread and just not send chunks to players while it's being done?
Thank you for your consideration
Yes, there's a mechanism to pause generation, so I could do that
Hang on, let me push a change to disable it until it can be handled better
Disable DB optimization alltogether?
Is it not totally essential
Not really. It's a "just in case". It only needs to be done after a trim operation, and that's done expliciltly
Riight
Download the latest nightly when this build completes
Are you familiar with Spark reports?
Asking since I want to send over some allocation reports and see what you make of them.
Yes I am
Seems that the plugin allocates a lot of memory. Like the most out of all plugins in my spark-report.
https://spark.lucko.me/etDPsfFcEG
spark
spark is a performance profiler for Minecraft clients, servers, and proxies.
got this with
/spark profiler start --timeout 300 --alloc --thread *Also me and my players get this every now and then.
Side-note the world looks amazing with this
You appear to be doing generation during this profiling, so high memory allocation is expected. Please remember that allocation and actual usage are not the same thing. If you have 2 scheduler threads, then the plugin can run at most 2 builders at any given moment, and hence will use only the memory required to load 32 chunks and build 2 LODs. If your heap is small then GC will run frequently during this time. If your heap is large, then Java will take advantage of that to run GC less frequently. In other words, the more memory you give it, the more memory it will use. This is not unique to this plugin, but is how Java (or any other garbage collected language) works.
The "error while fetching full data source" message is known, but I am unsure what's causing it.
If you're using a web panel to manage the server, and the panel is killing the server because it uses too much memory, then that's a misconfiguration of your panel. Some Pterodactyl eggs for instance will set the max heap size (
-Xmx) equal to the server's memory size. This is incorrect and will cause the panel to kill the server even when it has not ran out of memory
The heap size is how much memory the Java application itself can use, but the Java VM itself also needs some memory to function. If you set a heap size of 1 GB, the total memory used by the Java process is likely to be somewhere close to 1.2 or 1.3 GBsI am aware of this actually
I give my containers "infinite" memory and set the Xmx in the startup flags
Are you experiencing any specific issues with memory allocation, or was the usage just a surprise?
Well I have very frequent GC. I feel like that is an issue, but I suppose that's because of LOD generation?
I would say so, yes. Ideally you would set a world border, or a border in the plugin config specific to LOD generation, then use the pregen command to pre-generate the LODs for that area. Once that's done you will only see generation for chunks that change, ie. where players are actively building
Might real-time updates cause memory allocation too?
Because I pregenned the overworld
Pregenned in chunky and then in
/dhs"Real-time updates" is only the act of sending updated LODs to players immediately after an existing LOD has been updated. The server will do generation for chunk updates even if real-time updates are disabled.
If you disable this, then players will only receive the updates after reconnecting (or moving out of range of the updated LODs and moving back)
And does the LOD get regenerated after every single block change?
No, generation is "debounced" by 5 seconds
Can that be configurable?
Because I have real-time updates off anyway so people won't be noticing that the LOD generation has a delay of 5/10/60 seconds
Plus only like ~5 people on the server have the mod installed
Fair point. The more you delay it, the more work it has to do for every cycle, but some adjustability makes sense
I had a similar issue with Pl3xmap allocating a lot of memory because it generated the map constantly after minor changes. I switched it to regenning the map each time the server starts (daily) and it now has far less performance impact
Plus I have a feeling that the performance impact is better for updating an LOD once with 10x as many changes than updating an LOD 10 times for 1 change each time. Just an example ofc.
Yes, it will depend on how your players are behaving. With a lot of activity in a small number of chunks, then it would be better with a larger interval
Try this build when it completes
That was quick, holy cow
🙏 I am at work, so haven't tested it
What was the config option meant to be called
Should it just automatically be in my config.yml after loading the server with the new jar
Or do I have to add it
Because I don't see the new option and I still see
config-version: 7 when the commit says config-version: 8Config files are not automatically updated. The version number is used to detect if your config is older than the plugin expects, and if it is - then missing values will be loaded from the default config
You can either delete your existing config and have the plugin create a new one on next startup, or you can add
lod_refresh_interval: 5 to your existing file.
5 here being the number of seconds between updatesright right
Can you provide a
/spark heapsummary when your server is using more memory than you expect?At the moment it's running fine now that I've set
lod_refresh_interval to 300. I have another question. Does increasing the amount of scheduler threads increase the RAM usage even when DH isn't doing much generating? Like for example if I make it so that it uses 12 threads but those threads are only really used once every 5 minutes, do they still take up a lot of RAM? Or do they only take up RAM when they're actually being utilised?The threads don't do anything if it's not generating, but more threads means it can generate more in parallel
So peak usage during generation will probably be higher with more threads
And are the threads only created when there's stuff to generate? Or are the threads just always there waiting to given a task?
Both, sort of. Threads are created when needed, and reused once they become idle, but not shut down until the server shuts down (when the plugin is disabled). So if you set the number of threads to 100 it won't immediately create 100 threads
But if it has 100 generation tasks to perform at the same time, then it will create all 100 threads
And is there a limited amount of threads the server can allocate?
It's probably possible to enforce some sort of limit on the number of threads, but hundreds or even thousands is possible.
I can introduce a thread timeout if you think it'll have any value, although I suspect it'll only reclaim a few megabytes of memory at idle
There you go. Threads will now shut down if they're idle for 60 seconds.
Thanks
And yeah I'm not too familiar with threaded programming
https://mclo.gs/036FRaY
Getting this errorspam in console on leaf 1.21.8 with the latest build of DHSupport from gitlab
Comment out that event in the config
It's not anywhere in my config.yml for me to uncomment it
Then presumably you're getting the "your config is for an older version" warning on startup
Should I delete my config.yml and let it regenerate?
@Jckf (GMT+2) Is there a better place in discord to discuss the DH server plugin?
What are you looking for?
Well I wanted to discuss the plugin more generally rather than simply just about performance. I decided to just make an issue on the repo, but I still think a dedicated channel in this server is necessary since the plugin seems to have a sizeable user-base.
GitLab
Feature Request:
/dhs clients command (#40) · Issues · Distant-...Just a simple command that tells you how many connected clients have DistantHorizons installed, and maybe perhaps even some extra info. Mainly I just want to know how...
What you're requesting already exists
See "/dhs status"
Fair enough. Thanks
Also I'm still getting weird warnings in the console even though I reset my
config.yml
https://mclo.gs/KnYztV5
Its trying to calculate the color of a beacon, and the server is reporting a block color as "LIGHT"
Weird
Light what, server? Light is not a color
Maybe somewhere it was meant to be written
LIGHT_BLUE and got cut offMaybe the color name comes after a space and I have processed the text incorrectly
Hm you are correct. I am splitting on underscore for some reason
Cannot recall why
Happens
Ah no I remember. The color name is taken from the block name, so BLUE_WOOL is supposed to return just BLUE
But of course then LIGHT_BLUE_WOOL turns into just LIGHT
The color issue should be fixed now. Grab the latest nightly.
How often are LODs regenerated when edits are made to the world? I run a 64-player speedrunning tournament and might be interested in adding the DH plugin, but if LODs are constantly being regenerated and may cause issues, I will not add it
From reading the discussion here, seems like it's all done off main thread so maybe not a concern? Running a 9950x with 5 threads + -Xmx 32GB
Currently pre-genning world with Chunky, then will have DH go back over and generate LODs once Chunky is done
@Isaac Pretty much everything this plugin does is off the main thread. LOD updates are by default processed every 15 seconds, but this is adjustable in the config. Be sure to grab the latest nightly build to get the most recent features and fixes.