undoublethink
JCHJava Community | Help. Code. Learn.
•Created by undoublethink on 4/10/2025 in #java-help
Uploading a large file using a HttpClient
I'm using Methanol (an extension over the standard
HttpClient) to upload a large file to a server.
When I track the file upload progress is appears to buffer 2GB at a time. This causes OOM errors unless I increase the memory allocation pool for the JVM which is undesirable.
I've looked through all the documentation and all mentions of internal buffers have the default around 16KB which is much smaller than what I am seeing.
I initially thought the issue was either due to sending it as multipart form data or because it was trying to read the entire file into a buffer before sending it but I see the same behavior with just a regular BodyPublishers.ofFile.
I'd appreciate any insight into this behavior.4 replies
JCHJava Community | Help. Code. Learn.
•Created by undoublethink on 3/31/2023 in #java-help
Best way to implement a thread-safe reactive collection elements?
I have a backend service that performs a long running task. The task is multi-threaded and produces
Results. Every time a Result is produced it gets pushed into a collection.
My frontend should be able to view these Results as they are submitted but it must also be able to get the previously submitted results. It should be able to get the results in the order they were submitted.
My plan is to use server side events/websockets to sent the results to the frontend. On connection it should receive a "catch up" of all previous results then it will handle new ones as they come in.
I think I can accomplish this using something like reactor-core Sinks.many().replay().all() but I'd prefer to avoid bringing that in.
My problem is that I'm not sure how to make the previous results safely (and simply) "observable" in this manner. Ensuring that a new client never receives a duplicate result (if a result is submitted while it is in the "catch up" phase) and that it gets events in the correct order.
Is something like a custom SubmissionPublisher a good starting point?
Does anyone have any suggestions for how to go about this?16 replies