Migrate from local minio to backblaze as an s3 provider
Hello, I was writing another question about issues that I thought had to do with my Backblaze configuration, because I had copied the contents of the minio bucket to the Backblaze bucket, and things were not working.
After further testing, I have realized that the way files are uploaded to minio is different to how they are uploaded to Backblaze, which causes the issues. However, I'm trying to migrate my data away from my local minio instance.
Is there any way I can do so? I have scoured the documentation and issues but found nothing. I'm not expecting a fully detailed set of instructions, just for a pointer as to how to do it. Perhaps through the replication mechanism that ente has? Or would that be invalidated given that I previously had set
are_local_buckets to true? Or is there a way that I can convert the minio file structure to what should be in backblaze?
Thank you in advance, love the product, I've been using ente for years now.5 Replies
I'm sorry, I couldn't find the answer in the documentation.
-# If your issue is resolved, feel free to use the /solved command to close this thread. If you'd like to ask me another question use /docsearch
Entes replication should be fine for that. I dont have my museum.yaml at hand rn, but you can have a look at the samples in github.
Technically it works sth like this: you upload a picture and museum forwards it to the main s3 instance. Every like 30 seconds runs a small routine which then grabs the pictures from tmp/replication or from the s3 itself and puts it to the second s3.
Same goes for the third
Both MinIO and Backblaze are S3 conpatible storage and you can use rclone or s5cmd or other CLI tool for syncing up while using museum.yaml to set B2 as the primary bucket, and have 2 MinIO buckets as replica (you need exactly 3 buckets in total)
With latest server image being pushed, you can set are_local_buckets and use_path_style_urls for individual buckets which reduces the initial hassle with using MinIO alongside other external storage providers.
In case you're unable to restore, you can use a plaintext backup using CLI and use that for restoring.
Finally managed to get it working, I was being dumb. In case anyone comes across this in the future, I had my minio instance running locally, and was copying the contents of the filesystem to Backblaze using rclone, instead of setting up the minio instance on rclone as a remote, and copying via the S3 API from minio to Backblaze. I didn't understand the difference between block storage and object storage, and that was causing my issues. Thank you for the support.
Thread marked as solved. It will be closed in <t:1755555548:R>.
This thread is now closed.