Immich•2mo ago

Advise on backup strategy with restic

After a huge ordeal (as I'm running Immich in Home Assistant) to run restic, I finally got everything working. I'm amazed at how good restic is (way faster than rclone, and way more efficient storage wise, and it also solves a file count limit problem for Google Shared Drives). It's creating my first full snapshot now. I'm looking for several opinions to learn (as everyone will do things differently and there's not an absolute right way here). 1. I wanted some advice on my retention policy, as tbh I've never had a backup system like this. What's a good one like? I've set this one for now, but idk if it's overkill.

restic \
    forget \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 12 \
    --prune

restic \
    forget \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 12 \
    --prune

2. At the same time, I'm generating a db dump for every backup (daily), but I'm not currently removing old db dumps. Since the backups are just snapshots, would it be right to just delete all old db dumps once the new one has been created? Since the restic snapshots will handle the "staging" anyway. It feels wasted transfer every time (even if it's not a huge size relatively). 3. Since I'm also building these db dumps manually (see screenshot), it also feel like the automated default dumps from the server feel useless, and again add to the transfer. They won't be in sync anyway for the backup. Would you stop them being created? Thanks!

7 Replies

Immich•2mo ago

:wave: Hey @Chaos, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

bo0tzz•2mo ago

1. Overkill retention is rare; because of the deduplicated & incremental backups, keeping an extra copy is very cheap, so might as well keep many 2. Leave them as just sql (not gzipped or such) and restic should be able to deduplicate them, but it's also fine to only keep one sql file in a version of the backup as you say yes. There shouldn't be wasted transfer because of the incremental nature. 3. Up to you. It's not much extra space & an extra copy has never hurt me, but it's fine to turn off too

ChaosOP•2mo ago

1. Can I ask to see your version? Just out of curiosity. 2. No, the command in the docs is gzipped IIRC. They are def compressed. Then I guess it cannot dedup, so?

docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=<DB_USERNAME> | gzip > "/path/to/backup/dump.sql.gz"

3. Ok Thanks!

bo0tzz•2mo ago

1. https://github.com/bo0tzz/kube/blob/main/apps/immich/backup/backup.yaml#L13-L16 this is very little tbh, I should probably increase it 2. Yes, I'm saying to skip the compression if you want to use deduplication optimally. But also the files are small enough that it probably doesn't matter a lot

Immich•2mo ago

    retain:
      daily: 2
      weekly: 1
      monthly: 2

    retain:
      daily: 2
      weekly: 1
      monthly: 2

ChaosOP•2mo ago

are these the right commands to backup and restore the uncompressed db dump?

docker exec -t immich_postgres \
  pg_dumpall --clean --if-exists --username=<DB_USERNAME> \
  > "/path/to/backup/dump.sql"

docker exec -t immich_postgres \
  pg_dumpall --clean --if-exists --username=<DB_USERNAME> \
  > "/path/to/backup/dump.sql"

cat "/path/to/backup/dump.sql" \
| sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" \
| docker exec -i immich_postgres psql --dbname=postgres --username=<DB_USERNAME>

cat "/path/to/backup/dump.sql" \
| sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" \
| docker exec -i immich_postgres psql --dbname=postgres --username=<DB_USERNAME>

bo0tzz•2mo ago

Looks like it yes Always worth testing your restore procedure though

Gaming

Programming

Advise on backup strategy with restic

Did you find this page helpful?