Move to new database vector extension
Hi there 🙂
I've been struggling to understand how to migrate to the new database vector extension as I don't have a super standard setup 😉
Could one of you help me with this migration process?
Setup:
- OS: TrueNAS Scale (23.10.2)
- Deployment: Integrated Kubernetes (k3s)
- Configuration: https://github.com/v3DJG6GL/truecharts_archive/blob/main/stable/immich/15.18.3/ix_values.yaml
- DB: CNPG
This is the .yaml part that configures the CNPG currently:
Note: I am the maintainer this repository because the initial maintainers dropped support for TrueNAS SCALE a year ago.
9 Replies
:wave: Hey @ysmfwns3,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:I have looked into this a little more closely:
1. It seems that I have to modify
common/templates/class/cnpg/_cluster.tpl to preload needed libraries:
- Currently, this code loads the needed vectors.so library if objectData.type is vectors:
- I suppose this modification would additionally load vchord.so library during transition:
But I suppose the objectData.type will change too with the new library, right? But I haven't found which objectData.type is used for vchord.so
2. Additionally, I suppose I need to modify common/templates/lib/cnpg/cluster/_bootstrapStandalone.tpl:
- As of now, this code creates the extension vectors if objectData.type is vectors:
- This modification would also create the extension vchord during transition:
But I am facing the same issue here: what objectData.type is used for the new extension?
Or is this a value something which the initial maintainers set by themself? If that's the case, I just could set a new type; e.g. vchord...🤔https://github.com/immich-app/immich-charts/issues/149 has some discussion on how to run the new db extension with cnpg
I can't help with any of that truecharts mess
objectData.type seems like some custom value used to only run certain sql conditionally during the cluster booststrap. I can’t say if there’s a deeper meaning behind the specific name (like if it’s interpolated into a command somewhere)
thanks for your replies 🙂
I am trying to setup a new immich chart with the new vector extension configured. Seems saver to first get a new installation running than migrate my current installation.
I think I have found almost all the parts that have to be modified inside common.
But I am currently struggling wit this error when deploying the chart:
I have added these containers to
common/values.yaml:
And it seems as if the deployment finds the value postgresVectorchord16Image but the image is not pulled...? 🤔
tbh, I definitely should move to a real kubernetes deployment and stop messing around with that. it's terrible - but I don't have the time for that currently 😅
alright, test instance is running 😄
it was a typo: I had to set postgresVectorchord16Image instead of postgresVectorChord16Image
Furthermore I changed the repository and had to use at least v0.3.0:
Now I only need to figure out how to flawlessly migrate old instances to the new vector extension 🫠
update:
I managed it by manually importing & exporting all tables to the new instance with new DB vector extension.
Seems to work so far 🙂
is there a reference how the db should look like?
some tables were not present on the new setup
- typeorm_metadata
- migrations
I suppose the first table is related to TypeORM that was dropped with immich v1.132.0 and thus not used anymore
but I am not sure whether the second table should exist... 🤔I think the second one is also a typeorm table
Hi! I have a problem. Trying to migrate to new vector things. But image is always rebooting. Here is some logs
ok, good to know!
And I suppose that immich somehow checks the database integrity and would report/log if there are irregularities?
that's the content of the
migrations table btw:
Yes, it checks the migrations and kysely_migrations tables among others and will abort if there’s anything unexpected