I
Immich•3w ago
Dr America

Trying to upgrade from unknown previous version

I'm trying to upgrade up to latest from a previous version, last updated sometime before the vectorchord change, but I'm running bare metal on Arch and running into a problem of not having certain tables/relations, namely "face_index" and "clip_index". It is refusing to start without these, and I don't know how to generate them. I also never used the machine-learning portion before, so it shouldn't have any data worth keeping anyways, if what's needed is dropping some tables. If someone would be able to help me or point me to a resource to help me either get these new relations, be it by creating them myself, or by re-creating a new DB and importing in my old data and doing any necessary upgrades to it, I'd be very appreciative.
12 Replies
Immich
Immich•3w ago
:wave: Hey @Dr America, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.
Dr America
Dr AmericaOP•3w ago
Dr America
Dr AmericaOP•3w ago
The face_search and smart_search tables/relations also do not exist Also the "corrupted migrations" were from trying to revert to an earlier version from latest, but I just copied my backup again to try another earlier version
Immich
Immich•3w ago
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Dr America
Dr AmericaOP•3w ago
Immich
Immich•3w ago
[Discussion] TypeError: Cannot read properties of undefined (reading 'dimsize') (immich-app/immich#17332)
Dr America
Dr AmericaOP•3w ago
at least on version 1.30.3 is there any simple way to build a new immich db and then just import all of my old data into it? if so, then I could just 1. save old data for backup/restore 2. delete old immich db and create a new one 3. import all of my old data into new table and then it should work (hopefully)
Sergey Katsubo
Sergey Katsubo•3w ago
get these new relations, be it by creating them myself, or ...
You can check DB schema drift with 
docker exec --user root immich_server sh -c 'cd /tmp && DB_URL=postgres://$DB_USERNAME:$DB_PASSWORD@database:5432/$DB_DATABASE_NAME node /usr/src/app/server/dist/bin/migrations.js debug && cat migrations.sql'
docker exec --user root immich_server sh -c 'cd /tmp && DB_URL=postgres://$DB_USERNAME:$DB_PASSWORD@database:5432/$DB_DATABASE_NAME node /usr/src/app/server/dist/bin/migrations.js debug && cat migrations.sql'
(assuming default database service name in compose and version 1.136+, for earlier versions remove server/ from the path to migrations.js) The tables face_search and smart_search may be missing/lost if you restored from backup without the required ML/vector extension. PG restore would try to create tables. And fail. Because these tables depend on extensions.
Zeus
Zeus•3w ago
I recommend starting here by figuring out what version you’re coming from. Then you’ll need to make all needed breaking changes Good chance you jumped over a lot of stuff
Dr America
Dr AmericaOP•7d ago
So, I've got version 1.25.2 working, which is about the last version I remember working The problem is that I never actually went and enabled the machine-learning part of it, as I'm technically on the unofficial Arch Linux build, and the ML side was breaking a lot with my GPU when I last tried to use it I really just never had anything in these tables, so would there be a way to just create them in this version and enable the ML stuff so that I can just bring everything to newest? Idk exactly how immich has/had their SQL table creation set up, but if I can just copy/paste however they're doing it on the docker side, it should work theoretically, no? In any case though, the easiest solution from where I'm standing might just be dumping my DB to a big ass SQL file, letting immich recreate the entirety of the tables from scratch (with ML enabled), then dumping that SQL file back into the new DB so all of my data now exists. I'm gonna give it a whirl and see if that works, and that immich doesn't have any problem with me dumping data from an old DB into the freshly created DB. If it does, hopefully upgrading is fairly straight forward from there.
Sergey Katsubo
Sergey Katsubo•7d ago
^^ That's an option, right. Alternatively, have you tried to check the DB schema drift as I described above? In best case, you'll get SQL statements to create missing entities. In worst case, just nothing.
Dr America
Dr AmericaOP•6d ago
I'd have to look for where all that would be in Arch, as they move a lot of those files around I'm getting a whole lot of errors related to backslash commands, but if I can get that figured out, I might be able to just get it done more or less 
 Schema |           Name            | Type  |  Owner   
--------+---------------------------+-------+----------
 public | activity                  | table | postgres
 public | album_users_audit         | table | immich
 public | albums                    | table | postgres
 public | albums_assets_assets      | table | postgres
 public | albums_audit              | table | immich
 public | albums_shared_users_users | table | postgres
 public | api_keys                  | table | postgres
 public | asset_faces               | table | postgres
 public | asset_files               | table | postgres
 public | asset_job_status          | table | postgres
 public | asset_stack               | table | postgres
 public | assets                    | table | postgres
 public | assets_audit              | table | immich
 public | audit                     | table | postgres
 public | exif                      | table | postgres
 public | geodata_places            | table | immich
 public | kysely_migrations         | table | immich
 public | kysely_migrations_lock    | table | immich
 public | libraries                 | table | postgres
 public | memories                  | table | postgres
 public | memories_assets_assets    | table | postgres
 public | migrations                | table | postgres
 public | move_history              | table | postgres
 public | naturalearth_countries    | table | immich
 public | notifications             | table | immich
 public | partners                  | table | postgres
 public | partners_audit            | table | immich
 public | person                    | table | postgres
 public | session_sync_checkpoints  | table | immich
 public | sessions                  | table | postgres
 public | shared_link__asset        | table | postgres
 public | shared_links              | table | postgres
 public | system_metadata           | table | postgres
 public | tag_asset                 | table | postgres
 Schema |           Name            | Type  |  Owner   
--------+---------------------------+-------+----------
 public | activity                  | table | postgres
 public | album_users_audit         | table | immich
 public | albums                    | table | postgres
 public | albums_assets_assets      | table | postgres
 public | albums_audit              | table | immich
 public | albums_shared_users_users | table | postgres
 public | api_keys                  | table | postgres
 public | asset_faces               | table | postgres
 public | asset_files               | table | postgres
 public | asset_job_status          | table | postgres
 public | asset_stack               | table | postgres
 public | assets                    | table | postgres
 public | assets_audit              | table | immich
 public | audit                     | table | postgres
 public | exif                      | table | postgres
 public | geodata_places            | table | immich
 public | kysely_migrations         | table | immich
 public | kysely_migrations_lock    | table | immich
 public | libraries                 | table | postgres
 public | memories                  | table | postgres
 public | memories_assets_assets    | table | postgres
 public | migrations                | table | postgres
 public | move_history              | table | postgres
 public | naturalearth_countries    | table | immich
 public | notifications             | table | immich
 public | partners                  | table | postgres
 public | partners_audit            | table | immich
 public | person                    | table | postgres
 public | session_sync_checkpoints  | table | immich
 public | sessions                  | table | postgres
 public | shared_link__asset        | table | postgres
 public | shared_links              | table | postgres
 public | system_metadata           | table | postgres
 public | tag_asset                 | table | postgres
 public | tags                      | table | postgres
 public | tags_closure              | table | postgres
 public | user_metadata             | table | postgres
 public | users                     | table | postgres
 public | users_audit               | table | immich
 public | version_history           | table | postgres
 public | tags                      | table | postgres
 public | tags_closure              | table | postgres
 public | user_metadata             | table | postgres
 public | users                     | table | postgres
 public | users_audit               | table | immich
 public | version_history           | table | postgres
this is my current schema, and for some reason I'm now getting "column entity,isArchived does not exist" errors seems my last version was actually 1.133.1, so I might try using that instead OK, did some more digging, and I figured out that what broke was not having fixed my tables problem before trying to upgrade past whichever version integrated the ML service into the main service, and my old backup from 1.129.0 is now working with the tables set up correctly. Dumping only the data from my old tables, recreating a fresh DB with the ML tables, and importing my data with all triggers disabled (thanks stack overflow!) seems to have done the trick. FYI, the version_history table is very useful! I'm going to dump this DB and use one of my more recent postgres backups with some other unrelated db's in it and make a new backup with the fixed DB. Then begins the process of finally upgrading, since I haven't since March lol. Updated to 1.136, then to 1.142 (don't think it was necessary), then to 2.2.1 (because that was the latest I already had built), and seems to be working fine other than I'm going to have to figure out updating pgvector lol

Did you find this page helpful?