Immich•2y ago

Bulk Import stops unexpectedly

I am trying to use the bulk import to move my photos from photoprism to immich, but it "failes?" after some time.

17 Replies

PS Z:\photoprism> immich upload --key <api-key> --server http://192.168.68.3:2283/api --recursive -y .\2021\10\
Checking connectivity with Immich instance...
Server status: OK
Checking credentials...
Login status: OK
Successful authentication for user [email protected]
Indexing local assets...
Indexing complete, found 95 local assets
Comparing local assets with those on the Immich instance...
A total of 1 assets will be uploaded to the server
Start uploading...
Upload Progress | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 1% || 1/95 || Current file [Z:\photoprism\2021\10\20211031_192137_31B11656.jpg]
PS Z:\photoprism>

PS Z:\photoprism> immich upload --key <api-key> --server http://192.168.68.3:2283/api --recursive -y .\2021\10\
Checking connectivity with Immich instance...
Server status: OK
Checking credentials...
Login status: OK
Successful authentication for user [email protected]
Indexing local assets...
Indexing complete, found 95 local assets
Comparing local assets with those on the Immich instance...
A total of 1 assets will be uploaded to the server
Start uploading...
Upload Progress | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 1% || 1/95 || Current file [Z:\photoprism\2021\10\20211031_192137_31B11656.jpg]
PS Z:\photoprism>

This appears at the same time in docker:

immich_postgres            | 2023-06-24T11:45:03.457514893Z 2023-06-24 11:45:03.457 UTC [50] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich_postgres            | 2023-06-24T11:45:03.457565918Z 2023-06-24 11:45:03.457 UTC [50] DETAIL:  Key ("ownerId", checksum)=(c4214509-73ca-4814-b771-fc63f5a14018, \xe1677f8099abf3cd54d3310c84e8d2676eb8d5c9) already exists.
immich_postgres            | 2023-06-24T11:45:03.457613335Z 2023-06-24 11:45:03.457 UTC [50] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "isArchived", "mimeType", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, DEFAULT, DEFAULT, $10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, $19) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isArchived", "isVisible"

immich_postgres            | 2023-06-24T11:45:03.457514893Z 2023-06-24 11:45:03.457 UTC [50] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich_postgres            | 2023-06-24T11:45:03.457565918Z 2023-06-24 11:45:03.457 UTC [50] DETAIL:  Key ("ownerId", checksum)=(c4214509-73ca-4814-b771-fc63f5a14018, \xe1677f8099abf3cd54d3310c84e8d2676eb8d5c9) already exists.
immich_postgres            | 2023-06-24T11:45:03.457613335Z 2023-06-24 11:45:03.457 UTC [50] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "isArchived", "mimeType", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, DEFAULT, DEFAULT, $10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, $19) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isArchived", "isVisible"

Alex Tran•2y ago

it is a duplication file so it won't be uploaded

LaserKasparOP•2y ago

So it is normal, that is will just exit the whole upload?

Alex Tran•2y ago

No, the CLI says a total of 1 assets will be uploaded so it only perform the uploading of that 1 file

LaserKasparOP•2y ago

Is there a success message after it finishes?

Alex Tran•2y ago

not at the moment 😛 we are working on rewriting the CLI to make it more user friendly though

LaserKasparOP•2y ago

Because it won't stop trying to upload this one file. It always says it is going to upload one file.

Alex Tran•2y ago

yeah we haven't implemented client side hashing, and the CLI is stateless so when it checks with the server and see the combination of filename and file size is not in the database it queues the file to be upload. Then upon uploading to the server, the hash of the file is calculated and it appears there is a different file under different name has the same file content. So it rejects this file

LaserKasparOP•2y ago

Thanks for your answer. If the CLI exits it means it has sucessfully uploaded every file. If there is an error it would say something right?

Alex Tran•2y ago

Yes

LaserKasparOP•2y ago

And why is it that it only finds 11k uploads in 14k files?

Indexing complete, found 14676 local assets
Comparing local assets with those on the Immich instance...
A total of 11436 assets will be uploaded to the server

Indexing complete, found 14676 local assets
Comparing local assets with those on the Immich instance...
A total of 11436 assets will be uploaded to the server

Alex Tran•2y ago

because you might have some files format that the CLI is filtering out. For example, raws format We haven't updated the CLI to match with the server in term of supporting files yet

Alex Tran•2y ago

Alex Tran•2y ago

Current supported file on CLI

Alex Tran•2y ago

Current supported files on the server

Alex Tran•2y ago

we will need to update the CLI to match with this 😄

LaserKasparOP•2y ago

Ok, thanks for your support! This project is really nice (:

Gaming

Programming

Bulk Import stops unexpectedly

Did you find this page helpful?