[Android] Some video assets not considered uploaded by the application causing duplicates
Hello! As I already mentioned in the #support-crew channel, some videos (10) on my dad's phone aren't considered uploaded by the app while they're indeed on the server causing the app to upload them tens/hundreds of times. I don't really know where to look for as they look to be hashed in some ways as the server gives the following error:
I updated the app to the latest available on TrueNAS (using the catalog, which is 2.2.3) and the android app version is the latest. I tried today and still have the issue even though the logs date back a few days ago.
41 Replies
:wave: Hey @Xiticks,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.If there is anything I could do to help (maybe giving more "up to date" logs or else, lmk
The app is otherwise seem to work correctly on other android/ios devices. Even on my dad phone, it's only these 10 assets. I could share some of them if needed.
@shenlong-tanwen Do you mind help Xiticks with this?
just for sanity check, you did logout and log back in on your dad's device, correct?
I cleared the sqlite database which forced him to reconnect and rehash all assets
I don't see any hash failures on the logs. Does the status page show them as unhashed or does it show all the assets as hashed?
The status page is under Settings -> Sync Status
All local assets look to be all hashed
Do you have remainder count in the backup page?
Yes
the 10 that keep being uploaded each time
Does the checksum match the one in the server?
How do I check that on the server?
If you open the asset in the web app, the asset response should have the checksum. Or the easiest way to check is to query them against the server's DB
Or, you can also enable Advance troubleshooting in the mobile app, find the cloud only video in the timeline, long press it, advanced action and it should show the checksum of the remote asset
last image is the local video
They don't seem to match
Can you calculate the actual SHA-1 hash of the video on the device? Or if it does not have anything private, you can also zip and share the video over to me and I can see if I can reproduce it from my end
What might be happening is that the hash calculated on the device is somehow incorrect but when it gets uploaded to the server, the hash calculated there matches up with the one already on the server and this gets rejected. But since the on-device hashes are different, the app does not link them together and always tries to reupload them :monkaHmm:
I'll zip it so if you need it for other troubleshooting reason you'll have it
But how could I calculate the sha-1 on the device? always like to learn
I hope it's the "correct" version
I downloaded from ui when I saw it bug
Android? There are few file explorers that present this info. It's been so long since I've actively used an Android device. But I used to use an app called "Mixplorer" which IIRC, showed different checksums when you check the info of a file
well no since the "original" never uploaded
You can check if
sha1sum works on the terminal app called termux. Or there should be plenty of apps on the playstore that calculates it for you
Yep. Can you zip the one from your mobile device and share it as well?Will do, the phone is slow as hell
Wait I'm dumb I send the wrong one too xD
If that helps, the media are stored on a sd card on the phone
Idk if this could have lead to some corruption as the video is 5 years old
well all 10 are from 2019, except 1 from 2017 and 2 other from 2020
How was it uploaded to the server in the first place? Was it through the mobile app or was it through a different bulk import?
Mobile app
Maybe the one from 2017 was from adding external library that I downloaded to the app to show how it works
but all other were all on the phone
https://www.swisstransfer.com/d/028f0a86-a125-4e45-8445-1f28d58f1c22
Ok so this one should contain the same video
original from the phone
server being one of the 100's video that was a duplicate on the server a few days ago
Doesn't look like it after double check, can't rememebr how I transfered his photos when he changed phone
The ones shared is the same as the one from the screenshots, right?
for the original, yes, not for the "server" one
but I could get the hash of the server version through web if need, had to give my father's phone back
No problem, I'm heading to bed now. We can continue this tomorrow if that's okay with you? You can share me the checksum from the web app when you find some time
There is now 5 assest on the server (I stopped the backup after the screenshots)
(redacted email)
:Deadge: And all these are same videos?
The one's you've shared have a different checksum that is not in this list
Yes the same exact one uploaded 5 times (before it was like 150 times so I removed them)
Yes, original, which is on the phone is never considered uploaded so that’s the reason of this ticket
As for the other one, it’s a video from before I clean, I’ll send the 5 ones in a bit
https://www.swisstransfer.com/d/cb870268-e97f-4246-8195-6932c5211834
So these are the 5 on the server which should match the ones there
But the one on the phone is "missing" as it is never considered uploaded
Is it possible for you to move this video to your internal storage and see if the issue is still reproducing? This is really weird and I'm not sure if moving it internally makes a difference. But we can try it to rule out any issues from the SD Card
Moving the assets back and forth from the SD card fixes the issue
I tried
A) Move to local storage -> upload (works) -> move back to SD Card (video is detected as already uploaded)
B) Move to local storage -> move back to SD card -> Upload (assets work)
bug seems to be from the Android api to get assets from the SD card
I assume they might have change the modification date within the EXIF itself
Yes, in a file explorer, the videos are now marked as "today" and not their old date (Immich is fine as it read the creation date I believe)
Weird that it happens for 10 assets (9 video and 1 photo)
I've kept 7 files not moved yet, if more troubleshooting is needed
If you think that's a one of issue, I'm fine to close the ticket and moving back and forth the rest of the media
Thanks for the help!
Let's keep this open for a few days. I'll see if I can find something related to this :pepeargh:
If there is anything I could provide that could be of any help lmk
Android version, the other assets with the issue or anything
Can you try unmounting the SDCard and mount it back again and see if that fixes the issue with the other files? You can probably remove it, let MediaStore reindex it's DB, reinsert it and check
Unmount/mount reindex
and
Unmount clear cache & remove sqlite, mount back reindex
Both still upload the assets once, then when I relaunch the app uploads them again
I cannot find anything related to what we observe here. What's the device make and model? And the android version running on it?
The phone is a bit "old and niche brand"
It's a
Crosscall Core-x4
Android 10