Reprocess specific transcoded videos?
Hello found a album where all of the videos do not want to play in the web ui (something about mime support). It's odd as it's the same phone I've used for 1000s of other videos but this one day of videos does not work. On PC I tried 3 browsers and same
Anyhow I see in we ui a way to progress the video again, awesome. However doing this did not change any results.
I guess I was wondering 2 things.
1. If I find the matching file in encodedvideo folder and just delete it, will immich recreate it for me? (I guess if the answer is yes I am curious how to identify the name and path to the specific file in encodedvideo as well)
2. Similarly, I may want to just delete all the encodedvideo folder and start over. Would this work! My thought here is that deleting and starting over is more concrete to see the files were regenerated, vs trusting the reprocess thing actually doing something. I also separately have issues where that folder takes up space and never let go it's its space after deleting (separate issue I posted a couple of weeks backl
Thanks
52 Replies
:wave: Hey @RoachForLife,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs
docs
- Container Status: docker ps -a
docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l
and df -h
).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close
command, and re-open it later if needed.Raw files on external library on the Nas, immich (dB and encodedvideo) sits on same Nas but ssd drive
Using 1.35 immich
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Curious if any of the mods could help on this one, thanks
1) No I don't believe so. It will just error when attempting to play the transcode as it won't be able to find it. (You should not manually touch anything in the upload location)
2) To minimize touching the folder directly. You can turn the trnascode policy to "Don't transcode" then run the "Transcode All" Job. This should delete all items in the encoded-vidoes folder. Then you can double check and see if there's extra stuff in the folder
Ok thanks I will give #2 a shot later this week. Thanks
@schuhbacca So I did just what you said finally and results were a bit shocking but seem to support my concern. Top image is before setting to Do not transcode, bottom is after processing

Why would anything still remain in the encoded-video folder at all? There are still 113gig of videos in there
Not sure, what I said should remove all encoded videos that immich is tracking. Can you see what's in that folder?
It ended up being videos of course. Should I try to cross reference the id back to immich, or maybe to the DB?
I was going to reprocess everything soon so if this is something I should do as a test I probably need to do it soon
Yeah I would. Or look for errors in server log when running the transcode job with none set
Sorry but where do I get to the server log? Normally I just check dockge but not sure it will have the history from yesterday but ill check
terminal can just run
docker logs immich_server
or whatever your server container name isok cool let me do that.
looks like i cant go fair enough back for it all (some is cut off the screen). Is there a way to have it save this to a file maybe?
Here is a portion of it at least
That all looks fine according to the logs (At least related to video transcoding deletion). You can try and query the db to see if any of the transcode video columns are not nulled out
Have you ever re-installed or anything like that?
@schuhbacca Reinstalled, no, just standard updates thru the months. I am unsure how to query the db. Is there a set of instructions somewhere on this?
ok thanks. Regarding the query should I be looking up specific files or can I run a query to look up the null in the transcoded video column specifically? Or maybe do this? SELECT * FROM "assets" WHERE "assets"."type" = 'VIDEO';

I used docker exec -it immich_postgres psql --dbname=immich --username=postgres, to get into it, which I believe is right. Doesnt seem to find the assets table
It seems the table is asset not plural. Anyhow I did run this. Cant seem to copy all of it, is it possible to export to a file?
Anyhow heres a small snippet

the table is asset not plural.You correctly spotted it. Plural to singular was changed in 1.136 and not yet reflected in the docs.
Thanks for the asset snippet!
1.
encodedVideoPath
is null, this is good / expected as we don't transcode anymore.
2. But there are assets with originalPath
containing upload/encoded-video/<...>.mp4
which is rather unexpected. Encoded-video should not appear in originalPath. It looks like transcoded video files were actually added into Immich as original videos, hmm
@Sergey Katsubo Thanks. I do have the setting to NOT encode if a mp4 already so unsure how it would get in there. If I use this setting, should I expect zero .mp4s in the encodedvideo folder?
Let's split concerns into 2 topics.
1. Encoded or not encoded videos / mp4.
2. The fact that ".../encoded-video/...mp4" appear as original assets. (This is troubling me)
For (1) could you grab stats from the database: count of videos
- by extension
- and whether they have been encoded or not (if they have non-empty encodedVideoPath)
Thanks Sergey. I had moved on and deleted everything and reprocessed (see this post as it hasnt gotten any traction but is also concerning and related - https://www.reddit.com/r/immich/comments/1m8hfib/my_journey_into_immich_not_recovering_itself/ ) but I did back it up in proxmox. I will make a point of creating a new lxc restore of everything and run the sql on it and post back here. Appreciate it!
Regarding backups I have the following backups, let me know which is best. I assume the one after I turned off transcoding and it removed most items but still a bunch remained in the encodedvideo folder?
- Prior to any edits
- After setting transcoding to OFF and it deleted nearly half of the files in encodedvideos folder
The files with -MP suffix are extracted motion videos from Android motion photos and should not be deleted.
@Sergey Katsubo Ok took a bit but I was able to load a new proxmox ct with the state right after I turned transcoding to OFF and it deleted a bunch, but not all, of the videos. And ran your statement. See attached

Thanks, this is helpful. On my other machine (where I reprocessed everything, not the one I just told sergey about), it seems I have 13k motion photos videos with the -mp (40gig). I had put in a FR a month ago for the option to have Immich not process these. I obviously have these 13k where I actually would prefer the space back, and the only way I know to fix them is on my phone, meaning Id have to transfer them to and from my phone to strip the video portion. Having immich just not process the video would be nearly as good as I at least wouldnt have the extra video.
It's actually kinda crazy that my entire immich library folder is 183gb and 40gb of that are motion videos I dont want 🙁
We can probably add a system config option for that.
Thank you for considering that! I didnt realize it was nearly 1/4 of all of my server space just in motion videos so that would be great
@Sergey Katsubo Hola. Just curious if that output helped? Thanks
Hey.
Yes, it has become clearer now, thanks to the SQL statement output + your comments + Jason's note about MP being extracted motion videos.
So it turned out that both my prev questions are connected, not independent:
1. Encoded or not encoded videos / mp4. 2. The fact that ".../encoded-video/...mp4" appear as original assets. (This is troubling me)The short answer: this is the way how Immich handles MP (motion photos). It is expected. If I got it right: - Why do mp4/mov files appear in
encoded-video
despite Video Transcoding is disabled? (Intuitive expectation: encoded-video should be empty). Answer: because Immich extracted them from MPs and put there. This processing of MPs happens independently of Transcoding policy.
- Why do mp4/mov files in encoded-video
have "original" asset mark? (Intuitive expectation: encoded-videos are not originals. They are transcoded versions of actual original assets living in other locations, internal or external libraries). Answer: they appear as "original" assets because they were extracted from MPs and now live a separate live as "originals". This motion videos from MPs live in encoded-video
because this is how it's implemented.
@jrasm91 Is this a correct explanation?
We can add an extra dimension - MP or not - to the stats from asset
table in DB:

@Sergey Katsubo Here is the latest sql query. I am not certain this all is making sense tho. So in my live production Immich which has a lot more data on it (reprocessed everything after turning transcoding off). There I have 40gb of Motion Photo videos (with -mp). On this other box we are using, right when I disabled transcoding, it has about 113gb of space being used in the encodedvideo folder. That leaves a difference of about 73gb of space that is not a motion photo video and should not exist.
I am re-running to see how many MP files exist right now in the previous state system I am working on with you and will report back in 10min when it goes through its check
Can you confirm that if transcoding is OFF, that, less the MP files, it should clear out all of the videos? The system does have this setting on currently

So if something is a h.264, does it just use the file in the external library or does it make its own copy for the encodedvideo folder?
I suppose if the answer to that is, it makes its own copy (which feels redundant to not just reference the same file over on my external library) but anyhow, then would unchecking H.264 and setting to DONT TRANSCODE, would it then have nothing (zero) in the encoded video folder?
Ultimately, reprocessing the exact same data has produced an encodedvideo folder that is 30-40gb smaller so something extra was definitely lingering in there
Yes, we need the motion video portion of the live photo to be it's own asset with an original path, but that original asset is actually generated from the android motion jpeg. So it's an original path, but generated. So we decided to put it in encoded-videos, to indicate that it is generated.
https://github.com/immich-app/immich/blob/ce2ea989265e878eedf4aaf1e1b73802b1c53c21/server/src/services/media.service.ts#L492-L499
- It removes the transcode when TranscodePolicy.Disabled
- It writes a log message at default LOG level, so it will be visible in the Immich container logs.
Did you run 1 pass of reprocessing (Transcode Videos job) or 2 passes and the second pass freed additional space?
From the last query we see that there do exist a lot of videos having their transcoded counterpart (marked as "encoded" on the screenshot).
Supposedly they should be removed (eventually?) when transcode policy is OFF. Unless they do not match criteria for removal, e.g. isRemuxRequired from the linked source code.
This is how I understand it, though reality can be surprising.
@Sergey Katsubo tbh, I dont recall how many times I processed. I suppose I could hit process again now and it would guarantee it ran at least 2x? Im guessing I ran it just once after switching to dont transcode and then it took it bit but began dropping in space
Also if it matters, I guess this version (before reprocessing) had a lot more MP than after reprocessing, which is also odd since the data it pulls from (my ext library) is identical. Anyhow here is what it has

Whereas its more like 40gb in its reprocessed state (basically started over after deleting the encodedvideo folder and turning transcoding back on)
Im wondering now if part of the issue of Immich not recovering its data after deleting items is more centered around these MP video files. The difference in MPs is 65gb which is very close to the total difference between the old system (before even turning off transcoding) and after I reprocessed everything. I wonder if Immich does not properly delete MP videos when an item is deleted from the external library?
if Immich does not properly delete MP videos when an item is deleted from the external library?Yeah, that's possible. For example these two open bugs are about external library and live photos, albeit they are different from your issue: - https://github.com/immich-app/immich/issues/17731 - https://github.com/immich-app/immich/issues/20031
[Issue] MOV files left behind if deleted while External LIbrary Watching enabled (immich-app/immich#17731)
[Issue] Live Photo Video and Sidecar getting deleted when renaming external library folders (immich-app/immich#20031)
@Sergey Katsubo I suppose I would ask if more investigation could be done. Not so much for my issue as clearing everything and reprocessing has fixed the issue (for now) but to make the product better long term. I would not want to HAVE to delete everything and reprocess all just to fix these types of issues every so often. Having immich correctly deal with the deleted files in its DB would be a far superior solution. I am sure there are lots out there that have these orphaned files sitting in their library folder and don't even realize it. I only took a vested interest as my 500gb SSD was climbing fast and not ever going down which prompted me to look into it more.
If there is anything more I can do on my end to assist I am happy to, especially if it leads to some improvements to the product.
One thought would be a utility of sorts that could examine the DB vs library files vs external library, so compare and possibly clean itself up? Kinda like a reprocess but without having to truely delete things from scratch but more just to clean/delete only the stuff it needs to. I say this as a non dev so no idea how difficult this is
Internally we have already talked at length about building more "integrity checks" into the application. The idea is to basically convert external library scanning into general file scanning on a per library basis to verify that file paths in the database correspond to a corresponding files on disk and that checksums match between the two. This would lead to detecting differences. Either files without database records or database records with corresponding files.
Or records that mismatch. But this is not an easy or trivial change and will require significant engineering effort to implement, but it is pretty high on our list.
@jrasm91 Thank you so much for the candid information. I'm happy to hear this. Yeh, I am definitely not a dev (although I work with developers in my day job) so I get it. Been using Immich nearly a year now, and its been wonderful, and hope to continue using it for many more. I do feel like a lot of the users (don't have any metrics to support this just based on reddit post volume) use external libraries with immich as the front end, so things like this go a long way to supporting that.
Anyhow thanks again, and I will save these Proxmox CTs so if I can be of help checking stuff I am happy to help.
PS - Thanks also on your mention the other day on considering an option to disable Motion Videos all together. If my issue is linked to MP it may resolve that just by disabling it (assuming it cleans up the left behind ones of course). Anyhow, all good stuff, thanks guys!
@js'
@jrasm91 If it makes it any easier, I had put in a FR for the motion photo thing, url is https://github.com/immich-app/immich/discussions/19560 . I've began taking notice more of the devs on each release and see you have added so much towards Immich so just wanted to say thank you again for the amazing product you and the team have built
@jrasm91 I guess if you have a minute at some point and can take a look at this post on reddit I did. I am also exploring manually updating the photo itself to remove the motion. Its not my preference since I am okay with the file size being larger on my external library (NAS) for the original file but trying to reduce what is stored on the immich library side (on a SSD). Anyhow I was trying to confirm what should happen if I were to update these, perhaps you would know? https://www.reddit.com/r/immich/comments/1mqumai/what_will_happen_if_i_strip_an_already_processed/
Still definitely my preference for an option in the settings to just disable processing of Motion Photos all together but exploring all options. Thanks again!