Immich•2y ago

Make Storage Template the default feature

I wonder why the storage template is not enabled by default. I think most people would be okay if we use a format like ‘year/month/date’ as the default. And they can change that if needed.

Struggling to understand why the ‘upload’ folder is the default one. I see a question now and then on how does the folder structure looks so weird (referring to the upload folder).

Immich•7/28/24, 4:55 PM

:wave: Hey @radh,

Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich :immich:.

References

Container Logs:
```
docker compose logs
```
```
docker compose logs
```
docs
Container Status:
```
docker compose ps
```
```
docker compose ps
```
docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy

## Checklist

:blue_square: I have verified I'm on the latest release(note that mobile app releases may take some time).
:blue_square: I have read applicable release notes.
:blue_square: I have reviewed the FAQs for known issues.
:blue_square: I have reviewed Github for known issues.
:blue_square: I have tried accessing Immich via local ip (without a custom reverse proxy).
:blue_square: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.
:blue_square: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

If this ticket can be closed you can use the

/close

/close

command, and re-open it later if needed.

Zeus•7/28/24, 4:56 PM

Files can't be put into the correct location on upload because the EXIF data isn't known. So they have to go somehwere to start. And any time we move it after that there is a tiny chance of a failure

Zeus•7/28/24, 4:57 PM

It is very stable lately and we may do this at some point but that is the logic

radhOP•7/28/24, 4:58 PM

Right. I understand that even when I use the ‘storage template’, the upload folder is used as a buffer. But just that it seems confusing for people that get started with immich. Since they need to manually toggle the feature to have a sensible folder structure.

radhOP•7/28/24, 4:59 PM

I agree on the stability part of it. But using Storage Template has also been stable correct?

radhOP•7/28/24, 5:00 PM

I have been using the storage template for more than a year, and everything seems good.

Zeus•7/28/24, 5:00 PM

It's stable as your backing filesystem

Zeus•7/28/24, 5:00 PM

for a lot of people, that is "not very". lol

radhOP•7/28/24, 5:02 PM

Oh. I did not know about that.

Zack•7/28/24, 5:21 PM

Filesystems are generally fine, and the storage template system is built to be hardened against failures like that now

Zack•7/28/24, 5:21 PM

However, you'd be surprised how many people actually don't care what their files look like on disk

Zack•7/28/24, 5:22 PM

The beauty of something like immich is that you can see everything and search it all, etc

Zack•7/28/24, 5:23 PM

The problem with the storage template system is it requires a move operation, and that can have pretty intensive IO impacts depending on your mount setup or the underlying filesystem

ZZack However, you'd be surprised how many people actually don't care what their files...

radhOP•7/28/24, 5:53 PM

I dont look at it either

Zack•7/28/24, 5:54 PM

lol, so why do you care what they look like?

ZZack The beauty of something like immich is that you can see everything and search it...

radhOP•7/28/24, 5:55 PM

I think if people want to move out of immich. They would most certainly run that job once.

Also, on the IO - this applies only to the first import right. Any future uploads are going to be relatively small to make a huge impact.

Rradh I think if people want to move out of immich. They would most certainly run that...

Zack•7/28/24, 5:55 PM

It's just unnecessary, if people want to move out, they can run the storage template then

radhOP•7/28/24, 5:56 PM

The thought was to simplify the feature set if possible. No real use case for me

Zack•7/28/24, 5:56 PM

We originally had it enabled by default, but it's just not necessary for a majority of users

Zack•7/28/24, 5:56 PM

So it seems needless

Zack•7/28/24, 5:56 PM

With certain setups you can end up with the move being an entire copy of the file, maxing your IO if you're running on hard drives

Zack•7/28/24, 5:56 PM

It significantly impacts initial import times if that happens

mertalev•7/28/24, 5:57 PM

My concern is that having it disabled makes Immich (and specifically the DB) more of a single point of failure. DB gone = you're left with a bunch of random uuids for files

radhOP•7/28/24, 5:58 PM

Never thought of that. But that is definitely a concern!

Mmertalev My concern is that having it disabled makes Immich (and specifically the DB) mor...

Zack•7/28/24, 5:58 PM

Simply, have backups, we're not going to re-enable it by default lol

Zack•7/28/24, 5:59 PM

We had a good reason for disabling it by default, I don't intend on us flip-flopping between those

mertalev•7/28/24, 6:00 PM

There was someone whose DB backup didn't work and I had to tell them to write a script with exiftool to salvage the files. I get why it was disabled but this is also a bad place to be that a user isn't necessarily aware they're signing up for

Zack•7/28/24, 6:00 PM

We are planning to add in an automatic backup system for the database by default anyway, so that's not really a concern for me

Zack•7/28/24, 6:02 PM

If the concern is that it isn't clear, we can add more detail to the onboarding page

Zack•7/28/24, 6:02 PM

Explaining what the default format looks like

radhOP•7/28/24, 6:12 PM

I would argue that for the end user, the storage template adds a huge peace of mind. It is like the immich team says ‘hey, even if immich gets messed up -> database is lost, or the database backup is lost/corrupted, your memories are still safe and in a format that makes sense to the human eye’. If this was the case, the user knows that if something like that happens, they can just give up on that app (here immich) and move on. Without the struggle of getting into a process to get their data into a sensible format - even though that process could be explained in detail in the docs.

Zack•7/28/24, 6:22 PM

I'm up for clarifying this in onboarding, and then people can make that choice for themselves

Zack•7/28/24, 6:22 PM

I think that will make it clear enough

mertalev•7/28/24, 6:23 PM

This would definitely help. I don't think the onboarding section makes it really clear what the storage template feature does or what disabling it really means. It explains more if you enable it, but a lot of users end up skipping it because the description sounds vague and ominous. When I mention the storage template to those users, they were looking for just that but never actually knew about that feature

Zack•7/28/24, 6:23 PM

Yea, just sounds like a UX problem to me

Zack•7/28/24, 6:24 PM

Storage template was the reason onboarding was originally built

Zack•7/28/24, 6:24 PM

So can probably use some work

ZZack The problem with the storage template system is it requires a move operation, an...

radhOP•7/30/24, 4:21 AM

Hey. Something just popped in my mind. I have no software background, so take this with a grain of salt.

What if the clients that are uploading an asset, extract the exif from that asset - send that first to the server. Then immich knows where to put that asset according to the storage template. Now once everything is smooth and ready on the server side, the client can then push the actual asset. This asset can go directly to say ‘year/month/date’ - instead of upload and then ‘year/month/date’. I suppose something like this might fix the IO issues?

mertalev•7/30/24, 6:10 AM

It's a valid option and some projects do go for a more client-side strategy. But it means having separate implementations for metadata extraction on clients, which need to be maintained and can have different behavior from each other. It also means that uploads are now branched into two cases that need to be tested separately.

And since we still have an API that other projects use, we can't expect each third-party client to handle EXIF metadata as well, meaning the current implementation would still exist anyway.

radhOP•7/30/24, 11:25 AM

Yeap. You make a lot of sense.

References

Container Logs:

docker compose logs

docker compose logs

docs

Container Status:

docker compose ps

docker compose ps

docs

Reverse Proxy: https://immich.app/docs/administration/reverse-proxy

# Checklist

:blue_square: I have verified I'm on the latest release(note that mobile app releases may take some time).

:blue_square: I have read applicable release notes.

:blue_square: I have reviewed the FAQs for known issues.

:blue_square: I have reviewed Github for known issues.

:blue_square: I have tried accessing Immich via local ip (without a custom reverse proxy).

:blue_square: I have uploaded the relevant logs, docker compose, and .env files, making sure to use code formatting.

:blue_square: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as

"complete

" by reacting with the appropriate number

)

If this ticket can be closed you can use the

/close

/close

command

, and re

-open it later if needed

Make Storage Template the default feature

References

Similar Threads

Similar Threads

References

Similar Threads