Trouble starting Typesense - volume not writable?
Hi,
this is painful to debug, I am also in cotact with the typesense comnunity here:
https://github.com/typesense/typesense/issues/1278
affected service
https://railway.app/project/09c93cce-d2c9-4843-80e8-d068e6719f1e/service/0db5ed7f-6ac5-4619-8521-907cd610b023?id=e321b389-43e9-4a07-a543-3890698b096d
I am trying to create template (and service for myself) from the typesense doceker image. It requires a volume, but even if I do not mount a volume and use /tmp for data storage the image fails to boot.
I have been asked to verify that the mounted path is writable (duh?!) of course it is, but how I am to verify that it REALLY IS?
In non-railway setting the typesense docker image works just fine.
GitHub
Unable to run docker image: terminate called recursively · Issue #1...
Description I am trying to start a typesense instance on Railway.app via the docker image. The image will not start. Here is the logs I am getting: I20231004 10:23:55.523072 1 typesense_server_util...
Railway
404 - Page not found
Railway is an infrastructure platform where you can provision infrastructure, develop with that infrastructure locally, and then deploy to the cloud.
23 Replies
Project ID:
09c93cce-d2c9-4843-80e8-d068e6719f1e,0db5ed7f-6ac5-4619-8521-907cd610b023,e321b389-43e9-4a07-a543-3890698b096dcan you give the solution in this thread a try?
https://discord.com/channels/713503345364697088/1158951309777567814
did not help
okay I'll look into this later today!
from the logs in that github issue, what makes you think this is a volume issue?
yes, it crashes just as it is trying to write something into the volume
yes but right before that it was able to remove something on the volume without issues
my guess is that the rm "worked" because there was nothing to remove
is there any way to acces the volume from outside of railway container?
there is not, all volume access has to be done via the service it's attached to, Railway doesn't provide any sort of sftp access at this time
could you try running a container typesense/typesense:0.25.1
with
TYPESENSE_DATA_DIR=/tmp
TYPESENSE_API_KEY=is-very-secret
there is no reaseon this would not work, the image is basically an executable wrapped in a container, it does not anything special, no reason it should to be able write into the filesystem
will do
🚀
note: tehere was a typo in one of the env vars
If it’s accessible, just not writable, can you try chmod’ing it?
you’ll likely have to add the chmod command as a line in your start command/dockerfile
didint work, i get the same errors as OP
looking more like an issue with typesense
does typesense have any known issues with ext4 mounted storage?
no idea, will check, I am doing some experiments myself as well
I have crated a repo here for testing to actually check what things look like once the container is running
of course everything is fine, so I will folloup with typesense team
https://github.com/xgroot-dev/typesense-runner
it was the number of threads typesense was trying to start
Thread pool size: 256The default is 8*CORES I guess there is some limitation in number of native threads for raway containers? What is the max number? I will find out! typesense_server_utils.cpp:384] Thread pool size: 200 is the max, with 201 it fails
ah then typesense is incorrectly detecting the amount of cores available
does it provide a way to overwrite the auto detection with a flag?
Solution
@honzasterba
its detecting the cores correctly, because the default is 8*CORES and there are 32 cores available
I made one too
https://railway.app/template/pCQ-HZ?referralCode=xN06sk
and added safeguards not to start too many threads and also some hacks around configuring HA
may I ask you to unpublish that template as having two of the same templates is very confusing to the users, your template uses a github repo and that is significantly less desirable then a template that deploys from a docker image
ok
thank you for understanding