Cannot get ChromaDB working
Added ChromaDB template to my project: https://railway.app/template/tifygm
Chroma DB seems to be running fine:
Tried to access ChromaDB from another service with host: http://chroma.railway.internal:8000 but got no response
To make it easier to debug, I created a public domain for chroma and made a curl request that works with my local Chroma DB:
Result
Any ideas? Documentation about the chroma template just says:
"Uses official chroma docker image and mounts a volume, you can talk to chroma on internal networking port 8000."
To me it looks like the internal/external domains are not succesfully linking with http://0.0.0.0:8000
Project ID: 31616189-a805-4097-9eb6-61efdf3f0920
46 Replies
Project ID:
31616189-a805-4097-9eb6-61efdf3f0920Tried to access ChromaDB from another service with host: http://chroma.railway.internal:8000/ but got no responsewhat error did you get when you tried this?
"Fetch failed". But that was already an error of the application itself trying to access the Chroma API.
What are best practices to debug internal communication? Do I need to deploy something that does a curl request?
giving me a better error message would surely help 🤣
So I guess I need to create a curl or something in my app to get a better error message?
Being able to execute something via CLI in the environment/service would of course be the easiest.
I am a bit confused why there does not seem to be an inbuilt option for this.
But yeah will do that later.
@Brody : ECONNREFUSED
This /docs route serves the swagger docs for chroma and works perfectly fine on my local installation
Both services are in my staging environment, so they should be able to talk to each other.
connection refused means chroma isn't listening on an ipv6 address
Ok, thx. Sadly I cannot figure out how to change that.
you'd need to change the default start command
I was looking at that. But the Railway UI is not showing it to me.
I'm talking about the default start command in use by the image itself
Basically, create my own version of the image, right?
not at all
but upon further inspection there's a
CHROMA_HOST_ADDR environment variable
though I'm very curious as to why you think that would be necessary when you know that you can set the start command in the service settings (not that it's needed at this point since there is an applicable environment variable)Because I am a noob. All devops topics are new to me.
fair enough
did you have an idea of what to set that environment variable to?
Tried: CHROMA_HOST_ADDR=::
But does not help. I also found that one before and could not make it work.
more specifically why didn't it help, what changed? what errors did you get?
Still the same error: {"message":"Connection failed","error":"connect ECONNREFUSED fd12:8c32:4503::2d:8e35:dc90:8000"}
And deploy logs still show: [17-02-2024 16:26:44] Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Shouldn't it be like this: "Server running at http://:::3100/" ?
probably, yeah. show me how and where you set that environment variable
And then I applied the changes
did you originally deploy this from a template?
maybe the author has set a start command for it, please check that
Just made a fresh deploy from the template, so that I did not overwrite anything, but there seems to be nothing
- No startup command
- No variables
alright, I'm out of ideas, I shall deploy this myself and play around with it when I'm back at my computer
Thx a lot! I get the feeling the author was a bit lazy and just created the tempalte (to get the rev share) without actually testing it himself
And templates have no rating feature so people could leave feedback
I was just typing to say something similar.. the template not coming with any variables what so ever is kinda lazy, they mention being able to connect to it via the private network, but that doesn't work, and there's not even convenience variables that build you public and private urls like what the official databases have
not to mention it doesn't even come with token authentication enabled, so I think you're right on the money with your reasoning
There are a lot of templates like this. "Chatbot-UI" is another example.
yeah I know, there's far more lower quality templates than you'd ever imagine, for reasons a lot more complex then this as well. sadly there's no template policy (other than tos) so railway can't just remove a template unless it breaks tos
Can I add another one for the same service and just call ist "Chatbot-UI (actually working)"?
Because I made a working chatbot-ui template.
that's honestly a hard subject
for example, I have a typebot template, it (imo) is very high quality and fully up to date, yet there exists a direct republish of the template named "typebot (updated)" or something, yet mine definitely is up to date in every way possible
Ok, yeah that sucks if someone abuses it. They should really add a simple "upvote/downvote" feature instead of only showing if it deployed succesfully (which means nothing)
eventually there will be policies in place for duplicates, extremely low quality, outright broken, etc that will allow railway to take down the templates
but for now, I'm really not sure what the best move is in the case where you want to publish a template that is intended to replace a pre-existing low quality template
at the very least don't put "(actually works)" in the title as that's extremely unprofessional, despite it's truth
Yeah maybe more something like Chatbot-UI (incl. supabase connection + database setup)
oh question on that, why supabase? why not postgres on railway?
That is not my choice. Chatbot-UI currently requires supabase
Even if you run it locally
That is why it is hard to setup and nothing works without env variables + bash scripts being run on deployment
ah it uses supabase for more than just a postgres database?
Yeah also sending out confirmation emails for new users etc.
But not that I know the details. I did not create the project, just using it
one day you'll be able to run supabase on railway!
There is a docker version of it. Might already be doable. But I am not going to try 😛
its not, there are two services that require mounting a single volume to the two services, something that railway does not yet support doing at the moment
update, the changes to allow setting the host chroma runs on with environment variables where changes that where only made last week, and a new image has not been since published, will get you a solution to change the host for the latest release though
environment variables
start command
keep in mind uvicorn does not support dual stack binding, so while this will work to get communication on the (ipv6) private network working, communication from the (ipv4) public network wont work, its one or the other,
:: for private network, 0.0.0.0 for public
and im fairly certain the volume should instead be mounted to /chroma/chroma@Brody : Wow thank you so much! Will check it out tomorrow
happy to help, I haven't tested it myself but I know from many past experiences with uvicorn that if it's logging that it's listening on
[::] (it is) it will work through the private network (only)Internal communication is working! Still have some other issues, but I am sure I can solve those 🙂 Thx again!
awesome!