R
Railway11mo ago
OhLyln

Cannot Get Private Networking working for Go app and ChromaDB

ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f So I have a Chroma Vector DB with that ID and I have it's PORT variable set to 8000 which is what it naively listens on. I have it as chroma.railway.internal but my app cannot seem to hit that. The strange thing is that when I enable the public networking, it can access it on the generated domain:443. I'll paste the error I'm getting down here:
error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f
error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f
Does anyone have some suggestions?
Solution:
try setting this as the start command
/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"
/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"
...
Jump to solution
35 Replies
Percy
Percy11mo ago
Project ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f,41f01962-2e35-4e48-a28b-1f107d378c6f
Brody
Brody11mo ago
are you building with a dockerfile
OhLyln
OhLyln11mo ago
Hey Brody! Yes i am here's my dockerfile :
FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]
FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]
I switched from the alpine build to bookworm because I saw alpine can cause issues with private networking
Brody
Brody11mo ago
then I guess you already knew what I was going to say
OhLyln
OhLyln11mo ago
tbh I'm not a docker expert so Idk if bookworm gets rid of alpine. I can try with a nixpack
Brody
Brody11mo ago
try changing your CMD line to CMD sleep 3 && /biblesearch bookworm is a Ubuntu base
OhLyln
OhLyln11mo ago
Okay cool, running that rn
Brody
Brody11mo ago
you also don't need any of those ARG or ENV lines, that's only needed if you need a variable during build
OhLyln
OhLyln11mo ago
Oh okay I did not know that
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown
I am not sure if I typed that incorrectly maybe CMD ["sleep 3 && /biblesearch"]
Brody
Brody11mo ago
show me your new dockerfile without the brackets like I wrote it
OhLyln
OhLyln11mo ago
Okay I will try that thanks okay new error I think this is app level though maybe:
error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78
error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78
Brody
Brody11mo ago
chroma needs to be listening on ::
OhLyln
OhLyln11mo ago
Can i set that in the port env var for my chroma instance?
Brody
Brody11mo ago
depends, does it have an applicable environment variable to set the host it listens on?
OhLyln
OhLyln11mo ago
I don't think so I can take a loot at the docs, I am just using the default template right now I'll try it anyway
Brody
Brody11mo ago
default template leaves a lot to be desired I'm told
OhLyln
OhLyln11mo ago
Yeah it basically just sets up the default dockerfile and a volume it seems
Brody
Brody11mo ago
does it deploy from a github repo?
OhLyln
OhLyln11mo ago
No just from their github docker repo
Brody
Brody11mo ago
well thats something at least yeah im looking at the data for the template, doesnt come with any service variables for things like public or private database url
OhLyln
OhLyln11mo ago
Yeah it had 0 variables set which is a little frustrating because I am not sure what I can configure in it
Brody
Brody11mo ago
yeah unfortunately railway does not yet have any quality control for templates
OhLyln
OhLyln11mo ago
Still building I am not sure if it's just taking a long time or what just an fyi no logs either I might restart that deploy it failed anyway
Brody
Brody11mo ago
we talking about the chroma service?
OhLyln
OhLyln11mo ago
Yeah correct I just hit redeploy on the latest deployment Okay it's me making it fail I was setting PORT to :: on the chroma service so I think you cannot do that lol
Brody
Brody11mo ago
:: is not a port lol its a host and the host is set to 0.0.0.0 https://github.com/chroma-core/chroma/blob/main/bin/docker_entrypoint.sh#L7
OhLyln
OhLyln11mo ago
Hm okay I am wondering If i can set it's host to :: somehow
Solution
Brody
Brody11mo ago
try setting this as the start command
/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"
/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"
Brody
Brody11mo ago
No description
OhLyln
OhLyln11mo ago
deploying, I did not know you could edit start command for stuff in docker lol that is super useful promising
Brody
Brody11mo ago
yeah opens up a lot of doors
OhLyln
OhLyln11mo ago
I am an idiot i set that as start for my app one sec gotta switch it around THAT WORKS
Brody
Brody11mo ago
uvicorn doesn’t support dual stack binding, so chroma will now only work on the private ipv6 network and not the public ipv4 network
OhLyln
OhLyln11mo ago
Okay that is totally valid, thank you so much man I totally appreciate you, you've helped me before as well just wanted to give you such a shoutout
Brody
Brody11mo ago
haha no problem, im happy to help
Want results from more Discord servers?
Add your server