Railway•7mo ago

Cannot Get Private Networking working for Go app and ChromaDB

ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f So I have a Chroma Vector DB with that ID and I have it's PORT variable set to 8000 which is what it naively listens on. I have it as chroma.railway.internal but my app cannot seem to hit that. The strange thing is that when I enable the public networking, it can access it on the generated domain:443. I'll paste the error I'm getting down here:

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp: lookup chroma.railway.internal on 8.8.8.8:53: no such host"
level:
"error"
time:
1702269309
@replica: 41f01962-2e35-4e48-a28b-1f107d378c6f

Does anyone have some suggestions?

Solution:

try setting this as the start command

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

...

Jump to solution

35 Replies

Percy•7mo ago

Project ID: c6a718e0-23e7-4d39-85b8-cd951a53c53f,41f01962-2e35-4e48-a28b-1f107d378c6f

Brody•7mo ago

are you building with a dockerfile

OhLyln•7mo ago

Hey Brody! Yes i am here's my dockerfile :

FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]

FROM golang:1.21.5-bookworm

WORKDIR /app

COPY . .

ARG CHROMA_URL
ENV CHROMA_URL=$CHROMA_URL

ARG OPENAI_API_KEY
ENV OPENAI_API_KEY=$OPENAI_API_KEY

RUN CGO_ENABLED=0 GOOS=linux go build -o /biblesearch
CMD ["/biblesearch"]

I switched from the alpine build to bookworm because I saw alpine can cause issues with private networking

Brody•7mo ago

then I guess you already knew what I was going to say

OhLyln•7mo ago

tbh I'm not a docker expert so Idk if bookworm gets rid of alpine. I can try with a nixpack

Brody•7mo ago

try changing your CMD line to CMD sleep 3 && /biblesearch bookworm is a Ubuntu base

OhLyln•7mo ago

Okay cool, running that rn

Brody•7mo ago

you also don't need any of those ARG or ENV lines, that's only needed if you need a variable during build

OhLyln•7mo ago

Oh okay I did not know that

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "sleep 3 && /biblesearch": stat sleep 3 && /biblesearch: no such file or directory: unknown

I am not sure if I typed that incorrectly maybe CMD ["sleep 3 && /biblesearch"]

Brody•7mo ago

show me your new dockerfile without the brackets like I wrote it

OhLyln•7mo ago

Okay I will try that thanks okay new error I think this is app level though maybe:

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78

error:
"Post "http://chroma.railway.internal:8000/api/v1/collections": dial tcp [fd12:eab5:1cb0::73:9796:713]:8000: connect: connection refused"
level:
"error"
time:
1702270542
@replica: 53697e50-faf1-479f-8f5b-7660c0c49c78

Brody•7mo ago

chroma needs to be listening on ::

OhLyln•7mo ago

Can i set that in the port env var for my chroma instance?

Brody•7mo ago

depends, does it have an applicable environment variable to set the host it listens on?

OhLyln•7mo ago

I don't think so I can take a loot at the docs, I am just using the default template right now I'll try it anyway

Brody•7mo ago

default template leaves a lot to be desired I'm told

OhLyln•7mo ago

Yeah it basically just sets up the default dockerfile and a volume it seems

Brody•7mo ago

does it deploy from a github repo?

OhLyln•7mo ago

No just from their github docker repo

Brody•7mo ago

well thats something at least yeah im looking at the data for the template, doesnt come with any service variables for things like public or private database url

OhLyln•7mo ago

Yeah it had 0 variables set which is a little frustrating because I am not sure what I can configure in it

Brody•7mo ago

yeah unfortunately railway does not yet have any quality control for templates

OhLyln•7mo ago

Still building I am not sure if it's just taking a long time or what just an fyi no logs either I might restart that deploy it failed anyway

Brody•7mo ago

we talking about the chroma service?

OhLyln•7mo ago

Yeah correct I just hit redeploy on the latest deployment Okay it's me making it fail I was setting PORT to :: on the chroma service so I think you cannot do that lol

Brody•7mo ago

:: is not a port lol its a host and the host is set to 0.0.0.0 https://github.com/chroma-core/chroma/blob/main/bin/docker_entrypoint.sh#L7

OhLyln•7mo ago

Hm okay I am wondering If i can set it's host to :: somehow

Solution

Brody•7mo ago

try setting this as the start command

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

/bin/sh -c "uvicorn chromadb.app:app --workers 1 --host :: --port 8000 --proxy-headers --timeout-keep-alive 30"

Brody•7mo ago

OhLyln•7mo ago

deploying, I did not know you could edit start command for stuff in docker lol that is super useful promising

Brody•7mo ago

yeah opens up a lot of doors

OhLyln•7mo ago

I am an idiot i set that as start for my app one sec gotta switch it around THAT WORKS

Brody•7mo ago

uvicorn doesn’t support dual stack binding, so chroma will now only work on the private ipv6 network and not the public ipv4 network

OhLyln•7mo ago

Okay that is totally valid, thank you so much man I totally appreciate you, you've helped me before as well just wanted to give you such a shoutout

Brody•7mo ago

haha no problem, im happy to help

Gaming

Programming

Cannot Get Private Networking working for Go app and ChromaDB