R
Railway•13mo ago
nico

Need help setting up private network w/ uvicorn and docker

I have a frontend that tries to fetch data from my backend, which is running in a docker container using uvicorn. I'm able to get everything functional if using an ipv4 address, but not if im using the private network. My Dockerfile:
...
EXPOSE 80
EXPOSE 4000

CMD bash -c "set -a && source .env && set +a && uvicorn main:app --reload --host 0.0.0.0 --port 4000 --host '::' --port 80"
...
EXPOSE 80
EXPOSE 4000

CMD bash -c "set -a && source .env && set +a && uvicorn main:app --reload --host 0.0.0.0 --port 4000 --host '::' --port 80"
In my frontend code, i try to fetch from backend.railway.internal/
147 Replies
Percy
Percy•13mo ago
Project ID: e4f222ca-fce1-437e-a848-11bcb7c17dbd
nico
nico•13mo ago
e4f222ca-fce1-437e-a848-11bcb7c17dbd
Brody
Brody•13mo ago
if the users browser will be making this request you need to use the public domain
nico
nico•13mo ago
its not im calling it internally through like a proxy in the nextjs api router
Brody
Brody•13mo ago
sever side rendering?
nico
nico•13mo ago
not exactly in nextjs i have /api/py/[...path]/route.ts
import { NextApiResponse } from 'next';
import { NextRequest, NextResponse } from 'next/server';
import { getServerSession } from 'next-auth';
import { authConfig } from '@/lib/auth';
import axios from 'axios';
import { StreamingTextResponse } from 'ai';

const handler = async (req: NextRequest, res: NextApiResponse) => {
try {
const { method, url, headers, body } = req;

const session = await getServerSession(authConfig);
if (!session) {
return new NextResponse('Unauthorized', { status: 401 });
}

const { user } = session;
const { name, email, image, id } = user as any; // TODO: fix type

// pass in user session details in headers
const proxiedHeaders = {
...headers,
'x-user-name': name,
'x-user-email': email,
'x-user-image': image,
'x-user-id': id,
};

if (!url) return res.status(500).json({ error: 'Missing URL' });

// grab path after /api/py/:path*
const pathSuffix = url.split('/py/')[1];

// header for useStream
const useStream = headers.get('x-use-stream') === 'true';
console.log("variable:", process.env.PYTHON_BACKEND_URL)
const response = await axios({
method,
url: process.env.PYTHON_BACKEND_URL + pathSuffix,
headers: proxiedHeaders as any,
data: body,
responseType: useStream ? 'stream' : undefined,
});
console.log("response", response)

if (useStream) {
return new StreamingTextResponse(response.data);
} else {
return new Response(JSON.stringify(response.data), { status: 200 });
}
} catch (err) {
return new Response((err as Error).message, { status: 500 });
}
};

export { handler as GET, handler as POST };
import { NextApiResponse } from 'next';
import { NextRequest, NextResponse } from 'next/server';
import { getServerSession } from 'next-auth';
import { authConfig } from '@/lib/auth';
import axios from 'axios';
import { StreamingTextResponse } from 'ai';

const handler = async (req: NextRequest, res: NextApiResponse) => {
try {
const { method, url, headers, body } = req;

const session = await getServerSession(authConfig);
if (!session) {
return new NextResponse('Unauthorized', { status: 401 });
}

const { user } = session;
const { name, email, image, id } = user as any; // TODO: fix type

// pass in user session details in headers
const proxiedHeaders = {
...headers,
'x-user-name': name,
'x-user-email': email,
'x-user-image': image,
'x-user-id': id,
};

if (!url) return res.status(500).json({ error: 'Missing URL' });

// grab path after /api/py/:path*
const pathSuffix = url.split('/py/')[1];

// header for useStream
const useStream = headers.get('x-use-stream') === 'true';
console.log("variable:", process.env.PYTHON_BACKEND_URL)
const response = await axios({
method,
url: process.env.PYTHON_BACKEND_URL + pathSuffix,
headers: proxiedHeaders as any,
data: body,
responseType: useStream ? 'stream' : undefined,
});
console.log("response", response)

if (useStream) {
return new StreamingTextResponse(response.data);
} else {
return new Response(JSON.stringify(response.data), { status: 200 });
}
} catch (err) {
return new Response((err as Error).message, { status: 500 });
}
};

export { handler as GET, handler as POST };
it will call the python_backend_url directly
Brody
Brody•13mo ago
gotcha
nico
nico•13mo ago
basically proxying through directly
Brody
Brody•13mo ago
uvicorn doesn't support dual stack binding, so you'd either have to configure it to listen to ipv6 (internal network) or ipv4 (public network) not both, if you're okay with just listening on the internal network I can tell you how, or if you need to listen on both the internal and public I can also tell you how, your choice
nico
nico•13mo ago
yea im fine with configuring it to only internal network ive only added the public ipv4 one bc i couldnt get it to work with the internal one i previously had
...
EXPOSE 80

CMD bash -c "set -a && source .env && set +a && uvicorn main:app --reload --host '::' --port 80"
...
EXPOSE 80

CMD bash -c "set -a && source .env && set +a && uvicorn main:app --reload --host '::' --port 80"
but i dont think it worked either
Brody
Brody•13mo ago
sounds good, first, in the service that runs uvicorn set a PORT service variable to 3000
nico
nico•13mo ago
alright, should i change this uvicorn command to point to port 3000 as well?
Brody
Brody•13mo ago
nope, have you set the variable yet though?
nico
nico•13mo ago
just set, waiting for redeploy rn
No description
nico
nico•13mo ago
how should i change my docker cmd?
Brody
Brody•13mo ago
there's more steps, don't bother testing just yet
nico
nico•13mo ago
alright also ty for taking the time to help, ive already spent hours going through the very limited docs on private networks in the railway site and trying to debug this <:AA_Vanilla_Pray:905177543953498112>
Brody
Brody•13mo ago
your new CMD command should be this
CMD uvicorn main:app --host :: --port $PORT
CMD uvicorn main:app --host :: --port $PORT
note that I've intentionally left out the .env thing, please don't use .env files, always use service variables when on railway. (there are still more steps) is this the service for the python app? there's variables for next?
nico
nico•13mo ago
alr the only variable i need to update on the frontend is PYTHON_BACKEND_URL and set that to the private network i believe
Brody
Brody•13mo ago
you have separate services for next and python right?
nico
nico•13mo ago
yes
Brody
Brody•13mo ago
and the screenshot above is the python service?
nico
nico•13mo ago
yea
Brody
Brody•13mo ago
okay cool
nico
nico•13mo ago
No description
nico
nico•13mo ago
backend config ^
Brody
Brody•13mo ago
in the python service, set the PYTHON_BACKEND_URL to http://${{RAILWAY_PRIVATE_DOMAIN}}:${{PORT}} cancel the redoploy popup
nico
nico•13mo ago
that variable is for the frontend for the proxy
Brody
Brody•13mo ago
I know, just trust
nico
nico•13mo ago
done
Brody
Brody•13mo ago
once you set that, click the little eye icon on that variable to make sure it renders properly
nico
nico•13mo ago
No description
Brody
Brody•13mo ago
perfect that should be all the config needed for your python service, can you confirm you can no longer access it publicly?
nico
nico•13mo ago
uhh it hasn't been redeployed?
Brody
Brody•13mo ago
have you pushed the code when you changed the CMD command?
nico
nico•13mo ago
yes
Brody
Brody•13mo ago
then it should have redoployed/rebuilt
nico
nico•13mo ago
i pushed it before we did this change ill repush something so it redeploys
Brody
Brody•13mo ago
if it has already rebuilt from the CMD command change then you're fine just open the public domain and see what you get
nico
nico•13mo ago
it was failing the health checks
No description
Brody
Brody•13mo ago
oh yeah I completely forgot about that, my bad the health check is ipv4, so since you're listening on ipv6 only, you do actually need dual stack binding
nico
nico•13mo ago
sorry i just repushed and cancelled the other think
Brody
Brody•13mo ago
we need to not use uvicorn anymore so remove it from your requirements.txt and install hypercorn instead, making sure it's in the requirements.txt file
nico
nico•13mo ago
alrighty
Brody
Brody•13mo ago
hypercorn should be a near drop in replacement for uvicorn
nico
nico•13mo ago
alright this code but uvicorn => hypercorn?
Brody
Brody•13mo ago
and your new CMD command with hypercorn would be
CMD hypercorn main:app --bind [::]:$PORT
CMD hypercorn main:app --bind [::]:$PORT
nico
nico•13mo ago
that handles dual stack binding?
Brody
Brody•13mo ago
indeed
nico
nico•13mo ago
alright repushed with requirements and dockerfile updated
Brody
Brody•13mo ago
healthcheck should now work, and the public and private domains should too so let's move to next step
nico
nico•13mo ago
alright, ill also let u know once it finishes deploying
Brody
Brody•13mo ago
you have a PYTHON_BACKEND_URL service variable on the next service too right?
nico
nico•13mo ago
yes used in this proxy code
Brody
Brody•13mo ago
in the next service variables, change that value to ${{backend.PYTHON_BACKEND_URL}} and then click the eye icon to make sure that renders out properly
nico
nico•13mo ago
No description
Brody
Brody•13mo ago
and that's the next service?
nico
nico•13mo ago
yes
nico
nico•13mo ago
No description
Brody
Brody•13mo ago
okay then I think it should work, unless in forgetting something again (likely)
nico
nico•13mo ago
alright ill pray for it then
Brody
Brody•13mo ago
did your frontend just re-deploy?
nico
nico•13mo ago
will let u know once everything deploys
Brody
Brody•13mo ago
ah okay, sounds good
nico
nico•13mo ago
well it has to rebuild after the variable change backend finished deploying and is working. for some reason frontend is still building...been 12 mins usually finishs in like 5...
nico
nico•13mo ago
question, for the backend, can i remove public networking entirely?
No description
Brody
Brody•13mo ago
yes you can
nico
nico•13mo ago
frontend deploy literally stuck here idk
No description
nico
nico•13mo ago
ill just wait it out and hope it'll actually finish sometime will let u know once its done
Brody
Brody•13mo ago
haha just abort it then use this to redoploy https://bookmarklets.up.railway.app/service-redeploy/
nico
nico•13mo ago
its actually building now but that looks useful thx so both finished deploying but i'm getting "invalid URL" on the actual requests
nico
nico•13mo ago
again, the frontend has this
No description
nico
nico•13mo ago
oh wait its missing a trailing slash
Brody
Brody•13mo ago
hold on the url variable itself should not have a trailing slash
nico
nico•13mo ago
the way i use it in the code i use it
Brody
Brody•13mo ago
here pathSuffix should have a prefixed slash the service variable definitely should not have a trailing slash
nico
nico•13mo ago
ok ill update my code to accomodate for it then
Brody
Brody•13mo ago
its just how its always done
nico
nico•13mo ago
ya ur right its really weird but all the requests r now timing out? both front and backend were deployed successfully and it passes health check as well
Brody
Brody•13mo ago
what is the image your next's dockerfile uses?
nico
nico•13mo ago
nextjs one uses nixpacks
Brody
Brody•13mo ago
what does it timeout on? can you send the error?
nico
nico•13mo ago
any api request that proxies through and tries to hit that backend for example, https://tryrapidly.com/api/py/hi
Brody
Brody•13mo ago
no like what is the specific timeout error? dns lookup, conn timeout? etc
nico
nico•13mo ago
shrugg
No description
nico
nico•13mo ago
i had it running through a cloudflare proxy i just purged cache and turned proxy off but its still not loading anything
Brody
Brody•13mo ago
im looking for errors printed by next in the deployment logs
nico
nico•13mo ago
nothing at all on next side i never disabled the public domain so theoretically shouldn't it be accessible at https://backend-production-bc84.up.railway.app:3000/ ? that one also times out
Brody
Brody•13mo ago
nope, you can only access https services externally from port 443 the first step id take would be to log the url that next requests during a proxy request
nico
nico•13mo ago
No description
Brody
Brody•13mo ago
^
nico
nico•13mo ago
alr
Brody
Brody•13mo ago
and didnt you remove the public backend domain?
nico
nico•13mo ago
not yet
Brody
Brody•13mo ago
what do the logs of the python service look like?
nico
nico•13mo ago
empty
No description
nico
nico•13mo ago
nothing relevant rlly except that its running would anything change if we changed all the ports to port 80?
Brody
Brody•13mo ago
we dont wanna do that
nico
nico•13mo ago
i think prev i had issues with using port 4000 but fixed when i used ASthink ok
Brody
Brody•13mo ago
add --error-logfile - to the hypercorn start command (CMD)
nico
nico•13mo ago
like this? CMD hypercorn main:app --error-logfile --bind [::]:$PORT or with that extra -
Brody
Brody•13mo ago
the little dash is needed
nico
nico•13mo ago
CMD hypercorn main:app --error-logfile - --bind [::]:$PORT alr redeploying
nico
nico•13mo ago
frontend logs, i printed out the url we try to hit (ignore the error its irrelevant to this, just had to reset cloudflare cache)
No description
nico
nico•13mo ago
backend logs meanwhile still look like this
No description
Brody
Brody•13mo ago
seems like you are running some kind of blocking code thats preventing any requests from working
nico
nico•13mo ago
hmm there shouldnt be anything blocking the backend code is basically unchanged since we've started and its working locally
Brody
Brody•13mo ago
maybe add more workers? --workers 4
nico
nico•13mo ago
even for endpoints that just return a hardcoded result
Brody
Brody•13mo ago
have you tried hypercorn locally?
nico
nico•13mo ago
not locally trying now
Brody
Brody•13mo ago
after all, i did say should be a drop in replacement 😉
nico
nico•13mo ago
hypercorn main:app --error-logfile - --bind \[::\]:3000 works locally
No description
nico
nico•13mo ago
shrugg mean what really suprises me is that it does pass the healthcheck too
Brody
Brody•13mo ago
do you get the same logs from hypercorn locally too?
nico
nico•13mo ago
zero logs whatsoever
nico
nico•13mo ago
No description
nico
nico•13mo ago
im not actually sure what the query-engine thing is i dont think it should be a problem tho?
Brody
Brody•13mo ago
id guess the query-engine thing is the blocker here, i have seen similar things be an issue
nico
nico•13mo ago
NotLikeThis
Brody
Brody•13mo ago
have you tried increasing the workers?
nico
nico•13mo ago
no but do u think that could solve the problem? i mean it was working before also i think query-engine is some printout by prisma db
Brody
Brody•13mo ago
i dont think it could hurt
nico
nico•13mo ago
i add this to the hypercorn cmd right?
Brody
Brody•13mo ago
yep
nico
nico•13mo ago
alr redeployed once more ur patience is impeccable wow
Brody
Brody•13mo ago
haha thank you
nico
nico•13mo ago
unfortunately...no change
nico
nico•13mo ago
No description
nico
nico•13mo ago
still timing out
nico
nico•13mo ago
wait i think the server just isnt even starting...
No description
nico
nico•13mo ago
this was from an older deploy er wait no thats just difference between logging of uvicorn and hypercorn
Brody
Brody•13mo ago
yeah thats just uvicorns logs
nico
nico•13mo ago
AHHHH we're so close to getting it working too ok lemme try to isolate the problem im going to make a fastapi server that is super simple and has just one single endpoint and see if that works
Brody
Brody•13mo ago
it would but hypercorn isnt applicable there
nico
nico•13mo ago
wait wdym?
Brody
Brody•13mo ago
haha sorry my bad, for some reason i thought you said flask api
nico
nico•13mo ago
nw if this simpler version works then its def just something in code thats resulting in some blocking behavior like u mentioned
Brody
Brody•13mo ago
thats likely the case, though still strange that it worked with uvicorn
nico
nico•13mo ago
yep it works with the simple server https://tryrapidly.com/api/py/hi
from fastapi import FastAPI, Header, Query

app = FastAPI()

@app.get("/hi")
def t():
return {"Hello": "World"}
from fastapi import FastAPI, Header, Query

app = FastAPI()

@app.get("/hi")
def t():
return {"Hello": "World"}
Brody
Brody•13mo ago
unauthed but i believe you seems like uvicorn was monkeypatching some blocking code
nico
nico•13mo ago
ok now im testing it locally and its also timing out guess i just have some code screwed up somewhere very interesting tho
Brody
Brody•13mo ago
always easier to debug locally though!
nico
nico•13mo ago
so much sigh unfortunately the game of whack a mole continues
Brody
Brody•13mo ago
youll have that in the software dev world lol
nico
nico•13mo ago
Hey just wanted to let u know I ended up resolving the error. There was an await function for disconnecting from the db after making a connection that was causing it to stall tysm brody <:HGLoli_heart:640312018511069184>
Brody
Brody•13mo ago
haha yep that would for sure do it, happy to help!
Want results from more Discord servers?
Add your server