R
Railway10mo ago
SHxKM

Running one off commands in Docker container

Super weird to me that Railway doesn't support simply SSHing into a VM. Anyway, I'm deploying a Django project. In Django there's something called management commands and it's imperative for me to be able to use it. Locally, I do it with:
python manage.py [command] [args]
python manage.py [command] [args]
But since I'm using a containerized environment, I do it like this:
./run manage [command] [args]
./run manage [command] [args]
I thought I can do the same remotely with railway link. When I do:
railway run ./run manage [command] [args]
railway run ./run manage [command] [args]
The command seemingly runs successfully. But I know it didn't run on the railway production instance because it's supposed to populate DB rows, and nothing like that happened in the Railway environment. Am I out of luck?
99 Replies
Percy
Percy10mo ago
Project ID: 87f6d50b-7bab-488e-b802-02f9edc442e3
SHxKM
SHxKM10mo ago
87f6d50b-7bab-488e-b802-02f9edc442e3 Even running:
railway run ./run manage createsuperuser
railway run ./run manage createsuperuser
Only creates the superuser locally (it's super weird this commands affects both local and production environments). So, I just don't see a way to use Railway if we can't run commands remotely on the VM. I don't see myself sharing commands between local and prod environments (not that the above is working). I guess this is rooted in the fact that run in my case does these actions in a container, even if I did remove that and just make it run python3 manage.py createsuperuser, it will fail because the it's executed against the local source code, and locally I'm running my Python interpreter in a docker-compose container. OK, so I went as far as logging into the docker container locally, installing railway CLI inside the container, and from within the container, do railway link, and try to run:
railway run python3 manage.py createsuperuser
railway run python3 manage.py createsuperuser
But I get:
django.db.utils.OperationalError: connection failed: FATAL: password authentication failed for user "postgres"
django.db.utils.OperationalError: connection failed: FATAL: password authentication failed for user "postgres"
This can't be a misconfig because the app is able to read and write normally to the DB.
Brody
Brody10mo ago
railway does not use vm's, they run your code in empherial containers. railway run runs the given command locally but with the linked service variables available, so all you would need to do is run railway run python manage.py createsuperuser locally in your project's folder after linking to the appropriate service, no need to be inside a container or anything complicated
SHxKM
SHxKM10mo ago
Thanks @Brody. I’d suspected this is what’s happening, and that it won’t work with the way my repo is laid out. The entire Python env is contained in a web container in my case, so I guess when I run my script, it always logs into that instead of the remote env vars.
Brody
Brody10mo ago
yeah if you had a local .env the variables from that would overwrite the environment variables set "inside" railway run
SHxKM
SHxKM10mo ago
Any workarounds that come to mind? I’d really like to keep “duplication for the purpose of deployment” to a minimum.
Brody
Brody10mo ago
stop using .env files and store your environment variables in railway instead
SHxKM
SHxKM10mo ago
Hmmm, huh? Are you suggesting I also hold my local (developlment environment) ENV variables on Railway as well?
Brody
Brody10mo ago
absolutely, why not? but those would be stored as services variables under a development environment in your railway project you use the cli to switch environments, then you run your app locally through railway run, then switch back to production to create a superuser account on the production database
SHxKM
SHxKM10mo ago
Well I don’t think it’s horrible? Except it couples me to your platform a little But I might just try to go with that. Or maybe there’s a magic way to copy the python env from inside the Docker container ugh…
Brody
Brody10mo ago
it does, but you can then just export the entire service variables as a .env file if you want to leave the platform
SHxKM
SHxKM10mo ago
I think im gonna give this a try. Can I create an environment without any services?
Brody
Brody10mo ago
nope, there needs to be services, otherwise you have no place to store your variables
SHxKM
SHxKM10mo ago
Wait. Doesn’t this force me to pay for Postgres + Redis + Web needlessly? Because right now I’m running those locally with docker compose
Brody
Brody10mo ago
no, you don't need to actually run them, they can just be there as skeleton services
SHxKM
SHxKM10mo ago
hmm nice, I think. How would I go about intializing a copy of my production ENV?
Brody
Brody10mo ago
in fact with the new environment changes you may be able to just have a single blank service that holds your service variables from the projects settings under the environments tab
SHxKM
SHxKM10mo ago
this is gonna be awesome if works nicely enough
Brody
Brody10mo ago
well this is why railway run and railway shell exist
SHxKM
SHxKM10mo ago
Do I now need to deploy in order for the variables to take effect?
No description
Brody
Brody10mo ago
just hold your alt key down while pressing deploy so the variables are saved
SHxKM
SHxKM10mo ago
Sorry for the dumb question, is that option or control on Mac? also, will this not spin up the service?
Brody
Brody10mo ago
lol let me go Google a picture of a Mac keyboard yeah it's the option key as far as I can tell the alt deploy is just supposed to save variables, but you can always go remove the source if you want before you save
SHxKM
SHxKM10mo ago
yeah so now I have those envs same as the local .env file, I do:
railway environment
railway environment
Chooose local, but unfortunately it doesn't seem like it's able to pass the variables to docker compose up --build
Brody
Brody10mo ago
ah I see what you're doing, you may need to be inside the container that runs your app then, or just use compose to run the databases and run your app on your computer
SHxKM
SHxKM10mo ago
Sorry, and I appreciate your patience. But I understood neither options. If I understand you correctly:
or just use compose to run the databases and run your app on your computer
or just use compose to run the databases and run your app on your computer
This involves setting up a non-dockerized Python env locally. Not an option here because I wouldn't want all the duplication.
you may need to be inside the container that runs your app then
you may need to be inside the container that runs your app then
If I need to ocassionally SSH into a local container to run some one of commands I'm fine with that, but this is what I've been trying to do since noon.
Brody
Brody10mo ago
seems like you understood both options just fine, don't underestimate yourself!
SHxKM
SHxKM10mo ago
Yeah, except I've been trying to do #2 for a good few hours. Maybe I'm missing something there.
Brody
Brody10mo ago
so try option 2 again, but this time, make sure there is no .env file to be found, and link to the production service
SHxKM
SHxKM10mo ago
oooo Should I be using railway shell? or just link to the correct service/environment? option 2 isn't gonna work, in order to run ./run manage createsuperuser, the web container should be up locally. But the web container can't spin up locally without the .env file.
Brody
Brody10mo ago
if you have multiple commands you want to run, sure, but for one off commands use railway run though either command you will need to be linked to the desired project, environment and service you've gotten yourself into quite the pickle with how you're doing things personally I'd just use docker to run the databases, then run the app locally in a venv, it's far less complex that way
SHxKM
SHxKM10mo ago
I'm basing my project off of this excellent repository for deploying Django w/ docker compose: https://github.com/nickjj/docker-django-example I guess our item of interest is the run file (https://github.com/nickjj/docker-django-example/blob/main/run). The issue here isn't with how the project is set up, it's just that Railway doesn't support docker-compose, and I'm trying to force it to work still. @Brody we both know Docker isn't needed to run Redis/DB, as that's usually the simpler part (which you don't deploy with compose anyway in production, unless you self-manage)
Brody
Brody10mo ago
modify that run file to have the cli dynamically create a local .env file?
SHxKM
SHxKM10mo ago
Wait, if I do railway link and link to the local environment with the designated service, and then:
python
>> import os
>> os.getenv("SECRET_KEY")
python
>> import os
>> os.getenv("SECRET_KEY")
Then if all env variables were successfully populated from the railway local environment, it should outout my local env secret key, no? Because that's empty Interesting, it's only there if I enter railway shell. Let's try again now.
Brody
Brody10mo ago
railway run/shell isn't going to make an .env file, you need to run something completely different
SHxKM
SHxKM10mo ago
but the run file doesn't depend on an env file mostly, and I commented out the parts in docker-compose that do. I'm trying to pass the env variables from railway local env
Brody
Brody10mo ago
something like railway variables --kv > .env
SHxKM
SHxKM10mo ago
So railway link local-service, then docker compose up --build doesn't see the variables. Even a simple echo $SECRET_KEY returns empty. If do railway shell however, echo $SECRET_KEY does work. But for some reason docker compose up --build still complains.
Brody
Brody10mo ago
because it's looking for a physical .env file right?
SHxKM
SHxKM10mo ago
the docker-compose file? no, I removed all references to that
Brody
Brody10mo ago
then how will it know what environment variables to pass into the container hence my suggestion here, build this into the run file some way so it's automated
SHxKM
SHxKM10mo ago
OK, confused on how that'll help us achieve stuff. But will try
Brody
Brody10mo ago
unless you know of another way to pass environment variables into a container run by docker compose?
SHxKM
SHxKM10mo ago
no, I appreciate your help. Unfrotunately this is not working (incorporating railway variables --kv > .env). The container is built not by the run script, but simply by running docker-compose up --build, by the time we want to run railway run ./run manage.py createsuperuser, the envs are already baked in.
Brody
Brody10mo ago
why don't we want the envs to be baked in?
SHxKM
SHxKM10mo ago
because these are coming from the .env file.
Brody
Brody10mo ago
we had the cli make that env file, I'm not seeing an issue the end goal is to get the correct set of environment variables (the production environment variables) into your container that's running django so that you can shell into the container and create a superuser
SHxKM
SHxKM10mo ago
Railway wants to run the createsuper against a local container, running with env vars that are baked in (with local env_file), but it wants to reference production env vars. I don't think it's possible. Unless you're telling me I need to spin up containers referencing production's env vars whenever I want to change anything with production? so something like:
railway variables --kv > .env
docker compose up --build
railway run ./run do foo
railway variables --kv > .env
docker compose up --build
railway run ./run do foo
Brody
Brody10mo ago
once you have the production environment variables baked into the local container you don't need to use the railway cli inside the local container
SHxKM
SHxKM10mo ago
Let's try this
Brody
Brody10mo ago
unfortunately since you have quite the complex setup, yes. most people just have their project files and use the cli right in the project
SHxKM
SHxKM10mo ago
I'm just gonna do:
railway variables --kv > .env
docker compose up --build
railway variables --kv > .env
docker compose up --build
And then try to run the commands.
Brody
Brody10mo ago
I mean I was just guessing on that syntax to create a .env file, have you tried it?
SHxKM
SHxKM10mo ago
looks good
Brody
Brody10mo ago
yeah but does it create an env file lol
SHxKM
SHxKM10mo ago
yes, I meant the file looks good
Brody
Brody10mo ago
ah cool
SHxKM
SHxKM10mo ago
however after doing railway link with production, running docker compose up --build says no service selected.
Brody
Brody10mo ago
did you break something with your local docker compose setup?
SHxKM
SHxKM10mo ago
Doesn't look like it I don't know why it complains specifically after doing railway link
Brody
Brody10mo ago
docker compose seems to disagree
SHxKM
SHxKM10mo ago
oh wait OK, was missing profiles. But unfortunately this still isn't working.
django.db.utils.OperationalError: connection failed: FATAL: database "railway" does not exist
django.db.utils.OperationalError: connection failed: FATAL: database "railway" does not exist
Brody
Brody10mo ago
that's the output of the superuser command inside the django container?
SHxKM
SHxKM10mo ago
nah, that's trying to spin up local containers with PROD env vars
Brody
Brody10mo ago
that's what we want
SHxKM
SHxKM10mo ago
yeah except we want the container to not exit 😦 it's not able to establish connections. I guess not having a public DNS for my railway postgres has something to do with it?
Brody
Brody10mo ago
print the environment variables inside the container any dns resolver can resolve the public proxy domain
SHxKM
SHxKM10mo ago
I don't have a public proxy for it
Brody
Brody10mo ago
you removed the tcp proxy on the database?
SHxKM
SHxKM10mo ago
yes, I'm using private networking
Brody
Brody10mo ago
definitely would have been worth mentioning... a long time ago
SHxKM
SHxKM10mo ago
Very sorry about this, how would this affect anything else in the process except for this step? Also, I thought it's the most obvious thing to do with a DB.
Brody
Brody10mo ago
because its a private network you cant connect to the database locally if its not exposed publicly, you will need to expose it publically just for this operation
SHxKM
SHxKM10mo ago
I didn't any explicitly remove proxy or something, I only removed the public DNS. OK, I appreciate your help very much. It seems I need to rethink stuff here. Either I change the project structure, or move off to another host. To open my DB to the outer world and go through all the hoops every time I need to do this seems a bit too much. Very much appreciated @Brody !
Brody
Brody10mo ago
no problem!
SHxKM
SHxKM10mo ago
@Brody one idea that sprung up right now, is what if I comment out the dependency on the .env_file temporarily. Then, write a bash script to build with docker compose in a way that would pass the PROD env vars one by one: docker compose … -e FOO=BAR -e BAR=BAZ I still have to expose the database momentarily. What do you think?
Brody
Brody10mo ago
that looks like it would achieve the exact same thing railway variables --kv > .env does but 10x harder, and you still need to expose the database publicly
SHxKM
SHxKM10mo ago
Well if Railway doesn’t provide access to the “ephemeral container”, the decision is to either leave, or open temporarily. No matter what I do. Ok, 3rd option: spin up a virtual env specifically for talking with Railway. Non dockerized, nothing. Just has the env vars.
Brody
Brody10mo ago
the 3rd option is the best I personally don't see the need to run the django app in a docker container for local development when you can use a venv still need to connect to the database publicly though you did say you didn't remove a tcp proxy, so if you're correct in saying that, then it's still publicly available (though with a password)
SHxKM
SHxKM10mo ago
I didn’t remove a proxy explicitly. I did remove the public HTTP domain. And even after enabling it back, it said that Railway couldn’t connect.
Brody
Brody10mo ago
databases don't have http domains show me what the public networking section of the database's settings says please
SHxKM
SHxKM10mo ago
No description
No description
Brody
Brody10mo ago
yeah you removed the tcp proxy
SHxKM
SHxKM10mo ago
If by TCP proxy you mean the public networking domain, that’s true. It was an HTTP domain
Brody
Brody10mo ago
it wasnt lol, it was a tcp proxy
SHxKM
SHxKM10mo ago
I stand corrected
SHxKM
SHxKM10mo ago
@Brody 20 minutes after opening the TCP proxy, Railway is still not able to connect. Is that a sign of trouble?
No description
SHxKM
SHxKM10mo ago
Anyway, I'm trying to export POSTGRES_DB=[New Proxy URL], and then run python3 manage.py createsuperuser. No dice. Feel like I'm really close though The env variables are populated properly in railway shell And regarding what the UI asks me:
PGPASSWORD=******** psql -h -U postgres -p -d railway
PGPASSWORD=******** psql -h -U postgres -p -d railway
Doesn't work, it complaints that the port -p is not an int, since it isn't. The PGPORT env var is an empty string I can't psql into the postgres instance anyway, using the public proxy TCP address
SHxKM
SHxKM10mo ago
Oh interesting, I spun up a new postgres database just to see what has gone wrong. And it seems like the proxy setting looks different. @Brody please tell me I can open a proxy back after having removed it...
No description
No description
SHxKM
SHxKM10mo ago
OK, found it's just a button to the right. Clearly have been trying to make this work for too long. Success.
Brody
Brody10mo ago
yeah you can add and remove the proxy at any point also, what tool are you using to edit your screenshots with those nice little tooltips
SHxKM
SHxKM10mo ago
@Brody The very awesome Shottr, indispensable! (and free, though I found it so useful I felt compelled to pay).
Brody
Brody10mo ago
oh mac app 😦
SHxKM
SHxKM10mo ago
One more reason to switch
Brody
Brody10mo ago
ha like i have that kind of money
SHxKM
SHxKM10mo ago
don't feel bad. macOS and Apple have been going downhill for the last 4-5 years or so.
Brody
Brody10mo ago
so has windows
Zazh
Zazh10mo ago
sharex has that on windows
Brody
Brody10mo ago
I'll check it out, I currently use greenshot
Want results from more Discord servers?
Add your server