Trouble connecting Django to PostgreSQL database
Hello,
I am very much a new developer trying to deploy a Django app with PostgreSQL. After reading the docs, it seemed to me that I should do the following:
* Provision a PostgreSQL database (which I have done, with Project ID: b90272c3-dbc8-4122-b259-9d4f79d6642d)
* Connect it to my local Django project (which has been shown to work with my local PostgreSQL).
* Eventually, I would then set these environment variables with a .env and it should just work after deploying. I'm sort of following this tutorial: https://kowe.io/guide/deploy-django-on-railway-postgresql
However, I'm having trouble connecting my Railway provisioned database to my local Django project. When I try to run python3 manage.py migrate, the command just hangs for a very long time with no visible errors. I would really appreciate any help or direction.
Deploy Django on Railway - PostgreSQL - by clintonmatics - Kowe.
This tutorial is about deploying your Django web application to Railway, this time with a PostgreSQL database.
137 Replies
Project ID:
b90272c3-dbc8-4122-b259-9d4f79d6642dis your postgres database in the same project as your django service?
Hi Brody, thanks for answering. I think that's maybe what I'm trying to set up. I haven't hosted my Django project anywhere, it's just on my local machine. I am hoping to have it be hosted by Railway and use the Postgres DB, if that makes sense. I thought I would connect my local django project to the Railway Postgres DB first and then continue on. Does that make sense? Maybe I'm not expressing myself well of I have a misunderstanding in how this should work.
yeah it makes sense, you only have the database on railway right now
show me the database thing in your settings.py please
Ah, I see...I think I can see how I can make a Django deployment that includes an already linked Postgres database now...
In any case, this is my database setup in settings.py:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql_psycopg2",
"NAME": os.environ.get("PGDATABASE"),
"USER": os.environ.get("PGUSER"),
"PASSWORD": os.environ.get("PGPASSWORD"),
"HOST": os.environ.get("PGHOST"),
"PORT": os.environ.get("PORT"),
}
}
And in my .env file, I have the actual values, copied over from the Variables section of the database
Thank you so much for your help, by the way! I really appreciate it
lets try to do this in a way that doesnt involve putting the login to your database in a plaintext file
in the same project as your postgres database, add an empty service
OK, thank you. I added it
screenshot please
give it a better name haha
ok, I just went for the default
Will this empty service host the environment files? Or would it be the django project? What makes sense as a name?
this empty service for now will hold the variables, and it will also eventually be where you deploy your app to
so lets see a new screenshot after the name change
Ok, I see
Here you go
for the sake of simplicity, just name it "bibblio"
no need for complex names here
Got it
open up the bibblio service variables and its raw editor, paste this in -
save and deploy
Got it, I was reading about shared variables, this is setting the variables from the Postgres DB to the local environment of what will be the django project, right?
It saved and deployed correctly
click their eye icons, do they all show the variables correctly?
yup!
is there any other variables your app needs? if so, go ahead and add them there
the end goal of this is so that you are able to delete your local .env file
I see. Yes, I have a few others, a Secret Key and a variable for whether the Django project is in Debug mode or not. I'll add them
ah you have debug set by a variable, okay you can keep the local .env file with only that debug variable set inside of it
in the service variables youd want to have that debug variable set so your app runs in production mode when on railway
Ok, yes. So in my local env, I'll set DEBUG to True but when I add it to the service variable I set it to false.
correct
so show me your local .env file now, it should only contain the debug line
OK, I added and deployed the changes and here's the .env file. Crucially, the DJANGO_DEBUG on the service is False
Ah sorry for the bad screenshot of the env file
looks good to me
do you have the railway cli installed?
Yes, I just installed it
And I logged in and all
then in your project directory, run
railway link and link to the bibblio servicedone
do you have migrations ready to apply?
Yes, I believe so
okay, run
railway run python manage.py migrate
this runs the given command locally, but with the linked service variables available, much better than keeping the database credentials in plaintextGot it. That makes sense. It's running
Or at least, I haven't gotten a completion nor error message
awsome
Thank you so much again, by the way. I've really been struggling with trying to deploy this app for months (on and off) and it's been frustrating. I really appreciate you helping me out
you sure waited a long time to ask for help haha
Haha, yea, I think my mistake was jumping from one deployment environment to another (Google Cloud, Vercel, Azure) and not finding ways to ask for helpthere
yeah i dont think any of those services have a good community
Yea, it was a lot of forum hunting and being frustrated, even trying to reach out to people who wrote tutorials on LinkedIn to no avail
but anyway, I unfortunately have to run to a meeting for my day job soon (within 10 minutes)
Do you think I might be able to ask for some help later? Or generally, what next steps do i take? Should I simply run hopefully once the migrations run?
do you have your code on github?
Yea, I do here: https://github.com/dalbanhi/bibbl.io
GitHub
GitHub - dalbanhi/bibbl.io: A repository for Bibbl.io, a digital bo...
A repository for Bibbl.io, a digital book/journal organizer - dalbanhi/bibbl.io
Do you think I might be able to ask for some help later?of course! then youd want to go the github deployment route and not railway up, we can get that hooked up when you are free again
Ok, sounds good! Thanks so much, Brody!
Another quick question: why does the migration take so long? Is it simply because it's running this on the Railway server?
as mentioned, its not running this on the railway server, its ran locally
Ah ok. I think I'm understanding a bit better. However, I'm a little concerned as the migrations never ended/failed
screenshot please
So this is the terminal where I ran the railway command (first screenshot). However, the second screenshot is where I was running my project locally ( I forgot to kill it beforehand). I got the attached error. I'm thinking I should stop my local server and then re-try the railway command?
yes
Ok
I'm running it again. Approximately how long do you usually take? The migrations for my project usually take very little time when I ran it locally before so I'm a little confused as to why it's taking significantly longer
nah theres for sure something wrong here
what verison of the cli are you on
railwayapp 3.5.2
just for fun, what happens if you run
python manage.py migrateAh. There, I get this error:
okay thats normal
what do you get if you run
railway --helpit seems to be working well
Hey Brody, again, I really appreciate your help. I had an idea since I'm not really sure how to proceed from here. Would it make sense to try to set up a django with postgres template and then try to add my github repo to the django service? Or would that possibly interfere with what we're trying to do here
nah something is going on here with your project and the cli, issue isnt with the railway side of things
what does
railway variables print
well dont send that here, this is publicOh, sorry--yea I should have realized
check the database on railway, does it have tables?
Yes it does
im kinda stumped right now
Hmm, ok, no worries Brody. I appreciate you trying to help me out anyway
Do you think it would at all make sense to delete the services on my project and re-try it? I'm really not sure what else I could do
no, theres no issues with railway here
OK. Well, I think I will try to uninstall the CLI since you mentioned the issue seems to be there
and re-install it
Actually, one question. I should be connecting to the 'bibblio' service, right? not the "Postgres" service?
correct
Should I have a Procfile or anything in my root directory?
yes, but that has nothing to do with the cli
Hm ok, yea. Thanks for your help! I'll reach out again if some of my tinkering does anything
try
railway shell and then run the migration command (without railway run)When I run railway shell, it seems to immediately boot me off of it
or like, I'm not able to type in a command before getting back to my regular command line
you are in the shell
what terminal is this?
Its zsh
mac?
yea
okay so what happens if you run the migration command in the railway shell
I see... I didn't understand that I was in the shell. I thought it would be more like when you run a python repl or like when connecting to postgres with psql. I ran the command in the railway shell and it's still taking a long time, just like with the
railway run optionsomething funky is happening
can you temporarily remove your local .env file
ok
I removed it and ran it again with
railway shell . It seems to still be stuckthis is super odd
okay desperate attempt, copy the raw variables from railway into a local .env file. then run the migration command without railway run or railway shell
ok
So, wait, you mean to copy the actual values of the Postgres service, not the related variables like ${{Postgres.PGDATABASE}}, right?
yeah you'd need to replace the references with the values
Yea, it still didn't work. This is what my .env file looks like and the migrations are still hanging
you aren't in a railway shell right?
no, I don't think so-- when I ran railway shell it removed my virtual environment (venv), so I'm pretty sur e I'm in my regular shell
okay so then there is something wrong with your project
or your network
go download dbgate and use it to try to connect to the database
Hmm... ok. I have another meeting soon that I have to run to. So I'll try to do that and get back to you
Brody, I really deeply do appreciate your help
It's possible that its my network-- I'm a teacher so I work at a school and there might be some network restrictions
oh yes from personal experience, schools do have very restrictive firewalls
Ok. I'll try this back home and see if it works on my home network. If not, I'll try to work through DB Gate. I do have a bit of a busy schedule the next few days due to the holiday, but if I encounter any more issues, I'll be sure to ping you
thanks so much again!
happy to help, and yep feel free to ping
Hey Brody! Guess what? Trying it out at home not on the school's firewall got me different (and fast) results! I tried to run both the migration command with/without railway run and in both cases, I got the same error (see attached).
I ran into something similar when switching databases from local sqlite to postgres. What I did there is that I dumped my database into a json (it wasn't very large), then deleted the migrations, and then ran my migrations, and then finally reloaded my data using loaddata in django.
I feel like that might work, but something in me thinks that there might be something obvious I am missing that would not involve me doing this, but somehow 'syncing' or merging my databases instead.
What do you think, do you have any insights onto how to solve this new issue? This SO post (https://stackoverflow.com/questions/44651760/django-db-migrations-exceptions-inconsistentmigrationhistory) gave me some ideas last time but I'm wary of dropping my database since I know I wouldn't actually be able to do that with a team/larger database.
Stack Overflow
django.db.migrations.exceptions.InconsistentMigrationHistory
When I run python manage.py migrate on my Django project, I get the following error:
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.a...
do you have any data in the database? like besides the tables existing?
I do have some data, like 'testing' data from when I was testing out my app. I wouldn't be sad if I had to drop those tables completely, but generally, I guess I've never really gone from dev to production and so I'm not really sure what the process for your data is supposed to go. If it were possible to keep that testing data, that would be nice but I'm not like wedded to it or anything.
Also, Brody, thanks so much for your help. Do you have something like a Ko-Fi and/or like Venmo or something? You've really helped me out a bunch and honestly just made me feel like this was possible. I'd love to like buy you a cup of coffee in gratitude or something
I'm assuming your data in the database got messed up somehow, I don't know your project so I wouldn't really know on how to advice fixing it without wiping the database and re-running the migration again.
i do have a bmac in my bio, but please don't feel any need to send me anything
got it, yea, I don't feel the need to, I just appreciate it! So, just as a follow up question, does just wiping my database mean deleting my migrations folder and trying to run teh migrations again?
wiping your database would mean you need to go into the volume's settings and wipe it, then running the migrations again with
railway runHey Brody, thanks for getting back to me. I'm not sure what you exactly mean by 'wipe the volume's settings'. Would that be the Postgres database I just made on Railway? If I go into my Railway project, I have some tables there (see attached) but I don't actually have like any tables that correspond to any data that I might have created, like by creating users or anything. So I'm a little confused as to what/why I would wipe this volume
so to recap you where having issues with the database and i recommended wiping it so that you can start with a fresh migration
Yes, exactly
sorry, I should have mentioned that
so is that something you are going to do or are you going to try to resolve the errors manually?
I'm going to try to do that because I have a back up of my data/this is a testing project. I want to learn how to deploy this project and I'm not too fussed about the data
okay then go ahead and wipe the database's volume
So, what do you mean by that? Do I go into the Railway project and just delete all of these tables or is that something I need to do on my like local environment. I don't know if that makes sense
you simply press the wipe volume button in the databases's volume settings
OK. I'm having some trouble finding that but I will look for it. But afterwards, I should be able to run
railway run python manage.py migrate and then the same with runserver and if I have no errors, I can push to my github repo, then connect my repo to the service and it should work?click on the volume of the postgres service
and yes you would want to run
railway run python manage.py migrate and then you can redeploy the railway serviceOK. I think I did both of those things. In order to redeploy the Railway service, you mean the Postgres database, right?
Hey Brody, so I'm able to create a new user with my Postgres database service from my local machine, which is exciting! However, I then tried to deploy by connecting to my repo and it said my deploy finished. In my deploy logs for that service, all I see is this (attached), which doesn't seem good. Do you have any tips as to what to explore? I'm not entirely sure why python would be a command that is not found on Railway's python
please send your build logs
Ok. I think it's possible my issue is this: https://www.answeroverflow.com/m/1113611990238240778 (someone very knowledgeable giving some good tips there 😉 ), so I'm going to try what you are suggesting there.
I too have a package.json too because I guess I tried to make a hybrid react/django app and I'm using webpack to bundle my react files into a index-bundle.js files. In the future, I'm probably sticking with more of a cleanly split react front end / django cors framework app, but I'm digressing. Below is my build log:
/bin/bash: line 1: python: command not found - Railway
I'm facing an issue while deploying my Django project on Railway. The deployment logs show the error message "/bin/bash: line 1: python: command not found," even though I have both a requirement.txt file and a runtime.txt file in my project. I'm unsure about the cause of this problem and would appreciate some guidance from a professional develop...
Pushing [==================================================>] 94.63MB 3bea1f1ca704
Pushing [==================================================>] 96.77MB 3bea1f1ca704
Pushing [==================================================>] 98.95MB 3bea1f1ca704
Pushing [==================================================>] 101.1MB 3bea1f1ca704
Pushing [==================================================>] 103.2MB 3bea1f1ca704
Pushing [==================================================>] 105.9MB 3bea1f1ca704
Pushing [==================================================>] 106.5MB 3bea1f1ca704
Pushed 3bea1f1ca704
750024d5-965f-44e2-9fd6-d06bda6f036d: digest: sha256:9efe2f346744dc0f2552ce1a4c817d49b2c63d92a3f9f902a8ede9444b948049 size: 2421
Publish time: 9.84 seconds
(or at least the ending of it)
Hmm, so I tried this railway.json file (the one you suggested) and my build failed this time
It doesn't seem to have found a version of Django on my requirements.txt but I just did pip freeze > requirements.txt to save my package requirements (this is my build log):
Any help or insights at all you can point me to are super appreciated
full build logs please
im pretty sure i have sent you this, but you havent been using it so ill send it again -
https://bookmarklets.up.railway.app/log-downloader/
Ah, no I dont think I'd seen that. A useful tool
what version of python do you use locally?
Python 3.11.8
then you would want to add a runtime.txt to your repo with just
3.11 in ithm ok. Gotcha, thanks for the tip -- I'll let you know how it goes!
So the deployment worked but I'm getting a TCP/IP connection error (log attached). I'm trying to debug the issue and any help would be appreciated!
omg, wow, I don't know how I would have caught that. Thanks for the second set of eyes
you arent the first to make that mistake
haha thanks
I'll let you know how it goes, but I'm about to have dinner. Thank you sooo much again for all of your help
It's truly amazing
happy to help
Thanks so much! After fixing some more typos, I'm getting no build nor deployment errors (I think... is it normal for a deploy log to look like this?):
yeah that looks good to me
Awesome! Yea, I think I'm not serving my static files for my django project correctly, but I think that's more of a django thing than a railway thing. I really appreciate all of the help, again!
yeah read up on the whitenoise docs
will do! Have a good night!
you too!
Hey Brody, I hope you're doing well. I had a quick question I was hoping you might be able to point me in the right direction of. I'm having some csrf troubles on my django app still (it's showing up now but I can't post anything). So, I'm trying to write to my own debugging logging file. I'm able to write to my own file on my local machine, but whenever I'm not sure where that logging file would exist on the Railway server. Would you happen to know anything about that? Any help/insight would be greatly appreciated
you would want to log to stdout/stderr not a file