Railway•7mo ago

How to serve media files in Django app on Railway ? Nginx ? Please help

Hello everyone, I’m a software development student and me and my peers we have written our first web app with Django. We are absolutely new to Django, Railway and Nginx but we was learning on the road and after almost a week of deployement efforts we have managed to deploy our app on Railway. We used Dockefile, gunicorn and whitenoise for static files. All this is working great but the app doesn't serve media files. After research we discovered that apparently we need to use Nginx web server for serving media files (those uploaded via users and admins). So we'are trying to configure a Nginx webserver to do that but as we never worked with nginx our conf file is not working. All the static assets are served ok (js, css, etc) but files in media directory are not found. Please, could anyone help us to configure nginx or other to serve media files ? Project id: d616bd58-0440-4197-b741-24b36e54d293 settings.py:

STORAGES = {
    "default": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
    },
    "staticfiles": {
        "BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
    },
}

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

# The absolute path to the directory where collectstatic will collect static files for deployment.
STATIC_ROOT = BASE_DIR / "staticfiles"

# The URL to use when referring to static files (where they will be served from)
STATIC_URL = 'static/'

# Media files
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

STORAGES = {
    "default": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
    },
    "staticfiles": {
        "BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage",
    },
}

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

# The absolute path to the directory where collectstatic will collect static files for deployment.
STATIC_ROOT = BASE_DIR / "staticfiles"

# The URL to use when referring to static files (where they will be served from)
STATIC_URL = 'static/'

# Media files
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

47 Replies

Percy•7mo ago

Project ID: d616bd58-0440-4197-b741-24b36e54d293

stkr•7mo ago

nginx conf file:

upstream depistclic {
    server 0.0.0.0:80;
}

server {
    listen 80;

    server_name depistclic-production.up.railway.app;

    location / {
        proxy_pass http://depistclic;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location = /favicon.ico { access_log off; log_not_found off; }

    location /media/ {
        alias /app/DepistClic/media/;
    }
}

upstream depistclic {
    server 0.0.0.0:80;
}

server {
    listen 80;

    server_name depistclic-production.up.railway.app;

    location / {
        proxy_pass http://depistclic;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location = /favicon.ico { access_log off; log_not_found off; }

    location /media/ {
        alias /app/DepistClic/media/;
    }
}

dockerfile:

FROM python:3.10.12

RUN apt-get update
RUN apt-get -y install python3-pip python3-cffi python3-brotli libpango-1.0-0 libpangoft2-1.0-0 nginx

WORKDIR /app

RUN pip install pipenv

# Copiez le Pipfile et le Pipfile.lock dans le répertoire de travail
COPY Pipfile Pipfile.lock ./

# Installez les dépendances Python à l'aide de Pipenv
RUN pipenv install --deploy --ignore-pipfile

COPY . ./
COPY depistclic /etc/nginx/sites-available/depistclic

ENTRYPOINT ["sh", "./entrypoint.sh"]

FROM python:3.10.12

RUN apt-get update
RUN apt-get -y install python3-pip python3-cffi python3-brotli libpango-1.0-0 libpangoft2-1.0-0 nginx

WORKDIR /app

RUN pip install pipenv

# Copiez le Pipfile et le Pipfile.lock dans le répertoire de travail
COPY Pipfile Pipfile.lock ./

# Installez les dépendances Python à l'aide de Pipenv
RUN pipenv install --deploy --ignore-pipfile

COPY . ./
COPY depistclic /etc/nginx/sites-available/depistclic

ENTRYPOINT ["sh", "./entrypoint.sh"]

entrypoint.sh:

#!/bin/sh

cd DepistClic
pipenv run python manage.py migrate --no-input
pipenv run python manage.py collectstatic --no-input
ln -s /etc/nginx/sites-available/depistclic /etc/nginx/sites-enabled
nginx -t
service nginx start
service nginx status
pipenv run gunicorn DepistClic.wsgi:application --bind 0.0.0.0:$PORT

#!/bin/sh

cd DepistClic
pipenv run python manage.py migrate --no-input
pipenv run python manage.py collectstatic --no-input
ln -s /etc/nginx/sites-available/depistclic /etc/nginx/sites-enabled
nginx -t
service nginx start
service nginx status
pipenv run gunicorn DepistClic.wsgi:application --bind 0.0.0.0:$PORT

Brody•7mo ago

(use triple backticks for code blocks)

stkr•7mo ago

done, I hope is more readable now. thanks

Brody•7mo ago

but have a look at this template, nginx isnt strictly needed https://railway.app/template/AWUIv6 specifically https://github.com/vfehring/django-volumes/blob/master/mysite/settings.py#L133-L134 and https://github.com/vfehring/django-volumes/blob/master/mysite/urls.py#L27 mount the volume to /app/media and there is no need to set that RAILWAY_VOLUME_MOUNT_PATH variable as it is set automatically on deploy

stkr•7mo ago

thank you, I'm reading it

stkr•7mo ago

I did this but I'm not sure if it's right: -I've mounted a volume for my app with mount path: /root/DepistClic/DepistClic/media -I didn't change anything in settings.py -And this is my urls.py:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('mainapp.urls')),
    path("__reload__/", include("django_browser_reload.urls")),
]+static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('mainapp.urls')),
    path("__reload__/", include("django_browser_reload.urls")),
]+static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

But it's still doesn't work ? Have I missed something ? Sorry it's the first time I'm doing this

Brody•7mo ago

I've mounted a volume for my app with mount path: /root/DepistClic/DepistClic/media

please mount the volume to the path i specified

I didn't change anything in settings.py

i have shown you what to change in your settings.py

And this is my urls.py

i have shown you what to change in your urls.py

I did this but I'm not sure if it's right

it seems like you didnt follow any of the instructions i gave?

stkr•7mo ago

fixing it. thanks Ok, this is what I did, I followed instructions as precisely as I can understand it: -The volume's mount path is: /app/media -The settings.py:

# The absolute path to the directory where collectstatic will collect static files for deployment.
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# The URL to use when referring to static files (where they will be served from)
STATIC_URL = 'static/'

# Media files
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

# The absolute path to the directory where collectstatic will collect static files for deployment.
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# The URL to use when referring to static files (where they will be served from)
STATIC_URL = 'static/'

# Media files
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

-urls.py:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('mainapp.urls')),
    path("__reload__/", include("django_browser_reload.urls")),
]

urlpatterns += [ re_path(r'^media/(?P<path>.*)$', serve, {'document_root': settings.MEDIA_ROOT})]

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('mainapp.urls')),
    path("__reload__/", include("django_browser_reload.urls")),
]

urlpatterns += [ re_path(r'^media/(?P<path>.*)$', serve, {'document_root': settings.MEDIA_ROOT})]

` But I think I still miss something. I should put media files manually on the volume right ?

Brody•7mo ago

your app would need to provide a way to upload the images, please referance the template's code i linked since it does just that

stkr•7mo ago

This code from the template https://railway.app/template/AWUIv6

new_directory = os.path.join(settings.MEDIA_ROOT, 'directory')
if not os.path.exists(new_directory):
  os.makedirs(new_directory)

new_directory = os.path.join(settings.MEDIA_ROOT, 'directory')
if not os.path.exists(new_directory):
  os.makedirs(new_directory)

I don't get it. This is what I understand: -There is an image field in my database entries: for an item it is currently at images/file.png for example. -Currently this file is stored at ~/DepistClic/DepistClic/media/images/file.png and the app is unable to serve it to the browser from https://depistclic-production.up.railway.app/media/images/file.png --> "The requested resource was not found on this server." -We have mounted a volume with mount path app/media so the the MEDIA_ROOT = app/media because MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"] But currently I cannot find this path app/media. I've read the doc about volumes https://docs.railway.app/reference/volumes and I should be able to find it. Now I don't understand how the media files will be served from the volume and how I put those file there, or how I tell Django to serve them from there. As an admin or user I upload the image to the app. It goes to media directory of the app right ? Could you help me to understand ? I think I'm missing something fundamental here Thank you for your time and patience, I'm still learning

Railway Docs

Volumes | Railway Docs

Documentation for Railway

Brody•7mo ago

I'll just tag in the creator of that template when I see him online @Vin - we need assistance 🙂

MantisInABox•7mo ago

lemme catch up Can you share your repo?

stkr•7mo ago

sure https://github.com/v-dav/DepistClic

stkr•7mo ago

the branch from which I'm trying to deplmoy is this: https://github.com/v-dav/DepistClic/tree/deployment_vlad

GitHub

GitHub - v-dav/DepistClic at deployment_vlad

Decision aid tool to help healthcare professionals screen for complications in patients affected by Diabetes Mellitus Type 2 - GitHub - v-dav/DepistClic at deployment_vlad

MantisInABox•7mo ago

Few things I can see right off the top: Edit settings.py lines 135-145 to be as follows

STATIC_URL = 'static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static")]
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# Media Files (uploaded from users)
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

STATIC_URL = 'static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static")]
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# Media Files (uploaded from users)
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

This will set the app properly for serving the images.. You have the urls.py file setup correctly. which of your mainapp/urls.py routes will be displaying images from the mounted volume? Or are you trying to serve the images that you have in DepistClic/DepistClic/media?

stkr•7mo ago

url: path('download_ordo/<int:screening_test_id>/', views.download_ordo, name='download_ordo') This allows to open the pdf file if it's loaded in the database, via this view:

def download_ordo(request, screening_test_id):
    screening_test = ScreeningTest.objects.get(pk=screening_test_id)
    ordo_url = screening_test.get_ordo_url()
    
    if ordo_url:
        return redirect(ordo_url)

def download_ordo(request, screening_test_id):
    screening_test = ScreeningTest.objects.get(pk=screening_test_id)
    ordo_url = screening_test.get_ordo_url()
    
    if ordo_url:
        return redirect(ordo_url)

And this is the part that rendering the button with the link to the pdf on the page:

{% if screening_test.ordo %}
                                <form action="{% url 'download_ordo' screening_test.id %}" method="get" class="inline" target="_blank">
                                    <button class="mt-4 bg-red-500 hover:bg-red-800 text-white font-bold py-2 px-4 rounded-lg">Ordonnance</button>
                                </form>

{% if screening_test.ordo %}
                                <form action="{% url 'download_ordo' screening_test.id %}" method="get" class="inline" target="_blank">
                                    <button class="mt-4 bg-red-500 hover:bg-red-800 text-white font-bold py-2 px-4 rounded-lg">Ordonnance</button>
                                </form>

And for the images that are rendered on the page if available in database:

{% if screening_test.image %}
                            <img src="{{ screening_test.image.url }}">
{% endif %}

{% if screening_test.image %}
                            <img src="{{ screening_test.image.url }}">
{% endif %}

It's our first app so I understand if the design may appear quiet horrible 😐

MantisInABox•7mo ago

And to confirm... What is the mount path of your volume?

stkr•7mo ago

/app/media

stkr•7mo ago

ok I put this in settings:

# Static files (CSS, JavaScript, Images)

STATIC_URL = 'static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static")]
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# Media files
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

# Static files (CSS, JavaScript, Images)

STATIC_URL = 'static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static")]
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")

# Media files
MEDIA_URL = "media/"
MEDIA_ROOT = os.environ["RAILWAY_VOLUME_MOUNT_PATH"]

MantisInABox•7mo ago

Okay, it looks like it may conflict with the media folder that you have in your app already... you can change the mount point to /app/uploads and then the MEDIA_URL = "uploads/" in your settings.py file... I am trying to rule out the various issues

stkr•7mo ago

Ok, it's done. This is the deployement log:

System check identified some issues: WARNINGS: ?: (staticfiles.W004) The directory '/app/DepistClic/static' in the STATICFILES_DIRS setting does not exist. Operations to perform: Apply all migrations: admin, auth, contenttypes, mainapp, sessions Running migrations: No migrations to apply. System check identified some issues: WARNINGS: ?: (staticfiles.W004) The directory '/app/DepistClic/static' in the STATICFILES_DIRS setting does not exist. 130 static files copied to '/app/DepistClic/staticfiles'. [2023-11-08 20:15:52 +0000] [36] [INFO] Starting gunicorn 21.2.0 [2023-11-08 20:15:52 +0000] [36] [INFO] Listening at: http://0.0.0.0:5959 (36) [2023-11-08 20:15:52 +0000] [36] [INFO] Using worker: sync [2023-11-08 20:15:52 +0000] [38] [INFO] Booting worker with pid: 38

MantisInABox•7mo ago

Okay, that's positive now... You are using a dockerfile when deploying, yes?

stkr•7mo ago

yes, here:

FROM python:3.10.12

RUN apt-get update
RUN apt-get -y install python3-pip python3-cffi python3-brotli libpango-1.0-0 libpangoft2-1.0-0

WORKDIR /app

RUN pip install pipenv

# Copiez le Pipfile et le Pipfile.lock dans le répertoire de travail
COPY Pipfile Pipfile.lock ./

# Installez les dépendances Python à l'aide de Pipenv
RUN pipenv install --deploy --ignore-pipfile

COPY . ./

ENTRYPOINT ["sh", "./entrypoint.sh"]

FROM python:3.10.12

RUN apt-get update
RUN apt-get -y install python3-pip python3-cffi python3-brotli libpango-1.0-0 libpangoft2-1.0-0

WORKDIR /app

RUN pip install pipenv

# Copiez le Pipfile et le Pipfile.lock dans le répertoire de travail
COPY Pipfile Pipfile.lock ./

# Installez les dépendances Python à l'aide de Pipenv
RUN pipenv install --deploy --ignore-pipfile

COPY . ./

ENTRYPOINT ["sh", "./entrypoint.sh"]

` With the entrypoint.sh:

#!/bin/sh

cd DepistClic
pipenv run python manage.py migrate --no-input
pipenv run python manage.py collectstatic --no-input
pipenv run gunicorn DepistClic.wsgi:application --bind 0.0.0.0:$PORT

#!/bin/sh

cd DepistClic
pipenv run python manage.py migrate --no-input
pipenv run python manage.py collectstatic --no-input
pipenv run gunicorn DepistClic.wsgi:application --bind 0.0.0.0:$PORT

MantisInABox•7mo ago

Okay, all your entrypoint commands look right. can you try adding a mkdir static before your migrate and collectstatic? Just for shits and giggles?

stkr•7mo ago

okey, doing It's nothing to do with the database ? Is it ? Just to make sure that it's not a problem with the imageField link pointing to the file ok the deploy log is happier:

Operations to perform: Apply all migrations: admin, auth, contenttypes, mainapp, sessions Running migrations: No migrations to apply. 130 static files copied to '/app/DepistClic/staticfiles'. [2023-11-08 20:24:16 +0000] [37] [INFO] Starting gunicorn 21.2.0 [2023-11-08 20:24:16 +0000] [37] [INFO] Listening at: http://0.0.0.0:6789 (37) [2023-11-08 20:24:16 +0000] [37] [INFO] Using worker: sync [2023-11-08 20:24:16 +0000] [39] [INFO] Booting worker with pid: 39

MantisInABox•7mo ago

Okay, so it looks like it actually succeeded now

stkr•7mo ago

ok now the missing image link is following: https://depistclic-production.up.railway.app/uploads/images/Grade_podologique.png it seem have changed from media to uploads now I just have to put the files there right ?

MantisInABox•7mo ago

Yes, you would want to upload the images through your admin dashboard, and they should go to the volume now, and the links should work I'll hang around while you give that a try

stkr•7mo ago

yes thanks a lot anyway, for the hour you've spent helping me on that. I really appreciate that

MantisInABox•7mo ago

No problem.

stkr•7mo ago

just a question, is it normal that I cannot find the uploads folder while I'm in the railway shell ?

Brody•7mo ago

railway shell is a local only shell from railway --help:

shell        Open a local subshell with Railway variables available

shell        Open a local subshell with Railway variables available

stkr•7mo ago

ok, and how I could know if it's effectively stored on the monted volume ?

Brody•7mo ago

the occupied size would increase on the volume metrics

stkr•7mo ago

yes it has slightly increased. So I guess the files are there. I've added all files manually through the admin panel, and now redeploying and crossing fingers

MantisInABox•7mo ago

Awesome!

stkr•7mo ago

Is it possible to visualise somewhere server logs on railway, like whith the development server ?

Brody•7mo ago

these are logs? wdym visualize?

stkr•7mo ago

i mean all the requests

Brody•7mo ago

youd need to enable access logging with gunicorn https://docs.gunicorn.org/en/stable/settings.html#logging add --access-logfile - to your start command the dash directs logging to stdout so that it can be picked up by railway

stkr•7mo ago

well, it's not working... I really don't understand The files were uploaded on the admin panel and the volume metrics moved. The link to the file points to the uploads as here: https://depistclic-production.up.railway.app/uploads/ordonnances/Bilan_annuel_4EjwBX7.pdf

stkr•7mo ago

The requested resource was not found on this server. So frustrating... But I don't want to give app Thank you guys both of you this is from the access logging:

192.168.0.3 - - [08/Nov/2023:21:12:49 +0000] "GET /uploads/ordonnances/Bilan_annuel_4EjwBX7.pdf HTTP/1.1" 404 179 "https://depistclic-production.up.railway.app/synthese/";

pandas•7mo ago

Did you try changing

STATIC_URL = 'static/'

STATIC_URL = 'static/'

STATIC_URL = '/static/'

STATIC_URL = '/static/'

It's the path or permission issue always

stkr•7mo ago

I tried with and without first slashes both static and media, and for each case I tried the mount path /app/uploads and /app/DepistClic/uploads. Nothing works, not any media file is accessible through urls of type https://depistclic-production.up.railway.app/uploads/ordonnances/file.pdf. The files are on the mounted volume, I checked it through ls -la in entrypoint.sh. Mystery. Tomorrow I'll try again with nginx I guess, maybe it could serve these files Maybe add os.path.join(BASE_DIR, "uploads") to the STATICFILES_DIRS... I tried also with nginx with this config:

server {
    listen 80;

    server_name depistclic-production.up.railway.app;

    location / {
        proxy_pass http://web:${PORT};
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location = /favicon.ico { access_log off; log_not_found off; }

    location /uploads/ {
        alias /app/DepistClic/uploads;
    }
}

server {
    listen 80;

    server_name depistclic-production.up.railway.app;

    location / {
        proxy_pass http://web:${PORT};
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location = /favicon.ico { access_log off; log_not_found off; }

    location /uploads/ {
        alias /app/DepistClic/uploads;
    }
}

` Stilll not working. Could anyone say how people manage media files in production in Django app ? Thanks

pandas•7mo ago

In production I always resort to boto3 and cdn network Setting up, caching, serving, network cost is not worth it to self manage / host This might help if you stil want to use django wsgi.py

from whitenoise import WhiteNoise
from my_project import MyWSGIApp

application = MyWSGIApp()
application = WhiteNoise(application, root="/path/to/static/files")
application.add_files("/path/to/more/static/files", prefix="more-files/")

from whitenoise import WhiteNoise
from my_project import MyWSGIApp

application = MyWSGIApp()
application = WhiteNoise(application, root="/path/to/static/files")
application.add_files("/path/to/more/static/files", prefix="more-files/")

Gaming

Programming

How to serve media files in Django app on Railway ? Nginx ? Please help