Railway•13mo ago

Dockerfile caching?

Hello, I have done some modifications on my dockerfile but seems it is catching the changes and not "reading" the new values, so It's not adding them to the built image, any ideas how to avoid catching or how to force to clear cache? Thanks!

54 Replies

Percy•13mo ago

Project ID: 40e44f3c-0508-45c6-b255-a4558921d6f5

ENT3I <3•13mo ago

40e44f3c-0508-45c6-b255-a4558921d6f5

Brody•13mo ago

the builder should detect there is a change and invalidate the cache, your problem is likely something else

ENT3I <3•13mo ago

Thanks @Brody for the response, let me explain what I'm trying to do. I have an n8n instance running on queue mode. I need to install some npm dependencies that I would like to use and those need to be installed on my /usr/local/lib/node_modules/n8n I added this line to my Dockerfile: RUN /tmp/install_modules.sh This is my script (install_modules.sh):

#!/bin/sh

# Set the custom modules you want to install
CUSTOM_MODULES="axios,cheerio,request,aws-sdk,mime"

# Check if any custom modules are defined
if [ -z "$CUSTOM_MODULES" ]; then
    echo "No custom modules specified."
    exit 0
fi

# Split the modules into an array
MODULES=$(echo $CUSTOM_MODULES | tr "," "\n")

# Install each module
for module in $MODULES; do
    npm install $module
done

echo "Custom modules installed."

#!/bin/sh

# Set the custom modules you want to install
CUSTOM_MODULES="axios,cheerio,request,aws-sdk,mime"

# Check if any custom modules are defined
if [ -z "$CUSTOM_MODULES" ]; then
    echo "No custom modules specified."
    exit 0
fi

# Split the modules into an array
MODULES=$(echo $CUSTOM_MODULES | tr "," "\n")

# Install each module
for module in $MODULES; do
    npm install $module
done

echo "Custom modules installed."

Brody•13mo ago

(enclose the code in triple backticks)

ENT3I <3•13mo ago

This is the deployment log:

#9 naming to us-west1.registry.rlwy.net/5c6bd6c0-8985-4062-bffd-1c9a8482c9f6:d8cf2b99-1dcc-4a61-befd-66cc1e2031a5 done

#9 DONE 0.2s
 

Build time: 31.57 seconds

 

================

Publishing Image

================


The push refers to repository [us-west1.registry.rlwy.net/5c6bd6c0-8985-4062-bffd-1c9a8482c9f6]

Preparing  d05c6fa6a783

Preparing  0d53fcfbcbb3

Preparing  5f70bf18a086

Preparing  a5105cb83e40

Preparing  41553384751d

Waiting  d05c6fa6a783

Preparing  7fe47c2d4fc2

Waiting  0d53fcfbcbb3

Preparing  a9aa4cfc7b2d

Waiting  cc2447e1835a

Pushing [>                                                  ]  196.6kB/17.64MB d05c6fa6a783

Pushing [======>                                            ]   2.37MB/17.64MB d05c6fa6a783

Pushing [================>                                  ]  5.849MB/17.64MB d05c6fa6a783

Mounted from 350d60ed-b27b-4b8c-ba7a-d4785d364f5d  0d53fcfbcbb3

Pushing [==============================>                    ]  10.75MB/17.64MB d05c6fa6a783

Pushing [============================================>      ]  15.87MB/17.64MB d05c6fa6a783

Layer already exists  5f70bf18a086

Pushing [==================================================>]   18.2MB d05c6fa6a783

Layer already exists  a5105cb83e40

Layer already exists  41553384751d

Layer already exists  7fe47c2d4fc2

Layer already exists  ba0368c069bc

Layer already exists  6f61d0e9b9bb

Pushed  d05c6fa6a783

Layer already exists  a9aa4cfc7b2d

Layer already exists  25156ca953a7

Layer already exists  0d5e1f8c333d

Layer already exists  cc2447e1835a

d8cf2b99-1dcc-4a61-befd-66cc1e2031a5: digest: sha256:c244cdf62902ea584f2cf8062a6061b0587c56781a37d01de179e2f966f780d6 size: 3240

 

Publish time: 4.29 seconds

 

====================

Starting Healthcheck

====================


Path: /healthz

Retry window: 5m0s

 

Attempt #1 failed with service unavailable. Continuing to retry for 4m59s

[1/1] Healthcheck succeeded!

#9 naming to us-west1.registry.rlwy.net/5c6bd6c0-8985-4062-bffd-1c9a8482c9f6:d8cf2b99-1dcc-4a61-befd-66cc1e2031a5 done

#9 DONE 0.2s
 

Build time: 31.57 seconds

 

================

Publishing Image

================


The push refers to repository [us-west1.registry.rlwy.net/5c6bd6c0-8985-4062-bffd-1c9a8482c9f6]

Preparing  d05c6fa6a783

Preparing  0d53fcfbcbb3

Preparing  5f70bf18a086

Preparing  a5105cb83e40

Preparing  41553384751d

Waiting  d05c6fa6a783

Preparing  7fe47c2d4fc2

Waiting  0d53fcfbcbb3

Preparing  a9aa4cfc7b2d

Waiting  cc2447e1835a

Pushing [>                                                  ]  196.6kB/17.64MB d05c6fa6a783

Pushing [======>                                            ]   2.37MB/17.64MB d05c6fa6a783

Pushing [================>                                  ]  5.849MB/17.64MB d05c6fa6a783

Mounted from 350d60ed-b27b-4b8c-ba7a-d4785d364f5d  0d53fcfbcbb3

Pushing [==============================>                    ]  10.75MB/17.64MB d05c6fa6a783

Pushing [============================================>      ]  15.87MB/17.64MB d05c6fa6a783

Layer already exists  5f70bf18a086

Pushing [==================================================>]   18.2MB d05c6fa6a783

Layer already exists  a5105cb83e40

Layer already exists  41553384751d

Layer already exists  7fe47c2d4fc2

Layer already exists  ba0368c069bc

Layer already exists  6f61d0e9b9bb

Pushed  d05c6fa6a783

Layer already exists  a9aa4cfc7b2d

Layer already exists  25156ca953a7

Layer already exists  0d5e1f8c333d

Layer already exists  cc2447e1835a

d8cf2b99-1dcc-4a61-befd-66cc1e2031a5: digest: sha256:c244cdf62902ea584f2cf8062a6061b0587c56781a37d01de179e2f966f780d6 size: 3240

 

Publish time: 4.29 seconds

 

====================

Starting Healthcheck

====================


Path: /healthz

Retry window: 5m0s

 

Attempt #1 failed with service unavailable. Continuing to retry for 4m59s

[1/1] Healthcheck succeeded!

Brody•13mo ago

please

ENT3I <3•13mo ago

how?

Brody•13mo ago

you are currently doing one backtick, do 3 instead

ENT3I <3•13mo ago

done

Brody•13mo ago

does n8n not have a native way of doing this?

ENT3I <3•13mo ago

nope... I wish it had one but for external modules (npm) you need to install them manually

Brody•13mo ago

send the dockerfile

ENT3I <3•13mo ago

Dockerfile.txt

install_modules.sh

Brody•13mo ago

do you see the Custom modules installed. message in the build logs

ENT3I <3•13mo ago

yes

ENT3I <3•13mo ago

but also I see errors so probably modules not being installed

Brody•13mo ago

looks like the modules you are installing conflict with the already installed modules, you might have to specify version of the modules that are compatible

ENT3I <3•13mo ago

let me try only with aws-sdk which I'm pretty sure should not be in the list @Brody is railway down? I can't see variables

ENT3I <3•13mo ago

Brody•13mo ago

yes they are lol

ENT3I <3•13mo ago

and everytime I refresh page looks like this

Brody•13mo ago

yeah just try again in 15 mins

ENT3I <3•13mo ago

Ok, I have to afk for 45-60 minutes then I will try again. Thank you for your help. I hope I can solve this soon I'm not doing anything overcomplex, just I want to install npm libraries in a specific folder

Brody•13mo ago

sounds good, ill be around

ENT3I <3•13mo ago

I'm trying to install one by one and see #5 [2/2] RUN npm install aws-sdk #5 2.496 npm WARN deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.

Brody•13mo ago

well that is just a warning

ENT3I <3•13mo ago

yeah I know just wondering that won't cause any future issues, I guess should not be the case

Brody•13mo ago

it shouldn't

ENT3I <3•13mo ago

still having same issues of this image:

ENT3I <3•13mo ago

dunno why... looks like something is not compatible with these packages

Brody•13mo ago

that image has like 3 pixels total, I can't read a single word

ENT3I <3•13mo ago

Brody•13mo ago

still have dependency issues how do other people install extra node modules?

ENT3I <3•13mo ago

Omg I fixed it what a dumb this made the difference

ENT3I <3•13mo ago

the command USER root

Brody•13mo ago

interesting, I didn't see anything about permission errors

ENT3I <3•13mo ago

yep... but now it works do you know if I can pass a variable content to dockerfile?

Brody•13mo ago

you mean for the ARGs

ENT3I <3•13mo ago

yeah instead of hard-texting the modules there I want to use variable if possible

Brody•13mo ago

all service variables are available via the ARGS

ENT3I <3•13mo ago

so in this case my variable name is CUSTOM_MODULES should i use $CUSTOM_MODULE ? RUN npm install -g $CUSTOM_MODULE

Brody•13mo ago

read this please https://docs.railway.app/deploy/dockerfiles#environment-variables

ENT3I <3•13mo ago

Thank you!

ENT3I <3•13mo ago

I think I got it:

Brody•13mo ago

fun fact, you dont need most of the ARG lines, and all of those ENV lines, as railway will automatically inject the variables to the environment when it runs the built image you only use CUSTOM_MODULES during the build, so youd only need to do ARG CUSTOM_MODULES

ENT3I <3•13mo ago

oh really? Interesting

Brody•13mo ago

im like 98% sure thats how it works

ENT3I <3•13mo ago

Thank you so much mate for your help. I really appreciate the time you spend here with me. Thank you! I have started using Railway a few days ago. I'm actually running an instance on AWS, but it's very time-consuming each time I need to make changes / deploy / upgrade, etc. I'm super happy to have discovered this fantastic tool and super sad I didn't discover it before, LOL

Brody•13mo ago

love to here it!! 🙂

ENT3I <3•13mo ago

https://tenor.com/view/thank-you-thanks-hey-bro-fist-bumb-gif-16262296206650437916

Tenor

Gaming

Programming

Dockerfile caching?