N8N Migration
Hi Railway team.
I know this isn't directly related to your product, but I would appreciate any guidance. I installed N8N from the template many moons ago. I have had problems where on server restart, my encrypted connection details failed to save and I had to replace every credential in there and reconnect every node.
I have a new instance of this software running on a different project with an updated template and the default encryption key is set as an environment variable on that one.
My question is, is there a decent way I can go about migrating this without taking down my services?
My second question is, I am randomly not able to log into it today using my saved credentials, and I'm not sure what's up... Does anyone know if I can reset my password from the database on this software, as I never set email up with it?
I know this is not Railway specific, but any help and guidance on this one would be appreciated...
20 Replies
Project ID:
e1f04c05-5bea-4d11-b787-1f1e98108f2be1f04c05-5bea-4d11-b787-1f1e98108f2b
Would changing this password here screw anything up?
wdym not railway specific, you're asking a question about a service you host on railway, seems railway specific enough to me
Well, It's just that I know (from my own company), that I hate supporting complimentary software to my business that is outside of our actual company bounderies lol
but regardless, the template creator @jtmaveryk would be far better suited to answer these questions
you're good don't worry
Thank you, love this team + community đź‘Ť
I've tagged jack, let's just wait until he's available!
I dunno anything about n8n, but is it possible for you to export data from old instance and import that to the new?
There is an export-import feature in it I think (although this was still partially under construciton in the version I have installed).
I will likely need to do this one workflow at a time for a successful migration, but I still have the issue with the password, as that has to be done from the dashboard I think, and I'm also not sure if that it is the best way or not.
However, yes I do think this is a very grusome but possible option in the software, as long as it is meticulously done (I have nearly 300 workflows, and they all call each other based on the workflow id number)
I do need to find a safe way to reset my password without screwing anything up as a top priority for this at the least.
Mmm if you set up email now, would you be able to reset through that? It should be possible if they’re configurable through env vars
Is there a way for me to change the environment variables without restarting the application? I imagine that wouldn't be possible... The scariest part of this is that all of my service's credentials lost their encryption (and the nodes stopped functioning correctly) when I tried to update this server a few weeks back, and our services went offline for hours while I frantically tried to add the credentials back at a top golf lol
I was somewhat under the impression when I set this server up originally that this would work more like a Wordpress install where that kind of thing is just "taken care of", I now am aware that I was wrong...
Uhhhh, that’s not good… anything is rarely managed to the extent of WordPress installations if you self-host it.
Changing env vars will trigger a redeploy/restart, there’s no way around that & n8n needs to restart to pick up those variables anyway. If I were you, if your installation is so brittle that you’re unsure if data will survive a restart, I’d focus on exporting data right now.
Yeah I agree that it's overly brittle (hence why I am trying to move it to a different install).
I see that there is a CLI tool for exporting the credentials/workflows. (https://docs.n8n.io/hosting/cli-commands/#workflows)
Can I setup and use the railway CLI to interact with this without restarting the server?
Can I setup and use the railway CLI to interact with this without restarting the server?
CLI commands | n8n Docs
CLI commands available in n8n.
I've figured out how to solve this issue (partially) with the n8n CLI, thank you folks. You can mark this ticket complete. I will create a new ticket if I need it.
Apologies, I need to reopen this (I don't think it's been closed yet). So I'm trying to use the Railway CLI to make a backup of my N8N data. However, when I type in a command such as n8n inot the shell set to the service,
I get a response back:
zsh: command not found: n8n.
Running this command:
railway run n8n export:workflow --output=workflows-aug-10.json
I get response:
No such file or directory (os error 2)
How can I run that command and store the output in my local folder where I run the command? Is it in a docker container or something?
there's no way to run commands on the container, railway run and railway shell only run commands locally with the service variables available
Hm... So there's no way for me to actually execute that command to create a backup of my credentials and workflows?
Hey! Changing your password there shouldn't screw anything up as thats for the deprecated and outdated basic owner account.
As to migration, all your workflows should be stored within the database as your deployment is fresh and doesn't retain data upon redeployment, so you should be fine to just swap the GitHub repo it pulls images from to
jtmaveryk/railway-n8n I believe. Before that though, just write down or regenerate your credentials as those won't save from your old deployment until you add the ENCRYPTION_KEY variable. In the migration, I'd create that variable with whatever you want to set it to (just keep in mind changing it will result in loss of credentails, you'll have to reinput), and you should be all set. Assuming you're not on that old of a version, you should be fine to remove the basic user authentication variables seeing you dont have a variable telling n8n to use it, so you should be good to remove your USERNAME and PASSWORD variables.Hi Jack, thank you for your help. I need a couple clarifications.
My goal is a zero -downtime migration from one n8n instance to another that already has the encryption key I want to use setup. I have two major issues at the moment: - I don't know how, but for some reason when I moved my account from my personal to the new team plan, I seem to have lost the ability to log into this n8n instance (My login details are saved in 1pass, so they didn't change, was kind of confused by this when I tried to log in today). It might be a coincidence. As far as I'm aware, this instance is still properly responding to my webhooks, which is good because I can't let all of these N8N workflows disappear, they represent important business logic for us. If I had access to the workflows, I could make the switch over by migrating the workflows one by one, and I am willing to spend some time to do this. I never setup an email server on this deployment, so I can't use the forgot password function. - I can not seem to restart this server (or change the environment variables) without loosing the in-memory encryption key, which is really bad news for me as far as I can tell, because it takes all of my webhooks managed through this server down. Here are some things I am thinking might be a solution: - If I download a set of workflows from this DB, can I, in theory, deploy it to a new n8n instance just by altering the DB? If so, this would be a good start I think. All of these workflows will have invalid credentials, but I can just create new ones using the new deployment. - If there is a way to use the CLI? That also seems to have a very easy native backup/restore solution. Thank you so much for your help on this issue. I feel very dumb for putting myself in this position. I should also clarify, I do not need the credentials themselves, as I have these already stored in a 1password instance, I just don't want to take the services down while I convert 300 nodes to use the new credentials, so I guess what I really need is just the ability to move the workflow definitions from one deployment to the next. Is there maybe a way for me to see the .n8n config file from the railway CLI? So I can pull the encryption key off of it and use that as the environment variable on restart? Or can someone from railway team share this file securely with me? I found a way to get my n8n generated encryption key without dashboard access and without file access, if anyone searches this in the future and is in my situation, here is how I got it... (I am NOT recommending this, but it worked for me) I was able to accomplish this by Creating a workflow with the API, reading the config file from within an N8N workflow, set that key data to a new variable using a set node, and then lastly send that response back to an API endpoint on a different n8n deployment (Make sure to delete this execution afterwards from both N8N servers so it is not stored in plain text). The file location for the N8N server is at: /home/node/.n8n/config . You need this for setting up the workflow. You also probably should have another n8n instance setup. Warning, this is not an easy process... Firstly: You need to have previously created and have your API key saved somewhere. Secondly: You can use your API key to create a new workflow. You might want to practice this creation via api a few times in a different n8n deployment to test your created flow. (I created this workflow first in the other n8n instance so I knew for the most part what the JSON would look like) Thirdly: Use the API to create a workflow that looks like this: Webhook listener -> read binary file -> set data -> call webhook (to your second N8N server). Make sure you have executions logging on the second n8n server. Fourth: Make a call to that endpoint and wait for it to call back with the encryption key 🎉. You can now decrypt anything that's in the postgres database at the very least. What I am scared to do is set this encryption key environment variable, as I am not sure if that will mess up the data on the server in any way.
My goal is a zero -downtime migration from one n8n instance to another that already has the encryption key I want to use setup. I have two major issues at the moment: - I don't know how, but for some reason when I moved my account from my personal to the new team plan, I seem to have lost the ability to log into this n8n instance (My login details are saved in 1pass, so they didn't change, was kind of confused by this when I tried to log in today). It might be a coincidence. As far as I'm aware, this instance is still properly responding to my webhooks, which is good because I can't let all of these N8N workflows disappear, they represent important business logic for us. If I had access to the workflows, I could make the switch over by migrating the workflows one by one, and I am willing to spend some time to do this. I never setup an email server on this deployment, so I can't use the forgot password function. - I can not seem to restart this server (or change the environment variables) without loosing the in-memory encryption key, which is really bad news for me as far as I can tell, because it takes all of my webhooks managed through this server down. Here are some things I am thinking might be a solution: - If I download a set of workflows from this DB, can I, in theory, deploy it to a new n8n instance just by altering the DB? If so, this would be a good start I think. All of these workflows will have invalid credentials, but I can just create new ones using the new deployment. - If there is a way to use the CLI? That also seems to have a very easy native backup/restore solution. Thank you so much for your help on this issue. I feel very dumb for putting myself in this position. I should also clarify, I do not need the credentials themselves, as I have these already stored in a 1password instance, I just don't want to take the services down while I convert 300 nodes to use the new credentials, so I guess what I really need is just the ability to move the workflow definitions from one deployment to the next. Is there maybe a way for me to see the .n8n config file from the railway CLI? So I can pull the encryption key off of it and use that as the environment variable on restart? Or can someone from railway team share this file securely with me? I found a way to get my n8n generated encryption key without dashboard access and without file access, if anyone searches this in the future and is in my situation, here is how I got it... (I am NOT recommending this, but it worked for me) I was able to accomplish this by Creating a workflow with the API, reading the config file from within an N8N workflow, set that key data to a new variable using a set node, and then lastly send that response back to an API endpoint on a different n8n deployment (Make sure to delete this execution afterwards from both N8N servers so it is not stored in plain text). The file location for the N8N server is at: /home/node/.n8n/config . You need this for setting up the workflow. You also probably should have another n8n instance setup. Warning, this is not an easy process... Firstly: You need to have previously created and have your API key saved somewhere. Secondly: You can use your API key to create a new workflow. You might want to practice this creation via api a few times in a different n8n deployment to test your created flow. (I created this workflow first in the other n8n instance so I knew for the most part what the JSON would look like) Thirdly: Use the API to create a workflow that looks like this: Webhook listener -> read binary file -> set data -> call webhook (to your second N8N server). Make sure you have executions logging on the second n8n server. Fourth: Make a call to that endpoint and wait for it to call back with the encryption key 🎉. You can now decrypt anything that's in the postgres database at the very least. What I am scared to do is set this encryption key environment variable, as I am not sure if that will mess up the data on the server in any way.
The only data it should mess up if incorrectly configured is possible user credentials and workflow credentials. If I were you, I’d deploy a replica n8n environment and test it and see if it persists or issues arise; I would for you but I’m at work currently.