R
Railway15mo ago
Beggars

Node.js 18 and node-gyp compatibility

Howdy folks. I am loving Railway, the interface and overall experience is fantastic. I am trying to deploy an API project which uses NestJS and running into some issues with some Node packages. The first is an issue with a package I am using langchain which throws a warning because it requires Node.js 18 and above. Is there a way I can tell my Railway app to use the latest LTS version of Node? Version 16 is quite old now.
#12 1.911 npm WARN EBADENGINE Unsupported engine {
#12 1.911 npm WARN EBADENGINE package: 'langchain@0.0.81',
#12 1.911 npm WARN EBADENGINE required: { node: '>=18' },
#12 1.911 npm WARN EBADENGINE current: { node: 'v16.18.1', npm: '8.19.2' }
#12 1.911 npm WARN EBADENGINE }
#12 1.911 npm WARN EBADENGINE Unsupported engine {
#12 1.911 npm WARN EBADENGINE package: 'langchain@0.0.81',
#12 1.911 npm WARN EBADENGINE required: { node: '>=18' },
#12 1.911 npm WARN EBADENGINE current: { node: 'v16.18.1', npm: '8.19.2' }
#12 1.911 npm WARN EBADENGINE }
Secondly, the dreaded node-gyp error with another package.
#12 23.32 npm ERR! code 1
#12 23.32 npm ERR! path /app/node_modules/hnswlib-node
#12 23.32 npm ERR! command failed
#12 23.32 npm ERR! command sh -c -- node-gyp rebuild
#12 23.32 npm ERR! gyp info it worked if it ends with ok
#12 23.32 npm ERR! gyp info using node-gyp@8.4.1
#12 23.32 npm ERR! gyp info using node@16.18.1 | linux | x64
#12 23.32 npm ERR! gyp ERR! find Python
#12 23.32 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
#12 23.32 npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
#12 23.32 npm ERR! gyp ERR! find Python checking if "python3" can be used

#12 23.32 npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
#12 23.32 npm ERR! gyp ERR! find Python checking if "python" can be used
#12 23.32 npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
#12 23.32 npm ERR! gyp ERR! find Python
#12 23.32 npm ERR! gyp ERR! find Python **********************************************************
#12 23.32 npm ERR! gyp ERR! find Python You need to install the latest version of Python.
#12 23.32 npm ERR! code 1
#12 23.32 npm ERR! path /app/node_modules/hnswlib-node
#12 23.32 npm ERR! command failed
#12 23.32 npm ERR! command sh -c -- node-gyp rebuild
#12 23.32 npm ERR! gyp info it worked if it ends with ok
#12 23.32 npm ERR! gyp info using node-gyp@8.4.1
#12 23.32 npm ERR! gyp info using node@16.18.1 | linux | x64
#12 23.32 npm ERR! gyp ERR! find Python
#12 23.32 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
#12 23.32 npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
#12 23.32 npm ERR! gyp ERR! find Python checking if "python3" can be used

#12 23.32 npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
#12 23.32 npm ERR! gyp ERR! find Python checking if "python" can be used
#12 23.32 npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
#12 23.32 npm ERR! gyp ERR! find Python
#12 23.32 npm ERR! gyp ERR! find Python **********************************************************
#12 23.32 npm ERR! gyp ERR! find Python You need to install the latest version of Python.
These issues are preventing me from deploying my project. I would hate to have to use another platform if I can't find a solution to these issues. Thank you for your time.
Solution:
```json { "$schema": "https://railway.app/railway.schema.json", "build": { "builder": "NIXPACKS",...
Jump to solution
87 Replies
Percy
Percy15mo ago
Project ID: dc2d2825-bef5-4c42-8d8b-646d7f1898f9
Beggars
Beggars15mo ago
dc2d2825-bef5-4c42-8d8b-646d7f1898f9
Brody
Brody15mo ago
in your package.json set engines.node = 18 as for the gyp issue, send me your railway.json file and i will try to modify it to stop installing gyp
Beggars
Beggars15mo ago
@Brody That was an incredibly fast response. Thank you. I'll see if bumping Node.js version to 18 maybe fixes the issue with the package.
Brody
Brody15mo ago
i doubt it will fix the issue with gyp
Beggars
Beggars15mo ago
Yeah, you were right. Gyp issue still prevalent. I don't seem to have a railway.json file because I created the project through the web interface and then linked it using railway link in my project.
Brody
Brody15mo ago
every deployment gets an auto generated railway.json file, you can copy it from the latest deployment under the details tab
Beggars
Beggars15mo ago
Ah, gotcha. I didn't realise that. This is my railway.json file: { "$schema": "https://railway.app/railway.schema.json", "build": { "builder": "NIXPACKS", "buildCommand": "npm run build" }, "deploy": { "numReplicas": 1, "startCommand": "npm start", "restartPolicyType": "ON_FAILURE", "restartPolicyMaxRetries": 10 } }
Brody
Brody15mo ago
fun fact, you dont need to define those build and start commands yourself if you already have build and start scripts in your package.json file
Beggars
Beggars15mo ago
Ah, cool. Always good to not have to configure things.
Brody
Brody15mo ago
so now you can create a railway.json file in your project folder, and use this
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"install": {
"dependsOn": ["setup"],
"cmds": ["NPM_CONFIG_PRODUCTION= npm ci --omit=dev --omit=optional"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"install": {
"dependsOn": ["setup"],
"cmds": ["NPM_CONFIG_PRODUCTION= npm ci --omit=dev --omit=optional"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
i have not tested this personally, but this is what has worked for another user https://discord.com/channels/713503345364697088/1106436043827335169/1110792220627451994
Beggars
Beggars15mo ago
I thought maybe something good was happening, but alas:
#12 23.97 npm ERR! code 1
#12 23.97 npm ERR! path /app/node_modules/hnswlib-node
#12 23.97 npm ERR! command failed
#12 23.97 npm ERR! command sh -c -- node-gyp rebuild
#12 23.97 npm ERR! gyp info it worked if it ends with ok
#12 23.97 npm ERR! gyp info using node-gyp@8.4.1
#12 23.97 npm ERR! gyp info using node@18.12.1 | linux | x64
#12 23.97 npm ERR! gyp ERR! find Python
#12 23.97 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
#12 23.97 npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
#12 23.97 npm ERR! gyp ERR! find Python checking if "python3" can be used
#12 23.97 npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
#12 23.97 npm ERR! gyp ERR! find Python checking if "python" can be used
#12 23.97 npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
#12 23.97 npm ERR! gyp ERR! find Python
#12 23.97 npm ERR! gyp ERR! find Python **********************************************************
#12 23.97 npm ERR! gyp ERR! find Python You need to install the latest version of Python.
#12 23.97 npm ERR! gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#12 23.97 npm ERR! gyp ERR! find Python you can try one of the following options:
#12 23.97 npm ERR! code 1
#12 23.97 npm ERR! path /app/node_modules/hnswlib-node
#12 23.97 npm ERR! command failed
#12 23.97 npm ERR! command sh -c -- node-gyp rebuild
#12 23.97 npm ERR! gyp info it worked if it ends with ok
#12 23.97 npm ERR! gyp info using node-gyp@8.4.1
#12 23.97 npm ERR! gyp info using node@18.12.1 | linux | x64
#12 23.97 npm ERR! gyp ERR! find Python
#12 23.97 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration
#12 23.97 npm ERR! gyp ERR! find Python Python is not set from environment variable PYTHON
#12 23.97 npm ERR! gyp ERR! find Python checking if "python3" can be used
#12 23.97 npm ERR! gyp ERR! find Python - "python3" is not in PATH or produced an error
#12 23.97 npm ERR! gyp ERR! find Python checking if "python" can be used
#12 23.97 npm ERR! gyp ERR! find Python - "python" is not in PATH or produced an error
#12 23.97 npm ERR! gyp ERR! find Python
#12 23.97 npm ERR! gyp ERR! find Python **********************************************************
#12 23.97 npm ERR! gyp ERR! find Python You need to install the latest version of Python.
#12 23.97 npm ERR! gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#12 23.97 npm ERR! gyp ERR! find Python you can try one of the following options:
Brody
Brody15mo ago
send your package.json?
Beggars
Beggars15mo ago
{
"name": "askaurelia",
"version": "0.0.1",
"description": "",
"author": "",
"private": true,
"license": "UNLICENSED",
"scripts": {
"build": "nest build",
"format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
"start": "nest start",
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"test": "jest",
"test:watch": "jest --watch",
"test:cov": "jest --coverage",
"test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
"test:e2e": "jest --config ./test/jest-e2e.json"
},
"engines": {
"node": "18.x"
},
"dependencies": {
"@nestjs/common": "^9.0.0",
"@nestjs/config": "^2.3.2",
"@nestjs/core": "^9.0.0",
"@nestjs/jwt": "^10.0.3",
"@nestjs/passport": "^9.0.3",
"@nestjs/platform-express": "^9.0.0",
"@nestjs/typeorm": "^9.0.1",
"@octokit/rest": "^19.0.11",
"dotenv": "^16.0.3",
"hnswlib-node": "^1.4.2",
"jwt-simple": "^0.5.6",
"langchain": "^0.0.81",
"passport": "^0.6.0",
"passport-github2": "^0.1.12",
"passport-google-oauth20": "^2.0.0",
"passport-jwt": "^4.0.1",
"pg": "^8.11.0",
"reflect-metadata": "^0.1.13",
"rxjs": "^7.2.0",
"sqlite3": "^5.1.6",
"typeorm": "^0.3.16"
},
"devDependencies": {
"@nestjs/cli": "^9.0.0",
"@nestjs/schematics": "^9.0.0",
"@nestjs/testing": "^9.0.0",
"@types/express": "^4.17.13",
"@types/jest": "29.5.1",
"@types/node": "18.16.12",
"@types/supertest": "^2.0.11",
"@typescript-eslint/eslint-plugin": "^5.0.0",
"@typescript-eslint/parser": "^5.0.0",
"eslint": "^8.0.1",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-prettier": "^4.0.0",
"jest": "29.5.0",
"prettier": "^2.3.2",
"source-map-support": "^0.5.20",
"supertest": "^6.1.3",
"ts-jest": "29.1.0",
"ts-loader": "^9.2.3",
"ts-node": "^10.0.0",
"tsconfig-paths": "4.2.0",
"typescript": "^5.0.0"
},
"jest": {
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".*\\.spec\\.ts$",
"transform": {
"^.+\\.(t|j)s$": "ts-jest"
},
"collectCoverageFrom": [
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
}
}
{
"name": "askaurelia",
"version": "0.0.1",
"description": "",
"author": "",
"private": true,
"license": "UNLICENSED",
"scripts": {
"build": "nest build",
"format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
"start": "nest start",
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"test": "jest",
"test:watch": "jest --watch",
"test:cov": "jest --coverage",
"test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
"test:e2e": "jest --config ./test/jest-e2e.json"
},
"engines": {
"node": "18.x"
},
"dependencies": {
"@nestjs/common": "^9.0.0",
"@nestjs/config": "^2.3.2",
"@nestjs/core": "^9.0.0",
"@nestjs/jwt": "^10.0.3",
"@nestjs/passport": "^9.0.3",
"@nestjs/platform-express": "^9.0.0",
"@nestjs/typeorm": "^9.0.1",
"@octokit/rest": "^19.0.11",
"dotenv": "^16.0.3",
"hnswlib-node": "^1.4.2",
"jwt-simple": "^0.5.6",
"langchain": "^0.0.81",
"passport": "^0.6.0",
"passport-github2": "^0.1.12",
"passport-google-oauth20": "^2.0.0",
"passport-jwt": "^4.0.1",
"pg": "^8.11.0",
"reflect-metadata": "^0.1.13",
"rxjs": "^7.2.0",
"sqlite3": "^5.1.6",
"typeorm": "^0.3.16"
},
"devDependencies": {
"@nestjs/cli": "^9.0.0",
"@nestjs/schematics": "^9.0.0",
"@nestjs/testing": "^9.0.0",
"@types/express": "^4.17.13",
"@types/jest": "29.5.1",
"@types/node": "18.16.12",
"@types/supertest": "^2.0.11",
"@typescript-eslint/eslint-plugin": "^5.0.0",
"@typescript-eslint/parser": "^5.0.0",
"eslint": "^8.0.1",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-prettier": "^4.0.0",
"jest": "29.5.0",
"prettier": "^2.3.2",
"source-map-support": "^0.5.20",
"supertest": "^6.1.3",
"ts-jest": "29.1.0",
"ts-loader": "^9.2.3",
"ts-node": "^10.0.0",
"tsconfig-paths": "4.2.0",
"typescript": "^5.0.0"
},
"jest": {
"moduleFileExtensions": [
"js",
"json",
"ts"
],
"rootDir": "src",
"testRegex": ".*\\.spec\\.ts$",
"transform": {
"^.+\\.(t|j)s$": "ts-jest"
},
"collectCoverageFrom": [
"**/*.(t|j)s"
],
"coverageDirectory": "../coverage",
"testEnvironment": "node"
}
}
Brody
Brody15mo ago
would you happen to know what package is using node-gyp as a dep?
Beggars
Beggars15mo ago
I do actually. It's the hnswlib-node dependency.
Brody
Brody15mo ago
oh so you actually do need python then?
Beggars
Beggars15mo ago
I think it uses Python to build it or something. You've given me an idea to see if there is a way to specify a non-source version of it actually.
Brody
Brody15mo ago
i can also give you a railway.json file that will install python too?
Beggars
Beggars15mo ago
Oh, that would be awesome if that's possible. Installing Python would sort it I think. If this all works out, do you know if there is a way I can contribute back to Railway docs and write up some docs on dealing with Node.js and node-gyp issues?
Brody
Brody15mo ago
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
i honestly cant think of a good place for you to write docs for this, we need a community blog lol
Beggars
Beggars15mo ago
Haha, yes. A community blog or wiki would be great. Okay, that made some progress. Man, node-gyp really is a stain on the Node.js ecosystem, huh? I have no idea what any of this even means, haha.
#12 22.94 npm ERR! gyp http 200 https://nodejs.org/download/release/v18.12.1/SHASUMS256.txt
#12 22.94 npm ERR! gyp info spawn /root/.nix-profile/bin/python3
#12 22.94 npm ERR! gyp info spawn args [
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/node-gyp/gyp/gyp_main.py',
#12 22.94 npm ERR! gyp info spawn args 'binding.gyp',
#12 22.94 npm ERR! gyp info spawn args '-f',
#12 22.94 npm ERR! gyp info spawn args 'make',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/hnswlib-node/build/config.gypi',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/node-gyp/addon.gypi',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/root/.cache/node-gyp/18.12.1/include/node/common.gypi',
#12 22.94 npm ERR! gyp info spawn args '-Dlibrary=shared_library',
#12 22.94 npm ERR! gyp info spawn args '-Dvisibility=default',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/18.12.1',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_gyp_dir=/app/node_modules/node-gyp',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/18.12.1/<(target_arch)/node.lib',
#12 22.94 npm ERR! gyp info spawn args '-Dmodule_root_dir=/app/node_modules/hnswlib-node',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_engine=v8',
#12 22.94 npm ERR! gyp info spawn args '--depth=.',
#12 22.94 npm ERR! gyp info spawn args '--no-parallel',
#12 22.94 npm ERR! gyp info spawn args '--generator-output',
#12 22.94 npm ERR! gyp info spawn args 'build',
#12 22.94 npm ERR! gyp info spawn args '-Goutput_dir=.'
#12 22.94 npm ERR! gyp info spawn args ]
#12 22.94 npm ERR! gyp info spawn make
#12 22.94 npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
#12 22.94 npm ERR! make: g++: No such file or directory
#12 22.94 npm ERR! make: *** [addon.target.mk:109: Release/obj.target/addon/src/addon.o] Error 127
#12 22.94 npm ERR! gyp ERR! build error
#12 22.94 npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess.onExit (/app/node_modules/node-gyp/lib/build.js:194:23)
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28)
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:291:12)
#12 22.94 npm ERR! gyp http 200 https://nodejs.org/download/release/v18.12.1/SHASUMS256.txt
#12 22.94 npm ERR! gyp info spawn /root/.nix-profile/bin/python3
#12 22.94 npm ERR! gyp info spawn args [
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/node-gyp/gyp/gyp_main.py',
#12 22.94 npm ERR! gyp info spawn args 'binding.gyp',
#12 22.94 npm ERR! gyp info spawn args '-f',
#12 22.94 npm ERR! gyp info spawn args 'make',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/hnswlib-node/build/config.gypi',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/app/node_modules/node-gyp/addon.gypi',
#12 22.94 npm ERR! gyp info spawn args '-I',
#12 22.94 npm ERR! gyp info spawn args '/root/.cache/node-gyp/18.12.1/include/node/common.gypi',
#12 22.94 npm ERR! gyp info spawn args '-Dlibrary=shared_library',
#12 22.94 npm ERR! gyp info spawn args '-Dvisibility=default',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/18.12.1',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_gyp_dir=/app/node_modules/node-gyp',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/18.12.1/<(target_arch)/node.lib',
#12 22.94 npm ERR! gyp info spawn args '-Dmodule_root_dir=/app/node_modules/hnswlib-node',
#12 22.94 npm ERR! gyp info spawn args '-Dnode_engine=v8',
#12 22.94 npm ERR! gyp info spawn args '--depth=.',
#12 22.94 npm ERR! gyp info spawn args '--no-parallel',
#12 22.94 npm ERR! gyp info spawn args '--generator-output',
#12 22.94 npm ERR! gyp info spawn args 'build',
#12 22.94 npm ERR! gyp info spawn args '-Goutput_dir=.'
#12 22.94 npm ERR! gyp info spawn args ]
#12 22.94 npm ERR! gyp info spawn make
#12 22.94 npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
#12 22.94 npm ERR! make: g++: No such file or directory
#12 22.94 npm ERR! make: *** [addon.target.mk:109: Release/obj.target/addon/src/addon.o] Error 127
#12 22.94 npm ERR! gyp ERR! build error
#12 22.94 npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess.onExit (/app/node_modules/node-gyp/lib/build.js:194:23)
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28)
#12 22.94 npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:291:12)
Brody
Brody15mo ago
fun
Solution
Brody
Brody15mo ago
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
you know what to do lol
Beggars
Beggars15mo ago
Haha
Brody
Brody15mo ago
let me know how that goes
Beggars
Beggars15mo ago
Oh shit. I think this worked. Brody, you're a genius.
Brody
Brody15mo ago
woohoo
Beggars
Beggars15mo ago
Okay. So, what I am going to do is write up a Markdown document. And then, if you's want to take it and put it somewhere in the docs on the site, might save you a tonne of time in the future, because I know node-gyp is a pain for Node.js users.
Brody
Brody15mo ago
i can put it into the project help section of https://help.railway.app/ at least i can write my own guides there, i will ask angelo if i could put a guide you authored there too
Beggars
Beggars15mo ago
Sounds great. Thank you for not only your fast response but your patience in getting this sorted out as well.
Brody
Brody15mo ago
no problem, happy to help 🙂
Zvezke
Zvezke8mo ago
Just for good measure: I also ran into the same error ('gyp ERR! find Python). This solved it. Thanks.
Zvezke
Zvezke8mo ago
Well, I am back. Initially, adding "python38" worked. Later, I got the same (as mentioned in this thread) the missing "gcc" error. "gcc" added, but now deployments fail due to missing "libvips". I have been trying to add all the suggested libvips (i.e. "dev", "tools") to "nixPkgs", but Nix just seems to not recognise the packages. All help is highly appreciated.
No description
Zvezke
Zvezke8mo ago
This is my railway.json:
No description
Brody
Brody8mo ago
send the json file in text please
Zvezke
Zvezke8mo ago
Of course. Railway:
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
... and if needed; package.json and pnpm-lock.yaml. Package.json:
{
"name": "default",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@formatjs/intl-localematcher": "^0.2.32",
"@headlessui/react": "^1.7.17",
"@heroicons/react": "^2.0.18",
"@hookform/resolvers": "^3.3.2",
"@supabase/ssr": "^0.0.10",
"@supabase/supabase-js": "^2.39.0",
"@tailwindcss/forms": "^0.5.7",
"@tailwindcss/line-clamp": "^0.4.4",
"@tiptap/pm": "^2.1.13",
"@tiptap/react": "^2.1.13",
"@tiptap/starter-kit": "^2.1.13",
"@types/node": "18.14.6",
"@types/react": "18.2.42",
"@types/react-dom": "18.0.11",
"date-fns": "^2.30.0",
"eslint": "8.35.0",
"eslint-config-next": "^14.0.3",
"eventsource": "^2.0.2",
"eventsource-polyfill": "^0.9.6",
"lucide-react": "^0.287.0",
"negotiator": "^0.6.3",
"next": "14.0.3",
"pocketbase": "^0.15.3",
"punycode": "^2.3.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-hook-form": "^7.48.2",
"server-only": "^0.0.1",
"sharp": "^0.32.6",
"typescript": "4.9.5",
"zod": "^3.22.4"
},
"devDependencies": {
"@types/eventsource": "^1.1.15",
"@types/negotiator": "^0.6.3",
"autoprefixer": "^10.4.16",
"postcss": "^8.4.32",
"prettier": "^2.8.8",
"prettier-plugin-tailwindcss": "^0.3.0",
"prisma": "^4.16.2",
"tailwindcss": "^3.3.6"
}
}
{
"name": "default",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@formatjs/intl-localematcher": "^0.2.32",
"@headlessui/react": "^1.7.17",
"@heroicons/react": "^2.0.18",
"@hookform/resolvers": "^3.3.2",
"@supabase/ssr": "^0.0.10",
"@supabase/supabase-js": "^2.39.0",
"@tailwindcss/forms": "^0.5.7",
"@tailwindcss/line-clamp": "^0.4.4",
"@tiptap/pm": "^2.1.13",
"@tiptap/react": "^2.1.13",
"@tiptap/starter-kit": "^2.1.13",
"@types/node": "18.14.6",
"@types/react": "18.2.42",
"@types/react-dom": "18.0.11",
"date-fns": "^2.30.0",
"eslint": "8.35.0",
"eslint-config-next": "^14.0.3",
"eventsource": "^2.0.2",
"eventsource-polyfill": "^0.9.6",
"lucide-react": "^0.287.0",
"negotiator": "^0.6.3",
"next": "14.0.3",
"pocketbase": "^0.15.3",
"punycode": "^2.3.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-hook-form": "^7.48.2",
"server-only": "^0.0.1",
"sharp": "^0.32.6",
"typescript": "4.9.5",
"zod": "^3.22.4"
},
"devDependencies": {
"@types/eventsource": "^1.1.15",
"@types/negotiator": "^0.6.3",
"autoprefixer": "^10.4.16",
"postcss": "^8.4.32",
"prettier": "^2.8.8",
"prettier-plugin-tailwindcss": "^0.3.0",
"prisma": "^4.16.2",
"tailwindcss": "^3.3.6"
}
}
pnpm-lock.yaml:
Zvezke
Zvezke8mo ago




Brody
Brody8mo ago
try adding vips to your nix packages array
Zvezke
Zvezke8mo ago
I think I have tried, but I gave it a go again - failed, unfortunately:
No description
Zvezke
Zvezke8mo ago
My railway.json with vips:
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "vips", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "vips", "gcc", "python38"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
Brody
Brody8mo ago
try this, but if this doesnt work we'd need to move to a custom dockerfile
{
"$schema": "https://schema.up.railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"],
"aptPkgs": ["...", "libvips-dev"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://schema.up.railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"setup": {
"nixPkgs": ["...", "gcc", "python38"],
"aptPkgs": ["...", "libvips-dev"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
Zvezke
Zvezke8mo ago
Thanks, but unfortunately it didn't work. Still the exact same error - "fatal error: vips/vips8: No such file or directory". I am looking forward to look into a custom dockerfile, though I must admit that I don't really have any experience with dockerfiles. I'm fine with Linux, Bash and CLI (apart from web dev) - so let's bring it on. Thanks. 🙂
Brody
Brody8mo ago
if you can share your repo with me it would help me write the dockerfile
Zvezke
Zvezke8mo ago
Sure:
Zvezke
Zvezke8mo ago
GitHub
GitHub - Zvezke/TreKor
Contribute to Zvezke/TreKor development by creating an account on GitHub.
Brody
Brody8mo ago
can you also send me your latest build logs https://bookmarklets.up.railway.app/log-downloader/
Zvezke
Zvezke8mo ago
This should be it.
Brody
Brody8mo ago
thank you, dont know when ill be able to write the dockerfile, railway is having a bit of an issue right now
Zvezke
Zvezke8mo ago
Sure, no stress. Take your time ... we can wait a couple of days if needed. 🙂
Brody
Brody8mo ago
haha it wont be a couple of days
Zvezke
Zvezke8mo ago
Ok, just don't stress. 😉 And hey - thanks a lot. I really appreciate it.
Brody
Brody8mo ago
hey @Zvezke - give this a try, no clue if it will work first try, just copy this into a Dockerfile in your project, push the changes and railway will build with it instead of nixpacks
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL

RUN apk add --no-cache pnpm openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json pnpm-lock.yaml ./

RUN pnpm i --frozen-lockfile

COPY . ./

RUN pnpm run build

CMD pnpm run start
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL

RUN apk add --no-cache pnpm openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json pnpm-lock.yaml ./

RUN pnpm i --frozen-lockfile

COPY . ./

RUN pnpm run build

CMD pnpm run start
Zvezke
Zvezke8mo ago
Thanks! Unfortunately, it errored out immediatly:
No description
Zvezke
Zvezke8mo ago
Should I try with NPM instead, i.e.:
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL

RUN apk add --no-cache npm openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json ./

RUN npm i

COPY . ./

RUN npm run build

CMD npm run start
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL

RUN apk add --no-cache npm openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json ./

RUN npm i

COPY . ./

RUN npm run build

CMD npm run start
Brody
Brody8mo ago
well I assume you'd want to use pnpm, but you wouldn't need to install npm because it comes with the node image
Zvezke
Zvezke8mo ago
Yeah, of course - I don't need to install npm. I'm not sure whether I use pnpm or npm matters with regards to the docker file. Does it matter? I am using pnpm locally - but isn't the docker file only for deployment on Railway?
Brody
Brody8mo ago
it's always good to have the deployment use the same tooling as you use locally
Zvezke
Zvezke8mo ago
Ok, I'll just give it a try with npm - or no?
Brody
Brody8mo ago
sure why not, but remove npm from the apk add line
Zvezke
Zvezke8mo ago
Yeah, I did. 🙂
Brody
Brody8mo ago
keep me updated then!
Zvezke
Zvezke8mo ago
Well, it didn't crash immediately - it started generated the pages. The log doesn't show any fatal errors, but a bunch of error about missing env vars - 'Error: Your project's URL and Key are required to create a Supabase client!'. The docker file hos only 'NEXT_PUBLIC_RAILWAY_URL' whereas four in all:
No description
Zvezke
Zvezke8mo ago
... can I just add the three missing the docker file (ARG)?
Zvezke
Zvezke8mo ago
No description
Brody
Brody8mo ago
yes! i didnt know what other variables your build would need so i only used what i found in the .env.development file
Zvezke
Zvezke8mo ago
Oh, of course. How should I do it syntactically? All in one line?
No description
Zvezke
Zvezke8mo ago
... or a new line for each: ARG ... ARG ...
Brody
Brody8mo ago
i'm pretty sure either works, so up to you whatever you think looks best
Zvezke
Zvezke8mo ago
😄
Brody
Brody8mo ago
yes, just checked, either way works the same
Zvezke
Zvezke8mo ago
Hey! It's alive! 🙂 Thanks a billion - I'll buy you a beer next time you're in Copenhagen, Denmark - Carlsberg, of course.
Brody
Brody8mo ago
so do you want to get pnpm working on the dockerfile?
Zvezke
Zvezke8mo ago
If possible, yeah. I assume using the same pkg-manager is less prone to bugs.
Brody
Brody8mo ago
can you send me your current dockerfile?
Zvezke
Zvezke8mo ago
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_POCKETBASE_URL NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_SUPABASE_ANON_KEY NEXT_PUBLIC_SUPABASE_URL

RUN apk add --no-cache openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json ./

RUN npm i

COPY . ./

RUN npm run build

CMD npm run start
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_POCKETBASE_URL NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_SUPABASE_ANON_KEY NEXT_PUBLIC_SUPABASE_URL

RUN apk add --no-cache openssl build-base python3 vips-dev

WORKDIR /app

COPY package.json ./

RUN npm i

COPY . ./

RUN npm run build

CMD npm run start
Brody
Brody8mo ago
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_POCKETBASE_URL NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_SUPABASE_ANON_KEY NEXT_PUBLIC_SUPABASE_URL

RUN apk add --no-cache openssl build-base python3 vips-dev && \
npm install -g pnpm

WORKDIR /app

COPY package.json pnpm-lock.yaml ./

RUN pnpm i --frozen-lockfile

COPY . ./

RUN pnpm run build

CMD pnpm run start
FROM node:18.19.0-alpine

ARG NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_POCKETBASE_URL NEXT_PUBLIC_RAILWAY_URL NEXT_PUBLIC_SUPABASE_ANON_KEY NEXT_PUBLIC_SUPABASE_URL

RUN apk add --no-cache openssl build-base python3 vips-dev && \
npm install -g pnpm

WORKDIR /app

COPY package.json pnpm-lock.yaml ./

RUN pnpm i --frozen-lockfile

COPY . ./

RUN pnpm run build

CMD pnpm run start
give it a try
Zvezke
Zvezke8mo ago
Thanks. I'll be back i five. It works - thanks. I really appreciate it. I'll buy you two beers next time you're in Copenhagen - two Carlsbergs, of course. And I'll introduce you to Scandinavian women as well.
Brody
Brody8mo ago
lmfao thanks
Zvezke
Zvezke8mo ago
... so actually - running 'npm install -g pnpm' separately did it (compared to you first suggestion). Btw, do you know if this is a bug in Railway?
Brody
Brody8mo ago
its not, can you tell me why it didnt work? error message you got?
Zvezke
Zvezke8mo ago
Initially, I got "gyp ERR! find Python" - adding "python38" to railway.json worked. Then, I got the same (as mentioned in this thread) the missing "gcc" error. "gcc" added, but deployments started failing due to missing "libvips". Adding libvips (in all different variations) didn't work - pkg not recognized. Then, I asked for help, and here we are ...
Brody
Brody8mo ago
not too sure what any of that has to do with the dockerfile we are now using?
Zvezke
Zvezke8mo ago
Well, adding packages (libvips in different variations) to nixPkgs didn't help - finally, you suggested a docker file.
Brody
Brody8mo ago
you have significantly confused me does this dockerfile, as is, not work? or in other words, did you have to modify it, for it to work?
Zvezke
Zvezke8mo ago
The dockerfile works - all is fine again. Initially, you suggested 'RUN apk ... pnpm ... ', but that didn't work. The file works now. I am just wondering what is causing a normal deployment to fail. It's just my curiosity - don't worry about it. Thanks, much appreciated - I'll see you in the threads. And I'll get back if things fail again. 🙂
Brody
Brody8mo ago
ohhh you asked why apk add couldn’t install pnpm. apk is alpines package manager, like debian's apt, but better. pnpm is a package in alpines package manager, but its only in the testing repository and apk doesn’t pull from that by default so it couldn’t install it, i overlooked that. instead of switching what repository apk pulls the packages from, i opted to just have npm install pnpm.