R
Railway15mo ago
Grey

Node build failing

Created a project today - I initially had a Dockerfile then removed it to try and use the default build. Project is KoaJS on Node 14. The build errors with:
#11 19.44 gyp ERR! find Python (accepted by both node-gyp and npm)
#11 19.44 gyp ERR! find Python - Set the environment variable PYTHON
#11 19.44 gyp ERR! find Python - Set the npm configuration variable python:
#11 19.44 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
#11 19.44 gyp ERR! find Python For more information consult the documentation at:
#11 19.44 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#11 19.44 gyp ERR! find Python **********************************************************
#11 19.44 gyp ERR! find Python
#11 19.44 gyp ERR! configure error
#11 19.45 gyp ERR! stack Error: Could not find any Python installation to use
#11 19.45 gyp ERR! stack at PythonFinder.fail (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#11 19.45 gyp ERR! stack at PythonFinder.runChecks (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#11 19.45 gyp ERR! stack at PythonFinder.<anonymous> (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#11 19.45 gyp ERR! stack at PythonFinder.execFileCallback (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#11 19.45 gyp ERR! stack at exithandler (child_process.js:390:5)
#11 19.44 gyp ERR! find Python (accepted by both node-gyp and npm)
#11 19.44 gyp ERR! find Python - Set the environment variable PYTHON
#11 19.44 gyp ERR! find Python - Set the npm configuration variable python:
#11 19.44 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
#11 19.44 gyp ERR! find Python For more information consult the documentation at:
#11 19.44 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#11 19.44 gyp ERR! find Python **********************************************************
#11 19.44 gyp ERR! find Python
#11 19.44 gyp ERR! configure error
#11 19.45 gyp ERR! stack Error: Could not find any Python installation to use
#11 19.45 gyp ERR! stack at PythonFinder.fail (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#11 19.45 gyp ERR! stack at PythonFinder.runChecks (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#11 19.45 gyp ERR! stack at PythonFinder.<anonymous> (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#11 19.45 gyp ERR! stack at PythonFinder.execFileCallback (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#11 19.45 gyp ERR! stack at exithandler (child_process.js:390:5)
Unsure if this is a Nixpacks issue or something else?
46 Replies
Percy
Percy15mo ago
Project ID: 18c9abfc-def5-4aa2-8518-6b3d9567c97c
Grey
Grey15mo ago
18c9abfc-def5-4aa2-8518-6b3d9567c97c nixpacks info:
==============
Using Nixpacks
==============
context: 6e6ea8f73c1638910f6b4660b31a5cea

╔═════════ Nixpacks v1.5.1 ═════════╗
║ setup │ nodejs-14_x, npm-6_x ║
║───────────────────────────────────║
║ install │ npm ci ║
║───────────────────────────────────║
║ start │ npm run start ║
╚═══════════════════════════════════╝
==============
Using Nixpacks
==============
context: 6e6ea8f73c1638910f6b4660b31a5cea

╔═════════ Nixpacks v1.5.1 ═════════╗
║ setup │ nodejs-14_x, npm-6_x ║
║───────────────────────────────────║
║ install │ npm ci ║
║───────────────────────────────────║
║ start │ npm run start ║
╚═══════════════════════════════════╝
Brody
Brody15mo ago
did the dockerfile you had work?
Grey
Grey15mo ago
no, it was something I set up before and relied on old conventions it built but wasn't meant for this usecase
Brody
Brody15mo ago
okay fair, so your node project needs python installed is what I get from the logs?
Grey
Grey15mo ago
it seems like that's because node-gyp needs it, but node-gyp is only imported as a devDependency for the packages I've installed
Brody
Brody15mo ago
ah i see, can you send me your railway.json file from your most recent deployment? this can be found in the details tab
Grey
Grey15mo ago
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS"
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS"
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
Brody
Brody15mo ago
but also "When your NODE_ENV environment variable is set to production, using npm ci will not install devDependencies" railway sets that to production, are you perhaps overwriting it somehow
Grey
Grey15mo ago
yep that's a good point just checked env vars in railway and it's not being overridden there not sure how else it would be getting changed since this is the build stage
Brody
Brody15mo ago
strange, try this though
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"install": {
"dependsOn": ["setup"],
"cmds": ["npm ci --omit=dev"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"install": {
"dependsOn": ["setup"],
"cmds": ["npm ci --omit=dev"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}
i think ci will accept the omit flag, but im not too sure, im not a js dev but that at least should give you an idea on how to overwrite the install command and you can play around with that now
Grey
Grey15mo ago
ahh found the issue maybe https://github.com/nodejs/node-gyp/issues/2110#issuecomment-622326221
fsevents is macOS only. It specifies an os field in its package.json but this is apparently ignored by npm ci. fsevents used to provide prebuilt binaries up to fsevents@1.2.9 which meant for those versions it didn't attempt to compile
bumping fsevents to >= v1.2.13 should fix the issue
Brody
Brody15mo ago
let me know how that goes, ive seen this issue asked here before, where ci would install a dev dep that needed python
Grey
Grey15mo ago
Updating that skipped the fsevents package, but another devDependency was still being installed that required node-gyp under the hood. Spent a bunch of time trying things locally and saw that npm ci would try to install these packages unless either NODE_ENV=production was set, or the --production flag was passed in updated the command in railway.json to npm ci --production and it built fine so it must be that NODE_ENV=production isn't set at build time?
Brody
Brody15mo ago
--omit=dev didn't work? yeah must not be for some reason
Grey
Grey15mo ago
nah, wasn't recognized maybe it's bc some people want to do dev steps when they build
Brody
Brody15mo ago
interesting, well I will keep that in mind and recommend the --production flag thank you for this information
Grey
Grey15mo ago
np! thanks for the assist
Brody
Brody15mo ago
no prob 🙂
Grey
Grey15mo ago
I'm back 😄 Same problem, different node version updated to node 18 and am starting to see the same issue again #11 0.820 npm WARN config production Use --omit=dev instead. ^ I started to see this appear before the error about gdal/fsevents appearing the command I use is now npm ci --omit=dev --omit=optional in railway.json I've tried a bunch of different things but am still getting config production Use --omit=dev instead. errors separately, I moved the offending packages to optional and it still tries to install them (although fsevents does get skipped this way) "overrides": { "gdal": "^0.11.0" }, "optionalDependencies": { "fsevents": "^1.2.13", "gdal": "^0.11.0" }
Brody
Brody15mo ago
can I see your railway.json file?
Grey
Grey15mo ago
sure thing @Brody !
Brody
Brody15mo ago
and have you tried npm ci --production ?
Grey
Grey15mo ago
Yep I tried that too, same warning build failed too, can't remember which error but I think fsevents was causing it
Brody
Brody15mo ago
fsevents sure is causing a lot of troubles, how about just removing it all together? last release was Feb 2021, surely there's an equivalent more up to date package without these issues?
Grey
Grey15mo ago
Yeah I can start to go that route. It's required by something else further down the pipe, but should be able to figure it out I was able to resolve the fs events issue, but the gval package then became the issue. It really still feels like it's trying to install Dev dependencies, that's the part that I can't get around and feels like I need to solve first @Brody confirmed that it's still trying to install stuff like chokidar, which is a devdependency tried using npm install --production and npm install --omit=dev too
Grey
Grey15mo ago
https://stackoverflow.com/questions/72776325/npm-warning-config-production-use-omit-dev-instead this mentioned to change the .npmrc file to get rid of that warning
Stack Overflow
npm Warning: config production; Use --omit=dev Instead
Every time I run any kind of npm command, I get the following error before the actual output: npm WARN config production Use --omit=dev instead. Of course I've come across similar issues regardi...
Grey
Grey15mo ago
do you know how I could debug this further?
Brody
Brody15mo ago
check the NODE_ENV variable right before install with echo $NODE_ENV && npm install --omit=dev
Grey
Grey15mo ago
ooh I think I got something
Grey
Grey15mo ago
Node | Nixpacks
App source + Nix packages + Docker = Image
Grey
Grey15mo ago
The Node provider sets the following environment variables: CI=true NODE_ENV=production NPM_CONFIG_PRODUCTION=false: Ensure that dev deps are always installed
Brody
Brody15mo ago
now why tf would that be forced
Grey
Grey15mo ago
😄
Brody
Brody15mo ago
well try setting NPM_CONFIG_PRODUCTION = true
Grey
Grey15mo ago
tried to set at the service level didn't work. going to echo out a few variables to confirm what is sticking
Brody
Brody15mo ago
try setting it before the install command or i should say, set it inline with the install command
Grey
Grey15mo ago
ok, I think that setting it at the service level actually did stop it from trying dev deps because logs don't show a warning about chokidar anymore
Brody
Brody15mo ago
progress!
Grey
Grey15mo ago
yep! still getting stuff like
#11 8.744 npm ERR! code 1
#11 8.744 npm ERR! path /app/node_modules/gdal
#11 8.746 npm ERR! command failed
#11 8.746 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#11 8.746 npm ERR! Failed to execute '/nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/bin/node /nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/gdal/lib/binding/gdal.node --module_name=gdal --module_path=/app/node_modules/gdal/lib/binding --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
#11 8.747 npm ERR! node-pre-gyp info it worked if it ends with ok

#11 8.747 npm ERR! node-pre-gyp info using node-pre-gyp@1.0.10
#11 8.747 npm ERR! node-pre-gyp info using node@18.12.1 | linux | x64
#11 8.747 npm ERR! node-pre-gyp info check checked for "/app/node_modules/gdal/lib/binding/gdal.node" (not found)
#11 8.747 npm ERR! node-pre-gyp http GET https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for gdal@0.11.1 and node@18.12.1 (node-v108 ABI, glibc) (falling back to source compile with node-gyp)
#11 8.747 npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! gyp info it worked if it ends with ok
#11 8.744 npm ERR! code 1
#11 8.744 npm ERR! path /app/node_modules/gdal
#11 8.746 npm ERR! command failed
#11 8.746 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#11 8.746 npm ERR! Failed to execute '/nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/bin/node /nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/gdal/lib/binding/gdal.node --module_name=gdal --module_path=/app/node_modules/gdal/lib/binding --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
#11 8.747 npm ERR! node-pre-gyp info it worked if it ends with ok

#11 8.747 npm ERR! node-pre-gyp info using node-pre-gyp@1.0.10
#11 8.747 npm ERR! node-pre-gyp info using node@18.12.1 | linux | x64
#11 8.747 npm ERR! node-pre-gyp info check checked for "/app/node_modules/gdal/lib/binding/gdal.node" (not found)
#11 8.747 npm ERR! node-pre-gyp http GET https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for gdal@0.11.1 and node@18.12.1 (node-v108 ABI, glibc) (falling back to source compile with node-gyp)
#11 8.747 npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! gyp info it worked if it ends with ok
but that may be unrelated to this/something that has to do with a preinstall script or similar
Brody
Brody15mo ago
what is node-pre-gyp
Grey
Grey15mo ago
it's used to compile C++/other add-ons for node packages it looks like the line that triggers the error comes from "install": "node-pre-gyp install --fallback-to-build", in the package.json of gdal, which is a dev dependency of my project but maybe the install script gets triggered no matter what/isn't getting ignored will see if I can find more info on that ok got it this did the trick NPM_CONFIG_PRODUCTION= npm ci --omit=dev --omit=optional needed to unset the NPM_CONFIG_PRODUCTION variable, and pass --omit=dev will try without the optional to see if that changes anything nope it needed the optional flag too craziness
Brody
Brody15mo ago
thats very odd, but im glad you got to the solution in the end, and sorry i couldn't have helped more
Grey
Grey15mo ago
yea, I blame needing the flag on my setup, but the NPM_CONFIG_PRODUCTION thing was definitely the way to go don't know why it wasn't an issue before with Node 14 🤷‍♂️
Grey
Grey15mo ago
definitely helped to talk through it though! thanks a ton
Brody
Brody15mo ago
no problem! 🙂