R
Join ServerRailway
✋｜help
Node build failing
Created a project today - I initially had a Dockerfile then removed it to try and use the default build.
Project is KoaJS on Node 14.
The build errors with:
Unsure if this is a Nixpacks issue or something else?
Project is KoaJS on Node 14.
The build errors with:
#11 19.44 gyp ERR! find Python (accepted by both node-gyp and npm)
#11 19.44 gyp ERR! find Python - Set the environment variable PYTHON
#11 19.44 gyp ERR! find Python - Set the npm configuration variable python:
#11 19.44 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
#11 19.44 gyp ERR! find Python For more information consult the documentation at:
#11 19.44 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#11 19.44 gyp ERR! find Python **********************************************************
#11 19.44 gyp ERR! find Python
#11 19.44 gyp ERR! configure error
#11 19.45 gyp ERR! stack Error: Could not find any Python installation to use
#11 19.45 gyp ERR! stack at PythonFinder.fail (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#11 19.45 gyp ERR! stack at PythonFinder.runChecks (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#11 19.45 gyp ERR! stack at PythonFinder.<anonymous> (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#11 19.45 gyp ERR! stack at PythonFinder.execFileCallback (/nix/store/b62jqvlh26w09ypimlj0c9q19lx2i19j-nodejs-14.21.1/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#11 19.45 gyp ERR! stack at exithandler (child_process.js:390:5)Unsure if this is a Nixpacks issue or something else?
18c9abfc-def5-4aa2-8518-6b3d9567c97c
nixpacks info:
==============
Using Nixpacks
==============
context: 6e6ea8f73c1638910f6b4660b31a5cea
╔═════════ Nixpacks v1.5.1 ═════════╗
║ setup │ nodejs-14_x, npm-6_x ║
║───────────────────────────────────║
║ install │ npm ci ║
║───────────────────────────────────║
║ start │ npm run start ║
╚═══════════════════════════════════╝did the dockerfile you had work?
no, it was something I set up before and relied on old conventions
it built but wasn't meant for this usecase
okay fair, so your node project needs python installed is what I get from the logs?
it seems like that's because node-gyp needs it, but node-gyp is only imported as a devDependency for the packages I've installed
ah i see, can you send me your railway.json file from your most recent deployment?
this can be found in the details tab
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS"
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}but also
"When your NODE_ENV environment variable is set to production, using npm ci will not install devDependencies"
railway sets that to production, are you perhaps overwriting it somehow
"When your NODE_ENV environment variable is set to production, using npm ci will not install devDependencies"
railway sets that to production, are you perhaps overwriting it somehow
yep that's a good point
just checked env vars in railway and it's not being overridden there
not sure how else it would be getting changed since this is the build stage
strange, try this though
{
"$schema": "https://railway.app/railway.schema.json",
"build": {
"builder": "NIXPACKS",
"nixpacksPlan": {
"phases": {
"install": {
"dependsOn": ["setup"],
"cmds": ["npm ci --omit=dev"]
}
}
}
},
"deploy": {
"numReplicas": 1,
"restartPolicyType": "ON_FAILURE",
"restartPolicyMaxRetries": 10
}
}i think ci will accept the omit flag, but im not too sure, im not a js dev
but that at least should give you an idea on how to overwrite the install command and you can play around with that now
ahh found the issue maybe
fsevents is macOS only. It specifies an os field in its package.json but this is apparently ignored by npm ci. fsevents used to provide prebuilt binaries up to fsevents@1.2.9 which meant for those versions it didn't attempt to compile
bumping fsevents to >= v1.2.13 should fix the issue
let me know how that goes, ive seen this issue asked here before, where ci would install a dev dep that needed python
Updating that skipped the fsevents package, but another devDependency was still being installed that required node-gyp under the hood.
Spent a bunch of time trying things locally and saw that
npm ci would try to install these packages unless either NODE_ENV=production was set, or the --production flag was passed inupdated the command in railway.json to
npm ci --production and it built fineso it must be that NODE_ENV=production isn't set at build time?
--omit=dev didn't work?yeah must not be for some reason
nah, wasn't recognized
maybe it's bc some people want to do dev steps when they build
interesting, well I will keep that in mind and recommend the --production flag
thank you for this information
np! thanks for the assist
no prob 🙂
I'm back 😄
Same problem, different node version
updated to node 18 and am starting to see the same issue again
#11 0.820 npm WARN config production Use
^ I started to see this appear before the error about gdal/fsevents appearing
the command I use is now
--omit=dev instead.^ I started to see this appear before the error about gdal/fsevents appearing
the command I use is now
npm ci --omit=dev --omit=optional in railway.jsonI've tried a bunch of different things but am still getting
config production Use--omit=devinstead. errorsseparately, I moved the offending packages to optional and it still tries to install them (although fsevents does get skipped this way)
"overrides": {
"gdal": "^0.11.0"
},
"optionalDependencies": {
"fsevents": "^1.2.13",
"gdal": "^0.11.0"
}
"overrides": {
"gdal": "^0.11.0"
},
"optionalDependencies": {
"fsevents": "^1.2.13",
"gdal": "^0.11.0"
}
can I see your railway.json file?
sure thing @Brody !
and have you tried
npm ci --production ?Yep I tried that too, same warning
build failed too, can't remember which error but I think fsevents was causing it
fsevents sure is causing a lot of troubles, how about just removing it all together?
last release was Feb 2021, surely there's an equivalent more up to date package without these issues?
Yeah I can start to go that route. It's required by something else further down the pipe, but should be able to figure it out
I was able to resolve the fs events issue, but the gval package then became the issue. It really still feels like it's trying to install Dev dependencies, that's the part that I can't get around and feels like I need to solve first
@Brody confirmed that it's still trying to install stuff like chokidar, which is a devdependency
tried using
npm install --production and npm install --omit=dev toohttps://stackoverflow.com/questions/72776325/npm-warning-config-production-use-omit-dev-instead
this mentioned to change the .npmrc file to get rid of that warning
this mentioned to change the .npmrc file to get rid of that warning
do you know how I could debug this further?
check the
NODE_ENV variable right before install with echo $NODE_ENV && npm install --omit=devooh I think I got something
The Node provider sets the following environment variables:
CI=true
NODE_ENV=production
NPM_CONFIG_PRODUCTION=false: Ensure that dev deps are always installed
now why tf would that be forced
😄
well try setting
NPM_CONFIG_PRODUCTION = truetried to set at the service level
didn't work. going to echo out a few variables to confirm what is sticking
try setting it before the install command
or i should say, set it inline with the install command
ok, I think that setting it at the service level actually did stop it from trying dev deps because logs don't show a warning about chokidar anymore
progress!
yep!
still getting stuff like
still getting stuff like
#11 8.744 npm ERR! code 1
#11 8.744 npm ERR! path /app/node_modules/gdal
#11 8.746 npm ERR! command failed
#11 8.746 npm ERR! command sh -c -- node-pre-gyp install --fallback-to-build
#11 8.746 npm ERR! Failed to execute '/nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/bin/node /nix/store/7ppnyqmp1c92gkfrixbz99sw7q6ifkg1-nodejs-18.12.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/node_modules/gdal/lib/binding/gdal.node --module_name=gdal --module_path=/app/node_modules/gdal/lib/binding --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
#11 8.747 npm ERR! node-pre-gyp info it worked if it ends with ok
#11 8.747 npm ERR! node-pre-gyp info using node-pre-gyp@1.0.10
#11 8.747 npm ERR! node-pre-gyp info using node@18.12.1 | linux | x64
#11 8.747 npm ERR! node-pre-gyp info check checked for "/app/node_modules/gdal/lib/binding/gdal.node" (not found)
#11 8.747 npm ERR! node-pre-gyp http GET https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! node-pre-gyp WARN Pre-built binaries not installable for gdal@0.11.1 and node@18.12.1 (node-v108 ABI, glibc) (falling back to source compile with node-gyp)
#11 8.747 npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/naturalatlas/node-gdal/releases/download/0.11.1/node-v108-linux-x64-glibc.tar.gz
#11 8.747 npm ERR! gyp info it worked if it ends with okbut that may be unrelated to this/something that has to do with a preinstall script or similar
what is node-pre-gyp
it's used to compile C++/other add-ons for node packages
it looks like the line that triggers the error comes from
"install": "node-pre-gyp install --fallback-to-build", in the package.json of gdal, which is a dev dependency of my projectbut maybe the install script gets triggered no matter what/isn't getting ignored
will see if I can find more info on that
ok got it
this did the trick
NPM_CONFIG_PRODUCTION= npm ci --omit=dev --omit=optionalneeded to unset the NPM_CONFIG_PRODUCTION variable, and pass --omit=dev
will try without the optional to see if that changes anything
nope it needed the optional flag too
craziness
thats very odd, but im glad you got to the solution in the end, and sorry i couldn't have helped more
yea, I blame needing the flag on my setup, but the NPM_CONFIG_PRODUCTION thing was definitely the way to go
don't know why it wasn't an issue before with Node 14 🤷♂️
definitely helped to talk through it though! thanks a ton
no problem! 🙂