Wasp•3y ago

Optional node_modules installs

I dunno why but for some reason I thought that Wasp detects npm installs and keeps track of them. Maybe cus of this file

installedFullStackNpmDependencies.json

installedFullStackNpmDependencies.json

installedFullStackNpmDependencies.json

installedFullStackNpmDependencies.json but would it make sense for Wasp to track chances in package.json file and if there are any - only then do a

npm install

npm install

or have some sort of flag to disable the modules installation
I was playing around yesterday with deployment to Railway, and I needed to do a few small changes in the source code (not

.wasp

.wasp

.wasp

.wasp folder) and build the codebase frequently afterwards and node_modules were installed every single time, whereas I think there might be a different way to do it.
Like keep a ref to the dependencies and track any chances in there on each build command. If any detected - reinstall them entirely. If none - just delete everything apart from

node_modules

node_modules

folder.
Alternative approach, add a CLI flag to disable installs manually (might be easier and preferred way)

MEE6•4/19/23, 7:10 AM

Wohooo @IamIconLiving

, you just became a Waspeteer level 3!

IamIconLivingOP•4/19/23, 7:11 AM

keep getting levelling up :godfatherboi:

Vinny•4/19/23, 7:19 AM

ah interesting. I'll have to defer to one of the devs on this one, but I'm curious why you were building so frequently?

IamIconLivingOP•4/19/23, 7:48 AM

ofc, good question. so I decided to upload my todo app from the documentation to the railway, but a small type error in my codebase that I didn't notice, so linting screamed at me.
I fixed the type error and did wasp buildwasp build but the .wasp/out.wasp/out folder was not reflecting my changes. I thought it's very weird and started digging and doing multiple wasp buildwasp build, expecting my fix to come into this folder
later then I realise that the folder is wrong - I should have look at .wasp/build.wasp/build folder (a bit confusing why those 2 co-exist at the same time) but then again, my changes were not reflecting there either for some reason. so I had to manually change some conflicting bits in the /build/build to be able to deploy to Railway

IamIconLivingOP•4/19/23, 7:49 AM

but this morning /build/build folder was refreshed correctly on each wasp buildwasp build, so chances are - I did some stupid thing due to end of the day

IamIconLivingOP•4/19/23, 7:50 AM

the point is - that's the journey, I did a ton of wasp build yesterday and it was too slow

aaand, I am already thinking that for an app I wanted to do, I need to do wasp buildwasp build quite frequently too, to then upload the app onto a platform, so building will be a second nature

Vinny•4/19/23, 8:40 AM

Oh cool. This is really great info! I'm glad to see someone really stress testing Wasp

IamIconLivingOP•4/19/23, 8:46 AM

oh yeah, I have few short term goals for Wasp:

deploy on Railway - done
make sure if google auth on Railway works - TBD
port LSP from VSCode to neovim plugins so guys in terminals can enjoy their time writing Wasp files - this is not a short term goal, but a long one, cus I have never done such things before, so it's dark area but exciting
start my app with Wasp - and this will be a real stress test

Vinny•4/19/23, 9:02 AM

oh great! I'm really curious if Railway has solved the issue with their proxy servers and OAuth2.

martinsos•4/19/23, 11:11 AM

Ok let's see what happened here!

Difference between out/ and build/ -> ah yes, well that is somewhat typical in the tools today, usually you have one place where you generate artefacts during development, and then a clean place to do it when building for production. I don't think we could combine these things together easily, it would probably get quite messy, as some things are done different during wasp buildwasp build , which uses .wasp/build, compared to

wasp start

wasp start

, which uses .wasp/out.

If I got it correctly, what you found troublesome is that on each wasp buildwasp build you have to wait for node_modules to get reinstalled, completely from scratch, which is a PITA.
Here is the offending line: https://github.com/wasp-lang/wasp/blob/main/waspc/cli/src/Wasp/Cli/Command/Build.hs#L47 -> we delete the whole .wasp/build dir when creating new build. The idea behind this is to make sure we have clean start, that nothing from the previous build is by accident left over.

But yup this cleans up node_modules also!

Hmmm, so we could, as you suggested, potentially remove everything but node_modules. What if some redundant stuff is left in them from before? Well I think it doesn't matter, they are anyway not getting deployed, it is instead expected that when you deploy you do npm build (it is what we do in our Docker container for deploying server).
There is another thing I wonder here though -> why do we even need node_modules here? We are just producing code that is to be deployed -> Docker container is anyway building node_modules on its own from what I see, and doing prisma generation and stuff. I think we let it install node_modules because we normally do that after generating code, but it might actually be redundant in this case, and we could possibly just remove it!

I will create a GH issue with all this info and propose that we look into skipping installation of node_modules for wasp buildwasp build, or if that can't be done, then we at least try to re-use them.

martinsos•4/19/23, 11:25 AM

Created an issue here: https://github.com/wasp-lang/wasp/issues/1152 . The more I dig into it, the more it seems like we don't even need that npm install heh :D. Now I investigated it so much that I should probably implement it - I had other plans for today but I might give it a go anyway, will let you know if I do so.

Mmartinsos Ok let's see what happened here! Difference between out/ and build/ -> ah yes, ...

IamIconLivingOP•4/19/23, 11:35 AM

wow thanks so much for this detailed response, much appreciated! yeah that's what I thought, that modules folder is being cleaned up and each run, hence reinstallation was "needed"

Mmartinsos Created an issue here: https://github.com/wasp-lang/wasp/issues/1152 . The more ...

IamIconLivingOP•4/19/23, 11:36 AM

haha as long as you think it's worth your time and money right now, that'd be awesome, but it's not a pressing issue!
as long as you can still build with actually installing new dependencies when required, meaning when new ones were added (I don't have an insight how deps are installed internally), then yeah, this could be very good addition to the tooling

Optional node_modules installs

Similar Threads

Similar Threads

Similar Threads