wasp db start Issues
This is the response when I try to run wasp db start on my Mac
Status: Downloaded newer image for postgres:latest
Error: in 18+, these Docker images are configured to store database data in a
format which is compatible with "pg_ctlcluster" (specifically, using
major-version-specific directory names). This better reflects how
PostgreSQL itself works, and how upgrades are to be performed.
See also https://github.com/docker-library/postgres/pull/1259
Counter to that, there appears to be PostgreSQL data in:
/var/lib/postgresql/data (unused mount/volume)
This is usually the result of upgrading the Docker image without
upgrading the underlying database using "pg_upgrade" (which requires both
versions).
The suggested container configuration for 18+ is to place a single mount
at /var/lib/postgresql which will then place PostgreSQL data in a
subdirectory, allowing usage of "pg_upgrade --link" without mount point
boundary issues.
See https://github.com/docker-library/postgres/issues/37 for a (long)
discussion around this process, and suggestions for how to do so.
https://github.com/Kitware/CDash/pull/3217
GitHub
Change
PGDATA in 18+ to /var/lib/postgresql/MAJOR/docker by tia...This is a pretty large breaking change, which is why this only makes the change in 18+ (which is currently in pre-release stages, and not due for GA until September, and pre-release PGDATA director...
GitHub
Upgrading between major versions? · Issue #37 · docker-library/po...
There doesn't seem to be a good way to upgrade between major versions of postgres. When sharing the volume with a new container with a newer version of postgres it won't run as the data dir...
GitHub
Pin Postgres 17 by williamjallen · Pull Request #3217 · Kitware/C...
The Postgres 18 docker image contains changes to make it easier to upgrade which break our existing setup. See docker-library/postgres#37 for a long discussion of the issues being addressed.
This ...
15 Replies
Hi @t.azul.ai. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
----------------------
Based on the error message you're seeing, this is related to a breaking change in PostgreSQL Docker images version 18+. The issue is that you have existing PostgreSQL data in an older format that's incompatible with the newer image structure.
Here are a few solutions to resolve this:
Option 1: Delete the existing Docker volume (simplest)
If you don't need to preserve your development database data, you can delete the existing Docker volume and start fresh:
1. Stop any running Wasp processes
2. Find your volume name (it should be something like
wasp-dev-db-<your-app-name>-<hash>)
3. Delete the volume:
4. Run wasp db start again
The volume name is typically shown in the output when you run wasp db start. According to the knowledge sources, it appears in the "Additional info" section.
Option 2: Pin to an older PostgreSQL version
If you need to maintain compatibility with your existing data, you could temporarily use an older PostgreSQL version. However, this would require modifying how Wasp starts the database, which isn't directly configurable through Wasp commands.
Option 3: Follow PostgreSQL's upgrade path
If you need to preserve your data, you'll need to follow the upgrade instructions provided in the error message, which involves using pg_upgrade. This is more complex and typically only necessary if you have important development data.
For most development scenarios, Option 1 (deleting the volume) is the recommended approach since development databases can be easily recreated using wasp db migrate-dev.
After deleting the volume and running wasp db start successfully, you'll need to run wasp db migrate-dev to recreate your database schema.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Sorry @t.azul.ai, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist:
Perhaps try again in 15mins?I have tried that, among other trouble shooting techniques, but this is the error that shows up for that: Error response from daemon get wasp-dev-db-<I INSERTED MY APP NUMBER HERE> @kapa.ai
But I also managed to delete the volume and then I listed out any additional docker volumes and there were none, so I ran wasp db start again and got the exact same error that I listed out the first time
Wohooo @t.azul.ai, you just became a Waspeteer level 1!
If someone from the @Wasp Team is around, this issue looks serious as it's breaking the environment setup / CI builds for everyone.
Hey @Sergio Visinoni , this issue should be limited to
wasp start db, not to CI or builds. Specifically, what happened is that wasp start db pulled in newer version of the dockerfile, with newer posters version, and your Dev dB on the disk is using older version,so there is conflict.
Easiest solution is to not use wasp start db but instead directly start the dB via specific postgres image: check our database docs for exact instructions.
Btw sorry for slow answer we are all on yearly retreat and are a bit slower to respond.
In the future we will likely pin down the version of it to avoid surprises like this + we already have an option also in main to lock the image via cli flag, that will be released soon.
Others from the team might have more to write, I am writing this as a quick responseI haven't checked the latest OpenSaaS 2.x CI setup, but with the 1.x, which I guess is the one mostly used out there, it uses all the internal Wasp commands (rightly so) to set up and run e2e tests. I don't know how many folks are using that, but I guess I'm not alone. Anyway, I'll check how easily it is to either replace that or try to upgrade to OpenSaaS 2.x (if that can help solving the issue)
https://github.com/wasp-lang/open-saas/blob/main/.github/workflows/e2e-tests.yml this still appears to be the recommended way, also according to the docs https://github.com/wasp-lang/open-saas/blob/02d0c90eb1750c155ccef3a215e48de58f95a2bc/opensaas-sh/blog/src/content/docs/guides/tests.md
Just sharing more info.
I understand you're busy and all, but the impact might be broader than it looks like
GitHub
open-saas/.github/workflows/e2e-tests.yml at main · wasp-lang/open...
A free, open-source SaaS app starter for React & Node.js with superpowers. Full-featured. Community-driven. - wasp-lang/open-saas
GitHub
open-saas/opensaas-sh/blog/src/content/docs/guides/tests.md at 02d0...
A free, open-source SaaS app starter for React & Node.js with superpowers. Full-featured. Community-driven. - wasp-lang/open-saas
Thanks, yup it makes sense it affects CI if
wasp start db is run there! I meant to say it shouldn't affect anything beyond that command.
Thanks for sharing, it will help us when we look into this properly. I hope you manage to get unstuck in the meantime with directly running a dev database.@Sergio Visinoni let me jump in with some suggestions:
1.
wasp db start uses latest Postgres, so when a new major Postgres release comes out - it always will immediately use the new Postgres image
2. The files on disk that were made with Postgres 16 are incompatible with Postgres 17
3. When you have a Docker volume on your local machine for dev - it will have the old data, and when the image changes from 16 to 17 - it won't be able to work - since it's dev data, the easiest way is to manually delete the volume and recreate the local DB
Now, that's how I did it - but I understand it might not be what you want. So, alternative options are to run the DB yourself with Docker e.g.
In the CI, if it's Github Actions - this issues shouldn't really happen since there are not 'permanent volumes' and this can't happen. Unless you are running e2e tests in some sort of VPS kinda of situation where the volumes are kept between runs.
We just added in the dev branch the support for explicitly setting the database image for wasp db start so this will help. But we should also pin the database version by default.Thanks a lot @miho
The fact that it's failing also in the CI puzzled me too, as I'm not doing anything fancy, just plain GHA and
wasp db start
I believe you can reproduce the same issue by making a PR to the OpenSaaS repo, which seems to use the same setup.
That said, I'll go with the approach of manually setting up the DB for now. ThanksHey @Sergio Visinoni @t.azul.ai we misdiagnosed this as a previous issue we faced. I've created an issue for this now: https://github.com/wasp-lang/wasp/issues/3321 where I list some workarounds for older Wasp versions.
We are going to release a hotfix for version 0.18.X today/tomorrow.
GitHub
wasp db start fails with latest Postgres 18 version · Issue #332...Reproduction Wasp version: 0.18.1 (latest at the time of writing) Reproduction steps: # Force Docker to fetch PG 18 docker image rm postgres:latest # Create a new app cd /tmp wasp new saas-app -t s...
Wasp 0.18.2 is out which comes with a fix for the failing
wasp db start command!
Update to the latest version by running: curl -sSL https://get.wasp.sh/installer.sh | sh -sAwesome, thank you for addressing this. Appreciate the updates.
I'm glad you got around to fixing this!
Thanks so much for this! 🔥