Local Neon for Development
Hi - I'm building a platform where I programmatically create (Neon) projects for my users. Obviously, I want to test this locally for development without creating real neon projects. What's the recommended way to do this?
There seem to be a few ways to get Neon running locally:
- Directly through cargo: https://github.com/neondatabase/neon?tab=readme-ov-file#running-local-installation
- Via docker compose: https://github.com/neondatabase/neon/blob/main/docker-compose/docker-compose.yml
- With neon_local
Is there something compatible with the Neon web API? Should I use neon_local and just stub calls to that during testing? Run the docker-compose alongside my own and then connect somehow?
16 Replies
compatible-crimson•2y ago
Just to double-check, when you say "databases", you're referring to Neon projects? Or databases within a project
Looping in @ShinyPokemon
sensitive-blueOP•2y ago
Projects, yes.
compatible-crimson•2y ago
One pattern I've seen is having separate Neon accounts, but you'd still end up actually creating rojects. I'll check-in internally, and get back to you
cc @Atli Cervantes
sensitive-blueOP•2y ago
Yeah, we'll do that later for staging/prod split, but this is just local.
I don't want to require an internet connection for that
genetic-orange•2y ago
@Florian can you stub the calls out to return dummy data during local development? I'm starting work on a similar project soon and will have to figure something similar out.
sensitive-blueOP•2y ago
No, I need actual Neon projects/databases, just running locally. @ShinyPokemon
This can't be that hard.. Maybe I'm just tired, but there should just be some option to run it in Docker and point the CLI/API at it, right?
genetic-orange•2y ago
@Florian my understanding is that those constructs are part of the Neon control plane. You cannot run this locally since it's not open source. Maybe @Conrad Ludgate can correct me on this though.
I think using a development Neon account (as mentioned by Mahmoud), or stubbing them out will be the best approach for your use case.
harsh-harlequin•2y ago
What functionality do you need? Just a mock or do you actually hope to access a database after creating a project
sensitive-blueOP•2y ago
Yeah, I understand that all the web-facing stuff is kept proprietary.
But there are of course way to run Neon locally, and there must be some way to launch & interface with it documented somewhere?
A more or less full neon project running locally.
Ideally something I can just spin up and down in Docker, but I can also run it through cargo and wrap the CLI in my code (if that persists properly)
I can just run neon locally directly (https://github.com/neondatabase/neon?tab=readme-ov-file#running-neon-database)
But I would rather have it in a Docker container with my other dev dependencies
However 'neon_local init' fails with
2024-05-15 21:24:17 command failed: materialize initial neon_local environment on disk
(no further details unfortunately)
The (partial) created .neon folder only contains one file: pageserver.toml
I'm testing this in our docker-compose with:
neon:
image: neondatabase/neon:latest
restart: on-failure
volumes:
- neon-data:/data
command: >
neon_local init &&
neon_local startgenetic-orange•2y ago
@Florian this doesn’t include the Neon API. The open source pieces of Neon are the infrastructure pieces. The API is proprietary.
You could use those pieces you referenced to spin up databases, but you need to stub out the API or use a real account
sensitive-blueOP•2y ago
Got it, yeah, that’s fine, neon_local will do - do you have any reference for spinning up neon_local in the provided docker image? I couldn’t find much
genetic-orange•2y ago
@Florian probably a good question for the folks in #🔧｜self-hosted or #📂｜neon-oss . Sorry I can't say more, but I don't have experience running the OSS bits
eastern-cyan•2y ago
eastern-cyan•2y ago
After creating programtically, you can use this command to retrieve the connectionUri.
sensitive-blueOP•2y ago
No @Rishi Raj Jain, we were talking about the neon_local CLI.
This was mostly answered in #🔧｜self-hosted and #📂｜neon-oss
eastern-cyan•2y ago
Alright, I'll learn from the answers posted, thanks!