Self Hosting on Kubernetes with runAsNonRoot: true option
I would like to host Novu on a Kubernetes environment using the Novu Docker images.
I am using the following Novu images:
- ghcr.io/novuhq/novu/api
- ghcr.io/novuhq/novu/web
- ghcr.io/novuhq/novu/embed
- ghcr.io/novuhq/novu/widget
- ghcr.io/novuhq/novu/worker
- ghcr.io/novuhq/novu/w
as well as the Docker images for mongo and redis:alpine.
For security reasons, the
runAsNonRoot: true option is enabled in the Kubernetes environment. This means that I need to specify which non-root user the container and image should run with.
For mongo and redis:alpine image, the service user with ID 999 can be selected for this purpose.
For the Novu images, I have tried using the node user with ID 1000, which however, leads to problems with some images. For example, in the Dockerfile for the web image, some COPY operations are performed as the root user because there is no user switch to the node user.
When starting the Novu web container in Kubernetes, this results in the inability to access, e.g. the /app/env.sh file copied in the Dockerfile, as it is a file owned by the root user. The exact error message is:
@novu/web@0.19.0 envsetup:docker /app chmod +x ./env.sh && ./env.sh && mv ./env-config.js ./build/env-config.js chmod: ./env.sh: Operation not permitted ELIFECYCLE Command failed with exit code 1.Is it possible to modify the Dockerfiles so that there is a switch to the node user before the commands or to transfer permissions for these files to the node user afterwards? Alternatively, is it possible to introduce a custom Novu service user with the necessary permissions? If it is not possible to make these changes in the Novu Dockerfiles, I will need to create my own Dockerfiles that build on the Docker images and transfer the permissions to the node user. The images created from these Dockerfiles should then work for Kubernetes with the
runAsNonRoot: true option.10 Replies
Hi @rise.michael 👋🏻
Novu does not support Kubernetes deployment for the community self-host MIT licensed version.
The current Kustomize-based configuration was added by someone from the community.
I will request you to reach out to our sales team at sales@novu.co if you are looking for production-grade Kubernetes-compatible self-hosting version support.
We will be happy to help you 🙂
Hello @Rise Michael wondering if you managed to get it working. I am facing the same issue. Did you end up having to create your own Dockerfiles?
Hi @Osama Sayed
Yes, I have created my own Dockerfiles, but also submitted an issue today in the novu github repo. Would provide a PR if the issue gets accepted.
@Rise Michael @Osama Sayed Thank you for raising this issue.
This is a know issue on kubernetes as the web container is set up to build the project on startup and when you do not run as root the system can not touch the file system to biuld the front end.
We do have this issue list on our self-hosting help guide, https://docs.novu.co/self-hosting-novu/kubernetes
If this is not clear feel free to make a pr to clear up the misunderstanding.
@Pawan Jain If you see this come up again feel free to use this responce above or contact me to see if I can give a hand.
@Zac Clifton Thanks for the response!
But wouldn't it be enough to set the permissions for the node user and switch to the node user at the end of the Dockerfile for the web container? I have tried this locally and it works completely fine.
GitHub
🐛 Bug Report: Node user instead of root user in Dockerfiles · Issue...
📜 Description For security reasons, it would be great to switch to the node user (1000) and grant the permissions instead of using the root user in the Dockerfiles (especially in the Dockerfile for...
@Zac Clifton Or is there something against this change that I am not aware of?
@Rise Michael, you just advanced to level 1!
It should be but I do not have the time to verify and test at this moment. I do see with more enterprise deals this may come up but I can not promise.