Workers Vite in browser not trying to reach worker

I'm trying to rebuild my setup to deploy my React SPA together with my api in one worker. For this im using the cloudflare worker vite plugin with

json    
"assets": {
        "not_found_handling": "single-page-application"
}

now i have an issue that it always just shows not found on worker paths when accessing them in browser. This is a big issue since I have some /callback routes for auth flows I need the user to be redirected to. Anything I need to configure?
Was this page helpful?