CD
Cloudflare Developers•5mo ago
ravinesPlains

Unable to Configure Static Assets & Worker Script Routing using `run_worker_first`

Hi we are in the process of porting our Astro.js website which is currently on Cloudflare Pages to Cloudflare workers by using the @astrojs/cloudflare plugin and are facing a issue w.r.t how cloudflare static asset routing which is shooting up our overall workers bill. Being a static site generator, Astro.js outputs index.html files for every webpage on our site. Requests to these static webpages are correctly served by the assets handler and the rest trigger the worker script (which is compiled by the @astrojs/cloudflare plugin) located at ./dist/_worker.js/index.js. --- Our use case requires the use of Astro's Middleware feature to run on specific routes. Since the middleware's default behaviour is to run on all routes, a worker invocation is triggered on every request that does not match a static asset, i.e. a index.html file which includes: A) Routes that we want to hit the middleware e.g. /oauth/callback B) Non-existent routes leading to a 404 not found error As of now, there is no provision in the Astro middleware to run on specific routes, like on other frameworks such as Next.js. --- Case B) is causing thousands of worker invocations on our current staging Astro website. The majority of these requests are coming from malicious bots and crawlers looking for sensitive files such as .env, .htaccess, etc. We believe that these invocations are going to shoot up our Cloudflare Workers bill once we move this website to production. Hence we are trying to find a solution that allows the Astro middleware worker script to run only on Case A), i.e. only on explicitly whitelisted paths.
1 Reply
ravinesPlains
ravinesPlainsOP•5mo ago
In light of this issue, we have been closely following the _routes.json in Workers proposal #9143. Since it was merged last week, we have been trying make it work towards the above mentioned use case but haven't got any progress yet. Here is one of the only combinations of the html_handling and run_worker_first asset config options in the wrangler config which seemed to show a behavioral change in the static asset/worker script routing: 
    "assets": {
        "binding": "ASSETS",
        "directory": "./dist",
        "html_handling": "force-trailing-slash",
        "run_worker_first": ["!/*", "/oauth/callback"]
    }   
    "assets": {
        "binding": "ASSETS",
        "directory": "./dist",
        "html_handling": "force-trailing-slash",
        "run_worker_first": ["!/*", "/oauth/callback"]
    }
Before: worker would run on every route for which asset not found, including 404 Now: Worker is never invoked and an empty no-body 404 response is returned if no asset is found. This happens even on sending a request to /oauth/callback for which there is no index.html asset present. Reasoning: The negative !/* rule which is intended to prevent worker invocation on not found requests seems to be taking higher precedence that the positive rule. Intended: Worker script should (only) get invoked if the request path matches /oauth/callback. I think we want positive rules to have higher precedence over negative rules but the current implementation is the other way around. We would be glad if you could point us towards an asset config that might help us achieve our use case. We also experimented with the dist/_routes.json file since apparently this Cloudflare Pages only feature has now come to Cloudflare Workers. This is the config we tried but it doesn't seem to be doing anything: 
{ "version": 1, "include": ["/oauth/callback"], "exclude": ["/*"] }
{ "version": 1, "include": ["/oauth/callback"], "exclude": ["/*"] }

Did you find this page helpful?