Hey guys, came here from the d1 channel. I've been curious about this since miniflare 3 came out. W

AAdi Hey guys, came here from the d1 channel. I've been curious about this since min...

M

MrBBot•6/14/23, 9:37 AM

Hey!

Great question! Miniflare 3 only provides an implementation of the internal D1 API. Wrangler injects "shim" code into your worker to wrap this API into the regular D1 database API. There's an in-progress PR to move this "shim" out of Wrangler and into the runtime (https://github.com/cloudflare/workerd/pull/764), where it would be accessible to Miniflare. For now, your best bet is probably to just stay on Miniflare 2 until that PR is merged and a new version of

workerd

workerd

is released.

MMrBBot Hey! 👋 Great question! Miniflare 3 only provides an implementation of the inter...

A

AdiOP•6/14/23, 9:44 AM

Ahh ok, that makes sense. Thanks

H

Hasi•6/17/23, 12:38 PM

Ummm

H

Hasi•6/17/23, 12:38 PM

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

H

Hasi•6/17/23, 12:40 PM

Which package should I install?

H

Hasi•6/17/23, 12:41 PM

mmhm I think I've found it xD

H

Hasi•6/17/23, 12:45 PM

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libunwind.so.1: cannot open shared object file: No such file or directory

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libunwind.so.1: cannot open shared object file: No such file or directory

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libunwind.so.1: cannot open shared object file: No such file or directory

/usr/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd: error while loading shared libraries: libunwind.so.1: cannot open shared object file: No such file or directory

H

Hasi•6/17/23, 12:45 PM

And I have the lib installed

D

DaniFoldi•6/17/23, 12:47 PM

what distro & version is that? linux support is a little rough today, debian 12 and ubuntu 22.04 are the only ones I know workerd supports - they are working on improving this though, so soontm

things will be a lot smoother

DDaniFoldi what distro & version is that? linux support is a little rough today, debian 12 ...

H

Hasi•6/17/23, 12:47 PM

Btw I use

H

Hasi•6/17/23, 12:48 PM

so it doesn't support arch?

D

DaniFoldi•6/17/23, 12:49 PM

I don't know the answer to that, unfortunately - it might support it somehow but it's the weekend so I don't want to ping people on the team, I'll try to get back to you on Monday though!

DDaniFoldi I don't know the answer to that, unfortunately - it might support it _somehow_ b...

H

Hasi•6/17/23, 12:50 PM

if it doesn't support arch it doesn't exists.

H

Hasi•6/17/23, 12:52 PM

anyways, so i stick to --remote for now

V

vvo•6/27/23, 1:46 PM

Hey there, I found a behavior difference between miniflare (via wrangler latest version) and a deployed worker: when using

env.R2_BUCKET.put(key, request.body)

env.R2_BUCKET.put(key, request.body)

env.R2_BUCKET.put(key, request.body)

env.R2_BUCKET.put(key, request.body) you will get an error like "Provided readable stream must have a known length (request/response body or readable half of FixedLengthStream)".

1. It's working when deployed on Cloudflare workers
2. It's a request.body

T

TravisFrank•6/29/23, 4:39 PM

Are the v3 API options documented anywhere?

M

MrBBot•6/29/23, 5:27 PM

Miniflare 3 API docs: https://github.com/cloudflare/miniflare/blob/tre/packages/miniflare/README.md

MMrBBot 📚 Miniflare 3 API docs: https://github.com/cloudflare/miniflare/blob/tre/packa...

T

TravisFrank•6/29/23, 6:12 PM

Thanks @mrbbot!

T

TravisFrank•6/29/23, 10:11 PM

Any chance

sourcemap

sourcemap

sourcemap

sourcemap support is on the docket for the API? That's pretty much the only bit of moving to v3 that's got me bummed, the logging leaves a lot to be desired

TTravisFrank Any chance `sourcemap` support is on the docket for the API? That's pretty much...

M

MrBBot•6/29/23, 10:40 PM

Source mapping should be supported by default, but due to a bug in

workerd

workerd

, you'll need to wrap your worker with something that returns un-source-mapped stack traces as JSON using the

MF-Experimental-Error-Stack

MF-Experimental-Error-Stack

MF-Experimental-Error-Stack

MF-Experimental-Error-Stack header: https://github.com/cloudflare/workers-sdk/blob/main/packages/wrangler/templates/middleware/middleware-miniflare3-json-error.ts. Wrangler will do this automatically.

As for logging, completely agree. I'd like to try bring Node's log formatting to

workerd

workerd

at some point.

T

TravisFrank•6/29/23, 10:47 PM

Sweet, thanks for the tip

T

TravisFrank•6/29/23, 10:50 PM

Overall the upgrade feels good, it's like a warm safety blanket knowing the runtime is the same. I also actually find myself enjoying doing away with the

wrangler.toml

wrangler.toml

parsing and passing bindings in the config- it's less "magic", but for me it seems easier to debug

TTravisFrank Overall the upgrade feels good, it's like a warm safety blanket knowing the runt...

M

MrBBot•6/29/23, 10:53 PM

Nice, glad you're liking it! We're planning to standardise/simplify the

unstable_dev()

unstable_dev()

unstable_dev()

unstable_dev() API (https://developers.cloudflare.com/workers/wrangler/api/) at some point soon, which will allow you to use your

wrangler.toml

wrangler.toml

file, but customise as required in JavaScript too.

T

TravisFrank•6/29/23, 10:55 PM

Yes, love that!

T

TravisFrank•6/29/23, 10:56 PM

Btw, in case I'm missing something- there's no way to run multiple workers with

wrangler dev

wrangler dev

, right? If there is I can save myself the pain of manually putting together the

esbuild

esbuild

esbuild

esbuild watch -> MF reload script.

M

MrBBot•6/29/23, 10:58 PM

Yes there is, right now if you run a

wrangler dev

wrangler dev

session in each worker's directory, they should auto-magically connect. We're considering eventually moving to a Docker Compose style solution, where running

wrangler dev

wrangler dev

automatically starts up all required services though.

T

TravisFrank•6/29/23, 11:09 PM

Oh wow

T

TravisFrank•6/29/23, 11:25 PM

T

TravisFrank•6/29/23, 11:27 PM

It doesn't look like service bindings are working for me. I also don't get any sort of "Your worker has access to the following bindings: - Services:"

TTravisFrank It doesn't look like service bindings are working for me. I also don't get any ...

M

MrBBot•6/29/23, 11:31 PM

Hmmm ok, could you try restart your computer and then try again? This is kinda why I'd like to see us move to a Docker Compose like thing, sometimes this magic doesn't work.

T

TravisFrank•6/29/23, 11:41 PM

Damn, still no luck :/

T

TravisFrank•6/29/23, 11:42 PM

Where does the service registry live on my machine?

T

TravisFrank•6/30/23, 8:51 AM

Aha! Debugged! User error, bad

toml

toml

(I had just added the

placement

placement

table and had

services

services

below it, so

services

services

was interpreted as a key on

placement

placement

)

MMrBBot Source mapping should be supported by default, but due to a bug in `workerd`, yo...

T

TravisFrank•6/30/23, 10:08 AM

I'm still a bit confused by the logging - none of my

console.logs

console.logs

console.logs

console.logs appear when running via the API, but they are appearing via

wrangler dev

wrangler dev

. I have

log: new Log(LogLevel.VERBOSE),

log: new Log(LogLevel.VERBOSE),

log: new Log(LogLevel.VERBOSE),

log: new Log(LogLevel.VERBOSE), in my options and

verbose: true,

verbose: true,

verbose: true,

verbose: true,, is there something else I'm missing?

TTravisFrank I'm still a bit confused by the logging - none of my `console.logs` appear when ...

M

MrBBot•6/30/23, 10:09 AM

Those API options should enable

workerd

workerd

's logging. Wrangler actually uses the DevTools inspector protocol for logging instead.

T

TravisFrank•6/30/23, 10:11 AM

Aha, gotchya

M

matt•6/30/23, 11:43 AM

oh cool, there's a miniflare channel

I just worked on a prototype for developing multiple workers but had to use express. Tried to use the

workers

workers

array but the contents of

/scripts/

/scripts/

/scripts/

/scripts/ can change during dev, and this way the latest script is used for each request. is this an ok approach?

app.use('/:name/*', async (req, res) => {
  const mf = new Miniflare({
    scriptPath: `./scripts/${req.params.name}/index.js`,
    bindings: process.env,
  })
  const url = `http://localhost:8787${req.originalUrl}`
  const queryRes = await mf.dispatchFetch(url, {
    method: req.method,
    headers: req.headers,
    body: req.body,
  })
  res.send(await queryRes.text())
  await mf.dispose()
})

app.use('/:name/*', async (req, res) => {
  const mf = new Miniflare({
    scriptPath: `./scripts/${req.params.name}/index.js`,
    bindings: process.env,
  })
  const url = `http://localhost:8787${req.originalUrl}`
  const queryRes = await mf.dispatchFetch(url, {
    method: req.method,
    headers: req.headers,
    body: req.body,
  })
  res.send(await queryRes.text())
  await mf.dispose()
})

app.use('/:name/*', async (req, res) => {
  const mf = new Miniflare({
    scriptPath: `./scripts/${req.params.name}/index.js`,
    bindings: process.env,
  })
  const url = `http://localhost:8787${req.originalUrl}`
  const queryRes = await mf.dispatchFetch(url, {
    method: req.method,
    headers: req.headers,
    body: req.body,
  })
  res.send(await queryRes.text())
  await mf.dispose()
})

app.use('/:name/*', async (req, res) => {
  const mf = new Miniflare({
    scriptPath: `./scripts/${req.params.name}/index.js`,
    bindings: process.env,
  })
  const url = `http://localhost:8787${req.originalUrl}`
  const queryRes = await mf.dispatchFetch(url, {
    method: req.method,
    headers: req.headers,
    body: req.body,
  })
  res.send(await queryRes.text())
  await mf.dispose()
})

Mmatt oh cool, there's a miniflare channel 🙂 I just worked on a prototype for develop...

M

MrBBot•6/30/23, 12:15 PM

Hey!

This should work, but ideally you'd want to reuse the

Miniflare

Miniflare

instance across requests. Do you know the possible values for

req.params.name

req.params.name

req.params.name

req.params.name ahead of time? Could you cache

mf

mf

mf

mf values for each

name

name

in a

Map<string, Miniflare>

Map<string, Miniflare>

Map<string, Miniflare>

Map<string, Miniflare>? You'll probably also want to copy the headers across from

queryRes

queryRes

queryRes

queryRes to

res

res

, and might consider streaming the body instead of buffering it. See the

writeResponse

writeResponse

writeResponse

writeResponse function in Miniflare's own code for this: https://github.com/cloudflare/miniflare/blob/6984a7f7a35a5b91dea3bb62d657818ca3cb52a2/packages/miniflare/src/index.ts#L307-L354.

M

matt•6/30/23, 12:26 PM

this looks brilliant, thank you! The contents of

./scripts

./scripts

./scripts

./scripts will change quite a lot during development, e.g.
-

./scripts/<worker_name>

./scripts/<worker_name>

./scripts/<worker_name>

./scripts/<worker_name> folders created/deleted
- contents of

./scripts/<worker_name>/index.js

./scripts/<worker_name>/index.js

./scripts/<worker_name>/index.js

./scripts/<worker_name>/index.js updated
id be happy to cache the

mf

mf

mf

mf instances, but is there a way to invalidate the cache in either of these scenarios?

Mmatt this looks brilliant, thank you! The contents of `./scripts` will change quite a...

M

MrBBot•6/30/23, 12:27 PM

const promise = cache.get(key)?.dispose(); cache.delete(key); await promise;

const promise = cache.get(key)?.dispose(); cache.delete(key); await promise;

const promise = cache.get(key)?.dispose(); cache.delete(key); await promise;

const promise = cache.get(key)?.dispose(); cache.delete(key); await promise; should invalidate it

M

matt•6/30/23, 12:30 PM

that will definitely work if I can figure out how to wire up the fs events, is there anything exposed by miniflare that can monitor the worker script folders/files? I would imagine perhaps the livereload uses something in this vein

Mmatt that will definitely work if I can figure out how to wire up the fs events, is t...

M

MrBBot•6/30/23, 12:31 PM

Unfortunately not, live reload relies on you calling

setOptions()

setOptions()

setOptions()

setOptions() yourself with updates

M

matt•6/30/23, 12:32 PM

got it, thanks for your help! this has been a fun prototype to put together

Original message was deleted

M

MrBBot•6/30/23, 12:45 PM

Replied over there

T

TravisFrank•6/30/23, 2:11 PM

Is there anyway to view worker logs while running with

unstable_dev

unstable_dev

? I'm not seeing anything show up in DevTools

T

TravisFrank•6/30/23, 4:27 PM

Ah, nvm, I see in source that's it's not yet implented

MMrBBot Hey! 👋 This should work, but ideally you'd want to reuse the `Miniflare` instan...

M

matt•7/1/23, 6:12 AM

was just reading up on traditional streams vs web streams in node. I may be wrong but I wonder if you can replace the last few lines of this function with:

await response.body?.pipeTo(Writable.toWeb(initialStream))

await response.body?.pipeTo(Writable.toWeb(initialStream))

await response.body?.pipeTo(Writable.toWeb(initialStream))

await response.body?.pipeTo(Writable.toWeb(initialStream))

M

matt•7/1/23, 8:50 AM

Am I doing this wrong? I picked it up rom bindings.spec.ts

mounts: {
  books: {
    // here I define name, scriptPath, modules, compatDate, compatFlags, bindings
  } 
}

mounts: {
  books: {
    // here I define name, scriptPath, modules, compatDate, compatFlags, bindings
  } 
}

mounts: {
  books: {
    // here I define name, scriptPath, modules, compatDate, compatFlags, bindings
  } 
}

mounts: {
  books: {
    // here I define name, scriptPath, modules, compatDate, compatFlags, bindings
  } 
}

Worker "core:user:example"'s binding "BOOKS" refers to a service "core:user:books", but no such service is defined

Worker "core:user:example"'s binding "BOOKS" refers to a service "core:user:books", but no such service is defined

Worker "core:user:example"'s binding "BOOKS" refers to a service "core:user:books", but no such service is defined

Worker "core:user:example"'s binding "BOOKS" refers to a service "core:user:books", but no such service is defined

M

matt•7/1/23, 8:53 AM

for now I am getting away with ... its working but I worry about the overhead

serviceBindings.books = req => {
  const smf = new Miniflare(<same options as mount config>)
  return smf.dispatchFetch(req)  
}

serviceBindings.books = req => {
  const smf = new Miniflare(<same options as mount config>)
  return smf.dispatchFetch(req)  
}

serviceBindings.books = req => {
  const smf = new Miniflare(<same options as mount config>)
  return smf.dispatchFetch(req)  
}

serviceBindings.books = req => {
  const smf = new Miniflare(<same options as mount config>)
  return smf.dispatchFetch(req)  
}

Mmatt was just reading up on traditional streams vs web streams in node. I may be wron...

M

MrBBot•7/1/23, 8:53 AM

Yep, that should work in modern Node versions. Miniflare’s minimum supported version is 16.13.0, which doesn’t provide those web stream adapters.

Hey guys, came here from the d1 channel. I've been curious about this since miniflare 3 came out. W

Similar Threads

Similar Threads

Similar Threads