Looks similar to what I'm seeing - others have reported it at https://github.com/cloudflare/workerd/

D

dave•1/3/24, 1:29 PM

my theory (untested, I stopped looking into it since I finished my project) is that a thread crashes due to OOM or something, and the network connection lost error is a bit misleading.

D

dave•1/3/24, 3:41 PM

Is it expected that the cache doesn't work when using

fetch

fetch

locally?

K

kian•1/3/24, 3:44 PM

Yep.

D

dave•1/3/24, 3:45 PM

I thought the default

caches.default

caches.default

caches.default

caches.default and the cache that

fetch

fetch

uses were the same?

E

Erisa•1/3/24, 3:46 PM

When ran in Cloudflare they both access the same CDN cache (sort of, tiered doesnt work with cache api), but when running locally only Cache API is emulated, the

fetch

fetch

doesn't emulate Cloudflare's cache behaviour

D

dave•1/3/24, 3:47 PM

when you have tiered cache enabled, which cache does

fetch

fetch

use? Always tired cache and never the

caches.default

caches.default

caches.default

caches.default?

U

Unsmart•1/3/24, 3:50 PM

I’d assume it uses both as in if local cache doesn’t have it will go to the upper tier (if it isn’t)

U

Unsmart•1/3/24, 3:52 PM

I’m pretty sure that’s how tiered cache works at least is that every datacenter can still cache the value and not just upper tiers

UUnsmart I’d assume it uses both as in if local cache doesn’t have it will go to the uppe...

D

dave•1/3/24, 3:54 PM

ahhh, that makes sense! What happens if the local cache is the location of the tiered cache, and the cache API attempts to replace it?

U

Unsmart•1/3/24, 3:57 PM

Would just update its own cache the other tiers would eventually pull from it if needed

D

dave•1/3/24, 3:58 PM

but then that sounds like the Cache API does work with tiered caching?

U

Unsmart•1/3/24, 3:59 PM

Not really. You would have to use fetch to pull from tiered cache, unless the datacenter you are in already has the cache. At least thats how I understand it

C

Chaika•1/3/24, 4:03 PM

I assume it depends on the topology as well, if its a location other colos pull from you might update it for tiered cached to use, if its at the bottom/just a local colo it wouldn't ever be used, just a side effect of not storing things twice just for tiered caching/normal cache

N

Nikita Lapkov•1/3/24, 4:16 PM

Hey! Is there any chance for CloudFlare Workers to support compression algorithms other than just gzip?

We provide a library for developers which has a Rust core compiled to WASM. Currently, gzip is able to compress 3MB WASM module into roughly 1.2MB. We have exhausted all options of compressing the code size on our end (like dependency reduction, unused code elimination, etc). Considering other options, we noticed that LZMA and ZSTD algorithms provide us with 800KB and 770KB WASM modules respectively, which is very promising. Our target is roughly 800KB.

CloudFlare Workers do not allow to run WASM code generated in runtime, so we cannot just unpack the WASM module in runtime ourselves and run it. An ugly alternative to that is including a WASM runtime of our own (like wasm3) and running WASM inside WASM. We would like to avoid that, but this is the only option we have at the moment.

Thus, we are wondering if native support for compressed WASM modules can be added to the platform.

Z

zegevlier•1/3/24, 4:24 PM

What's preventing you from just using the 1.2MB WASM module?

N

Nikita Lapkov•1/3/24, 4:25 PM

Free tier worker limit size is 1MB. Since we are a library we want (total library size + total user code size) to be less than 1MB

Z

zegevlier•1/3/24, 4:34 PM

Ah, I see. I think your best bet is to create a discussion post on the workerd (workers runtime) repo: https://github.com/cloudflare/workerd. Unless you are an enterprise customer, in which case reaching out to support might work better.

D

dave•1/3/24, 4:36 PM

stupid question, does workerd allow you to load WASM from a URL?

D

dave•1/3/24, 4:36 PM

or does it have to be bundled?

NNikita Lapkov Hey! Is there any chance for CloudFlare Workers to support compression algorithm...

D

dave•1/3/24, 4:38 PM

have you tried compiling with

-Os

-Os

-Os

-Os?

Ddave have you tried compiling with `-Os`?

N

Nikita Lapkov•1/3/24, 4:39 PM

Yes, but the size reduction is only a couple percent

Ddave or does it have to be bundled?

N

Nikita Lapkov•1/3/24, 4:39 PM

It has to be bundled

N

Nikita Lapkov•1/3/24, 4:41 PM

Before I create a discussion at workerd repo, does anybody else have WASM code size issues? The more use cases we have, the better

D

docy•1/3/24, 7:42 PM

is there no way to access

env

env

outside of fetch scope? it's annoying having to pass it around everywhere

D

docy•1/3/24, 7:46 PM

is it a bad idea to just do

process.env.API_KEY = env.API_KEY

process.env.API_KEY = env.API_KEY

process.env.API_KEY = env.API_KEY

process.env.API_KEY = env.API_KEY in my fetch handler

D

dave•1/3/24, 7:56 PM

Why does this tutorial use the fetch adapter? https://developers.cloudflare.com/workers/tutorials/openai-function-calls-workers/#2-make-a-request-to-openai

Doesn't look like it's needed. https://developers.cloudflare.com/ai-gateway/tutorials/deploy-aig-worker/

K

kian•1/3/24, 8:03 PM

1st one is OpenAI SDK v3, 2nd is v4

K

kian•1/3/24, 8:14 PM

Given that the first tutorial doesn’t pin to the major version, I guess it’s broken since it’ll install v4. cc @rickyrobinett

Kkian Given that the first tutorial doesn’t pin to the major version, I guess it’s bro...

R

rickyrobinett•1/3/24, 8:17 PM

Hmm... yes, good catch. Let me work on a PR to update / fix.

D

daniel•1/3/24, 9:01 PM

is it possible to use local node modules in a worker (with wrangler)? i'm setting up a monorepo and i need some types from a local package in my worker

Kkian 1st one is OpenAI SDK v3, 2nd is v4

D

dave•1/3/24, 10:24 PM

Ahhh, thanks! Totally missed that.

H

Hazem•1/4/24, 12:53 AM

Is it to use cloudflare's workers for security research purposes such as brute forcing web hosts, fingerprinting, crawling, etc ?

H

Hazem•1/4/24, 12:53 AM

all legally ofcourse (bug bounty etc)

U

Unsmart•1/4/24, 12:53 AM

This has all you need: https://www.cloudflare.com/terms/

Cloudflare

Self-Serve Subscription Agreement | Cloudflare

This page is for those who are interested in our Self-Serve Subscription Agreement.

UUnsmart This has all you need: https://www.cloudflare.com/terms/

H

Hazem•1/4/24, 12:57 AM

thanks for this! i don't see anything in the Restrictions part that denies this, would love to get confirmation tho as i may have misread it

D

dave•1/4/24, 2:08 AM

I'm getting really weird console.debug results today.

D

dave•1/4/24, 2:11 AM

  di_output.analyzeResult?.paragraphs?.forEach((paragraph) => {
    if(paragraph.content.includes("found the basis for the appointment")) {
      console.debug("0_________")
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug("1_________")
      console.debug("hello?")
    }
  })
  console.debug("3_________")
  console.debug("_________")
  console.debug("_________")
  console.debug("_________")

  di_output.analyzeResult?.paragraphs?.forEach((paragraph) => {
    if(paragraph.content.includes("found the basis for the appointment")) {
      console.debug("0_________")
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug("1_________")
      console.debug("hello?")
    }
  })
  console.debug("3_________")
  console.debug("_________")
  console.debug("_________")
  console.debug("_________")

  di_output.analyzeResult?.paragraphs?.forEach((paragraph) => {
    if(paragraph.content.includes("found the basis for the appointment")) {
      console.debug("0_________")
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug("1_________")
      console.debug("hello?")
    }
  })
  console.debug("3_________")
  console.debug("_________")
  console.debug("_________")
  console.debug("_________")

  di_output.analyzeResult?.paragraphs?.forEach((paragraph) => {
    if(paragraph.content.includes("found the basis for the appointment")) {
      console.debug("0_________")
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug(paragraph.content)
      console.debug("1_________")
      console.debug("hello?")
    }
  })
  console.debug("3_________")
  console.debug("_________")
  console.debug("_________")
  console.debug("_________")

So this is my code... and this is my log:

0_________
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
3_________
_________
_________
_________

0_________
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
3_________
_________
_________
_________

0_________
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
3_________
_________
_________
_________

0_________
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
"If there is any risk that, pending the hearing of the petition, records may be lost or destroyed, that [object Object]
3_________
_________
_________
_________

D

dave•1/4/24, 2:11 AM

what on earth is going on?

C

Cyb3r-Jak3•1/4/24, 2:15 AM

There is a bug open related to log messages with

at

at

at

at in them. I know there is a fix PR up but don't remember the number

U

Unsmart•1/4/24, 2:16 AM

https://github.com/cloudflare/workers-sdk/pull/4683

GitHub

fix: ensure logs containing `at ` not truncated to `at [object Obje...

Fixes #4668.
What this PR solves / how to test:
Previously, logs containing at were always treated as stack trace call sites requiring source mapping. This change updates the call site detection t...

UUnsmart https://github.com/cloudflare/workers-sdk/pull/4683

D

dave•1/4/24, 2:19 AM

@Cyb3r-Jok3 thank you! Thought I was going crazy.

Is there a temp workaround?

U

Unsmart•1/4/24, 2:19 AM

do a replaceAll on console.log for "at" and replace with "@" or something

D

dave•1/4/24, 2:19 AM

lol

D

dave•1/4/24, 2:20 AM

hmm, downgrade to

3.19.0

3.19.0

3.19.0

3.19.0 is my only safe option right?

D

dave•1/4/24, 2:29 AM

I had a panic moment thinking I had incorrectly stored JSON in >100,000 files.

N

nuno•1/4/24, 11:34 AM

Heyo guys!
I'm developing/testing a worker project using the

wrangler dev --remote

wrangler dev --remote

wrangler dev --remote

wrangler dev --remote command, but it seems like my DOs aren't updating on the network, that's is intentional? Am I doing something wrong?

Hope someone could help me figure it this out. Thanks in advance.

R

richtera•1/4/24, 3:18 PM

Api tokens are still not associated with specific pages or workers projects. This is a rather large security problem when using gha or other build systems, because it allows any developer to deploy on any wrangler project with or without a branch tag.

R

Ranjan Mangla•1/4/24, 5:25 PM

I want to do some file operations using ffmpeg as soon as a file (it should only trigger when the uploaded file is mp4), is uploaded to my storage bucket, and then want to upload the resulting files to another R2 storage bucket using workers

I have gone through the doc, but can't figure out how to do it. can someody please help

R

Ranjan Mangla•1/4/24, 5:25 PM

I am migrating a script from aws lambda to cloudflare, it was effortless in aws, but cannot figure it out here

K

kian•1/4/24, 5:31 PM

Workers are just JS and can’t spawn child processes, ffmpeg isn’t going to work

Looks similar to what I'm seeing - others have reported it at https://github.com/cloudflare/workerd/

Similar Threads