oh i think I see... have to write to the writable and then the data becomes readable

Ddanny.m oh i think I see... have to write to the writable and then the data becomes read...

S

Sid•10/19/23, 6:23 PM

You should also be able to pipe your stream through a FixedLengthStream before sending it to R2

D

danny.mOP•10/19/23, 7:01 PM

so i was essentially giving that a shot... not sure if I'm doing things incorrectly, but for some reason my write statements seem to not be awaitable... will drop my code here in a sec

D

danny.mOP•10/19/23, 7:02 PM

while (true) {
  let foo = await reader.read();
  if (foo.value) {
  console.log(foo.value)
  let md = metadataExtractor.extract(foo.value)
  if (md.val) {
    let encoder = new TextEncoder()
    let strung = encoder.encode(JSON.stringify(md.val))
    console.log("MD", strung)
    console.log(strung.length)
    let {readable, writable} = new FixedLengthStream(strung.length)
    let writer = writable.getWriter()
    await writer.write(strung).then(e => console.log({e}))
    await env.CONTENT_BUCKET.put(`${key}-md`, readable)
    console.log("PUT METADATA IN R2")
  }
}

while (true) {
  let foo = await reader.read();
  if (foo.value) {
  console.log(foo.value)
  let md = metadataExtractor.extract(foo.value)
  if (md.val) {
    let encoder = new TextEncoder()
    let strung = encoder.encode(JSON.stringify(md.val))
    console.log("MD", strung)
    console.log(strung.length)
    let {readable, writable} = new FixedLengthStream(strung.length)
    let writer = writable.getWriter()
    await writer.write(strung).then(e => console.log({e}))
    await env.CONTENT_BUCKET.put(`${key}-md`, readable)
    console.log("PUT METADATA IN R2")
  }
}

while (true) {
  let foo = await reader.read();
  if (foo.value) {
  console.log(foo.value)
  let md = metadataExtractor.extract(foo.value)
  if (md.val) {
    let encoder = new TextEncoder()
    let strung = encoder.encode(JSON.stringify(md.val))
    console.log("MD", strung)
    console.log(strung.length)
    let {readable, writable} = new FixedLengthStream(strung.length)
    let writer = writable.getWriter()
    await writer.write(strung).then(e => console.log({e}))
    await env.CONTENT_BUCKET.put(`${key}-md`, readable)
    console.log("PUT METADATA IN R2")
  }
}

while (true) {
  let foo = await reader.read();
  if (foo.value) {
  console.log(foo.value)
  let md = metadataExtractor.extract(foo.value)
  if (md.val) {
    let encoder = new TextEncoder()
    let strung = encoder.encode(JSON.stringify(md.val))
    console.log("MD", strung)
    console.log(strung.length)
    let {readable, writable} = new FixedLengthStream(strung.length)
    let writer = writable.getWriter()
    await writer.write(strung).then(e => console.log({e}))
    await env.CONTENT_BUCKET.put(`${key}-md`, readable)
    console.log("PUT METADATA IN R2")
  }
}

D

danny.mOP•10/19/23, 7:03 PM

A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.
A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.

A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.
A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.

A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.
A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.

A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.
A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.

D

danny.mOP•10/19/23, 7:04 PM

was playing around with promisifying the

write

write

write

write statement and then using

then

then

then

then for the r2 upload... was seeming to work if I never await that promise, but I'd like the upload to occur before sending my response downstream... unsure why it determines that the promise will enver resolve

D

danny.mOP•10/19/23, 7:10 PM

yeah so weirdly this works...

D

danny.mOP•10/19/23, 7:11 PM

writer.write(strung).then(() => {
  env.CONTENT_BUCKET.put(`${key}-md`, readable)
}))

writer.write(strung).then(() => {
  env.CONTENT_BUCKET.put(`${key}-md`, readable)
}))

writer.write(strung).then(() => {
  env.CONTENT_BUCKET.put(`${key}-md`, readable)
}))

writer.write(strung).then(() => {
  env.CONTENT_BUCKET.put(`${key}-md`, readable)
}))

D

danny.mOP•10/19/23, 7:11 PM

and then I'm just never awaiting that promise... kindof odd

D

danny.mOP•10/19/23, 8:00 PM

no

D

danny.mOP•10/19/23, 8:00 PM

im parsing information out of it and putting that in r2

D

danny.mOP•10/19/23, 8:02 PM

im not on a free plan

D

danny.mOP•10/19/23, 8:02 PM

it's odd, making it thenable and trying to log errors, not getting any info about why my promises will never resolve

D

danny.mOP•10/19/23, 8:03 PM

im using FixedLengthStream

D

danny.mOP•10/19/23, 8:05 PM

it never responds when i don't use an await

D

danny.mOP•10/19/23, 8:07 PM

which makes sense to me... .write returns a promise

D

danny.mOP•10/19/23, 8:09 PM

it does write to r2

D

danny.mOP•10/19/23, 8:09 PM

but it doensn't respond

D

danny.mOP•10/19/23, 8:11 PM

i just didn't include this

D

danny.mOP•10/19/23, 8:11 PM

if (foo.done) {
                        break
                    }
                }
                metadataExtractor.clear()
                await bucketUpload;
                await env.CONTENT_BUCKET.put(`${key}-md`, readable)
        return new Response(`Put ${key} successfully! with count of ${count}`);

if (foo.done) {
                        break
                    }
                }
                metadataExtractor.clear()
                await bucketUpload;
                await env.CONTENT_BUCKET.put(`${key}-md`, readable)
        return new Response(`Put ${key} successfully! with count of ${count}`);

if (foo.done) {
                        break
                    }
                }
                metadataExtractor.clear()
                await bucketUpload;
                await env.CONTENT_BUCKET.put(`${key}-md`, readable)
        return new Response(`Put ${key} successfully! with count of ${count}`);

if (foo.done) {
                        break
                    }
                }
                metadataExtractor.clear()
                await bucketUpload;
                await env.CONTENT_BUCKET.put(`${key}-md`, readable)
        return new Response(`Put ${key} successfully! with count of ${count}`);

D

danny.mOP•10/19/23, 8:11 PM

it breaks out

D

danny.mOP•10/19/23, 8:15 PM

seems i needed to close my writer after

D

danny.mOP•10/19/23, 8:15 PM

writer.close()

C

Chris•10/20/23, 12:49 AM

Any R2 product managers/engineers in this channel? What's the official process for filing a bug with R2?

K

kian•10/20/23, 12:50 AM

Posting it here means an engineer might see it, but the best route is through a support ticket.

C

cloneth•10/20/23, 3:03 PM

Hey, I'm having problems with R2

Ccloneth Hey, I'm having problems with R2

C

Chaika•10/20/23, 3:08 PM

Can you be a bit more specific?

GGuyzeroth Hello, I am using the R2 multipart upload as per https://developers.cloudflare.c...

C

Chaika•10/20/23, 3:09 PM

wrangler 3.x dev uses local mode by default, which uses a local/simulated r2 bucket for testing. If you want to use your remote/real resources, you can use

wrangler dev --remote

wrangler dev --remote

C

Chaika•10/20/23, 3:12 PM

As far as I know, no. The simulation is just of a the bucket via the worker api, not of the entire s3 api. You could use

preview_bucket_name

preview_bucket_name

preview_bucket_name

preview_bucket_name which is what --remote would use, and have a seperate remote testing bucket that doesn't mess with your production data
https://developers.cloudflare.com/workers/wrangler/configuration/#r2-buckets

CChaika Can you be a bit more specific?

C

cloneth•10/20/23, 3:14 PM

We are trying to distribute movies with R2 for a movie site. We continued like this for a long time, there were no problems. However, our video player has not been working properly for 2 days due to problems caused by R2.

Ccloneth We are trying to distribute movies with R2 for a movie site. We continued like t...

C

Chaika•10/20/23, 3:18 PM

Do you get a specific error from R2 or any more details on how it's not working and you think it's R2?

CChaika Do you get a specific error from R2 or any more details on how it's not working ...

C

cloneth•10/20/23, 3:26 PM

We get these errors in the playerjs video player and we cannot directly access this link https://pub-a12f06d0f6d04269a5f5c378d1a005ab.r2.dev/Anizium%2FAnimeler%2Fdeath-note/death-note-1-1-2160-jp.mp4, but we can access it when we use VPN.

WhatsApp_Gorsel_2023-10-19_saat_22.56.33_fa3d4204.jpg

Ccloneth We get these errors in the playerjs video player and we cannot directly access t...

C

Chaika•10/20/23, 3:28 PM

The r2.dev domains are not recommend for production and have rate limiting on them

Public access through r2.dev subdomains are rate limited and should only be used for development purposes.

Connect a custom domain to your bucket: https://developers.cloudflare.com/r2/buckets/public-buckets/#connect-a-bucket-to-a-custom-domain
For example, cdn.yourwebsite.com, and then change your app to use that, same url structure

Public buckets · Cloudflare R2 docs

Public Bucket is a feature that allows users to expose the contents of their R2 buckets directly to the Internet. By default, buckets are never …

C

Chaika•10/20/23, 3:29 PM

That being said, that looks more like a block. You're from turkey? I think they might have banned r2.dev there. Using your Custom Domain should help with that eitherway

C

cloneth•10/20/23, 3:34 PM

Thank you very much. Yes I'm in Turkey. So why do you think it was banned in Turkey? I am of the same opinion.

C

Chaika•10/20/23, 3:51 PM

Are you asking how I came to the idea it might be banned? The specific ssl error you got, and some test online that checks if something is blocked from turkey also shows it.
Eitherway, try adding an r2 custom domain and using that, and then see

C

cloneth•10/20/23, 4:09 PM

well we have a custom domain called "cdn.anizium.com". But we don't know how to use it. Is there a suitable article on this subject?

Ccloneth well we have a custom domain called "cdn.anizium.com". But we don't know how to ...

H

Hello, I’m Allie!•10/20/23, 4:11 PM

Custom Domains

Public buckets · Cloudflare R2 docs

Public Bucket is a feature that allows users to expose the contents of their R2 buckets directly to the Internet. By default, buckets are never …

HHello, I’m Allie![Custom Domains](https://developers.cloudflare.com/r2/buckets/public-buckets/#cu...

C

cloneth•10/20/23, 4:15 PM

thank you so much

D

danny.mOP•10/20/23, 4:36 PM

is there a good way to debug the following? trying to run

wrangler dev --remote

wrangler dev --remote

which was working yesterday, but today I get an error. I have a bucket for local development that it should be talking to

D

danny.mOP•10/20/23, 4:36 PM

✘ [ERROR] Error on remote worker: ParseError: A request to the Cloudflare API (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview) failed.

      at throwFetchError
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121307:17)
      at fetchResult
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121259:5)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async createPreviewToken
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150628:29)
      at async createWorkerPreview
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150649:17)
      at async start
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150855:34)
  {
    text: 'A request to the Cloudflare API
  (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview)
  failed.',
    notes: [ { text: 'workers.api.error.internal_server [code: 10002]' } ],
    location: undefined,
    kind: 'error',
    code: 10002
  }

✘ [ERROR] Error on remote worker: ParseError: A request to the Cloudflare API (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview) failed.

      at throwFetchError
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121307:17)
      at fetchResult
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121259:5)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async createPreviewToken
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150628:29)
      at async createWorkerPreview
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150649:17)
      at async start
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150855:34)
  {
    text: 'A request to the Cloudflare API
  (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview)
  failed.',
    notes: [ { text: 'workers.api.error.internal_server [code: 10002]' } ],
    location: undefined,
    kind: 'error',
    code: 10002
  }

✘ [ERROR] Error on remote worker: ParseError: A request to the Cloudflare API (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview) failed.

      at throwFetchError
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121307:17)
      at fetchResult
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121259:5)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async createPreviewToken
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150628:29)
      at async createWorkerPreview
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150649:17)
      at async start
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150855:34)
  {
    text: 'A request to the Cloudflare API
  (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview)
  failed.',
    notes: [ { text: 'workers.api.error.internal_server [code: 10002]' } ],
    location: undefined,
    kind: 'error',
    code: 10002
  }

✘ [ERROR] Error on remote worker: ParseError: A request to the Cloudflare API (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview) failed.

      at throwFetchError
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121307:17)
      at fetchResult
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:121259:5)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async createPreviewToken
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150628:29)
      at async createWorkerPreview
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150649:17)
      at async start
  (/Users/interpretations/projects/extraction/dark-voice-8281/node_modules/wrangler/wrangler-dist/cli.js:150855:34)
  {
    text: 'A request to the Cloudflare API
  (/accounts/61d49c021d802c88abdb553e6f9342dd/workers/scripts/dark-voice-8281/edge-preview)
  failed.',
    notes: [ { text: 'workers.api.error.internal_server [code: 10002]' } ],
    location: undefined,
    kind: 'error',
    code: 10002
  }

D

danny.mOP•10/20/23, 4:37 PM

wrangler whoami

wrangler whoami

wrangler whoami

wrangler whoami knows who i am

D

danny.mOP•10/20/23, 4:39 PM

welp seems to have just stopped occuring anymore... odd

N

nathaniel•10/20/23, 4:41 PM

does serving files via public url count toward your quota of class a operations?

N

nathaniel•10/20/23, 6:10 PM

thanks

N

nathaniel•10/20/23, 6:10 PM

any way to replicate list objects functionality through a public url?

Nnathaniel any way to replicate list objects functionality through a public url?

K

Karew•10/20/23, 6:20 PM

It’s intentionally not offered AFAIK

K

Karew•10/20/23, 6:21 PM

The public domain is for serving individual files through the whole Cloudflare proxy/CDN with no S3 API in the way

K

Karew•10/20/23, 6:21 PM

If you keep track of your file uploads in a database or something though you can “list” them that way

oh i think I see... have to write to the writable and then the data becomes readable

Similar Threads

Similar Threads

Similar Threads