hello guys need help uploading images to R2 all at instead of uploading one by one. please suggest t
hello guys need help uploading images to R2 all at instead of uploading one by one. please suggest the way to upload images at once.
Jyou can upload a folder using cloudflare dashboard
NI thought of doing that using promise.all but what if some images failed to upload and it throw error. so the images will get uploaded will be there for life till I manually delete the uploaded image manually. what is the best approach to handle this scenario.
basically I have a travel package model where I am uploading images and package details.
basically I have a travel package model where I am uploading images and package details.
Nsir have to do it with code for client.
Jcloudflarestorage.com subdomains of this domain isn't accessible with some indian ISPs like Jio, if you are locally using Jio or related ISP you have to use vpn or proxy
JYou can do promise all with cloudflare workflows steps, it has auto retry mechanism
Nnice sir. could you please share article or doc link to achieve this
JYou don't have to use VPN in VPS server cause they will be able to access it, as they do not have any content restrictions
Jit's only ISP Jio who occasionally bans cloudflare domains and ips
JCloudflare Docs
Workflows is a durable execution engine built on Cloudflare Workers. Workflows allow you to build multi-step applications that can automatically retry, persist state and run for minutes, hours, days, or weeks. Workflows introduces a programming model that makes it easier to build reliable, long-running tasks, observe as they progress, and progra...
Nsir is this like queue system?
JNo you can read more about it on the official site
Nokay sir thank you so much for the help and time.
GAlternative method would be to clear that directory or bucket after a specific amount of time, depending on the use case.
I have a system for user uploads that gets moved after they are processed and otherwise they are deleted after 7 days by the bucket retention.
In other projects I take a different approach
I have a system for user uploads that gets moved after they are processed and otherwise they are deleted after 7 days by the bucket retention.
In other projects I take a different approach
GNot for video streaming, they have a separate service for that.
But for other files yes it's purely the amount of reads/writes you use that gets billed
If you use the cdn caching that basically becomes negligible so you really just pay for storage at that point
But for other files yes it's purely the amount of reads/writes you use that gets billed
If you use the cdn caching that basically becomes negligible so you really just pay for storage at that point
GYea but I don't think video streaming is allowed on the cdn
GWeird, I swear the TOD said that video streaming specifically wasn't allowed on the CDN but then again it's been pretty back and forth in the discussions I've read.
Even the updated TOS still seems to be confusing
Even the updated TOS still seems to be confusing
TAny idea why
const data = await env.R2_BUCKET.get(objectKey, {
range: { offset: part.start, length: part.length }
});
if (!data || !data?.body) {
logger.error({ partNumber: part.number, offset: part.start, length: part.length }, 'Missing part data');
throw new Error();
}
i get thrown cause data won't return, it works locally !
i also tried to retry 5 time with backoff and still nothing
const data = await env.R2_BUCKET.get(objectKey, {
range: { offset: part.start, length: part.length }
});
if (!data || !data?.body) {
logger.error({ partNumber: part.number, offset: part.start, length: part.length }, 'Missing part data');
throw new Error();
}
i get thrown cause data won't return, it works locally !
i also tried to retry 5 time with backoff and still nothing
VIs there a way to see how much bandwidth is being used per file/folder? Or is it total? Or is there some way for me to collect logs when a file gets hit?
I basically allow users to upload various files and currently there's no bandwidth limit ofc but I want to be able to see which users are heavy users in the case that something needs to happen so knowing which user is using up the most bandwidth would be nice to know
I basically allow users to upload various files and currently there's no bandwidth limit ofc but I want to be able to see which users are heavy users in the case that something needs to happen so knowing which user is using up the most bandwidth would be nice to know
VAnd just for reference, I serve my r2 files via a custom URL so it will hit the cache
VI'm on their pro plan actually!
VGotcha, so just to clarify, I'm looking at the dashboard under cache and I can see the paths which got the most traffic, so that's fine, however in my case I'd like to show this to my user as a "Here's how much bandwidth you've used this month" kind of statistic. So for each user their various files would have a path of like '/{user_id}/{folder_id}/somefile.jpg' which means I just need to aggregate the banwdith used by all paths that start with the specific user's ID.
Is it possible to retrieve that info via some sort of API?
Is it possible to retrieve that info via some sort of API?
VAh I see, but there's no dedicated simple way I guess? Also I know nothing about graphql tbh lol
I used to use BunnyCDN and they have a log forwarding option so it was pretty easy to figure out how many times a file was viewed, etc, by forwarding the logs to a simple $5 VPS and parsing them and adding them to a db
I used to use BunnyCDN and they have a log forwarding option so it was pretty easy to figure out how many times a file was viewed, etc, by forwarding the logs to a simple $5 VPS and parsing them and adding them to a db
VOh yeah I wouldn't let them interact with r2 directly, I just want to build an admin dashboard so I can keep track of which user is using how much bandwidth and reach out to them if needed, etc
Sim getting a CORS error.
[
{
"AllowedOrigins": [
"https://test.com",
],
"AllowedMethods": [
"GET",
"DELETE",
"POST",
"PUT",
"HEAD"
],
"AllowedHeaders": [
"Authorization",
"x-amz-date",
"x-amz-content-sha256",
"Content-Type",
"Length",
"Content-Length",
"content-type"
],
"ExposeHeaders": [
"ETag",
"Location",
"Access-Control-Allow-Origin",
"Content-Type",
"Content-Length"
],
"MaxAgeSeconds": 3000
}
]
i have no idea why. i adjusted my cors rules for my bucket. I simply send a signed transaction from my server to the client and the client just cannot use it due to this
[
{
"AllowedOrigins": [
"https://test.com",
],
"AllowedMethods": [
"GET",
"DELETE",
"POST",
"PUT",
"HEAD"
],
"AllowedHeaders": [
"Authorization",
"x-amz-date",
"x-amz-content-sha256",
"Content-Type",
"Length",
"Content-Length",
"content-type"
],
"ExposeHeaders": [
"ETag",
"Location",
"Access-Control-Allow-Origin",
"Content-Type",
"Content-Length"
],
"MaxAgeSeconds": 3000
}
]
i have no idea why. i adjusted my cors rules for my bucket. I simply send a signed transaction from my server to the client and the client just cannot use it due to this
Sany help please
VSo I'm not sure if you remember but you helped me with learning about HLS and serving video files (thanks for that)
I generate the manifest on the fly because I use waf so every time a manifest is requested, I know that the video has been played (or at least preloaded/requested/whatever). But what I don't know is how much bandwidth they used, because if the video file is 1GB lets say and they only watched 1 minute before exiting out, they'll incur less bandwidth than a video file that's 50MB but a user watch the whole thing, even though I've recorded 1 play for each file
I generate the manifest on the fly because I use waf so every time a manifest is requested, I know that the video has been played (or at least preloaded/requested/whatever). But what I don't know is how much bandwidth they used, because if the video file is 1GB lets say and they only watched 1 minute before exiting out, they'll incur less bandwidth than a video file that's 50MB but a user watch the whole thing, even though I've recorded 1 play for each file
VSo with the way it's being done, I can't do tracking at the layer closest to the user because the manifest file (which is the only 'file' that my server returns) simply links to direct files within the r2 bucket
VYeah bandwidth is not a concern, but I did read some stories about how some guy got reached out by cloudflare to hop on a large contract because he was using like 1-3 petabytes of data
Now I don't see myself even coming close to that, but in the off chance that some power house user uses my service to host their 1GB file that suddenly gets 1 million views, that's a petabyte of data right there, so if cloudflare reaches out to increase costs, I'd like to know who's using all the bandwidth and figure out extra pricing, etc
Now I don't see myself even coming close to that, but in the off chance that some power house user uses my service to host their 1GB file that suddenly gets 1 million views, that's a petabyte of data right there, so if cloudflare reaches out to increase costs, I'd like to know who's using all the bandwidth and figure out extra pricing, etc
VAnd they're still on the normal plan? They aren't being asked to pay more?
Vwow
Sim still getting the cors error....
VThat's insane haha, I mean like I'm just thinking about worst case scenario because I'm planning on charging a fixed price per 100GB lets say, but I'm not specifying bandwidth limits. But if some power user manages to get their video popular overnight and it gets like 10 million views, that's a lot of bandwidth, so I just want to try to be ready for a scenario like that if cloudflare comes knocking asking for a big contract when I'm still only charging them like a flat $10 a month
VBut yeah I've set up some pretty agressive caching and I've served 8GB so far and 7.2GB has been through the cache so I think it's setup well enough
VThey sure did haha, okay I guess it's a problem for future me and I don't need to look into tracking user bandwidth just yet
VIf the day comes, it'll be impressive if anything lol
Mcache shows what "none" means
AHi, I have a webapp playing video out of R2. I saw something odd that I couldn't find a reason for today. I was developing on a branch (and pushing to a preview URL ending in .dev). My video playback was a little slow, some requests for content seemed to hang. I merged it to my main branch (and my real domain ending in .com) and it seems fine. Is there some r2 throttling going on when the referrer is a .dev domain? Explicitly this isn't a dev R2 domain. I'm using the regular S3 API endpoint (ending in r2.cloudflarestorage.com).
Aguess I should give that a go, sensible suggestion, i'll try it out, thanks!
VIs there a standardized way of uploading files to R2? Especially large files? Currently I just generate a pre signed url and send that to the frontend but I want to handle scenarios where a users internet might cut out, or the upload fails for some reason, etc. is there a way to show progress or have the files be automatically cleaned up if it's stopped halfway or something?
Ghi. I am using R2 general storage to store images for my website. I have a lot of images and I want to give logged users all privileges to access any image. What would be the best way to do it? I dont want signed url for each image and if I understand correctly, you can not get signed url for whole repo. Right now I am using the workers to authenticate by JWT and then server the image. But for this I have to get the image blob using fetch in the frontend and then display the blob. Is there any way I can use <img> tags in the frontend to show images?
Bgetting this for uploading files from the client side:
Bmy cors policy:
Bthats just what aws-sdk gave me
B
Bnope didn't work
Bi removed ACL too
Bweird, now it uploads but the file is unplayable lmfao