@john.spurlock for visibility on the listing fixes.
@john.spurlock for visibility on the listing fixes.
K
Jnice, thanks!
Jsemi-permeable code freeze
BOh & deletion of multiple objects should now be faster too.
BCode freeze window ended this morning
Jright before the holiday eh
BSo I looked into it and there's this:
https://stackoverflow.com/questions/69899949/upload-with-high-level-multipart-still-gives-no-content-length-specified-warning
This indicates it is indeed required even for S3. I haven't figured out a good way to actually test out omitting the content-length
https://stackoverflow.com/questions/69899949/upload-with-high-level-multipart-still-gives-no-content-length-specified-warning
This indicates it is indeed required even for S3. I haven't figured out a good way to actually test out omitting the content-length
Stack Overflow
Even though I'm using the high level multipart I'm still getting the warning in the console:
WARN - com.amazonaws.services.s3.AmazonS3Client - No content length specified for stream data. Stream
WARN - com.amazonaws.services.s3.AmazonS3Client - No content length specified for stream data. Stream
BGitLab
New description proposal Based on gitlab-org/gitlab-ee#4184 we know that only GCS (Google Cloud Storage) accepts chunked-encoding, all the other S3-compatible
BBased on gitlab-org/gitlab-ee#4184 we know that only GCS (Google Cloud Storage) accepts chunked-encoding, all the other S3-compatible Object Storage requires Content-Length HTTP header.
BThat's not part of S3 AFAIK.
BSo you're saying that GitLab is working for you with AWS S3 & not with R2?
BOr are you using GitLab with GCS?
BDo you make sure to disable
enable_signature_v4_streaming ?BStreaming Sigv4 is the only part of the spec I see that potentially omits Content-Length
B(although you still provide the length, just via the
x-amz-decoded-content-length header)BDM me your account ID so I can double-check
BI'm seeing some 200 puts from your end
Bso not all are failing
BAll your multipart uploads are OK
BI don't know what the issue is but it doesn't seem like it's on the R2 side of things
BI do notice that the successful puts aren't presigned
Bso maybe that's logpush
BI would recommend reaching out to GitLab support
BAnother small release for helping out with Terraform:
- S3 CreateBucket request can specify
x-amz-bucket-object-lock-enabledwith a value of
and not have the requested rejected with afalseNotImplementederror. A value of
will continue to be rejected as R2 does not yet support object locks.true
CJust tested and it kinda of works but not really. You can create the bucket but the plan doesn't successfully complete and you can't refresh the plan or destroy the bucket. It tries to check the bucket ACL via
GetBucketACL which isn't implemented.BIs there any way to skip that?
CIt doesn't look like it. When reading a bucket resource it seems to always attempt to read the ACL.
https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/service/s3/bucket.go#L964-L970
https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/service/s3/bucket.go#L964-L970
BMight make sense to fix the terraform provider to be more graceful about servers not implementing various features? It handles empty responses so why not treat notimplemented as empty
CSounds good. I'll raise an issue there
BThe (potentially incomplete) set of commands that don't handle
NotImplemented gracefully:- GetBucketAcl
- GetBucketVersioning
- GetBucketLifecycleConfiguration
- GetBucketReplication
- GetBucketEncryption (but R2 does implement it)
- GetObjectLockConfiguration
- GetBucketTagging
BThe rest seem to gracefully ignore
NotImplementedCIssue raised: https://github.com/hashicorp/terraform-provider-aws/issues/25660
Thanks for your help on this!
Thanks for your help on this!
S?r2-roadmap
FThe following features are currently on the roadmap for R2 but not yet implemented.Pre-signed URLs are now implemented!
- Public Buckets
- Mapping a bucket to your custom domain
In the meantime, you can use a Worker to replicate the vast majority of functionality such as serving & caching public assets.
Take a look at the Demo Worker example here: https://developers.cloudflare.com/r2/examples/demo-worker/
Also, there's a great community project called Render that's made for R2:
https://github.com/kotx/render
Workers will not be required for making assets publicly available in the future, they are just an interim stop-gap.
Pany news on when we'll be able to upload single objects with more concurrency?
BNot sure.
KWhere is the ReadableStream coming from? If it’s the body of a fetch / request and it has a Content-Length then it shouldn’t be an issue
