10 replies

Batching - What triggers a batch being passed to resolver, and ability to have concurrent batches

I'm curious how Batching works under the hood. I've been experimenting with batching support for AWS clients, some of which have a 'batch' ability for certain commands.

For example, with DynamoDB, one can either submit a single 'get' request, or a batched get request, where multiple gets are submitted and returned from a single API call. This seems like a good candidate for batching, where we model the single request, but provide a batch resolver.

What isn't clear from the documentation or Effect API is when are individual requests passed to the resolver? For example:

    const requests = [
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition1', SK: 'sort1' } })),
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition2', SK: 'sort2' } })),
    ]
    const result = await Effect.all(requests, { batching: true }).pipe(
      Effect.provide(DynamoDBDocument.layer({ marshallOptions: { removeUndefinedValues: true } })),
      Effect.runPromiseExit,
    )

    const requests = [
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition1', SK: 'sort1' } })),
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition2', SK: 'sort2' } })),
    ]
    const result = await Effect.all(requests, { batching: true }).pipe(
      Effect.provide(DynamoDBDocument.layer({ marshallOptions: { removeUndefinedValues: true } })),
      Effect.runPromiseExit,
    )

In this case, the resolver has been given a

batchN(100)

batchN(100)

which is the service limit for DynamoDB, per batch request. Given that I could be generating the requests from separate fibers (e.g. a concurrent forEach over some input data, where each item needs an item from DynamoDB to process), it's not clear how the batching mechanism knows that there are no more requests before handing the batch to the resolver. If the upstream producers of the GetItem requests were slow, it might be desirable to send a partial batch, and indeed, it might then be desirable to have multiple batches being concurrently resolved (which seems against the resolver design as it states that batches are processed sequentially). In effect, alongside

batchN

batchN

there is a case for

batchWindow(duration)

batchWindow(duration)

that specifies the maximum time to elapse after the first request before the batch is resolved.

TIA

Batching - What triggers a batch being passed to resolver, and ability to have concurrent batches

    const requests = [
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition1', SK: 'sort1' } })),
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition2', SK: 'sort2' } })),
    ]
    const result = await Effect.all(requests, { batching: true }).pipe(
      Effect.provide(DynamoDBDocument.layer({ marshallOptions: { removeUndefinedValues: true } })),
      Effect.runPromiseExit,
    )

    const requests = [
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition1', SK: 'sort1' } })),
      GetItem(GetItemRequest({ TableArn: testTableArn, Key: { PK: 'partition2', SK: 'sort2' } })),
    ]
    const result = await Effect.all(requests, { batching: true }).pipe(
      Effect.provide(DynamoDBDocument.layer({ marshallOptions: { removeUndefinedValues: true } })),
      Effect.runPromiseExit,
    )

In this case, the resolver has been given a

batchN(100)

batchN(100)

batchN

batchN

there is a case for

batchWindow(duration)

batchWindow(duration)

that specifies the maximum time to elapse after the first request before the batch is resolved.

TIA

Batching - What triggers a batch being passed to resolver, and ability to have concurrent batches

Batching - What triggers a batch being passed to resolver, and ability to have concurrent batches

Similar Threads

Similar Threads

Similar Threads