Is there a performance penalty to enabling node_compat? Just wondering the reason for it not being e

TTitan Is there a performance penalty to enabling node_compat? Just wondering the reaso...

J

James•4/6/24, 10:04 PM

node_compat

node_compat

as a top-level option is legacy and should be avoided whenever possible.

J

James•4/6/24, 10:04 PM

nodejs_compat

nodejs_compat

as a compat flag is good and won't incur any kind of performance issues

A

Arnø•4/6/24, 10:04 PM

Hi, what would you recommend for importing a 1GB txt file from a cron to populate my D1 database? So that it doesn't exceed the resource limits.

J

johtso•4/6/24, 10:04 PM

oh wait no ignore that.. the ports just got swapped so I was hitting the other worker.. still not working

JJames Did you try giving the entrypoint a name with `entrypoint`?

J

johtso•4/6/24, 10:07 PM

I did try exporting it with a name, I'll give that another try though

J

johtso•4/6/24, 10:11 PM

@James I get this message when running the dev server

▲ [WARNING] This worker is bound to live services: PDFPARSER_SERVICE (pdfparser#PdfParserService)

▲ [WARNING] This worker is bound to live services: PDFPARSER_SERVICE (pdfparser#PdfParserService)

▲ [WARNING] This worker is bound to live services: PDFPARSER_SERVICE (pdfparser#PdfParserService)

▲ [WARNING] This worker is bound to live services: PDFPARSER_SERVICE (pdfparser#PdfParserService)

J

johtso•4/6/24, 10:11 PM

which seems promising

J

johtso•4/6/24, 10:11 PM

but then the Illegal invocation error

J

James•4/6/24, 10:11 PM

How are you calling it? Can you share a snippet?

JJames How are you calling it? Can you share a snippet?

J

johtso•4/6/24, 10:12 PM

async function membershipRequestHandler (request: Request, env: Env) {
    const body = await request.arrayBuffer();

    console.log(`Parsing PDF size ${body.byteLength / 1024 / 1024} MBs`)

    const result = await env.PDFPARSER_SERVICE.parsePdf(body);

async function membershipRequestHandler (request: Request, env: Env) {
    const body = await request.arrayBuffer();

    console.log(`Parsing PDF size ${body.byteLength / 1024 / 1024} MBs`)

    const result = await env.PDFPARSER_SERVICE.parsePdf(body);

async function membershipRequestHandler (request: Request, env: Env) {
    const body = await request.arrayBuffer();

    console.log(`Parsing PDF size ${body.byteLength / 1024 / 1024} MBs`)

    const result = await env.PDFPARSER_SERVICE.parsePdf(body);

async function membershipRequestHandler (request: Request, env: Env) {
    const body = await request.arrayBuffer();

    console.log(`Parsing PDF size ${body.byteLength / 1024 / 1024} MBs`)

    const result = await env.PDFPARSER_SERVICE.parsePdf(body);

J

johtso•4/6/24, 10:13 PM

export class PdfParserService extends WorkerEntrypoint<Env> {
  async fetch() { return new Response("Hello from Worker B"); }
  
  async parsePdf(data: ArrayBuffer) {
        return await pdf2array(Buffer.from(data));
    }
}

export class PdfParserService extends WorkerEntrypoint<Env> {
  async fetch() { return new Response("Hello from Worker B"); }
  
  async parsePdf(data: ArrayBuffer) {
        return await pdf2array(Buffer.from(data));
    }
}

export class PdfParserService extends WorkerEntrypoint<Env> {
  async fetch() { return new Response("Hello from Worker B"); }
  
  async parsePdf(data: ArrayBuffer) {
        return await pdf2array(Buffer.from(data));
    }
}

export class PdfParserService extends WorkerEntrypoint<Env> {
  async fetch() { return new Response("Hello from Worker B"); }
  
  async parsePdf(data: ArrayBuffer) {
        return await pdf2array(Buffer.from(data));
    }
}

J

James•4/6/24, 10:13 PM

And your toml is this?

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

J

johtso•4/6/24, 10:14 PM

exactly

J

James•4/6/24, 10:14 PM

Hmm, looks fine on the surface to me

J

johtso•4/6/24, 10:14 PM

I do have lots of linting errors in my TOML file.. I can't for a life of me remember what's responsible for that linting though

J

johtso•4/6/24, 10:15 PM

whatever it is doesn't know about

upload_source_maps

upload_source_maps

upload_source_maps

upload_source_maps or

services -> entrypoint

services -> entrypoint

services -> entrypoint

services -> entrypoint

J

johtso•4/6/24, 10:20 PM

@James from a bit of google I think it might actually have something to do with what I'm doing with the request body

J

johtso•4/6/24, 10:21 PM

I'll try just doing a hello world method on the service worker

J

johtso•4/6/24, 10:23 PM

nope, no luck with that either

K

kian•4/6/24, 11:18 PM

What's your full TOML?

Kkian What's your full TOML?

J

johtso•4/6/24, 11:26 PM

name = "newdo"
main = "src/index.ts"
compatibility_date = "2024-04-06"
compatibility_flags = [ "nodejs_compat" ]
logpush = true
upload_source_maps = true

[vars]
SERVICE_NAME = "datawrangler"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "memberships"
database_id = "xxxxxxxxxxx"
migrations_dir = "../database/src/migrations"

[[durable_objects.bindings]]
name = "MY_DURABLE_OBJECT"
class_name = "MyDurableObject"

[[migrations]]
tag = "v1"
new_classes = ["MyDurableObject"]

name = "newdo"
main = "src/index.ts"
compatibility_date = "2024-04-06"
compatibility_flags = [ "nodejs_compat" ]
logpush = true
upload_source_maps = true

[vars]
SERVICE_NAME = "datawrangler"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "memberships"
database_id = "xxxxxxxxxxx"
migrations_dir = "../database/src/migrations"

[[durable_objects.bindings]]
name = "MY_DURABLE_OBJECT"
class_name = "MyDurableObject"

[[migrations]]
tag = "v1"
new_classes = ["MyDurableObject"]

name = "newdo"
main = "src/index.ts"
compatibility_date = "2024-04-06"
compatibility_flags = [ "nodejs_compat" ]
logpush = true
upload_source_maps = true

[vars]
SERVICE_NAME = "datawrangler"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "memberships"
database_id = "xxxxxxxxxxx"
migrations_dir = "../database/src/migrations"

[[durable_objects.bindings]]
name = "MY_DURABLE_OBJECT"
class_name = "MyDurableObject"

[[migrations]]
tag = "v1"
new_classes = ["MyDurableObject"]

name = "newdo"
main = "src/index.ts"
compatibility_date = "2024-04-06"
compatibility_flags = [ "nodejs_compat" ]
logpush = true
upload_source_maps = true

[vars]
SERVICE_NAME = "datawrangler"

[[services]]
binding = "PDFPARSER_SERVICE"
service = "pdfparser"
entrypoint = "PdfParserService"

[[d1_databases]]
binding = "DB" # i.e. available in your Worker on env.DB
database_name = "memberships"
database_id = "xxxxxxxxxxx"
migrations_dir = "../database/src/migrations"

[[durable_objects.bindings]]
name = "MY_DURABLE_OBJECT"
class_name = "MyDurableObject"

[[migrations]]
tag = "v1"
new_classes = ["MyDurableObject"]

J

johtso•4/6/24, 11:27 PM

OH

J

johtso•4/6/24, 11:27 PM

I have an idea..

J

johtso•4/6/24, 11:30 PM

@kian aha! it was because I was instrumenting my worker with otel-cf-workers

J

johtso•4/6/24, 11:30 PM

makes total sense that that would break things

J

johtso•4/6/24, 11:31 PM

and now that I look at the stacktrace again I see it..

J

johtso•4/6/24, 11:32 PM

so I now get "Error: Cannot access

hello

hello

hello

hello as we couldn't find a

wrangler dev

wrangler dev

session for service "pdfparser" to proxy to"

S

Silent•4/6/24, 11:35 PM

How do i trigger a workers cron schedule code manually once?

W

Walshy•4/6/24, 11:47 PM

What I do is just have a fetch handler with some auth check and I just manually hit it

W

Walshy•4/6/24, 11:47 PM

In local dev I think you can hit like /__trigger_cron or something but prod doesn't have that

J

johtso•4/6/24, 11:47 PM

RPC is now working deployed! just not working locally with the dev server

J

johtso•4/6/24, 11:48 PM

not sure how wrangler tries to find the other dev session with the service.. but something ain't working

J

johtso•4/6/24, 11:48 PM

on a different note, is there a way to proxy the incoming request data to the service instead of serializing the data?

J

johtso•4/6/24, 11:49 PM

I'm handling files that exceed the 1MB limit

J

johtso•4/6/24, 11:49 PM

so just passing the request body ArrayBuffer as an argument to the service worker method doesn't work

J

johtso•4/6/24, 11:51 PM

the blog post says "You can send a byte stream over RPC, and the system will automatically stream the bytes with proper flow control."

J

johtso•4/6/24, 11:52 PM

I guess I just give it the whole

Response

Response

instead? the post doesn't seem to say

J

johtso•4/7/24, 12:04 AM

that works

J

johtso•4/7/24, 12:25 AM

also not sure what changed.. but it's working locally now too

J

johtso•4/7/24, 12:25 AM

the only thing that isn't working now is the types

B

boywonder350•4/7/24, 12:29 AM

Does anyone know if you need to read the

body

body

of a

fetch

fetch

response for it to be garbage collected properly? Sometimes I make requests but don't care about the body and want to make sure this is ok

U

user•4/7/24, 4:11 AM

I am suffering getting my CF pages site to be able to access my CF worker

U

user•4/7/24, 4:12 AM

cors my behated

A

addison•4/7/24, 6:50 AM

is anyone having issues with mailchannels on workers? getting

"Get \"http://customer-accounts-service.outbound.svc.cluster.local.:9292/v1/accounts/cloudflare\": read tcp 100.112.254.53:39318->100.70.108.250:9292: read: connection reset by peer"

"Get \"http://customer-accounts-service.outbound.svc.cluster.local.:9292/v1/accounts/cloudflare\": read tcp 100.112.254.53:39318->100.70.108.250:9292: read: connection reset by peer"

"Get \"http://customer-accounts-service.outbound.svc.cluster.local.:9292/v1/accounts/cloudflare\": read tcp 100.112.254.53:39318->100.70.108.250:9292: read: connection reset by peer"

"Get \"http://customer-accounts-service.outbound.svc.cluster.local.:9292/v1/accounts/cloudflare\": read tcp 100.112.254.53:39318->100.70.108.250:9292: read: connection reset by peer"

Is there a performance penalty to enabling node_compat? Just wondering the reason for it not being e

Similar Threads

Similar Threads

Similar Threads