Process a file is a little vague - what are you uploading and what do you want to do with it?

Kkian Process a file is a little vague - what are you uploading and what do you want t...

S

Socal•1/12/24, 12:01 AM

i want to send a csv or pdf to a cloudflare worker via a multipart/form-data post method. I then want to parse the file and do something with the result

K

kianOP•1/12/24, 12:02 AM

await request.formData() -> https://developer.mozilla.org/en-US/docs/Web/API/FormData

MDN Web Docs

FormData - Web APIs | MDN

The FormData interface provides a way to construct a set of key/value pairs representing form fields and their values, which can be sent using the fetch(), XMLHttpRequest.send() or navigator.sendBeacon() methods. It uses the same format a form would use if the encoding type were set to "multipart/form-data".

Kkian await request.formData() -> https://developer.mozilla.org/en-US/docs/Web/API/For...

S

Socal•1/12/24, 12:13 AM

ok, i can see the formData() response, but it doesn't have a get() method on it

K

kianOP•1/12/24, 12:16 AM

What's your code?

Kkian What's your code?

S

Socal•1/12/24, 12:18 AM

export default {
    async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
        const results: string[] = [];


        const options = {
            header: true,
            dynamicTyping: true,
        };


        const req = await request.formData();
        console.log(req);
        const result = papa.parse(req.get('blob'), options);
        console.log(result);

    
        return new Response('Hello World!');
    },
};

export default {
    async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
        const results: string[] = [];


        const options = {
            header: true,
            dynamicTyping: true,
        };


        const req = await request.formData();
        console.log(req);
        const result = papa.parse(req.get('blob'), options);
        console.log(result);

    
        return new Response('Hello World!');
    },
};

export default {
    async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
        const results: string[] = [];


        const options = {
            header: true,
            dynamicTyping: true,
        };


        const req = await request.formData();
        console.log(req);
        const result = papa.parse(req.get('blob'), options);
        console.log(result);

    
        return new Response('Hello World!');
    },
};

export default {
    async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
        const results: string[] = [];


        const options = {
            header: true,
            dynamicTyping: true,
        };


        const req = await request.formData();
        console.log(req);
        const result = papa.parse(req.get('blob'), options);
        console.log(result);

    
        return new Response('Hello World!');
    },
};

Kkian What's your code?

S

Socal•1/12/24, 12:19 AM

console.log(req) gives

FormData(1) {
  'blob' => File {
    lastModified: 1705018748508,
    name: 'csv_filename.csv',
    type: 'text/csv',
    size: 2007
  }
}

FormData(1) {
  'blob' => File {
    lastModified: 1705018748508,
    name: 'csv_filename.csv',
    type: 'text/csv',
    size: 2007
  }
}

FormData(1) {
  'blob' => File {
    lastModified: 1705018748508,
    name: 'csv_filename.csv',
    type: 'text/csv',
    size: 2007
  }
}

FormData(1) {
  'blob' => File {
    lastModified: 1705018748508,
    name: 'csv_filename.csv',
    type: 'text/csv',
    size: 2007
  }
}

K

kianOP•1/12/24, 12:20 AM

What makes you think there isn't a get() method though? Does the

papa.parse()

papa.parse()

call fail?

Kkian What makes you think there isn't a get() method though? Does the `papa.parse()` ...

S

Socal•1/12/24, 12:23 AM

ok, i did double check. it does have a get method that returns

File {
  lastModified: 1705018963031,
  name: 'csv_filename.csv',
  type: 'text/csv',
  size: 2007
}

File {
  lastModified: 1705018963031,
  name: 'csv_filename.csv',
  type: 'text/csv',
  size: 2007
}

File {
  lastModified: 1705018963031,
  name: 'csv_filename.csv',
  type: 'text/csv',
  size: 2007
}

File {
  lastModified: 1705018963031,
  name: 'csv_filename.csv',
  type: 'text/csv',
  size: 2007
}

. How do I access the file?

S

Socal•1/12/24, 12:24 AM

or what do i need to be sending into papaparse

K

kianOP•1/12/24, 12:25 AM

It's a

File

File

, which extends

Blob

Blob

, so there's methods like

arrayBuffer()

arrayBuffer()

,

stream()

stream()

and

text()

text()

K

kianOP•1/12/24, 12:26 AM

https://developer.mozilla.org/en-US/docs/Web/API/Blob

Kkian It's a `File`, which extends `Blob`, so there's methods like `arrayBuffer()`, `s...

S

Socal•1/12/24, 12:58 AM

thank you. I got it working. Apparently i needed to use text() text and then pass it into papaparse as papaparse tries to use FileReaderSync otherwise

R

Ranjan Mangla•1/12/24, 9:54 AM

How to setup. Cors in workers?

R

Ranjan Mangla•1/12/24, 9:55 AM

On doing a put request to my worker from react front end (store data in R2), I am getting cors error,

R

Ranjan Mangla•1/12/24, 9:55 AM

Tried setting up. Cors headers, still not working

RRanjan Mangla Tried setting up. Cors headers, still not working

H

Hello, I’m Allie!•1/12/24, 9:55 AM

Can you give us a code example to show what you are doing right now?

R

Ranjan Mangla•1/12/24, 9:59 AM

this is what i am doing:

export default {
  async fetch(request, env, ctx) {

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

export default {
  async fetch(request, env, ctx) {

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

export default {
  async fetch(request, env, ctx) {

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

export default {
  async fetch(request, env, ctx) {

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

RRanjan Mangla this is what i am doing: ``` export default { async fetch(request, env, ctx) {...

H

Hello, I’m Allie!•1/12/24, 10:01 AM

Oh, I see. Your browser is probably sending a

OPTIONS

OPTIONS

request to check for CORS headers, which you aren't handling, so it errors. Try adding an

if

if

statement for

OPTIONS

OPTIONS

requests that

return new Response(null, {
  headers: {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
    "Access-Control-Max-Age": "86400",
  }
});

return new Response(null, {
  headers: {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
    "Access-Control-Max-Age": "86400",
  }
});

return new Response(null, {
  headers: {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
    "Access-Control-Max-Age": "86400",
  }
});

return new Response(null, {
  headers: {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
    "Access-Control-Max-Age": "86400",
  }
});

and then see if it works.

Original message was deleted

H

Hello, I’m Allie!•1/12/24, 10:05 AM

https://tenor.com/view/wow-gif-20411229

Tenor

Original message was deleted

H

Hello, I’m Allie!•1/12/24, 10:06 AM

Thanks for the save!

R

Ranjan Mangla•1/12/24, 10:12 AM

still getting it , here is my code:

/**
 * Welcome to Cloudflare Workers! This is your first worker.
 *
 * - Run `npm run dev` in your terminal to start a development server
 * - Open a browser tab at http://localhost:8787/ to see your worker in action
 * - Run `npm run deploy` to publish your worker
 *
 * Learn more at https://developers.cloudflare.com/workers/
 */


export default {
  async fetch(request, env, ctx) {


    if (request.method === 'OPTIONS') {
        return new Response(null, {
          headers: {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
          }
        });
    }

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

/**
 * Welcome to Cloudflare Workers! This is your first worker.
 *
 * - Run `npm run dev` in your terminal to start a development server
 * - Open a browser tab at http://localhost:8787/ to see your worker in action
 * - Run `npm run deploy` to publish your worker
 *
 * Learn more at https://developers.cloudflare.com/workers/
 */


export default {
  async fetch(request, env, ctx) {


    if (request.method === 'OPTIONS') {
        return new Response(null, {
          headers: {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
          }
        });
    }

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

/**
 * Welcome to Cloudflare Workers! This is your first worker.
 *
 * - Run `npm run dev` in your terminal to start a development server
 * - Open a browser tab at http://localhost:8787/ to see your worker in action
 * - Run `npm run deploy` to publish your worker
 *
 * Learn more at https://developers.cloudflare.com/workers/
 */


export default {
  async fetch(request, env, ctx) {


    if (request.method === 'OPTIONS') {
        return new Response(null, {
          headers: {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
          }
        });
    }

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

/**
 * Welcome to Cloudflare Workers! This is your first worker.
 *
 * - Run `npm run dev` in your terminal to start a development server
 * - Open a browser tab at http://localhost:8787/ to see your worker in action
 * - Run `npm run deploy` to publish your worker
 *
 * Learn more at https://developers.cloudflare.com/workers/
 */


export default {
  async fetch(request, env, ctx) {


    if (request.method === 'OPTIONS') {
        return new Response(null, {
          headers: {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
          }
        });
    }

    if (request.method == 'GET') {
        return new Response('Hello');
    }
 
    if (request.method === 'PUT') {
        // Note that you could require authentication for all requests
        // by moving this code to the top of the fetch function.
        const auth = request.headers.get('Authorization');
        const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
  
        if (!auth || auth !== expectedAuth) {
          return new Response('Unauthorized', { status: 401 });
        }

        const corsHeaders = {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
            "Access-Control-Max-Age": "86400",
        };
        
        const url = new URL(request.url);
        const key = url.pathname.slice(1);
        await env.MY_BUCKET.put(key, request.body);
        const data = `Object ${key} uploaded successfully!`;
        // return new Response(`Object ${key} uploaded successfully!`);
        return new Response(JSON.stringify(data), {
            headers: {
                ...corsHeaders,
                'Content-Type': 'application/json;charset=UTF-8'
            }
        })
    }
   },
};

R

Ranjan Mangla•1/12/24, 11:20 AM

now it is solved, added PUT method in OPTIONS request as allowed methods

R

Ranjan Mangla•1/12/24, 11:20 AM

Thank you very much @HardlyHolidayin’ ,was stuck on it for past one day

R

Ranjan Mangla•1/12/24, 11:20 AM

and got know about pre-flight requests today facehappy

D

Deo Kumar•1/12/24, 12:43 PM

If anyone want to use photon-rs library which is available for node environment can try this fork. It works on Cloudflare Workers, Pages, Next js, node:
https://www.npmjs.com/package/@cf-wasm/photon

npm

@cf-wasm/photon

Photon library for Cloudflare workers. Latest version: 0.1.6, last published: an hour ago. Start using @cf-wasm/photon in your project by running

npm i @cf-wasm/photon

npm i @cf-wasm/photon

. There are no other projects in the npm registry using @cf-wasm/photon.

C

crossbeau•1/12/24, 10:07 PM

Is there a way to log out compatibility date and compatibility flags?

C

crossbeau•1/12/24, 10:08 PM

I manually created one worker a while ago and now trying to terraform it, and the code works on the manually created one and not the terraformed one. And i am thinking i need to match compatibility date

C

crossbeau•1/12/24, 10:08 PM

But unsure how to find

K

kianOP•1/12/24, 10:10 PM

what's the code that doesn't work/the error

K

kianOP•1/12/24, 10:11 PM

otherwise just open devtools & look at the scripts request

K

kianOP•1/12/24, 10:11 PM

R

Ranjan Mangla•1/13/24, 6:04 AM

I have links to files on my ftp server, I can't ssh into server

R

Ranjan Mangla•1/13/24, 6:04 AM

Is there any way I can download those files in workers

R

Ranjan Mangla•1/13/24, 6:05 AM

If I only pass the link as a post request and it then downloads it and then transfer to R2

R

Ranjan Mangla•1/13/24, 6:05 AM

Fetch is not working

R

Ranjan Mangla•1/13/24, 6:05 AM

In this regard

A

Arma•1/13/24, 7:21 AM

Fetch only does HTTP so you can’t do FTP with that

A

Arma•1/13/24, 7:22 AM

Maybe there is a serverless package that lets you do ftp ?

A

Arma•1/13/24, 7:27 AM

There is a TCP api for workers so it’s definitely possible https://developers.cloudflare.com/workers/runtime-apis/tcp-sockets

TCP sockets · Cloudflare Workers docs

Use the

connect()

connect()

API to create outbound TCP connections from Workers.

A

Arma•1/13/24, 7:36 AM

Btw if you download each file to memory on the worker, it will only work for files up to 128 MB

A

Arma•1/13/24, 7:37 AM

Since your use case is a one time operation, I would simply download the FTP contents on my machine and upload them with the wrangler cli

R

Ranjan Mangla•1/13/24, 7:43 AM

Actually it's not, we are migrating data from our ftp server to R2

RRanjan Mangla Actually it's not, we are migrating data from our ftp server to R2

H

Hello, I’m Allie!•1/13/24, 8:03 AM

You can use rclone to move from FTP to R2 too

H

Hello, I’m Allie!•1/13/24, 8:03 AM

You can’t run it on a Worker, but it can be a tad faster

H

Hello, I’m Allie!•1/13/24, 8:03 AM

And less complex

A

Arma•1/13/24, 8:04 AM

My thinking is, you can either

implement FTP minimally over the workers TCP api and use that to connect to your existing file server, then stream the response to R2
use your existing file server’s HTTP api (if it has any) via fetch then put the response on R2

R

Ranjan Mangla•1/13/24, 8:06 AM

Thanks @HardlyHolidayin’ & @Arma

R

Ranjan Mangla•1/13/24, 8:06 AM

Will explore rclone one

R

Ranjan Mangla•1/13/24, 8:07 AM

@Arma can you shed more light on the TCP method, didn't understand properly

Process a file is a little vague - what are you uploading and what do you want to do with it?

Similar Threads

Similar Threads

Similar Threads