Generating 1200x1200 10,000 unique images in lambda.

I'm using lambda for this one, I'm generating 1200x1200 10,000 unique images but it reaches the timeout limit. Any recommendation to make this faster? without removing status updates.
// generate images from layers
const canvas = createCanvas(dimension.width, dimension.height);
const ctx = canvas.getContext("2d");

for (let i = 0; i < metadata.length; i++) {
console.info(`<${userId}>`, `generated image #${i}`);
ctx.clearRect(0, 0, dimension.width, dimension.height);

const data = metadata[i];
const traitImages = data.attributes.map((attr) =>
layerMap.get(attr.trait_type + attr.value),
);

for (const image of traitImages) {
ctx.drawImage(image, 0, 0, dimension.width, dimension.height);
}

archive.append(canvas.toBuffer("image/png"), { name: `images/${i}.${format}` });

// if divisible by 5 update status
if (i % 5 === 0) {
await collection.updateOne({ userId }, { $set: { status: i.toString() } });
}
}
console.info(`<${userId}>`, "generated images");
await collection.updateOne({ userId }, { $set: { status: "processed" } });
// generate images from layers
const canvas = createCanvas(dimension.width, dimension.height);
const ctx = canvas.getContext("2d");

for (let i = 0; i < metadata.length; i++) {
console.info(`<${userId}>`, `generated image #${i}`);
ctx.clearRect(0, 0, dimension.width, dimension.height);

const data = metadata[i];
const traitImages = data.attributes.map((attr) =>
layerMap.get(attr.trait_type + attr.value),
);

for (const image of traitImages) {
ctx.drawImage(image, 0, 0, dimension.width, dimension.height);
}

archive.append(canvas.toBuffer("image/png"), { name: `images/${i}.${format}` });

// if divisible by 5 update status
if (i % 5 === 0) {
await collection.updateOne({ userId }, { $set: { status: i.toString() } });
}
}
console.info(`<${userId}>`, "generated images");
await collection.updateOne({ userId }, { $set: { status: "processed" } });
6 Replies
pagwin
pagwinā€¢11mo ago
I don't have any experience with lambdas or js canvas api but just to spitball some ideas 1. you could reimplement it in a faster language 2. WebGL? 3. screw making it faster and just have the lambda call out to a different lambda which generates one image 10,000 times
Typedef
Typedefā€¢11mo ago
I think one image per lambda call is crazy. Currently it can generate around 4200 images, almost half, when generating 1200x1200 10,000 images before it reaches the 15 minutes timeout. Let me know your thoughts šŸ™šŸ¼
pagwin
pagwinā€¢11mo ago
then just make a version of the lambda that can generate some number specified images and make however many calls to that lambda from the main lambda you need to get everything done fast enough, 2 calls generating 4k each then 1 more generating the last 2k or maybe 5 calls generating 2k
Neto
Netoā€¢11mo ago
take the first lambda, and spawn N more lambdas do handle the image generations
Neto
Netoā€¢11mo ago
kinda like this