Cost Efficient XL Generation for Large Products

Hey guys, title could have been better, but I am basically trying to setup an XML product feed for my store, which has 20k+ products including variants, and it constantly grows and shrinks throughout each day. I want the feed to rerun every 15 minutes, and I want to host this on vercel. I am worried about serverles costs since this is a function, below is my code with my approach I am doing caching, is this the most efficient way to handle this to reduce costs in Vercel? Is serverless (vercel) hosting bad for this use case?
// app/api/product-feed/route.ts

import { NextResponse } from 'next/server';
import { kv } from '@vercel/kv';
import { BigCommerceClient } from './bigcommerce-client';
import { generateXMLFeed } from './xml-generator';

const CACHE_KEY = 'product-feed-cache';
const CACHE_TTL = 15 * 60; // 15 minutes in seconds

export async function GET() {
try {
// Check cache first
const cachedFeed = await kv.get(CACHE_KEY);
if (cachedFeed) {
return new NextResponse(cachedFeed, {
headers: { 'Content-Type': 'application/xml' },
});
}

// If not in cache, generate new feed
const client = new BigCommerceClient();
const products = await client.getAllProducts();
const xmlFeed = generateXMLFeed(products);

// Cache the new feed
await kv.set(CACHE_KEY, xmlFeed, { ex: CACHE_TTL });

return new NextResponse(xmlFeed, {
headers: { 'Content-Type': 'application/xml' },
});
} catch (error) {
console.error('Error generating product feed:', error);
return new NextResponse('Error generating product feed', { status: 500 });
}
}

// Scheduled job to update cache (run every 15 minutes)
export async function POST() {
try {
const client = new BigCommerceClient();
const products = await client.getAllProducts();
const xmlFeed = generateXMLFeed(products);
await kv.set(CACHE_KEY, xmlFeed, { ex: CACHE_TTL });
return new NextResponse('Cache updated successfully', { status: 200 });
} catch (error) {
console.error('Error updating cache:', error);
return new NextResponse('Error updating cache', { status: 500 });
}
}
// app/api/product-feed/route.ts

import { NextResponse } from 'next/server';
import { kv } from '@vercel/kv';
import { BigCommerceClient } from './bigcommerce-client';
import { generateXMLFeed } from './xml-generator';

const CACHE_KEY = 'product-feed-cache';
const CACHE_TTL = 15 * 60; // 15 minutes in seconds

export async function GET() {
try {
// Check cache first
const cachedFeed = await kv.get(CACHE_KEY);
if (cachedFeed) {
return new NextResponse(cachedFeed, {
headers: { 'Content-Type': 'application/xml' },
});
}

// If not in cache, generate new feed
const client = new BigCommerceClient();
const products = await client.getAllProducts();
const xmlFeed = generateXMLFeed(products);

// Cache the new feed
await kv.set(CACHE_KEY, xmlFeed, { ex: CACHE_TTL });

return new NextResponse(xmlFeed, {
headers: { 'Content-Type': 'application/xml' },
});
} catch (error) {
console.error('Error generating product feed:', error);
return new NextResponse('Error generating product feed', { status: 500 });
}
}

// Scheduled job to update cache (run every 15 minutes)
export async function POST() {
try {
const client = new BigCommerceClient();
const products = await client.getAllProducts();
const xmlFeed = generateXMLFeed(products);
await kv.set(CACHE_KEY, xmlFeed, { ex: CACHE_TTL });
return new NextResponse('Cache updated successfully', { status: 200 });
} catch (error) {
console.error('Error updating cache:', error);
return new NextResponse('Error updating cache', { status: 500 });
}
}
Then I would setup a cron job in vercel
5 Replies
oz
oz5d ago
I guess a non serverless service will always be cheaper for an API like this
Neto
Neto5d ago
store as blobs do you need constant access to it?
oz
oz5d ago
No its for a search engine, they told us they need it. They will access it once every 15min.
Neto
Neto5d ago
if they need all the data and you won't be mutating and accessing it directly storing as a blob should be good enough, "super cheap"
oz
oz2d ago
Thanks!
Want results from more Discord servers?
Add your server
More Posts