Importer performance issue with ->firstOrNew() on big CSV files
Im trying to make an importer which can import reasonably fast +300k rows per import.
Im using the default Import action of filament with firstOrNew() method because i need to update existing records or create new ones. But i can see that for such big CSV files imports can take up to 4hrs because of the repetitive queries that get made by firstOrNew().
This is the code of resolveRecord() function im using:
I would appreciate any help with making this importer faster.
Im using the default Import action of filament with firstOrNew() method because i need to update existing records or create new ones. But i can see that for such big CSV files imports can take up to 4hrs because of the repetitive queries that get made by firstOrNew().
This is the code of resolveRecord() function im using:
public function resolveRecord(): ShopProduct
{
// Get the shop_id from the import's user
$shopId = $this->getImport()->user->shop_id;
// if the quantity is not set, set it to 0
if (!isset($this->data['stock_quantity'])) {
$this->data['stock_quantity'] = 0;
}
return ShopProduct::withoutGlobalScopes()->firstOrNew([
'barcode' => $this->data['barcode'],
'shop_id' => $shopId,
]);
}I would appreciate any help with making this importer faster.
