How to Import Large CSV File into a Database in LARAVEL 10

Spread the love

Hello Guys,

Discover the Process of Importing Large CSV Files into a Database with Laravel 10

In this article, we’ll guide you through the process of import large CSV files into your database using Laravel 10. Our step-by-step tutorial will demonstrate a practical example of importing substantial Excel files in CSV format with Laravel 10. If you’re dealing with massive CSV files, ranging from 1GB to 4GB in size, we have the perfect solution to ensure a seamless import into your database. Even when dealing with large files that may lead to timeouts or other issues within the Laravel framework, we’ve got you covered.

Our approach involves utilizing Laravel’s LazyCollection along with the powerful functions of DB, fopen(), and fgetcsv() to read the CSV file and seamlessly integrate its data into your database. Say goodbye to import-related hassles with our comprehensive guide.

Import Large CSV File into Database example:

Create Seeder

In this guide, we will walk you through the process of importing a large CSV file and crafting the ProductSeeder class.

Also Read: How to Crop an Image Before Uploading in LARAVEL 10

Ensure you’ve created products by populating the “Name,” “Amount,” and “Description” columns in a CSV file. Save this document within the public directory.

Let’s now construct the seeder class using the command:

php artisan make:seeder ProductSeeder
database/seeders/ProductSeeder.php
<?php
  
namespace Database\Seeders;
  
use Illuminate\Database\Console\Seeds\WithoutModelEvents;
use Illuminate\Database\Seeder;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\LazyCollection;
  
class ProductSeeder extends Seeder
{
    /**
     * Run the database seeds.
     */
    public function run(): void
    {
        DB::disableQueryLog();
        DB::table('products')->truncate();
  
        LazyCollection::make(function () {
            $handle = fopen(public_path("products.csv"), 'r');
            
            while (($line = fgetcsv($handle, 4096)) !== false) {
                $dataString = implode(", ", $line);
                $row = explode(',', $dataString);
                yield $row;
            }
  
            fclose($handle);
  
        })
        ->skip(1)
        ->chunk(1000)
        ->each(function (LazyCollection $chunk) {
  
            $records = $chunk->map(function ($row) {
                          return [
                            "name" => $row[0],
                            "amount" => $row[1],
                            "description" => $row[2]
                          ];
                      })->toArray();
            
            DB::table('products')->insert($records);
  
        });
    }
}

Now, you must issue the following command to all of the mentioned seeders:

php artisan db:seed --class=ProductSeeder

Notice the recent addition of a new row to your products table.

Understanding the functionality of data seeding underscores its importance in our Laravel application.

Explore our Laravel blog section for insightful content and helpful tips.

Leave a Comment