Eloquent chunk() missing half the results

I have a problem with Laravel’s ORM Eloquent chunk() method. It misses some results. Here is a test query :

$destinataires = Destinataire::where('statut', '<', 3)
    ->where('tokenized_at', '<', $date_active)
    ->chunk($this->chunk, function ($destinataires) {
        foreach($destinataires as $destinataire) {
            $this->i++;
        }
    }
echo $this->i;

It gives 124838 results.

But :

$num_dest = Destinataire::where('statut', '<', 3)
    ->where('tokenized_at', '<', $date_active)
    ->count();
echo $num_dest;

gives 249676, so just TWICE as the first code example.

My script is supposed to edit all matching records in the database. If I launch it multiple times, it just hands out half the remaining records, each time.

I tried with DB::table() instead of the Model. I tried to add a ->take(20000) but it doesn’t seem to be taken into account. I echoed the query with ->toSql() and eveything seems to be fine (the LIMIT clause is added when I add the ->take() parameter).

Any suggestions ?

Answer

Quick answer: Use chunkById() instead of chunk().

The explanation can be found in the Laravel documentation:

When updating or deleting records inside the chunk callback, any changes to the primary key or foreign keys could affect the chunk query. This could potentially result in records not being included in the chunked results.

Here is the solution example:

DB::table('users')->where('active', false)
    ->chunkById(100, function ($users) {
        foreach ($users as $user) {
            DB::table('users')
                ->where('id', $user->id)
                ->update(['active' => true]);
        }
    });

If you are updating database records while chunking results, your chunk results could change in unexpected ways. So, when updating records while chunking, it is always best to use the chunkById method instead. This method will automatically paginate the results based on the record’s primary key.

(end of the update)

The original answer:

I had the same problem – only half of the total results were passed to the callback function of the chunk() method.

Here is the code which had problems:

Transaction::whereNull('processed')->chunk(100, function ($transactions) {
    $transactions->each(function($transaction){
        $transaction->process();
    });
});

I used Laravel 5.4 and managed to solve the problem replacing the chunk() method with cursor() method and changing the code accordingly:

foreach (Transaction::whereNull('processed')->cursor() as $transaction) {
    $transaction->process();
}

Even though the answer doesn’t address the problem itself, it provides a valuable solution.