Skip to content
Advertisement

how to solve ‘Allowed memory size of’ error when streaming big files in php?

I have trying to make some “replacement-wrapper” over stream described in this : [article][1]

But when I tested it with not so big file (about 120M) it showed me an error:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 65011744 bytes) in C:Apache22htdocstutsphpstreamsfiltersHiddenNameFilter.php on line 28

Here is my wrapper:

class HiddenNameFilter extends PHP_User_Filter
{
private $data;
private $limiter;

function onCreate()
{
    $this->data = '';
    $this->limiter = 2;
    return true;
}


function filter($in, $out, &$consumed, $closing)
{
    while($bucket = stream_bucket_make_writeable($in))
    {
        $this->data .= $bucket->data;
        $this->bucket = $bucket;
        $consumed = 0;
    }
    
    if($closing)
    {
        $consumed += strlen($this->data);
        $pattern = "/Andrii/m";
        $str = preg_replace($pattern, '0_+', $this->data);
        $this->bucket->data = $str;
        $this->bucket->datalen = strlen($this->data);

        if(!empty($this->bucket->data))
        {
            stream_bucket_append($out, $this->bucket);
        }

        return PSFS_PASS_ON;
    }

    return PSFS_FEED_ME;
}

}

Here is my сlient code:

$handle = fopen(__DIR__ . '/big_file.txt', 'r');

stream_filter_register('myFilter', 'HiddenNameFilter');

stream_filter_append($handle, "myFilter");
$i = 0;
while($i != 30)
{
    $email = fgets($handle);
    if(!$email) break;

    $i++;
    $contents .= $email;
}
fclose($handle);
echo $contents;

Also I have found some PHP bug [here][2] but I dont know how to solve my problem.

I have PHP 7.3.7 (cli) (built: Jul 3 2019 14:34:10) ( ZTS MSVC15 (Visual C++ 2017) x64 ) On windows 10/ Please help) [1]: https://stackify.com/a-guide-to-streams-in-php-in-depth-tutorial-with-examples/ [2]: https://www.hashbangcode.com/article/php-custom-stream-filters

Advertisement

Answer

add your php code header

ini_set('memory_limit', '-1');
User contributions licensed under: CC BY-SA
9 People found this is helpful
Advertisement