“Request date header too old” error occurred when upload large number of files to blob storage

I have large number of image files on my server (about 900,000), and I’m using Azure Client Library for PHP to make a loop and upload my files to Azure Blob Storage. My upload script ran well for about 2 day, and then suddenly stoped and output below error.

Oct 19 01:28:14 ik1-315-17878 php: PHP Fatal error: Uncaught exception ‘MicrosoftAzureStorageCommonExceptionsServiceException’ with message ‘Fail:#012Code: 403#012Value: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.#012details (if any): AuthenticationFailedServer failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.#012RequestId:e7f247d6-001e-0022-142e-48abcc000000#012Time:2017-10-18T16:28:14.4980791ZRequest date header too old: ‘Wed, 18 Oct 2017 16:12:27 GMT’.’ in /home/t-matsumoto/GAZOU/vendor/microsoft/azure-storage/src/Common/Internal/ServiceRestProxy.php:486#012Stack trace:#012#0 /home/t-matsumoto/GAZOU/vendor/microsoft/azure-storage/src/Common/Internal/ServiceRestProxy.php(404): MicrosoftAzureStorageCommonInternalServiceRestProxy::throwIfError(Object(GuzzleHttpPsr7Respon in /home/t-matsumoto/GAZOU/vendor/microsoft/azure-storage/src/Common/Internal/ServiceRestProxy.php on line 486

I’ve searched about the cause, and it’s likely because timestamp when my script made that request was too old comparing to timestamp when Azure received the request. According to the error message above, time of making request is “Wed, 18 Oct 2017 16:12:27 GMT”, and time of receiving request is “2017-10-18T16:28:14.4980791Z” (about 15 minutes late). But what I still don’t understand is that why my script had run for a while with no error, and then the error occurred suddenly. Can anyone explain to me that phenomenon and solution to make my script run stably with no error.

My php script looks like below

    require_once 'vendor/autoload.php';

    use MicrosoftAzureStorageCommonServicesBuilder;
    use MicrosoftAzureStorageCommonServiceException;

    $connectionString = "DefaultEndpointsProtocol=http;AccountName=<accountNameHere>;AccountKey=<accountKeyHere>";

    // Create blob REST proxy.
    $blobRestProxy = ServicesBuilder::getInstance()->createBlobService($connectionString);

foreach ($myListFiles as $filename) {
    $content = fopen($filename, "r");
    $blob_name = "myblob";

    //Upload blob
    $blobRestProxy->createBlockBlob("mycontainer", $blob_name, $content);
}

I got uploading sample script from this link https://docs.microsoft.com/en-us/azure/storage/blobs/storage-php-how-to-use-blobs

Answer

I think @Gaurav Mantri has answered your question:

Azure storage error using Python SDK

But as you have a large number of image files to upload to Azure storage, I would recommend using AzCopy to do that. AzCopy is a command-line utility designed for copying data to and from Microsoft Azure Blob, File, and Table storage using simple commands with optimal performance.

Advertisement

Answer