Zip getting corrupted when uploaded to php as chunks

first of all, I am new to programming and starting to learn it, so my code is a mess or has flaws that you, experienced programmers, see, feel free to notice me. I will be grateful!

So to my problem. I am currently trying to write maybe too ambitious project for my level, but nonetheless. I am creating my own cloud-like solution for private usage and to learn how to code mostly. To overcome complications that I had while uploading large files to the server in one piece, I created .js on the client side to slice up files into chunks and send them one by one in order to merge them back on my server using PHP code. Everything is working great, I can easily upload files any sizes, any formats and files are readable. Except ZIP files. They tend to miss couple of chunks after merging on the server. And I d o not understand why. Isn’t zip file effectively the same file as every? By slicing them binary and merging blocks together I do not change structure of it. I would understand if parts would be shifted causing the corruption, but missing them? Every other format works just fine.. Is there some part in my code causing that?

My .js code:

    var xhr = new XMLHttpRequest;
    var formdata = new FormData;
    console.log(file);
    var name = file.name;
    var size = file.size;
    var chunksize = 10485760;
    var i = 0;
    var chunkname = '';
    var chunk = new Blob;
    var chunkammount = Math.ceil(size / chunksize);
    console.log(chunkammount);
    var chunks = new Array;
    var chunknames = new Array;
    var offset = 0;

    if (size < chunksize) {
        xhr.open('POST', 'singlefileupload.php', true);
        formdata.append('file', file);
        xhr.send(formdata);
        console.log('sent small file');

    } else {
        end = chunksize;
        while (i <= chunkammount) {
            offset = i * chunksize;
            chunkname = name + '_' + i;
            chunk = file.slice(offset, offset + chunksize);
            chunks.push(chunk);
            chunknames.push(chunkname);
            if (chunk.size == 0) {
                chunks.pop();
                chunknames.pop();
            }
            console.log("chunkname" + chunkname);
            console.log("size" + chunk.size);
            i++;
        }
        console.log(i + ';' + chunks.length);
        chunks = chunks.reverse();
        chunknames = chunknames.reverse();
        console.log(chunks);
        console.log(chunknames);
    }
    for (index = 0; index <= chunkammount; index++) {
        phpsend(chunknames[index], chunks[index], file, chunkammount);
        console.log(chunks[index]);
        console.log(chunknames[index]);
        console.log("index" + index);
        console.log("jsem na konci")
    }
}




function phpsend(chunkname, chunk, file, chunkammount) {

    var formdata = new FormData;
    formdata.append('name', file.name);
    formdata.append('chunkname', chunkname);
    formdata.append('file', chunk);
    formdata.append('chunkammount', chunkammount);
    addAjax({
        url: 'sliceupload.php',
        type: 'POST',
        data: formdata,
        contentType: false,
        processData: false,
        success: function () {
        },
    });
}

my php code:

<?php
//load
require_once('settings.php');
require_once('clean_folder.php');
require_once('writable_check.php');

session_start();
$tmp_dir = setting::$writepath . $_POST['name'] . "_" . $_SESSION['username'] . '/';
if (isset($_FILES['file'])) {
    if (!file_exists($tmp_dir)) {
        mkdir($tmp_dir);
    }

    $tmp_name = $_FILES['file']['tmp_name'];
    $file_name = $_POST['name'];
    $chunkname = $_POST['chunkname'];
    $filecount = $_POST['chunkammount'];
    $chunk_target = $tmp_dir . $chunkname;
    $file_target = setting::$writepath . $file_name;

    $chunkname = explode('_', $chunkname);
    $chunkindex = end($chunkname);

    move_uploaded_file($tmp_name, $chunk_target);


    $index = 0;

    if ($chunkindex == 0) {
        while ($index <= $filecount - 1) {

            $file_name_i = $tmp_dir . $file_name . '_' . $index;
            $range = filesize($file_name_i);
            $tmpfile = fopen($file_name_i, 'rb');
            $buffer = fread($tmpfile, $range);
            fclose($tmpfile);
            unlink($file_name_i);

            $mergedhandler = fopen($file_target, 'ab');
            $merged_file = fwrite($mergedhandler, $buffer);
            fclose($mergedhandler);

            $index++;
        }
        rmdir($tmp_dir);
    }
}

Answer

Solved it. For some reason (probably performance reason) my server skipped some of the files during upload of a large number of chunks. I do not have other hardware to test it out, but I assume it was it. At the same time, I figured out that copy stream to stream is much more efficient. It seems to work pretty reliably now.

Advertisement

Answer