Skip to main content

Uploading chunks of a large file using XHR2

I was having a conversation with my college friend about the enhancements in XHR2 and I was showing him how one can upload a file to the server using plain Ajax. He asked me whether I can upload a large file to the server such as a video file. I tried doing that but was bumped when the server reported with 400 error telling me that the 'POST size has exceeded maximum limit'. This got me thinking whether I can upload chunks of a large file to the server. I referred to the FileSystem API and came across the slice method that allows you to get a fragment of a file.

The slice method takes two parameters which specifies the start and end byte position of the fragment that is to be extracted. This fragment can then be uploaded to the server using XHR2.

Here’s the client code:

Here I’ve specified the chunk size as 1 MB (1024*1024) and then slicing the file such that each fragment is of 1 MB in size. Each of this fragments is then pushed to an array and then uploaded to the server using XHR2. Each fragment is a ByteArray and is sent to server synchronously.

Here’s the server side code:


The ByteArray is then written\appended to the file by calling the write method on the FileOutputStream object.

Output:

           Uploading chunk of size 1048576.
           Chunk of size 1048576 uploaded successfully.
           Uploading chunk of size 1048576.
           Chunk of size 1048576 uploaded successfully.
           Uploading chunk of size 1048576.
           Chunk of size 1048576 uploaded successfully.
           Uploading chunk of size 17540.
           Chunk of size 17540 uploaded successfully.

Here you can see that each fragment is of size 1 MB and the last fragment is of 17 KB. I've tried uploading large images and videos of size greater than 50 MB and it works fine.

Comments

  1. It is possible to send the file over Internet without uploading them anywhere by using a tool called Binfer. Binfer has no limitation on file size and quantity. You can send large files without any worry. Visit http://www.binfer.com for details

    ReplyDelete
  2. Hi, thank you for this useful script. (y)
    Won't it be better to send http headers(content-type & filename) only once ? or it does change with each sent chunk ???

    ReplyDelete
  3. Another thing, this will cause some server-side problems if we try to use it on asynchronous sending mode, maybe if you replace the "while(start<size)" with a callback there won't be any server-side conflicts since the chunks will be sent one by one even when using async mode.

    ReplyDelete
  4. Another thing, this will cause some server-side problems if we try to use it on asynchronous sending mode, maybe if you replace the while(start<size) with a callback on the upluadChunk function there wont be any server side conflict since the chunks will be sent one by one even while using async mode.
    (sorry for the double post)

    ReplyDelete
  5. Duuude ! WTF ?!!!!!!
    mail me O.o

    ReplyDelete
  6. oww my bad, it's the size tag lol

    ReplyDelete
  7. The server should be able to get the data in correct order. If async is used then the blobs would not reach the server in correct order. This is the reason why the third field is set to 'false' when sending the XHR request.

    ReplyDelete
  8. I think it is necessary to send the filename with each request. Once the first chunk of the file is uploaded it closes the stream and the session ends there. If there is a session maintained on the server then the it can store the filename in the session and the same may not be required to be sent in each request.

    ReplyDelete
  9. yes friend i get that, i was trying to say that maybe if you use a callback on the uploadChunk function instead of calling it using WHILE, there won't be any chunks' order conflict when using async mode :-D

    ReplyDelete
  10. it seems that i'am right,i just tested it :D
    you don't need to fix headers each time.

    ReplyDelete
  11. sorry for double posting :-/
    my tests were a little wrong, the headers get overwritten by default values.
    but there is a way to avoid sending it each time.
    this can be handled on the server side by memorizing the first request's http values and then using them when all the chunks are received.

    ReplyDelete
  12. Hi,


    Thanks for this post- can we also upload other fields along with this large file, in that case how it works should we send the data twice to server? once with only form-fields and and next large file-chunks.. please let me know if this is best approach to achieve, ?


    thanks
    Satish

    ReplyDelete

Post a Comment

Popular posts from this blog

Custom validation messages for HTML5 Input elements using the constraint validation API

HTML5 has introduced several input types such as EMAIL, URL, RANGE, SEARCH, DATE, TIME, etc,. Most of the modern browsers have implemented them and are ready to be used in a HTML document. Another exciting feature introduced in HTML5 is the form validation. Instead of writing JavaScript to validate users input, browsers can now validate it and show an appropriate message if the validation fails. The validation message is shown in line with the field for which the validation has failed. The default error message is shown when the validation fails. In this post I'll explain how these error messages can be changed.

De-obfuscating javascript code in Chrome Developer Tools

I had blogged about JavaScript debugging with Chrome Developer Tools some time back, wherein I have explained how these developer tools can help in debugging javascript code. Today Google Chrome 12 was released and my Chrome browser was updated to this version. As with every release, there have been some improvements made on performance, usability etc,. One feature that stood out for me is the ability to De-obfuscate the javascript code.

What is Minification?

Minification is the process of removing unnecessary characters such as white spaces, comments, new lines from the source code. These otherwise would be added to make the code more readable. Minifying the source code helps in reducing the file size and thereby reducing the time taken to download the file. This is the reason why most of the popular javascript libraries such as jQuery are minified. A minified jQuery file is of 31 KB in size where as an uncompressed one is about 229 KB. Unfortunately, debugging minified javascript file…

Adding beforeRender and afterRender functions to a Backbone View

I was working on a Backbone application that updated the DOM when a response was received from the server. In a Backbone View, the initialize method would perform some operations and then call the render method to update the view. This worked fine, however there was scenario where in I wanted to perform some tasks before and after rendering the view. This can be considered as firing an event before and after the function had completed its execution. I found a very simple way to do this with Underscore's wrap method.