Video Manipulation using HTML 5 and Javascript

August 5th, 2012 by

Some funny stuff can be done using HTML 5, canvas elements and the video events API.

In the following example we’re using these techniques to apply graphic effects to a video embedded in a HTML page..


 

The Goal

Please take a look at the following screencast to get an idea of what the final example looks like  or just take a look at the demo page.

As you can see a video is rendered and a clone is displayed with a strange swirl effect applied.

The Code

This is the example code – I have ommitted some CSS styles and stuff ..

First we’re loading a video using the HTML 5 Video API. I was too lazy here to deliver multiply sources for the video element to support other browsers than Chrome but in a production scenario we definitely should do that!

The second important element is the canvas. We’re using the canvas to draw and refresh a copy of the video’s image data.

To manipulate this image data I am using Joel Besada’s excellent JSManipulate library here that offers a lot of nice effects.

So what is happening when the video playback is started?

  • The event handler bound to the video’s play event triggers a function
  • The video’s data is drawn onto the canvas’ 2d context
  • We’re fetching this data and we’re using JSManipulate to apply some effects
  • Afterwards we’re repainting the canvas with the new data
  • The setTimeout is responsible for updates every 20 ms
<!DOCTYPE html>
<html>
<head>
 <title>hasCode.com HTML Canvas Video Manipulation Example</title>
 <script type="text/javascript" src="jsmanipulate.js"></script>
</head>
<body>
 <video controls id="video">
 <source src="video.mp4" type="video/mp4" />
 </video>
 <canvas id="canvas"></canvas>
 
 <script type="text/javascript">
 function setup(){
 var video = document.getElementById("video");
 var canvas = document.getElementById("canvas");
 
 var canvas2DContext = canvas.getContext("2d");
 
 video.addEventListener("play", function(){
 canvas.height = video.clientHeight;
 canvas.width = video.clientWidth;
 paintCanvas(video, canvas, canvas2DContext);
 }, false);
 }
 
 function paintCanvas(video, canvas, canvas2DContext){
 if(video.paused || video.ended) return false;
 
 canvas2DContext.drawImage(video,0,0);
 var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
 JSManipulate.triangleripple.filter(data, {});
 canvas2DContext.putImageData(data,0,0);
 
 setTimeout(function(){
 paintCanvas(video,canvas,canvas2DContext);
 }, 20);
 }
 
 setup();
 </script>
</body>
</html>

Running the Example

There are restrictions when opening the index.html from local so if you’ve got Python installed you’re able to start a webserver quick:

cd /path/to/project
python -m SimpleHTTPServer

Afterwards you’re able to watch the demo at http://localhost:8000/

Of course you’re free to use any of the well known web servers here.

Writing your own Effect Filter

It is indeed very easy to create your custom graphic filter – getImageData return an object containing an internal data object with an array of integers that contains the pixel’s data.

The format of this array is as this .. we have four bytes for each pixel .. the first one is the red-value, the second is the green-value, the third is the blue-value and the fourth byte is for the alpha-channel..

The RGB Image Data Format

The RGB Image Data Format

Now it is easy to write a custom filter – let’s say we just want to invert the video .. how is it done? Just substract the integer RGB value from 255 except for the alpha channel and use the new array to draw the canvas image.

function paintCanvas(video, canvas, canvas2DContext){
  if(video.paused || video.ended) return false;
 
  canvas2DContext.drawImage(video,0,0);
  var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
  var converted = data;
  for(var i=1;i<=data.data.length;i++){
    if(i % 4 == 0){ // alpha channel
      converted.data[i-1] = 255;
      continue;
    }
    converted.data[i-1] = 255-data.data[i-1]; // invert
  }
 
  canvas2DContext.putImageData(converted,0,0);
 
  setTimeout(function(){
    paintCanvas(video,canvas,canvas2DContext);
  }, 20);
}

Thats what the example finally looks like or just take a look at the demo page.

Tutorial Sources

Please feel free to to view and download the complete sources from this tutorial from my Bitbucket repository – or if you’ve got Mercurial installed just check it out with

 hg clone https://bitbucket.org/hascode/html5-js-video-manipulation

Resources

function paintCanvas(video, canvas, canvas2DContext){
if(video.paused || video.ended) return false;

canvas2DContext.drawImage(video,0,0);
var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
var converted = data;
for(var i=1;i<=data.data.length;i++){
if(i % 4 == 0){
converted.data[i-1] = 255;
continue;
}
converted.data[i-1] = 255-data.data[i-1];
}

canvas2DContext.putImageData(converted,0,0);

setTimeout(function(){
paintCanvas(video,canvas,canvas2DContext);
}, 20);
}

Tags: , , , , , ,

3 Responses to “Video Manipulation using HTML 5 and Javascript”

  1. Thomas M Says:

    This is very inspriring, thank you!

  2. Eric Says:

    Would it be possible to save the manipulated video as a new movie file to local storage?

  3. micha kops Says:

    I think it could be possible .. but the local storage api limits to 5mb in most modern browsers afaik (http://diveintohtml5.info/storage.html#limitations).

    Perhaps the new FileSystems API is a solution here but it’s still bleeding edge -> http://www.html5rocks.com/en/tutorials/file/filesystem/

Leave a Reply

Please leave these two fields as-is:

Protected by Invisible Defender. Showed 403 to 80,962 bad guys.

Search
Categories