Video Manipulation using HTML 5 and Javascript

August 5th, 2012 by Micha Kops

Some funny stuff can be done using HTML 5, canvas elements and the video events API.

In the following example we’re using these techniques to apply graphic effects to a video embedded in a HTML page..

The Goal
The Code
Running the Example
Writing your own Effect Filter
Tutorial Sources
Resources
Article Updates

The Goal

Please take a look at the following screencast on YouTube to get an idea of what the final example looks like or just take a look at the demo page.

As you can see a video is rendered and a clone is displayed with a strange swirl effect applied.

The Code

This is the example code – I have ommitted some CSS styles and stuff ..

First we’re loading a video using the HTML 5 Video API. I was too lazy here to deliver multiply sources for the video element to support other browsers than Chrome but in a production scenario we definitely should do that!

The second important element is the canvas. We’re using the canvas to draw and refresh a copy of the video’s image data.

To manipulate this image data I am using Joel Besada’s excellent JSManipulate library here that offers a lot of nice effects.

So what is happening when the video playback is started?

The event handler bound to the video’s play event triggers a function
The video’s data is drawn onto the canvas’ 2d context
We’re fetching this data and we’re using JSManipulate to apply some effects
Afterwards we’re repainting the canvas with the new data
The setTimeout is responsible for updates every 20 ms

<!DOCTYPE html>
<html>
<head>
 <title>hasCode.com HTML Canvas Video Manipulation Example</title>
 <script type="text/javascript" src="jsmanipulate.js"></script>
</head>
<body>
 <video controls id="video">
 <source src="video.mp4" type="video/mp4" />
 </video>
 <canvas id="canvas"></canvas>
 
 <script type="text/javascript">
 function setup(){
 var video = document.getElementById("video");
 var canvas = document.getElementById("canvas");
 
 var canvas2DContext = canvas.getContext("2d");
 
 video.addEventListener("play", function(){
 canvas.height = video.clientHeight;
 canvas.width = video.clientWidth;
 paintCanvas(video, canvas, canvas2DContext);
 }, false);
 }
 
 function paintCanvas(video, canvas, canvas2DContext){
 if(video.paused || video.ended) return false;
 
 canvas2DContext.drawImage(video,0,0);
 var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
 JSManipulate.triangleripple.filter(data, {});
 canvas2DContext.putImageData(data,0,0);
 
 setTimeout(function(){
 paintCanvas(video,canvas,canvas2DContext);
 }, 20);
 }
 
 setup();
 </script>
</body>
</html>

Running the Example

There are restrictions when opening the index.html from local so if you’ve got Python installed you’re able to start a webserver quick:

cd /path/to/project
python -m SimpleHTTPServer

Afterwards you’re able to watch the demo at http://localhost:8000/

Of course you’re free to use any of the well known web servers here.

Writing your own Effect Filter

It is indeed very easy to create your custom graphic filter – getImageData return an object containing an internal data object with an array of integers that contains the pixel’s data.

The format of this array is as this .. we have four bytes for each pixel .. the first one is the red-value, the second is the green-value, the third is the blue-value and the fourth byte is for the alpha-channel..

The RGB Image Data Format

Now it is easy to write a custom filter – let’s say we just want to invert the video .. how is it done? Just substract the integer RGB value from 255 except for the alpha channel and use the new array to draw the canvas image.

function paintCanvas(video, canvas, canvas2DContext){
  if(video.paused || video.ended) return false;
 
  canvas2DContext.drawImage(video,0,0);
  var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
  var converted = data;
  for(var i=1;i<=data.data.length;i++){
    if(i % 4 == 0){ // alpha channel
      converted.data[i-1] = 255;
      continue;
    }
    converted.data[i-1] = 255-data.data[i-1]; // invert
  }
 
  canvas2DContext.putImageData(converted,0,0);
 
  setTimeout(function(){
    paintCanvas(video,canvas,canvas2DContext);
  }, 20);
}

Thats what the example finally looks like as screencast on YouTube or just take a look at the demo page.

Tutorial Sources

Please feel free to to view and download the complete sources from this tutorial from my Bitbucket repository – or if you’ve got Mercurial installed just check it out with

 hg clone https://bitbucket.org/hascode/html5-js-video-manipulation

Resources

Article Updates

2018-06-01: Embedded YouTube video removed (GDPR/DSGVO).

function paintCanvas(video, canvas, canvas2DContext){
if(video.paused || video.ended) return false;

canvas2DContext.drawImage(video,0,0);
var data = canvas2DContext.getImageData(0,0,canvas.width, canvas.height);
var converted = data;
for(var i=1;i<=data.data.length;i++){
if(i % 4 == 0){
converted.data[i-1] = 255;
continue;
}
converted.data[i-1] = 255-data.data[i-1];
}

canvas2DContext.putImageData(converted,0,0);

setTimeout(function(){
paintCanvas(video,canvas,canvas2DContext);
}, 20);
}

Tags: canvas, codec, html, html5, javascript, js, video

This entry was posted on Sunday, August 5th, 2012 at 5:08 pm and is filed under Javascript, Snippet, Web. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

3 Responses to “Video Manipulation using HTML 5 and Javascript”

Thomas M Says:
September 1st, 2012 at 5:11 pm
This is very inspriring, thank you!
Eric Says:
April 26th, 2013 at 3:33 pm
Would it be possible to save the manipulated video as a new movie file to local storage?
micha kops Says:
April 26th, 2013 at 8:38 pm
I think it could be possible .. but the local storage api limits to 5mb in most modern browsers afaik (http://diveintohtml5.info/storage.html#limitations).

Perhaps the new FileSystems API is a solution here but it’s still bleeding edge -> http://www.html5rocks.com/en/tutorials/file/filesystem/