Chapter 4. Audio and video – Hello! HTML5 & CSS3: A user-friendly reference guide

Chapter 4. Audio and video

 

This chapter covers

  • Why audio and video are important on the web
  • Adding audio and video to your web pages
  • Encoding audio and video files for the web
  • Integrating video with other web platform features and content

 

Native media support is one of the best known as well as one of the most controversial HTML5 features. In this chapter, you’ll learn why HTML5 media support is great, why it’s frustrating, and the practical factors you need to consider when using it.

Audio and video on the modern web

Audio and video are key parts of the modern web. For many sites, video and audio are parts of the content as integral as the text and pictures—and in some cases, they’re more important.

Despite their rising importance, HTML4 offers no built-in method for adding audio or video to a web page. This makes embedding audio and video relatively complex. Compare the markup required to add an image to a web page with that typically required to add a video.

Image

Video

<img
 width="320"
 height="240"
 id="myimage"
 src="myimage.png">
<object
 classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000"
 codebase="http://download.macromedia.com/
    pub/shockwave/cabs/flash/
    swflash.cab#version=6,0,40,0"
 width="320" height="240"
 id="myvideoname">
    <param name="movie"
      value="myvideo.swf">
    <param name="quality" value="high">
    <param name="bgcolor" value=#ffffff>
    <embed href="myvideo.swf"
           quality="high" bgcolor="#ffffff"
           width="320" height="240"
           name="myvideoname"
           type="application/x-shockwave-flash"
           pluginspage="http://www.macromedia.com/
             go/getflashplayer">
    </embed>
</object>

Because there’s no native support for audio and video, web authors have had to resort to browser plug-ins. The web has largely settled on Adobe Flash as a de facto standard, but as the previous code shows, this is still a good deal more complex than putting an image on a page. And that’s not all the code that’s required: to add controls such as Play and Pause, there must be code written inside Flash, and even more code if the player needs to be integrated into other page content.

 

What is a plug-in?

A plug-in is a generic extension method for HTML that allows the page author to indicate embedded content that is to be rendered by an external program. The web browser hands over control of that region of the web page to the external program. This external program is referred to as a plug-in.

The content rendered by the plug-in is like a black box to the browser. Browser features like keyboard shortcuts, cookie preferences, and pop-up blockers don’t apply.

 

Following is the HTML5 code for embedding audio and video, which compares favorably with the <img> element. The screenshots show the default presentation of the <video> and <audio> elements in Firefox with the controls visible.

Audio

Video

<audio src="myaudio.ogg" controls>
</audio>
<video src="myvideo.ogv" controls>
</video>

Both elements in the previous example have been set to 320 x 240 pixels with CSS, although in the case of the <audio> element you can see this doesn’t achieve much. The <video> element is showing the first frame of the video. Note that the video is 320 x 180 pixels, but instead of stretching the video and changing the aspect ratio, the video is made as wide as possible and centered vertically.

The <audio> element

Audio on the web gained something of a bad rap in the 1990s as thousands of people happily attached background music to their GeoCities pages, but there are plenty of legitimate uses for audio in a web page. Sometimes, websites are entirely about sound—band home pages should have samples of the band’s work; dictionaries should allow you to listen to pronunciation; games need sound effects. As you’ve learned, until the <audio> element, the only available option was a browser plug-in. In this section, you’ll learn about the <audio> element, its supported attributes, file formats, and how to convert between them.

Browser support quick check: audio   Standard
4.0
3.5
9.0
10.5
4.0

Common attributes: controls, autoplay, loop, and preload

The <audio> element has several attributes that control its behavior, including src and controls, which you saw in the simple example. It’s possible to add the element with no attributes at all:

<audio src="myaudio.ogg">
</audio>

The screenshot isn’t interesting because there’s nothing to see—without the controls attribute, the element isn’t visible on the page. Having no visible controls means you have to start the audio playing by other means—either by using the autoplay attribute (discussed in a moment) or by providing your own controls (which you’ll learn about in the section “The <video> element”).

Adding the controls attribute means the <audio> element has visible properties:

<audio src="myaudio.ogg"
 controls>
</audio>

The preload attribute lets you hint to the browser whether a file is likely to be needed, so you can avoid excessive server load:

<audio src="myaudio.ogg" controls preload="metadata">
</audio>

It can take the following values:

  • none—You don’t believe the audio resource is likely to be used.
  • metadata—You don’t believe the audio is likely to be used, but the browser should fetch information such as the dimensions, first frame, and duration.
  • auto—The server will have no problem with the browser downloading the entire video even if the user doesn’t explicitly create it.

The browser is free to ignore the preload attribute—for example, a mobile browser may choose not to download any media over a limited cell connection unless the user explicitly requests it.

The autoplay and loop attributes specify that the file is to start playing as soon as the user loads the page and to continue to play repeatedly after it’s started:

<audio src="myaudio.ogg" controls
autoplay loop>
</audio>

Until recently the loop attribute didn’t have any effect in browsers, but you can simulate the effect in older browsers with a little JavaScript:

<audio src="myaudio.ogg" controls autoplay onended="this.play();">
</audio>

The ended event is fired when the video has finished playing. The code waits for that event and starts the audio playing again when it happens.

Be careful with autoplay. Remember, your users may be working in a quiet environment, or listening to music while they’re browsing, or depending on the audio provided by their screen-reader software, and they may not appreciate an audible interruption from a website they were only visiting to find a phone number.

The next example shows an <audio> element in IE8. The <audio> element itself is ignored, and just the contained content is displayed:

<audio src="myaudio.ogg" controls>
    Audio not supported.
</audio>

Normally you would want to display something more useful than just the fact that the <audio> element isn’t supported. At least if you show a link, the user has the chance to download the audio file and listen to it in an external player:

<audio src="myaudio.ogg" controls>
    <a href="myaudio.ogg">
        Download myaudio.ogg
    </a>
</audio>

Compare that with a browser like Firefox that does support the <audio> element, but not the media type specified in the src:

<audio src="myaudio.ogg" controls>
    Audio not supported.
</audio>
<audio src="myaudio.mp3" controls>
    Audio not supported.
</audio>

Firefox doesn’t support MP3, so the user still sees an audio control, but it’s inactive. In other browsers, you may see a broken image icon or some other indication that the media is invalid.

The <audio> element controls allow for a limited amount of styling with CSS. Here it’s been set to 200 pixels square in Firefox:

audio {
    width: 200px;
    height: 200px;
    outline: 1px solid #ccc;
}

The outline shows the extent of the element. As you can see, the width is applied to the controls but the height is ignored. The next example makes that clearer:

audio {
    width: 200px;
    height: 200px;
    outline: 1px solid #ccc;
}

The controls have a minimum intrinsic width. If you try to make them smaller than that, the width will be ignored, as you can see from the still-visible outline in this example:

audio {
    width: 100px;
    height: 50px;
    outline: 1px solid #ccc;
}

Other browsers display slightly differently, as you can see in the next examples. Chrome (left) behaves similarly to Firefox: the controls extend out of the defined width and height. But Opera (right) is a little cleaner-looking at narrow width.

It’s currently impossible to change the colors of the controls in any browser. CSS like the following sets the background color, but it doesn’t make any difference to the controls themselves:

audio {
    width: 50px;
    height: 25px;
    outline: 1px solid #ccc;
    background-color: #000;
    color: #fff;
}

But it’s possible to use the background in concert with the otherwise-ignored height property:

audio {
    width: 200px;
    height: 200px;
    outline: 1px solid #ccc;
    background:
      url('dust-puppy.svg')
      no-repeat top center;
    background-size: contain;
}

You could use this CSS to provide an image of the artist or some sort of cover art.

Codecs and license issues

Audio files are usually stored in a compressed format. To be stored on a computer, they must be encoded into that format; to be played back, they have to be decoded once again. The software that performs this encoding and decoding is called a codec. Music files on your computer usually have a file extension that identifies which codec is needed to decode them.

 

What is a codec?

In principal, it’s possible to describe the raw data of audio and video streams to an arbitrary accuracy. For audio, you’d store the amplitude of the sound wave for each moment in time you wanted to play the sound back; for video, you’d store the color of each pixel for each frame (usually 25–30 per second) as well as the sound. But this would lead to impossibly large files for anything of a useful length.

In practice, you want to compress the audio and video data in the same way you might compress a large file into a zip archive. A codec is what’s used to compress audio and video data for storage and later to decompress the same audio and video to be played through speakers and displayed on screens in real time.

Codecs can be split into two broad categories: lossless and lossy. Think about a zip archive: when you extract the content from it, you expect to get back the exact same files you put in—it’s a lossless compression. In the same way, some codecs are capable of compressing audio and video with no loss of information. But these files are necessarily still large. The more interesting set of codecs for the web are lossy—each time they’re used to encode a video stream, some information is thrown away, never to be seen again. These codecs can achieve far greater compression at the expense of some loss of audible or visual quality; the trick is to throw away data that makes as little difference to human perception as possible.

 

The problem with the MP3 codec is that there are several patents on it; and if you want to distribute software that encodes and decodes MP3, you need to pay to license those patents. Mozilla, the makers of Firefox, takes the position that the web should be built out of free and open standards and so doesn’t support MP3; instead, Firefox supports the open Ogg Vorbis (OGG) format. Opera agrees. Google also agrees in principal but for practical reasons distributes Chrome with MP3 support; Google has also released its own video format, WebM (which will be discussed further in the video section), which can also be used in audio-only mode. Apple and Microsoft both already have licenses to distribute MP3 codecs, so Safari and IE do support it; but, crucially, they don’t support the free and open OGG format out of the box.

The different format support is summarized in the next table. The short version is this: no one file format works on all browsers. You’ll need multiple files to support them all.

Browser support quick check: audio codecs   WAV OGG MP3 AAC WebM
8 5 5 5 8
3.5 3.5 ~ ~ 4
~ ~ 9 9 [*]
10.5 10.5 ~ ~ 11.1
4 [**] 4 4 [**]

* IE9 will support WebM if the user downloads an additional codec.

** Safari will support anything that can be played by Quick-Time. Users have to download additional codecs.

To encode a file to OGG, you can use the oggenc command-line utility available from www.rarewares.org. Use it to convert an uncompressed WAV file like this:

oggenc myaudio.wav

The output is a file called myaudio.ogg. To improve the quality of the encoding, use the -b flag to set the bitrate. The -o flag allows you to specify the output filename:

oggenc myaudio.wav -b 256 -o myhighqualityaudio.ogg

For MP3 audio, you can use the lame command-line utility, also available from www.rarewares.org:

lame myaudio.wav myaudio.mp3

Again, you can set the minimum bitrate with a command-line flag:

lame -b 256 myaudio.wav myhighqualityaudio.mp3

 

Bitrate

Bitrate is the number of bits (individual units of information) that are conveyed or processed per unit of time. Higher bitrates mean greater sound fidelity but also larger file sizes. Typical bitrates for CD-quality audio are in the 100–160 kbit/s range.

 

Command-line utilities are handy, especially if you have a large collection of audio files, because you can write a script to convert them all in a single batch. If you just have one or two files to convert, you may prefer a GUI-driven approach. For this, there’s a handy website: http://media.io. Visit the site, and select the file you want to encode from your hard drive.

After the file is uploaded, you’re given a choice of four options for the codec and, if appropriate, a choice for audio quality.

Select the options you need, and click Convert; a few seconds later, your encoded file will be available to download. Note that the 192 kbps OGG encoded file is approximately 20% of the file size of the lossless original.

You’ll have noticed that the file used in this example, despite being originally encoded with the free FLAC codec, isn’t free content. Although it’s in the OGG format supported by Firefox, Chrome, or Opera, I’m not allowed to upload it to my website because I have no rights to redistribute it. But it will now take up less space on my phone!

Even if I could upload it, the audio wouldn’t play in IE or Safari. Unless users have installed additional codecs in their operating systems, Safari and IE won’t play the OGG file—they need MP3.

If different browsers require different file types, how can you support multiple browsers with a single src attribute? HTML5 anticipates this issue and provides an easy mechanism for providing the correct source to each browser. Let’s look at that in the next section.

Using multiple sources

As you’ve just seen, you need to be able to provide different audio files to different browsers. But each <audio> element only allows you a single src attribute, so how can you manage that? The design of the <audio> element has anticipated this requirement. Multiple sources can be provided for the <audio> element by using the <source> element:

<audio id="myaudio" controls>
    <source src="myaudio.mp3" type="audio/mp3">
    <source src="myaudio.ogg" type="audio/ogg">
    No audio support!
</audio>

The following tables list the common file extensions and MIME types for audio.

Audio type

File extensions

MIME types

MP3 .mp3 audio/mpeg
MP4 .m4a, .m4b, .m4p, .m4v, .m4r, .3gp, .mp4, .aac audio/mp4
audio/aac
OGG .ogg, .oga audio/ogg
WebM .webm audio/webm
WAVE .wav audio/wave (preferred)
audio/wav
audio/x-wav
audio/x-pn-wav

Browsers are expected to scan the list of <source> elements from top to bottom and load the first one they believe they can play. By using some JavaScript, you can interrogate the <audio> element and find out which file is loaded.

Here’s the snippet of JavaScript used in the previous screenshots. Add it to the <audio> element as an attribute, and add a <div id="source"></div> after the <audio> element to display the output:

onloadeddata="document.getElementById('source')
    .innerHTML = 'Playing file ' +
         this.currentSrc.slice(this.currentSrc.lastIndexOf('/')+1);"

This code is executed in the loadeddata event, which means after the browser has loaded identifying information about the file. You’ll learn more about manipulating HTML5 media elements with JavaScript in the section “The <video> element.”

The <video> element

For the many people who don’t obsessively read standards groups’ mailing lists, the first time they became aware of HTML5 was when they found out that Flash video doesn’t work on the iPhone but HTML5 video does. As you saw in the introduction, the goal of the HTML5 <video> element is to make embedding video in your pages as easy as embedding images. This section looks at the details of making it work. We’ll follow a pattern similar to the previous section: first the allowed attributes, then the various encoding issues, and finally how to convert between file formats.

 

The sample video

The sample video used in this section’s examples is of the author playing American football. Because this video was taken by the author’s mother, we can neatly sidestep any issues involving media distribution rights.

 

Browser support quick check: video   Standard
4.0
3.5
9.0
10.5
4.0

<video> element attributes

The <video> element supports the same attributes as the <audio> element, with similar results. This section quickly runs through them in the same way as the section on audio.

The basic <video> element looks like this:

<video src="00092.webm"></video>

A <video> element without controls is a little more interesting than the equivalent <audio> element because you at least have the first frame of the video to look at. This screenshot was taken in Firefox; Opera and Chrome should work just as well for the WebM format.

As with audio, you can enable the standard controls with an attribute:

<video src="00092.webm"
  controls preload="metadata">
</video>

As with the <audio> element, the preload attribute provides a hint to the browser about how likely this video is to be played by the user.

If you add the autoplay attribute, on desktop browsers, the video will download and start playing as soon as possible:

<video src="00092.webm"
  controls autoplay loop>
</video>

The controls are available, but they auto-hide as the video starts playing.

As with the <audio> element, loop only works on the most recent browser versions, but it can be simulated with the same bit of JavaScript:

onended="this.play();"

Also as with the <audio> element, the <video> element can contain fall-back content. At the end of this chapter, we’ll look at using that content to embed an alternative player for your videos using a plug-in, falling back to HTML4 technologies for browsers that don’t support HTML5.

The <video> element also has its own specific attributes: poster, width, height, and audio. Let’s look at each of those in turn.

The poster attribute lets you control what’s shown in the <video> element when a video isn’t playing. By default, browsers show the first frame of the video, but you can supply your own image:

<video src="videofile.ogv"
  poster="posterimage.jpg">
</video>

The width and height attributes set the width and height of the <video> element:

<video src="videofile.ogv"
  width="400px" height="300px">
</video>

Note that this doesn’t directly set the width and height of the video itself; the aspect ratio of the video is always preserved. You can also set the width and height with CSS:

video {
    width: 400px;
    height: 300px;
}

If you set width and height attributes and also set the width and height with CSS, the CSS wins:

<video src="videofile.ogv"
  width="400px" height="300px"
  style="width: 320px;
         height: 180px;">
</video>

The muted attribute sets the default volume of the video to 0:

<video src="videofile.ogv" muted>
</video>

Unfortunately it doesn’t yet work in any browsers, but you can fake it with this bit of JavaScript:

onloadeddata="this.volume = 0;"

Containers, codecs, and license issues

The situation with video is even more complex than with audio because video files need both a visual and an auditory stream, so they need both video and audio codecs. A format to contain both audio and video also needs to be defined.

Browser support quick check: video formats   MPEG-4 Ogg/Theora WebM
5[*] 5 8
~ 3.5 4
9 ~ [**]
~ 10.5 11.1
4 [***] [***]

* Google has announced that Chrome will stop supporting MP4 in a future release.

** IE9 will support WebM if the user downloads an additional codec.

*** Safari will support anything that can be played by Quick-Time. Users have to download additional codecs

Following are the common file extensions and MIME types for video.

Video type

File extensions

MIME types

MPEG-4 .mp4 video/mp4
OGG .ogg, .ogv video/ogg
WebM .webm video/webm

 

MPEG-4 profiles

The MPEG-4 standard contains several different profiles in order to support a variety of different expected use cases, ranging from Blu-ray and HDTV to mobile phones with low screen resolutions. Mobile devices aren’t expected to support the same profiles as desktop PCs or dedicated home multimedia equipment, so when you’re encoding videos for use on iPhones make sure you’re targeting the Simple Profile.

 

Easy encoding with Miro Video Converter

Rather than mess around with the different codecs and encoding options yourself, there are tools that make things easy for you. One of the simplest is the Miro Video Converter, available from www.mirovideoconverter.com.

Miro Video Converter doesn’t present you with a lot of options—just a place to drop the file you want to convert; a drop-down list to say what you want to convert it to; and a button to start the conversion. After you drop a video file on the central area, only the output format needs to be selected before you’re ready to go.

The first three options are the main ones that interest us:

  • Theora is the OGG video format supported by Opera and Firefox.
  • WebM (VP8) is Google’s new video codec, supported by Chrome and newer versions of Opera and Firefox.
  • MP4 videoi s the format supported by Safari and IE.

The additional options are variations of these main three, except that the output video is scaled for the particular device.

After you set the option, click Convert and, depending on how large your video is, wait a few minutes or a few hours. Repeat the process for as many encodings as you require.

The main advantage of this approach is that it’s easy and requires little expertise. The disadvantage is that if you don’t like the results, you have to take a different approach—there are no configuration options for you to tweak.

Advanced encoding with FFmpeg

FFmpeg is a command-line tool originally written for the Linux operating system. It’s powerful and has thousands of options that you can set by passing options on the command line. Rather than get into the details of how FFmpeg works, which could easily take up a few chapters, let’s use the Miro Video Converter output as a starting point and look at some easy ways to tweak things.

If you click the FFMPEG Output button while encoding a WebM video, you’ll see that the command being used, on Windows, is something similar to this:

ffmpeg-bin\ffmpeg.exe -y -i "C:\00092.MTS" -f webm
     -vcodec libvpx -acodec libvorbis -ab 160000
     -crf 22 "C:\00092.webmvp8.webm"

You can run this yourself at the command prompt. Here’s a quick rundown of what the parameters mean (don’t worry too much about the details—for the most part you won’t need to change these):

  • -y—Overwrite any existing output without prompting.
  • -f—Container format.
  • -acodec—Audio codec to use.
  • -crf—Set the constant rate factor (crf). This automatically varies the bitrate to maintain a consistent quality.
  • -i—Input file.
  • -vcodec—Video codec to use.
  • -ab—Audio bitrate to use. Bigger numbers lead to larger files.

One easy change you might want to make is to change the size of the output video, using the -s parameter. This example sets the output to 320 pixels wide by 180 pixels high:

ffmpeg-bin\ffmpeg.exe -y -i "C:\00092.MTS" -f webm
     -vcodec libvpx -acodec libvorbis -ab 160000
     -s 320x180 -crf 22 "C:\00092.webmvp8.320.high.webm"

It’s also easy to adjust the quality of the output file by specifying a bitrate. To do so, use the -b parameter:

ffmpeg-bin\ffmpeg.exe -y -i "C:\00092.MTS" -f webm
     -vcodec libvpx -b 3600k -acodec libvorbis -ab 160000
     -s 320x180 "C:\00092.webmvp8.high.webm"

The same options can be applied to the command for iPhone MP4 encoding, although that has a few extra options specified by default:

ffmpeg-bin\ffmpeg.exe -i "C:\00092.MTS" -f mp4
     -acodec aac -ac 2 -strict experimental -ab 160k
     -s 320x180 -vcodec libx264
     -vpre slow -vpre ipod640 -b 1200k
     -threads 0 "C:\00092.iphone.320.mp4"

One thing to watch for when running the Miro version of FFmpeg is the location of the preset files. The slow and ipod640 presets used in the previous command correspond to the libx264-slow.ffpreset and libx264-ipod640.ffpreset files. Put these files in C:\usr\local\share\ffmpeg so that ffmpeg.exe can find them.

Using multiple sources

Now that you have a collection of video files to support all the different browsers and devices your users may be using, you can add them to the <video> element using the <source> element, just as with the <audio> element earlier:

<video id="myvideo" controls>
    <source src="00092.webm" type="video/webm">
    <source src="00092.mp4" type="video/mp4">
    <source src="00092.low.mp4" type="video/mp4">
    <source src="00092.ogv" type="video/ogg">
    No video support!
</video>

As with the <audio> element, it’s possible to find out which file the browser has chosen by looking at the currentSrc property of the <video> element in JavaScript. Adding this snippet of code to the <video> element reports the filename to an element with ID 'source' when the video has loaded:

onloadeddata=
 "document.getElementById('source')
    .innerHTML = 'Playing file ' +
         this.currentSrc.slice(this.currentSrc.lastIndexOf('/')+1);"

Loading the page in a variety of different browsers shows how the multiple source elements are picked up.

Android browser uses the low-quality MP4.

Firefox 4 uses the WebM video.

Firefox 3.6 uses the Ogg video.

Desktop Safari uses the high-quality MP4.

Controlling audio and video with JavaScript

Earlier in this chapter, you saw that, by default, <audio> and <video> elements don’t provide controls for the user to interact with them; it’s up to the web author to explicitly ask for controls to be provided. At the time, you’d be forgiven for thinking that this is a bit pointless—what good is a video if you can’t play it? In this section, you’ll discover exactly how useful it can be to have complete control over the video from JavaScript.

To begin with, let’s look at playing and pausing a video. This is straight-forward—the <video> element provides play() and pause() methods:

<button
onclick="document.getElementById('myvideo')
.play();">
  Play
</button>
<button
onclick="document.getElementById('myvideo')
.pause();">
  Stop
</button>

Instead of providing controls on the video, buttons are provided on the page. If your first thought when you saw the default controls was, “Ugh! I don’t like the look of those. How can I style them myself?” then here is the answer: create your own elements to control the video, and style them however you wish.

You don’t have to limit yourself to the standard operations of Play and Pause. This function starts the video play from a point in the middle of the stream; you pass in the point as a parameter:

function playFrom(secs) {
  var v = document
    .getElementById('myvideo');
  v.currentTime = secs;
  v.play();
}

You can then provide buttons to start playback from significant points:

<button onclick="playFrom(4);">Play from 4
secs</button>
<button onclick="playFrom(8);">Play from 8
secs</button>

Obviously there aren’t many significant points in a 15-second video clip, but this would be useful if you had a podcast or longer movie and wanted to provide bookmarks for when particular topics were being discussed or for the start of each scene.

In the example, although the buttons claim to start the video play at the fourth and eighth seconds, the user has no way of seeing if they really work. When the controls are hidden, you lose not just Play and Pause but also the timeline. Fortunately, HTML5 provides a new <meter> element that’s excellent for measuring how much of a video or audio clip has been played:

<meter id="mymeter" min="0"></meter>

The value of the meter needs to be continually updated as the video is playing. For this you use the timeupdate event, adding that to the <video> element alongside the loadeddata event already being used to capture the filename of the video being played:

<video id="myvideo" ontimeupdate="updateTime(this);"
       onloadeddata="dataLoaded(this);">

Here’s the dataLoaded function. It’s been updated to set the max value of the <meter> element so that it exactly matches the duration of the loaded video:

function dataLoaded(v) {
    document.getElementById('source')
        .innerHTML = 'Playing file ' +
            v.currentSrc.slice(v.currentSrc.lastIndexOf('/')+1);
    m = document.getElementById('mymeter');
    m.max = v.duration;
    m.value = 0;
}

The code to update the meter element is even simpler; it just sets the value of the <meter> element to the currentTime of the <video> element:

function updateTime(v) {
    m = document.getElementById('mymeter');
    m.value = v.currentTime;
}

Integrating media with other content

The <video> and <audio> elements are just like any other element on the web page. They can be styled with CSS and used in JavaScript. To begin, let’s look at applying CSS transforms and transitions. The following three screenshots show the same web page over the course of 10 seconds.

The code for this example, slightly elided, is shown next; see the full listing in ch04/video-css-transitions.html. Don’t worry too much about the details for now; read the sections “2D transforms” and “CSS transitions” in chapter 9 for a more in-depth discussion. In the meantime, remember that anything you can do to HTML elements with CSS can be done to the <video> element:

div video {
    transition-duration: 10s;
}
div:hover video:nth-child(1) {
    transform-origin: bottom
right;
    transform: rotate(16.5deg);
}
div:hover video:nth-child(2) {
    transform-origin: top right;
    transform: rotate(33deg);
}
div:hover video:nth-child(3) {
    transform-origin: top left;
    transform: rotate(66deg);
}
<div>
  <video id="myvideo1"
width="160"
         autoplay loop>
      <source src="00092.webm"
              type="video/webm">
      <source src="00092.mp4"
              type="video/mp4">
      <source src="00092.low.mp4"
              type="video/mp4">
      <source src="00092.ogv"
              type="video/ogg">
      No video!
  </video>
  <video id="myvideo2" ...
  <video id="myvideo3" ...
</div>

In the previous chapter, you learned that the <canvas> element can grab an image from anywhere on the page and subject it to various transformations. One of the more exciting features of HTML5 is that those same canvas manipulation tricks also work with the <video> element. The next example looks at the basic process of getting a frame from the video into a <canvas> element by making a frame grabber.

First, let’s get the HTML sorted out. Start with the usual <video> element, and add as many sources as required; you can use the code from “Using multiple resources” as a starting point.

<video id="myvideo" controls>
    <source src="00092.webm"
      media="video/webm">
    No video!
</video>

You need something for the user to click to signal that they want to grab a frame. A <button> element is easiest:

<button onclick="snap();">
  Snap
</button>

Place that before the <video> element. You also need a <canvas> element to put the frame in later:

<canvas id="mycanvas"></canvas>

Put the <canvas> element after the div with id 'source'. All the action happens in the snap() function:

function snap() {
  var video  = document
    .getElementById('myvideo');
  var canvas = document
    .getElementById('mycanvas');
  canvas.width  = video.videoWidth;
  canvas.height = video.videoHeight;
  var ctx = canvas.getContext('2d');
  ctx.drawImage(video, 0, 0);
}

Most of this function is plumbing—grabbing references to the relevant elements and setting the width and height of the canvas to match the video. The code that draws the current frame on the canvas is this:

ctx.drawImage(video, 0, 0);

It’s that simple! If you want to have some fun, refer back to the canvas transformations in chapter 3 and try them on frames of a video.

The other thing you learned about in the previous chapter was SVG—in particular applying SVG effects such as transforms, clips, and masks to HTML content with <foreignObject>. Because the <video> element is HTML content just like any other, those same effects can be applied. The following screenshots show a <video> element clipped to appear inside some text and then animated.

Here’s the content of the SVG embedded into the HTML. Note that the SVG has HTML embedded into it in turn:

 

Does HTML5 video replace Flash?

The short answer is, no. There are several things for which Flash is the only option now, and some things for which HTML5 video is never likely to be an option. Flash has support for Real Time Streaming Protocol (RTSP) and the Real Time Messaging Protocol (RTMP), which provide facilities such as adaptive streaming, switching the bitrate of the video stream as the available bandwidth varies, and digital rights management (DRM).

It’s possible that HTML5 video will one day support adaptive streaming, but it’s extremely unlikely that it will ever support any features for DRM in a cross-browser fashion. If you want to use DRM on your video and audio content, then you’ll need to continue using Flash.

 

Browser support

Support for both <video> and <audio> elements is universal across all current browsers. The problem at this point is finding the minimum number of different encodings for maximum browser compatibility.

 

  12 14 4 6 8 9 10 11.5 12 5 5.1
<audio> element  
WAV audio      
MP3 audio          
OGG audio  
<video> element  
OGG video  
MP4 video          
WebM video  
Key: • Complete or nearly complete support Incomplete or alternative support
Little or no support

Web server configuration for audio and video

The first thing you need to consider when serving video and audio is that you have to make sure the correct MIME types are sent in the headers. The MIME type sent by the server should match the value set in the type attribute. On the common Apache server, this means using the AddType directive. This can go in the server configuration files or in an .htaccess file in your website directory. The relevant values for HTML5 audio and video are as follows:

AddType audio/ogg     oga ogg
AddType audio/mp4     m4a

AddType video/ogg     ogv
AddType video/mp4     mp4 m4v
AddType video/webm    webm

Supporting legacy browsers with Flash video

It’s possible to get the best of both worlds: HTML5 video for browsers that support it and Flash for browsers that don’t. At its simplest, this is a matter of wrapping the code for Flash inside the <video> element:

Browsers that support the <video> element will ignore the fallback content, whereas browsers that don’t support the <video> element will ignore that and only see the <object> element that embeds the Flash plug-in. Browsers that support neither the <video> element nor the Flash player will see the link to download the video.

Summary

In this chapter, you’ve learned about multimedia on the web, playing audio and video with simple markup. You’ve seen the benefits of having multimedia content integrated with the rest of your web page content and looked at manipulating that multimedia with JavaScript.