The video provides the best introduction:

Basically, you can now have your typist create a transcript of your video and then directly upload that to YouTube, which will use speech recognition to time-align the transcript and turn it into captions. That’s awesome! You can even download those captions and keep them for some other purpose, such as add them to your local archive of videos.

For some select partners, YouTube is even experimenting with a fully automated captioning system, where the transcript is created using speech recognition. Since we know the quality of Google voice from searches performed through Google searches on mobile phones, let’s be skeptical of the quality of the fully automated solution. But it is certainly a great start and we can certainly expect Google to continue making improvements to their speech recognition technology.

In my long years of working with video I have been encouraging everybody who publishes video to also provide textual representations of video, which includes subtitles/captions, but also includes metadata and hyperlinks that will enable video to become part of the content networks of the Web.

The key advantage for me is not accessibility, but it is to increase the value of the content. Content that knows more about itself and can expose that to machines is inherently more valuable than content that is just a dark collection of bits.

Added Value 1: automated translation

Once a time-aligned transcript such as a caption file is available, the video can expose this to a translation engine and provide itself in any language. This capability is now available for some videos on YouTube, e.g. the following winning Eurovision song of this year:

You have to click on the triangle icon on the video player bottom right while the video is playing, and then follow the red “CC” menu to go to “Translate” and turn on subtitles in a chosen language.

Even if Google’s automated translations are not 100% accurate, they still make the content accessible to a much larger audience than if they were not available. And all of this basically “for free” through the automated translation engine.

Added Value 2: increased user attention

Interestingly, recent research has shown that captions and subtitles don’t only make content more accessible to the hard-of-hearing, but also to well-hearing people. Where a video file has captions, 80 percent more people watch the entire video to its completion.

Achieving complete views is one of the most difficult challenges video publishers face, since people loose interest and attention fairly easily in our modern world of media over-stimulation. So, anything that can help people focus their attention longer is great news.

Just imagine the increase this can bring e.g. to the value of post-roll ads and to closing titles that contain the brands that sponsored the creation of the video.

Added Value 3: video SEO

Typically for a video the only text that is available and indexed by search engines are “title”, “description”, “tags”, and “categories”. These are fairly limited when you consider all the action and information that is inherent in a video.

Once a time-aligned transcript such as a caption file is available for a video, search engines are able to index that text together with the minimalist other text related to a video, thus making the video a whole lot more discoverable.

It is a shame that YouTube’s caption files are not yet indexed by Google, but do not fear: Google already has the technology and is using it on the Google video site:

We can only expect that it will be available on YouTube soon, so if you want to give your videos a huge SEO boost, think about uploading a caption file.

Further applications will certainly emerge to make better use of the annotated video content, such as automated summarisation, search that points us directly to offsets (see media fragment URIs), and automatically created mashups based on keywords.

If you want to do the captioning yourself, there are now some nice tools that work with YouTube. In Australia we also have captioning services that can do it for you, such as the Australian Captioning Centre and caption.it. Also check out Media Access Australia who have a large collection of resources and information about captioning in Australia.