The most important consideration in video encoding is choosing a video codec. A future article will talk about how to pick the one that’s right for you, but for now I just want to introduce the concept and describe the playing field. (This information is likely to go out of date quickly; future readers, be aware that this was written in December 2008.)

When you talk about “watching a video,” you’re probably talking about a combination of one video stream, one audio stream, and possibly some subtitles or captions. But you probably don’t have two different files; you just have “the video.” Maybe it’s an AVI file, or an MP4 file. These are just container formats, like a ZIP file that contains multiple kinds of files within it. The container format defines how to store the video and audio streams in a single file (and subtitles too, if any).

When you “watch a video,” your video player is doing several things at once:

1. Interpreting the container format to find out which video and audio tracks are available, and how they are stored within the file so that it can find the data it needs to decode next
2. Decoding the video stream and displaying a series of images on the screen
3. Decoding the audio stream and sending the sound to your speakers
4. Possibly decoding the subtitle stream as well, and showing and hiding phrases at the appropriate times while playing the video

A video codec is an algorithm by which a video stream is encoded, i.e. it specifies how to do #2 above. Your video player decodes the video stream according to the video codec, then displays a series of images, or “frames,” on the screen. Most modern video codecs use all sorts of tricks to minimize the amount of information required to display one frame after the next. For example, instead of storing each individual frame (like a screenshot), they will only store the differences between frames. Most videos don’t actually change all that much from one frame to the next, so this allows for high compression rates, which results in smaller file sizes. (There are many, many other complicated tricks too, which I’ll dive into in a future article.)

There are lossy and lossless video codecs; today’s article will only deal with lossy codecs. A lossy video codec means that information is being irretrievably lost during encoding. Like copying an audio cassette tape, you’re losing information about the source video, and degrading the quality, every time you encode. Instead of the “hiss” of an audio cassette, a re-re-re-encoded video may look blocky, especially during scenes with a lot of motion. (Actually, this can happen even if you encode straight from the original source, if you choose a poor video codec or pass it the wrong set of parameters.) On the bright side, lossy video codecs can offer amazing compression rates, and many offer ways to “cheat” and smooth over that blockiness during playback, to make the loss less noticeable to the human eye.

There are tons of video codecs. Today I’ll discuss five modern lossy video codecs: MPEG-4 ASP, H.264, VC-1, Theora, and Dirac.

MPEG-4 ASP

a.k.a. “MPEG-4 Advanced Simple Profile.” MPEG-4 ASP was developed by the MPEG group and standardized in 2001. You may have heard of DivX, Xvid, or 3ivx; these are all competing implementations of the MPEG-4 ASP standard. Xvid is open source; DivX and 3ivx are closed source. The company behind DivX has had some mainstream success in branding “DivX” as synonymous with “MPEG-4 ASP.” For example, this “DivX-certified” DVD player can actually play most MPEG-4 ASP videos in an AVI container, even if they were created with a competing encoder. (To confuse things even further, the company behind DivX has now created their own container format.)

MPEG-4 ASP is patent-encumbered; licensing is brokered through the MPEG LA group. MPEG-4 ASP video can be embedded in most popular container formats, including AVI, MP4, and MKV.

H.264

a.k.a. “MPEG-4 part 10,” a.k.a. “MPEG-4 AVC,” a.k.a. “MPEG-4 Advanced Video Coding.” H.264 was also developed by the MPEG group and standardized in 2003. It aims to provide a single codec for low-bandwidth, low-CPU devices (cell phones); high-bandwidth, high-CPU devices (modern desktop computers); and everything in between. To accomplish this, the H.264 standard is split into “profiles,” which each define a set of optional features that trade complexity for file size. Higher profiles use more optional features, offer better visual quality at smaller file sizes, take longer to encode, and require more CPU power to decode in real-time.

To give you a rough idea of the range of profiles, Apple’s iPhone supports Baseline profile, the AppleTV set-top box supports Baseline and Main profiles, and Adobe Flash on a desktop PC supports Baseline, Main, and High profiles. YouTube (owned by Google, my employer) now uses H.264 to encode high-definition videos, playable through Adobe Flash; YouTube also provides H.264-encoded video to mobile devices, including Apple’s iPhone and phones running Google’s Android mobile operating system. Also, H.264 is one of the video codecs mandated by the Blu-Ray specification; Blu-Ray discs that use it generally use the High profile.

Most non-PC devices that play H.264 video (including iPhones and standalone Blu-Ray players) actually do the decoding on a dedicated chip, since their main CPUs are nowhere near powerful enough to decode the video in real-time. Recent high-end desktop graphics cards also support decoding H.264 in hardware. There are a number of competing H.264 encoders, including the open source x264 library. The H.264 standard is patent-encumbered; licensing is brokered through the MPEG LA group. H.264 video can be embedded in most popular container formats, including MP4 (used primarily by Apple’s iTunes Store) and MKV (used primarily by video pirates).

VC-1

VC-1 evolved from Microsoft’s WMV9 codec and was standardized in 2006. It is primarily used and promoted by Microsoft for high-definition video, although, like H.264, it has a range of profiles to trade complexity for file size. Also like H.264, it is mandated by the Blu-Ray specification, and all Blu-Ray players are required to be able to decode it. The VC-1 codec is patent-encumbered, with licensing brokered through the MPEG LA group.

Wikipedia has a brief technical comparison of VC-1 and H.264; Microsoft has their own comparison; Multimedia.cx has a pretty Venn diagram outlining the similarities and differences. Multimedia.cx also discusses the technical features of VC-1. I also found this history of VC-1 and H.264 to be interesting (as well as this rebuttal).

VC-1 is designed to be container-independent, although it is most often embedded in an ASF container. An open source decoder for VC-1 video was a 2006 Google Summer of Code project, and the resulting code was added to the multi-faceted ffmpeg library.

Theora

Theora evolved from the VP3 codec and has subsequently been developed by the Xiph.org Foundation. Theora is a royalty-free codec and is not encumbered by any known patents other than the original VP3 patents, which have been irrevocably licensed royalty-free. Although the standard has been “frozen” since 2004, the Theora project (which includes an open source reference encoder and decoder) only hit 1.0 in November 2008.

Theora video can be embedded in any container format, although it is most often seen in an Ogg container. All major Linux distributions support Theora out-of-the-box, and Mozilla Firefox 3.1 will include native support for Theora video in an Ogg container. And by “native”, I mean “available on all platforms without platform-specific plugins.” You can also play Theora video on Windows or on Mac OS X after installing Xiph.org’s open source decoder software.

The reference encoder included in Theora 1.0 is widely criticized for being slow and poor quality, but Theora 1.1 will include a new encoder that takes better advantage of Theora’s features, while staying backward-compatible with current decoders. (Info: 1, 2, 3, 4, 5, source code.)

Dirac

Dirac was developed by the BBC to provide a royalty-free alternative to H.264 and VC-1 that the BBC could use to stream high-definition television content in Great Britain. Like H.264, Dirac aims to provide a single codec for the full spectrum of very low- and very high-bandwidth streaming. Dirac is not encumbered by any known patents, and there are two open source implementations, dirac-research (the BBC’s reference implementation) and Schroedinger (optimized for speed).

The Dirac standard was only finalized in 2008, so there is very little mainstream use yet, although the BBC did use it internally during the 2008 Olympics. Dirac-encoded video tracks can be embedded in several popular container formats, including MP4, Ogg, MKV, and AVI. VLC 0.9.2 (released in September 2008) can play Dirac-encoded video within an Ogg or MP4 container.

And on and on…
Of course, this is only scratching the surface of all the available video codecs. Video encoding goes way back, but my focus in this series is on the present and near-future, not the past. If you like, you can read about MPEG-2 (used in DVDs), MPEG-1 (used in Video CDs), older versions of Microsoft’s WMV family, Sorenson, Indeo, and Cinepak.

The Internet is not simply a big place it is a huge place; new content is being created all the time. Google, Yahoo and Microsoft each have a finite number of resources, so when faced with the nearly-infinite quantity of content that’s available online, their various crawlers are only able to find and crawl a percentage of that content. Then, of all the content they’ve crawled, they’re only able to index a portion. Of course with the cheapness of storage, the search engines are able to index more and more content each day, but not at the pace the Web is growing.

URLs are like the bridges between your website and a search engine’s crawler: crawlers need to be able to find and cross those bridges (i.e., find and crawl your URLs) in order to get to your site’s content. If your URLs are complicated or redundant, crawlers are going to spend time tracing and retracing their steps; if your URLs are organised and lead directly to distinct content, crawlers can spend their time accessing your content rather than crawling through empty pages, or crawling the same content over and over via different URLs.

So, what can you do as a website developer or owner to reduce that labyrinth of URLs and helping crawlers find more of your content faster? Below are a few ideas:

• Remove unnecessary query string details from the URL.
  Parameters in the URL that don’t change the content of the page–like session IDs or list sort orders–can be removed from the URL and put into a cookie. By putting this information in a cookie and 301 redirecting to a clean URL, you retain the information and reduce the number of URLs pointing to that same content.
• Stop infinite pagination in, for example, lists and calendars.
  If you have a calendar with infinite past and future dates or a list with infinite pagination you have what is described as an infinite crawl space, which is a huge burden on crawlers. To resolve the calendar issue, you can add no-follow attributes to links to dynamically created future calendar pages. When creating pagination links, disable previous and next links when the first and last pages are reached and redirect users to an appropriate page if the query string in the URL is hacked (this may be a page not found static page).
• Utilise the robots.txt file to prevent actions the web crawlers can’t or shouldn’t perform.
  Using a robots.txt file, you can disallow crawling of login pages, contact forms, shopping carts, and other pages whose sole functionality is something that a crawler can’t and shouldn’t perform. This lets crawlers spend more of their time crawling content that they can actually do something with.
• Prevent duplicate content.
  An ideal scenario for crawlers is a one-to-one link between content an a URL. Each URL leads to a unique bit of content and each piece of content can be accessed by a unique URL. The closer your site can get to this scenario, the more streamlined your site will be for crawling and indexing. If your CMS makes this difficult to achieve, you can use the canonical tag to indicate a preferred URL for duplicate content.

More information on this topic can be found on the Google Webmaster Central Blog.

Carpe diem on any duplicate content worries: we now support a format that allows you to publicly specify your preferred version of a URL. If your site has identical or vastly similar content that’s accessible through multiple URLs, this format provides you with more control over the URL returned in search results. It also helps to make sure that properties such as link popularity are consolidated to your preferred version.

But what do they mean by canonical? One of the definitions of canonical is reduced to the simplest and most significant form possible without loss of generality.

What this means is that if you have a page–let’s take an e-commerce product page–and the simplest URL that you want it accessible by is:

http://www.site.com/category/product.html

you can add the canonical tag to that specific product. Google, Yahoo and Microsoft use this tag to tell their search engines which URL it should have for the current page.

Now, let’s say that the particular software you use also allows you to access the same product using:

http://www.site.com/company/product.html

and

http://www.site.com/different_category/product.html

Perhaps this one product is in multiple categories. With this tag in place when any of the alternate pages are loaded this tag notifies any search engine that this is really the same product as the page you defined in the canonical tag. So, you are still allowed to have the content available as generally needed (by categories, tags, or some other organisation system) and still avoid having the content duplicated and penalised.

To implement the canonical URL tag in your web application, you simply need to do the following inside the <head> section of the duplicate content URLs:

<link rel="canonical" href="http://www.site.com/category/product.html" />

As Google mention, this tag is a hint that they honour strongly. Google will take your preference into account, in conjunction with other signals, when calculating the most relevant page to display in search results.

• Don’t reinvent the wheel unless you really have to. Borrow code and ideas from elsewhere whenever it makes sense. The web community it great at sharing, just look at the various JavaScript libraries, the huge quantities of APIs or indeed the major players’ developer areas: Google Code, Yahoo! Developer Network, Mozilla Developer Center, Adobe Developer Connection and Dev Opera to name five I regularly refer to.
• Things should be as simple as possible, but no simpler (Einstein). This idea is really born out of and emphasised by 37Signals’ Getting Real book. Commonly, 90% of people using an application only use 10% of it’s functionality. The key therefore is to find what people use most often and only build that functionality. If there is a requirement to add more, then sobeit. This can also apply to the code-level, the essence here being a balance between over- and under-engineering something.
• Do one thing well (The UNIX philosophy). It is better to do one thing well, than several second-rate. This could be at the code level — think encapsulation, coupling and cohesion — or indeed at the application level — you’re never going to beat Microsoft Word, but Google and Zoho have developed compelling alternatives, but with far less features.
• Don’t fret too much about performance — understand how to write efficient code and plan to optimise later if or when needed.
• Don’t try for perfection because good enough is often just that. This of course is a matter for conjecture. If I were working on a personal project, I may be more stringent on perfection than say, for a client’s application. This doesn’t mean to say the client’s application would be any worse, but rather it is a question of dotting-the-is and crossing-the-ts. It also depends on your perspective and what gains can be made by aiming for perfection.
• (Hence) it’s okay to cut corners sometimes, only if you can do it right later. I rarely adhere to this! It makes sense to do it right the first time, since bodge-jobs often come back to haunt you and result in double the effort!
• Don’t fight it; go with the flow. This is somewhat clichéd, but the essence behind this is try to avoid getting stressed out. This isn’t always easy to achieve, but taking a step back from a situation and avoiding politics is important.

I often strive for perfection, which isn’t an entirely clever pursuit since it is almost impossible to achieve. However, in a realm of imperfection, the principles above have helped me to achieve a modicum of decent code throughout the years. They may also resonate and provide inspiration for you.

For example, the following is WordPress’ default URL configuration for a post:

http://www.domain.com/?p=1635

However, buy using a URL-rewriting available in the Apache webserver, we can achieve a far better result, such as the following:

http://www.domain.com/search-engine-safe-urls

NB. It is also possible to achieve a similar result with an ISAPI rewrite for Microsoft’s IIS webserver, but this topic will not be included in this post.

To get your website working with SES URLs you need to enable both the mod_rewite module and AllowOverride directive in the Apache configuration file.

Uncomment (remove #) from the following to enable the re-write rule:

LoadModule rewrite_module modules/mod_rewrite.so

Change the AllowOverride directive from none to all

<directory />
    Options FollowSymLinks
    AllowOverride all
    Order deny,allow
    Deny from all
</directory>
 
<directory "C:/WebRoot">
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.2/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks
 
    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   Options FileInfo AuthConfig Limit
    #
    AllowOverride All
 
    #
    # Controls who can get stuff from this server.
    #
    Order allow,deny
    Allow from all
</directory>

On Apache webservers, .htaccess (hypertext access) is the default name of directory-level configuration files. An .htaccess file is placed in a particular directory, and the directives in the .htaccess file apply to that directory, and all its subdirectories. It provides the ability to customize configuration for requests to the particular directory. In our case, enabling search engine safe (SES) URLs.

By setting the AllowOverride directive to All in effect defers configuration settings to the .htaccess file.

An example .htaccess file could include the following code to rewrite the URLs:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php/$1 [L,QSA]

Search engine friendly URLs are implemented with Rewrite engines. The rewrite engine modifies the URL based upon a number of rewrite conditions and rules.

The RewriteBase directive explicitly sets the base URL for per-directory rewrites. The RewriteCond directive defines a rule condition, so in this case handling missing files or directories. Finally, the RewriteRule directive is the real rewriting workhorse. In this example, we’re getting everything in the URI — i.e. not including the protocol (HTTP/S) and domain name — based upon a regular expression. This is then appended to the default file reference — index.php — as a back reference. The [L,QSA] refers to the rule being the last rule and append any query string parameters to the default file. It is important to note that this is all done on the server side, the user will never see the website address changing in the browser’s address bar. Furthermore, simply transposing the index.php filename with your default file name — e.g. index.cfm, default.aspx — will have the same result. Indeed, the above rewrite rules are becoming a de-facto standard for web applications.

To fully understand mod_rewrite rules above, look at the Apache mod_rewrite documentation.

Once you have your SES functionality in place on the webserver, it is then the responsibility of your application framework to understand the URL construction and handle it accordingly. Fortunately, frameworks such as ColdBox and Fusebox for ColdFusion, Zend and Symfony for PHP, all contain functionality to do this, but that is the subject of an entirely different post.

Users of web applications prefer short, neat URLs to raw query string parameters. A concise URL is easy to remember, and less time-consuming to type in. If the URL can be made to relate clearly to the content of the page, then errors are not only less likely to happen, but our good friends the search engine robots are able to draw a stronger assumption of the pages’ relevance and content.

To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file. Conversely and importantly, webmasters can also notify the search engines about the existence and importance of pages with a sitemap.xml file. (Both files are placed in the root directory of the domain.)

Fortunately for the webmaster, the major search engines provide various tools to help manage both Sitemap and Robot files.

To gain an understanding of both ‘protocols’, I’ll discuss them briefly below.

Sitemaps (Inclusion Protocol)

The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about each URL: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol.

The webmaster can generate a Sitemap containing all accessible URLs on the site and submit it to search engines. Since Google, MSN, Yahoo!, and Ask use the same protocol now, having a Sitemap would let the biggest search engines have the updated pages information.

Sitemaps supplement and do not replace the existing crawl-based mechanisms that search engines already use to discover URLs. By submitting Sitemaps to a search engine, a webmaster is only helping that engine’s crawlers to do a better job of crawling their site(s). Using this protocol does not guarantee that web pages will be included in search indexes, nor does it influence the way that pages are ranked in search results.

The following is a cut-down version of the sitemap.xml for this website. WordPress, via a plugin, automatically updates this file each time a new post or page is written.

<urlset xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>http://www.simonwhatley.co.uk/</loc>
<lastmod>2008-10-08T14:50:16+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>
http://www.simonwhatley.co.uk/big-city-little-people
</loc>
<lastmod>2008-10-08T14:50:16+00:00</lastmod>
<changefreq>monthly</changefreq>
<priority>0.1</priority>
</url>
</urlset>

More information about sitemaps can be found on the Sitemaps.org website.

Robots (Exclusion Protocol)

The robot exclusion standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website which is otherwise publicly viewable. Robots are often used by search engines to categorise and archive web sites. The standard complements Sitemaps, a robot inclusion standard for websites.

A robots.txt file on a website will function as a request that specified robots ignore specified files or directories in their search. This might be, for example, out of a preference for privacy from search engine results, or the belief that the content of the selected directories might be misleading or irrelevant to the categorisation of the site as a whole.

The protocol, however, is purely advisory. It relies on the cooperation of the web robot, so that marking an area of a site out of bounds with robots.txt does not guarantee privacy. Some web site administrators have tried to use the robots file to make private parts of a website invisible to the rest of the world, but the file is necessarily publicly available and its content is easily checked by anyone with a web browser.

For example, the following tells all crawlers not to enter four directories of a website:

User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /tmp/
Disallow: /private/

Exclusion can also be achieved on a page-level basis using a Meta-tag. This is a tag that would be placed in the HTML head of of a web page. The robots attribute controls whether search engine spiders are allowed to index a page, or not, and whether they should follow links from a page, or not.

A common example could be as follows:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en-GB" xml:lang="en">
 <head profile="http://gmpg.org/xfn/11">
	<title>Simon Whatley</title>
	<meta http-equiv="robots" content="index,follow" />
</head>
<body>
</body>
</html>

A word of caution though, Meta tags are not the best option to prevent search engines from indexing content of your website.

More information about Robots.txt files can be found on the Robotstxt.org website.

Webmaster Tools

The top 3 search providers all have their own webmaster tools admin interface. The Google offering is the most advanced, but it’s good practice to use and submit information to all three.

Links to their services are provided below:

• Google Webmasters
• Yahoo! Site Explorer
• Live Search Webmaster Centre

Ask doesn’t have an interface. However, you can still ping their Submission Service using the URL http://submissions.ask.com/ping?sitemap= in conjunction with your sitemap URL.

Further Information

• Sitemap Generator
• Sitemap 3rd Party porgrams and websites
• Webmaster Toolkit

The syntax for this string function is the same for SQL Server, Oracle and Microsoft Access. The syntax is as follows:

REPLACE(stringToLookIn, stringToMatch, replacementsString)

The syntax is fairly straight forward, the stringToMatch parameter is the character phrase that we want to replace, the replacementsString is the character phrase that will replace any occurence of the stringToMatch parameter. If the stringToMatch phrase occurs more than once in the string, then all instances of the phrase will be replaced with the replacement string. If no matches were found then the string is returned unaltered.

If we want to match multiple items, we need to nest the REPLACE function:

REPLACE(REPLACE(stringToLookIn, stringToMatch, replacementsString), stringToMatch, replacementsString)

or set the replaced string into a new variable multiple times:

stringReturned = REPLACE(stringToLookIn, stringToMatch, replacementsString)
stringReturned = REPLACE(stringReturned, stringToMatch, replacementsString)

This is far from ideal, especially the more strings there are to be matched. This is where User-Defined Functions (UDFs) can provide the answer.

A User-Defined Function, is a function provided by the user of a program or environment. In SQL databases, a user-defined function provides a mechanism for extending the functionality of the database server by adding a function that can be evaluated in SQL statements.

The Function Code

Below is the complete function definition:

<|/1/>CREATE <|/1/>FUNCTION dbo.udf_ReplaceChars
(
@ReplaceList		<|/2/>VARCHAR(<| style="color: #cc66cc;">50),
@String			<|/2/>VARCHAR(<| style="color: #cc66cc;">100)
)
<|/1/>RETURNS <|/2/>VARCHAR(<| style="color: #cc66cc;">100)
<|/1/>AS
<|/1/>BEGIN
	<|/1/>DECLARE	@<|/2/>CHAR		<|/2/>CHAR(<| style="color: #cc66cc;">1),
		@Loop		<|/2/>INT
 
	<|/1/>SET @Loop  = <| style="color: #cc66cc;">0
	<|/1/>WHILE @Loop < = LEN(@ReplaceList)
	<|/1/>BEGIN
		<|/1/>SET	@Loop = @Loop + <| style="color: #cc66cc;">1
		<|/1/>SET	@<|/2/>CHAR = SUBSTRING(@ReplaceList, @Loop, <| style="color: #cc66cc;">1)
		<|/1/>SET	@String = REPLACE(@String, @<|/2/>CHAR, '')
	<|/1/>END
 
	<|/1/>RETURN		@String
 
<|/1/>END
<|/1/>GO

The function simply loops over the replace list, finding each instance of the list item in the string in which we want to replace items. The new string is then returned out of the function.

The Function In Use

A very simple use of the replace function could be as follows:

<|/1/>SELECT dbo.udf_ReplaceChars('=,/,<,>,@,~,#', columnName) <|/1/>AS newColumn, columnName
<|/1/>FROM tableName

The function is not restricted to SELECT statements. Below is an example of an UPDATE statement utilising a variable:

<|/1/>UPDATE tableName
<|/1/>SET columnName = dbo.udf_ReplaceChars('=,/,<,>,@,~,#', @variableName)
<|/1/>WHERE idName = @myId

Download the Code

Download the code, rename the file to .sql and run on your database instance. You will then be able to reference the function in your Stored Procedures.

But what is a User Agent? A User Agent is the client application used with a particular network protocol; the phrase is most commonly used in reference to those which access the Web. Web user agents range from web browsers and e-mail clients to search engine crawlers (spiders), as well as mobile phones, screen readers and braille browsers used by people with disabilities. When Internet users visit a web site, a text string is generally sent to identify the user agent to the server. This forms part of the HTTP request, prefixed with user-agent: and typically includes information such as the application name, version, host operating system, and language. Bots, such as web crawlers, often also include a URL and/or e-mail address so that the webmaster can contact the operator of the bot.

By simply typing about:version into Chrome’s address bar you will be presented with the following information:

Google Chrome
0.2.149.29 (1798)
Official Build
Google Inc.
Copyright © 2006-2008 Google Inc. All Rights Reserved.
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13

As you can see Chrome’s version information provides limited detail about the browser. The last line is the important one. It is the HTTP User-Agent header:

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.2.149.29 Safari/525.13.

If you know the RFC 2616 specification on the HyperText Transfer Protocol — which incidentally, I gladly don’t — you would know that the User Agent, or more formally, product token, should be short and to the point:

Product tokens SHOULD be short and to the point. They MUST NOT be used for advertising or other non-essential information. Although any token character MAY appear in a product-version, this token SHOULD only be used for a version identifier (i.e., successive versions of the same product SHOULD only differ in the product-version portion of the product value).

Clearly this isn’t the case! One of Google’s reason’s behind creating the Chrome browser was to start afresh. It would have therefore been truely amazing if they had made the string simply Chrome/0.2.149.27.

Unfortunately, browser sniffing makes an ever-growing UA string the path of least resistance for browser vendors.

So, what does Chrome’s User Agent string actually mean:

• Mozilla/ – This means that browser has the kind of capabilities that Netscape 1.1 had compared to Mosaic and Lynx.
• 5.0 – This means that the browser engine is from the post-Browser War Web Standards era as opposed to being from the Browser War era.
• (Windows; – This means that general windowing system flavor the browser runs on is Windows (as opposed to, for example, Apple and X11).
• U; – This means that the browser has at least the level of cryptographic capability / encryption strength that U.S. versions of browsers had in the late 1990s.
• Windows NT 6.0; – This indicates the operating system the browser is running on. In this instance, the browser is running on Vista.
• en-US) – This indicates the user interface language of the browser (U.S. English in this case). This may be used to choose between different content languages even though HTTP has a different header for that purpose.
• AppleWebKit/ – This indicates that the engine of the browser is WebKit as opposed to being Gecko. Developers should not do user agent sniffing as a rule, but if they still do, this is what they should be sniffing.
• 525.13 – This is the WebKit version from which Chrome branched its copy. Site admins could use this to detect old versions with known bugs.
• (KHTML, like Gecko) – This introduces the substring Gecko into the UA string while pointing out to human readers that Webkit was forked from KHTML. Without this substring, Chrome might be put in the same category as IE and Netscape 4.
• Chrome/ – This string identifies the browser as actually Google Chrome.
• 0.2.149.27 – This is the Chrome version. This could be used to detect old versions with known bugs.
• Safari/ – This means that the browser is like Safari as opposed to being like Firefox.
• 525.13 – This just repeats the WebKit version in order to have some version but not the irrelevant Safari.app version.

In what is likely to be better news for men than women, Microsoft’s latest browser, Internet Explorer 8, boasts a feature that allows users to hide the trail of their web browsing.

The feature, predictably nicknamed “porn mode”, stops casual users and, crucially, online advertisers from seeing a browser’s audit trail. This means that advertisers cannot easily target adverts based upon a user’s viewing habits and conversely, advert providers, such as Google’s Adsense, cannot easily reimburse members of their content network since they do not know where the user clicked an advert.

In the dim and distant past, the job of publicising yourself was extremely difficult. Can you imagine life without the Internet, mobile telephones and email? How did people ever do business? With the advent of the World Wide Web and in particular search engines and blogging, this all changed and a wealth of opportunity has become available, especially to the freelancer.

But where do you start?

Create a Brand

Creating a brand is a great way to market yourself. This does not have to be the same as your company, and through time you may set up different brands for different sectors or ideas you may have. Brands serve to create associations and therefore, expectations of products you create, so a good brand name is a great way to get recognised in your community.

You can register the brand in the UK, Europe and the US, although the latter requires a US address. It is also not a given that your brand registration will be successful, making it a costly exercise. Careful consideration is what is needed here.

Create an Avatar

Avatars are images or icons that represent you in the online world. They are an extension of your brand. For example, the header of my website is also my favicon and avatar on various online services. It is a great way for people to draw an association between your online presence and you.

Create a Blog

Blogs are a great way to get yourself known and therefore heard amongst your peer group. Your blog should really be an extension of your brand and is a great avenue to showcase your skills, demo example applications, code and designs, or simply give your opinion on a subject.

I use the excellent WordPress blogging application, in a self-hosted environment. You don’t need to do this since there is a hosted version at WordPress.com, or you could use Blogger, another popular blogging platform, provided by Google.

The key to blogging is talk about what you enjoy, don’t just keep it technical. Blogs should be an extension of you, not an avenue for pretentious comment; you’ll soon be found out!

If you go the self-hosted route, you’ll need a domain name, hosting provider and obviously a blog application. I have listed a few below that can get you started.

Domain Names:

• Nominet
• Easily
• EuroDNS

Hosting Providers:

• HostMySite
• Titan Internet
• Flint Hosts
• UKHost4U
• 1and1

Blog Applications:

• WordPress (free)
• Moveable Type (free)
• ExpressionEngine (free)
• TypePad

If going the self-hosted is all too complicated for you or you simply don’t want the hassle that is associated with self-hosting, all is not lost. WordPress.com and Blogger are for you.

Blog Hosting Providers:

• WordPress.com
• Blogger

Both services take the onus away from the user when it comes to management (backups, plugins etc). At the simplest level, all you need to do is create and publish the content.

Join feed aggregators

To get noticed in the blogosphere, you can’t simply rely on the Google, Yahoo! and Microsoft search engines ranking your site. You will need to alert your peers to the fact that you’ve created some content that is worth reading. You can achieve this with feed aggregators.

Below I list a few that I use:

• Adobe
• Full-as-a-Goog
• ColdFusionBloggers
• Feed Squirrel
• London Bloggers

If you use WordPress, then you’re in luck. WordPress has a service called Ping-o-matic, which updates different search engines when your blog has been updated. You can also add your own services to ping and therefore notify the service of new content.

Comment on Blogs

Commenting on blogs is another great way of getting yourself known as well as offering an opinion. Since comments allow you to include a link back to your website, try and comment as your brand.

One tip, try not to be defamatory towards the blog owner, or others unless you have a strong justification for doing so. It’s all about the karma!

Join Micro-Blogging Services

If blogging is not your thing or you don’t have time to write articles, there are a number of blogging and, more importantly, micro-blogging services available to you that allow you to get your thoughts out into the wide-world.

Such services include the not-always-venerable Twitter, the feature rich Pownce, the new kid on the block Plurk and the blogging service, Tumblr.

Building a following will allow you to announce to your followers important events and ask questions of them.

What’s Next

In the next part of this series, I’ll talk about networking, a natural extension to publicising yourself on the web.