How to Learn from Failure

Too often we assume that a failed experiment is a wasted effort. But not all anomalies are useless.

  • Check your assumptions
    Ask yourself why this result feels like a failure. What theory does this contradict? Maybe the hypothesis failed, not the experiment.
  • Seek out the ignorant
    Talk to people who are unfamiliar with your experiment. Explaining your work in simple terms may help you see it in a new light.
  • Beware of failure blindness
    It’s normal to filter out information that contradicts our preconceptions. The only way to avoid that bias is to be aware of it.
  • Encourage diversity
    If everyone working on a problem speaks the same language, then everyone has the same set of assumptions.

Excerpt from February’s UK edition of Wired.

The Tragedy of the Commons

The tragedy of the commons refers to a dilemma described in an influential article by that name written by Garrett Hardin and first published in the journal Science in 1968. The article describes a situation in which multiple individuals, acting independently, and solely and rationally consulting their own self-interest, will ultimately destroy a shared limited resource even when it is clear that it is not in anyone’s long-term interest for this to happen.

Central to Hardin’s article is an example, of a hypothetical and simplified situation from medieval land tenure in Europe, of herders sharing a common parcel of land, on which they are each entitled to let their cows graze. In Hardin’s example, it is in each herder’s interest to put the next (and succeeding) cows he acquires onto the land, even if the carrying capacity of the common is exceeded and it is temporarily or permanently damaged for all as a result. The herder receives all of the benefits from an additional cow, while the damage to the common is shared by the entire group. If all herders make this individually rational economic decision, the common will be depleted or even destroyed to the detriment of all.

Meaning

The metaphor illustrates the argument that free access and unrestricted demand for a finite resource ultimately reduces the resource through over-exploitation, temporarily or permanently. This occurs because the benefits of exploitation accrue to individuals or groups, each of whom is motivated to maximize use of the resource to the point in which they become reliant on it, while the costs of the exploitation are borne by all those to whom the resource is available (which may be a wider class of individuals than those who are exploiting it). This, in turn, causes demand for the resource to increase, which causes the problem to snowball to the point that the resource is depleted (even if it retains a capacity to recover). The rate at which exhaustion of the resource is realized depends primarily on three factors: the number of users wanting to consume the common in question, the consumptiveness of their uses, and the relative robustness of the common.

Modern Commons

The tragedy of the commons can be applied to environmental issues such as sustainability. The commons dilemma stands as a model for a great variety of resource problems in society today, such as water, land, fish, and non-renewable energy sources like oil and coal. When water is used at a higher rate than the reservoirs are replenished, fish consumption exceeds its reproductive capacity, or oil supplies are exhausted, then we face a tragedy of the commons.

However, the commons that is likely to have the greatest impact on our lives in the new century is the digital commons, the information available on the Internet through the companies that provide access. The problem with digital information is the mirror image of the original grazing commons: Information is no longer costly to generate and organise. However, the information that is provided is inadequately catalogued and organised, whilst its value to individual consumers is too dispersed and small to establish an effective market. Furthermore, the Internet tends to fill with low-value information: The products that have high commercial value are marketed through revenue-producing channels, and the Internet becomes inundated with products that cannot command these values. Self-published books and music are good examples of this.

It is suggested that management of the digital commons is the most critical issue of market design that our market faces with several business models, all based around charging for content or access to content, being muted. This raises the issue of Network Neutrality. The free-spirited Internet user may balk at the market models in which consumers pay for content either through an ISP or monopoly control of information.

Controversy

Historical studies have shown that Hardin’s account of the breakdown of common grazing land was inaccurate, and that such commons were effectively managed to prevent overgrazing.

More significantly, controversy has been fueled by the application of Hardin’s ideas to current policy issues. In particular, some authorities have read Hardin’s work as specifically advocating the privatisation of commonly owned resources. Consequently, resources that have traditionally been managed communally by local organisations have been enclosed or privatised. Ostensibly, this serves to protect such resources, but it ignores the pre-existing management, often appropriating resources and alienating indigenous (and frequently poor) populations. In effect, private or state use may result in worse outcomes than the previous commons management.

Net neutrality advocates argue that allowing companies, or what is termed content gatekeepers, to demand a toll to guarantee quality or premium delivery would create and unfair business model. Advocates warn that by charging every Web site, from the smallest blogger to Google, network owners may be able to block competitor Web sites and services, as well as refuse access to those unable to pay.

One of the enchanting features of the Internet is its ability to harness free-wheeling innovation. ignoring network neutrality in favour of market models is bad news for innovation.

The web’s extreme openness, its capacity to allow anyone to connect to virtually anyone else, generates untold possibilities for collaboration. The more connected we are, the richer we should become (not just monetarily). This is because we will be able to connect to a diverse number of people, across continents, to combine their ideas, talents and resources in a way that should benefit everyone.

As Charles Leadbeater suggests in his book, We Think, the Web’s potential for good stems from the open, collaborative culture it inherited from its birthplace in the counter-culture of the 1960s, combined with folk culture and the commons as a shared basis for productive endeavour.

The Web allows for a massive expansion in individual participation in culture and the economy, all working towards solving issues such as the tradegy of the commons.

The Pareto principle (also known as the 80/20 rule, the law of the vital few, and the principle of factor sparsity) states that, for many events, roughly 80% of the effects come from 20% of the causes.

The original observation was in connection with income and wealth. Pareto noticed that 80% of Italy’s wealth was owned by 20% of the population. He then carried out surveys on a variety of other countries and found to his surprise that a similar distribution applied.

The principle has become a common rule of thumb in business; e.g. 80% of your sales come from 20% of your clients.

The principle is also an illustration of a Power law relationship, which occurs often in natural phenomena such as brush-fires and earthquakes. Because it is holds true over a wide range of magnitudes, it produces outcomes completely different traditional prediction schemes. It has been claimed, for example, that it explains the frequent breakdowns of sophisticated financial instruments. This is also likely true of any complex system, including social ones, and increasingly social and community dynamics are seen as falling under this rule in numerous ways, from participation pyramids to abandonment rates.

My Work Philosophy

Okay, so many of the points below aren’t purely my philosophy, but ideas and principles I have picked up along the way throughout my [development] career. Some relate to the UNIX philosophy, or even the Zen of Python, but wherever they’re from, they can be applied to many other domains.

  • Don’t reinvent the wheel unless you really have to. Borrow code and ideas from elsewhere whenever it makes sense. The web community it great at sharing, just look at the various JavaScript libraries, the huge quantities of APIs or indeed the major players’ developer areas: Google Code, Yahoo! Developer Network, Mozilla Developer Center, Adobe Developer Connection and Dev Opera to name five I regularly refer to.
  • Things should be as simple as possible, but no simpler (Einstein). This idea is really born out of and emphasised by 37Signals’ Getting Real book. Commonly, 90% of people using an application only use 10% of it’s functionality. The key therefore is to find what people use most often and only build that functionality. If there is a requirement to add more, then sobeit. This can also apply to the code-level, the essence here being a balance between over- and under-engineering something.
  • Do one thing well (The UNIX philosophy). It is better to do one thing well, than several second-rate. This could be at the code level — think encapsulation, coupling and cohesion — or indeed at the application level — you’re never going to beat Microsoft Word, but Google and Zoho have developed compelling alternatives, but with far less features.
  • Don’t fret too much about performance — understand how to write efficient code and plan to optimise later if or when needed.
  • Don’t try for perfection because good enough is often just that. This of course is a matter for conjecture. If I were working on a personal project, I may be more stringent on perfection than say, for a client’s application. This doesn’t mean to say the client’s application would be any worse, but rather it is a question of dotting-the-is and crossing-the-ts. It also depends on your perspective and what gains can be made by aiming for perfection.
  • (Hence) it’s okay to cut corners sometimes, only if you can do it right later. I rarely adhere to this! It makes sense to do it right the first time, since bodge-jobs often come back to haunt you and result in double the effort!
  • Don’t fight it; go with the flow. This is somewhat clichéd, but the essence behind this is try to avoid getting stressed out. This isn’t always easy to achieve, but taking a step back from a situation and avoiding politics is important.

I often strive for perfection, which isn’t an entirely clever pursuit since it is almost impossible to achieve. However, in a realm of imperfection, the principles above have helped me to achieve a modicum of decent code throughout the years. They may also resonate and provide inspiration for you.

Reasons why people working with computers tend to have lots of free time…

Computer Workers

Boris Johnson’s Wiff Waff

One of the most amusing quotes of 2008 must be this: At a party to mark the handover of the Olympic flag in Beijing, newly elected Mayor Boris Johnson laid claim to Britain’s sporting inventiveness.

Virtually every single one of our international sports were invented or codified by the British. And I say this respectfully to our Chinese hosts, who have excelled so magnificently at Ping-pong. Ping-pong was invented on the dining tables of England in the 19th century, and it was called Wiff-waff! And there, I think, you have the difference between us and the rest of the world. Other nations, the French, looked at a dining table and saw an opportunity to have dinner; we looked at it an saw an opportunity to play Wiff-waff. And I say to the Chinese, and to the world, that Ping-pong is coming home!

Embarking on a New Project?

Four resources to consider when embarking on a new project:

  1. Precedent (who’s done it already and how?);
  2. External data;
  3. Internal data and knowledge; and,
  4. Educated guesses

No. 4 is most valuable. After all, all the research in the world won’t tell you how to design it.

(Source: Cameron Moll)

In the next academic year, my Girlfriend’s school will be performing Carmina Burana and I’m really looking forward to it. Since seeing the choir perform Handel’s Messiah last year, I’m sure the performance will be not only dramatic but evocative.

But is it right for an all-boys choir and orchaestra to be performing Carmina Burana?

Why the hesitance you may ask. Well read on…

The BBCs h2g2 has a great write up on Carmina Burana, which begins:

It is not often that the little old ladies of the choral society scream since Fate strikes down the strong man, everyone weep with me!, or when they are in a more cheerful mood, My virginity makes me frisky, my simplicity holds me back. Oh, Oh, Oh, I am completely coming to life. All this while the men strain their dinner jackets singing a bawdy drinking song. However, this is Carmina Burana, one of the most popular works for choir and orchestra of the 20th Century.

But Carmina Burana is not all dramatic chanting – there is plenty of lyrical and gentle playing and singing, merry dances and of course the aforementioned drink and sex. The work lasts about an hour in performance and requires baritone, tenor and soprano solo singers as well as a boys’ chorus, an adult chorus and a massive orchestra.

UK residents will be familiar with the opening and closing music of Carmina Burana. The ‘O Fortuna’ chorus with its dramatic chanting against orchestral backing was used for many years in television advertising for a well-known brand of aftershave lotion (Old Spice). And, since imitation is the sincerest form of flattery, there is more than a hint of the same chorus in the sound track of the film The Fellowship of the Ring, particularly to accompany the Black Riders.

You can read more about Carmina Burana on the BBC website.

About h2g2

h2g2 is an unconventional guide to life, the universe and everything, an encyclopaedic project where entries are written by people from all over the world. h2g2 was launched in April 1999, and the BBC took over the running of the site in February 2001 as part of our drive to develop new and innovative online services.

The Guide is written by visitors to the website and already it has thousands of entries on all sorts of subjects. The result is a living, breathing guide that’s constantly being updated and revised, driven forward by the very people who use it.

Since getting the new iPhone 3G I’ve been downloading ‘useful’ applications like there was no tomorrow. I now have the very useful Vicinity app, various social networking apps and the best of all, a Light Sabre.

But there was something missing in my toolset; a WordPress app. Typepad have long since had one, so it was about time my favourite blog platform upped the ante!

Well, I’m glad to say, as of today an app has finally reached the Apple App Store that allows me to write posts: thank you Automattic.

In fact, just for good measure, this blog post has been written from the app.

The interface is intuitive and easy to use, although it does lack admin features, which for some may be a concern.

I’m sure I’ll get RSI if I continually blog via my iPhone, but for those moments when I’m caught away from a computer and desparately want to offer my 10 pence worth to the blogosphere, it’s a good tool to have.

Cool Hand Luke

What we’ve got here is failure to communicate

Following a bizarre conversation at work, that I had with a Polish colleague and in which I completed the following quote based upon a comment he made, I had to find out where the quote came from… It transpires it is from Cool Hand Luke, but has featured in numerous films throughout the years since the original film.

Cool Hand Luke
“What we’ve got here is failure to communicate.

Some men, you just can’t reach.

So you get what we had here last week — which is the way he wants it.

Well, he gets it.

And I don’t like it anymore than you men.”

« Older entries