Archive for September, 2009

Learning from failure or success

In a round up following KM Australia, back in August, Shawn Callahan has challenged the notion that we learn best from failure. I think he has a point — the important thing is learning, not failure.

Harris Hawk missing the quarry

Here’s Shawn’s critique.

During the conference I heard a some speakers recount the meme, “we learn best from failure.” I’m not sure this is entirely true. Anecdotally I remember distantly when I read about the Ritz Carlton approach to conveying values using stories and I’m now delivering a similar approach to a client on the topic of innovation. Here I’ve learned from a good practice. As Bob Dickman once told me, “you remember what you feel.” I can imagine memory being a key first step to learning. And some research shows it’s more complex than just learning from failure. Take this example. The researchers take two groups who have never done ten pin bowling and get them bowling for a couple of hours. Then one group is taken aside and coached on what they were doing wrong and how they could improve. The other group merely watches an edited video of what they were doing right. The second group did better than the first. However there was no difference with experienced groups.

I wish I could access the linked study — Shawn’s summary and the abstract sound very interesting. Here’s the abstract.

On the basis of laboratory research on self-regulation, it was hypothesized that positive self-monitoring, more than negative self-monitoring or comparison and control procedures, would improve the bowling averages of unskilled league bowlers (N =60). Conversely, negative self-monitoring was expected to produce the best outcome for relatively skillful league bowlers (N =67). In partial support of these hypotheses, positive self-monitors significantly improved their bowling averages from the 90-game baseline to the 9- to 15-game postintervention assessment (X improvement = 11 pins) more than all other groups of low-skilled bowlers; higher skilled bowlers’ groups did not change differentially. In conjunction with other findings in cognitive behavior therapy and sports psychology, the implications of these results for delineating the circumstances under which positive self-monitoring facilitates self-regulation are discussed.

Based on these summaries, I would draw a slightly different conclusion from Shawn’s. I think there is a difference between learning as a novice and learning when experienced. Similarly, the things that we learn range from the simple to the complex. (Has anyone applied the Cynefin framework to learning processes? My instinct suggests that learning must run out when we get to the chaotic or disordered domains. I think we can only learn when there is a possibility of repeatability, which is clearly the case in the simple and complicated domains, and may be a factor in moving situations from the complex to one of the other domains.)

The example Dave Snowden gives of learning from failure is actually a distinction between learning from being told and learning by experience.

Tolerated failure imprints learning better than success. When my young son burnt his finger on a match he learnt more about the dangers of fire than any amount of parental instruction cold provide. All human cultures have developed forms that allow stories of failure to spread without attribution of blame. Avoidance of failure has greater evolutionary advantage than imitation of success. It follows that attempting to impose best practice systems is flying in the face of over a hundred thousand years of evolution that says it is a bad thing.

In the burned finder scenario, success (not touching a burning match) is equivalent to lack of experience. Clearly learning from a lack of experience will be less effective than learning from (even a painful) experience. By contrast, the bowling example provides people with a new experience (bowling) and then gives them an opportunity to contemplate their performance (which was almost certainly poor). However, whatever the state of their performance, it is clear what the object of the activity is and therefore ‘success’ can be easily defined — ensure that this heavy ball leaves your hand in such a way that it knocks down as many pins as possible by the time it reaches the far end of the lane. As the natural tendency of learners at early stages in the learning process is to concentrate on the negative aspects of their performance (I can’t throw the ball hard enough to get to the end of the lane, or it keeps going in the gutter), it is understandable that a learning strategy which focuses on success could have better results than one that merely explains why the bad things happen.

In the bowling experiment, no difference was found between the negative and positive approaches when experienced bowlers were studied. All this suggests to me is that we need more work in this area, especially considering learning in the complicated or complex domains. Even for experienced bowlers, the set of variables that affect the passage of a bowling ball from one end of the lane to the other is a predictable one. There is not just one cause and effect, but the laws of physics dictate that the relationships between all the causes should have predictable outcomes. By contrast, much of what interests us with regard to knowledge and learning in organisational environments does not depend on simple causal relationships.

In those complicated or complex organisational situations, I think we can learn more from our own failures than other people’s successes (which I think is the point that Dave Snowden is making). I think Shawn is also right to suggest that we can learn from our own successes too. However, that can only be the case if we take the time to analyse exactly what was the cause of the success. So we need a commitment to learning (which brings us back to deliberate practice, amongst other things) and we need the insight into our actions and activities that allows us to analyse them effectively. I think the will to learn is often present, but insight is often missing when we consider successful initiatives, possibly because the greater distance between cause and effect means that we cannot be confident that success is a product of any given cause. On the other hand, it is usually easier to identify causes of failure, and the process of failure also provides an incentive to work out what went wrong.

As for the quality of the lessons learned from failure or success, I am doubtful that any firm conclusion could be drawn that as a general rule we learn better from failure or from success. However, as we become more experienced and when we deal with fewer simple situations, we will inevitably learn more from failure than success — we will have more experience of failure than success, and other people’s successes are of limited or no value. So, although we can learn from our successes, my guess is that more of our learning flows from failure.

It feels like there is more research to do into these questions.

Speaking of social software and KM

Last week, Headshift hosted an “insight event” to showcase the report on social software for law firms written by Penny Edwards and Lee Bryant. I was honoured to be asked to present, along with Sam Dimond of Clifford Chance and Steve Perry of Freshfields.

Nick Holmes wrote a great summary of the event on his blog, Binary Law, and I intended to post the notes for my session here, but Penny has now done a really impressive job of transcribing our three presentations, together with Lee’s opening remarks. I am particularly impressed because she was listening into the event from Amsterdam, and I gather the sound quality was not particularly good.

Penny’s four posts on the Headshift blog are as follows:

As well as the presentations, we had some great questions from the audience and an opportunity for offline social networking. I only wish we could have had longer to discuss all the issues that people raised. Many thanks to Penny for putting the event together, and to Lars Plougmann for hosting it. (By the way, I think the term “insight event” is a really good one.)

Book review: Story: Substance, structure, style, and the principles of screenwriting

I read Robert McKee’s book, Story: Substance, structure, style, and the principles of screenwriting during my holiday last month. It is a fascinating insight into a crucial part of the film-making process, and has helped me understand movie storytelling much better. If that was all it did, I would recommend it wholeheartedly — by shedding light on the mystery of film, it actually enhances one’s enjoyment of the medium, rather than degrading it. If you are at all interested in film, read it — you won’t regret it.

Bee at work

However, McKee’s book raised two distinct issues for me. One is internal to the book and concerns the way in which we do work. The second is an external one — what makes an expert?

Towards the end of the book, having worked through the elements that make up a satisfying story, McKee turns to the actual mechanics of screenwriting. It’s not as simple as starting at the beginning and working your way to the end. This model is described by McKee as writing from the outside in.

The struggling writer tends to have a way of working that goes something like this: He dreams up an idea, noodles on it for a while, then rushes to the keyboard…

He imagines and writes, writes and dreams until he reaches page 120 and stops…

So the struggling writer gathers friends’ reactions and his own thoughts to start the second draft with this strategy: “How can I keep the six scenes that I love and that everyone else loves and somehow pretzel this film through them in a way that’ll work?” With a little thought he’s back at the keyboard…

He imagines and writes, writes and dreams, but all the while he clings like a drowning man to his favorite scenes until a rewrite comes out the other end. …

The writer then does a third draft and a fourth and a fifth but the process is always the same: He clings to his favorite scenes, twisting a new way of telling them in hopes of finding a story that works. Finally … back come reader reports: “Very nicely written, good crisp actable dialogue, vivid scene description, fine attention to detail, the story sucks. PASS ON IT.”

By contrast, writing from the inside out is a much more structured process in which the story stays at the heart.

If, hypothetically and hopefully, a screenplay can be written from first idea to last draft in six months, [successful] writers typically spend the first four of those six months writing on stacks of three-by-five cards: a stack for each act — three, four, perhaps more. On these stacks they create the story’s step-outline.

Essentially, the step-outline is used by the writer to describe a single scene on each card. These cards come and go — a scene may be written and rewritten a dozen times on a dozen different cards. The relationship between the cards — which are key scenes and which promote sub-plots, for example — may change over this time. Finally, when the writer is satisfied that the whole things hangs together properly. At this point, the story can be pitched to someone else.

The writer never shows his step-outline to people because it’s a tool, too cryptic for anyone but the writer to follow. Instead, at this critical stage, he wants to pitch or tell his story so he can see it unfold in time, watch it play on the thoughts and feelings of another human being.

Once the story is seen to work, it is time for the treatment: each scene is expanded into a readable description. That process allows the story to be honed further until it is ready to be turned into a screenplay — dialogue, directions and all.

The wise writer puts off the writing of dialogue for as long as possible because the premature writing of dialogue chokes creativity.

Writing from the outside in — writing dialogue in search of scenes, writing scenes in search of story — is the least creative method.

The description of this process is fascinating. I think there are elements that can be drawn out for wider use in the work that we do. Rather than draft agreements which are then batted back and forth between parties (in much the same way as the screenplay in the “outside in” example), could we envisage the legal documentation of a transaction from the inside out? Perhaps the following key points might be useful.

  1. Why is this deal being done? (In McKee’s screenwriting terms, this might be the “Inciting Incident”)
  2. What are the major points of agreement (and disagreement) between the parties? (Tension between protagonist and antagonist; character and characterisation)
  3. What other issues are at play? Are there any external pressures — time, regulators, etc?
  4. Can the deal (and the answers to the preceding questions) be summarised easily? (The pitch)
  5. Does everyone agree with the pitch? (Develop into a treatment)
  6. Once the treatment is agreed, the documentation (formal contractual provisions) should easily flow from the treatment.

To be honest, I have no idea whether this would work. However, I have seen enough frustration borne of endless argument over the minutiae of legal drafting to be interested in seeing if an alternative would be any better at conveying the commercial meaning of a transaction into legally enforceable wording. Just as dialogue restricts creativity, so does legal drafting. Once a clause is set in Word and becomes the subject of argument, it is difficult for lawyer and client alike to think creatively about alternative ways of achieving the same object, or even whether that object is actually a desired one — consistent with the ‘story’ of the commercial transaction that is being documented.

The other issue that McKee’s work (his book and the seminars that he runs) raises relates to expertise. McKee is an adept critic and analyst of screenwriting, but he is not a great screenwriter himself. His record at the Internet Movie Database indicates that he has written a couple of TV movies and some TV series episodes. Some of the comments on his work suggest that this apparent lack of success undermines his authority on screenwriting. Others use the traditionally snarky riposte “those who can, do; those who can’t, teach.” Both of these reactions miss, I think, the fact that McKee never says “copy me.” Instead, he brings a thorough reading of a range of good and poor films. (In fact, one could be concerned by the fact that he relies excessively on Chinatown for examples, were it not for the impressive list of films referenced in less detail but obviously equally well understood.) Out of that reading, he extracts incredible insight, which should be regarded more highly, not less, for the simplicity with which it is distilled into a set of clearly understood principles.

That, I think is part of the essence of expertise — insight translated into clarity, so that one’s audience can hope to achieve the same insight. That is the opposite of the traditional obfuscation which many experts (many in the field of law, I am sorry to say) typically indulge in.

Another aspect of expertise, which needs to be harnessed in the service of insight and clarity, is passion or enthusiasm for the subject. That passion is clearly evident in McKee’s book. He wants to eradicate poor storytelling in the movies by making the basic element — the screenplay — better. According to the Wikipedia article on him, McKee’s insights are not all especially original. I do not think this necessarily matters — his passion brings them to life more vividly than their originators were able to.

There is clearly more to expertise than just insight and passion, but McKee’s work shows how those elements in combination with even a limited body of material can generate real value.

Storing our future knowledge?

Over the summer, I read a couple of blog posts about knowledge storage that I marked to come back and comment on. Separately, Mary Abraham and Greg Lambert have suggested a fairly traditional approach to selection of key knowledge for storage and later access.

Dover Castle

First, Greg issued a clarion call for selectivity in information storage:

Knowledge Management should not be based on a “cast a wide net” approach to the information that flows in and out of our firms. In fact, most information should be ephemeral in nature; addressing only the specific need of the moment and not be thought of as a permanent addition to the knowledge of the firm. When we try to capture everything, we end up capturing nothing. In the end we end up losing the important pieces of knowledge because they are buried in a mountain of useless data filed under the topic of “CYA”.

I had to Google “CYA”. And thereby hangs a lesson. How can we know when we make a decision about recording the present for posterity that the things we choose will be (a) comprehensible to those who come after us and (b) meet their as yet unknowable needs?

For centuries, the study of history relied on official records and was therefore a story of kings and queens, emperors and presidents, politicians and popes. The things that were left behind — castles, cathedrals, palaces and monuments as well as documents — actually provided us with only slender insight into the real lives of the majority of people who lived at any given point in time. Only when archaeologists and social historians started to untangle more trivial artefacts like potsherds, clay pipes, bone pits and everyday documents like manorial rolls, diaries, or graffiti were we given a more rounded picture of the world of our predecessors. At the time, those things were ephemeral — not created for posterity. The lesson we always forget to learn is that we don’t get to write our history — the future does.

Because Google has access to a vast mass of ephemera, I was able to learn what “CYA” means. In Greg’s context, it is the stuff we think we might need to keep to protect ourselves — it is an information security blanket.

Mary Abraham picked up the thread by addressing the Google question:

Folks who drink the super search kool-aid will say that the cost of saving and searching data is becoming increasingly trivial, so why spend any time at all trying to weed the collection?  Rather, save it all and then try Filtering on the Way Out.  On the other hand, look at the search engine so many of us envy — Google.  It indexes and searches enormous amounts of data, but even Google doesn’t try to do it all.  Google doesn’t tackle the Deep Web.

So why are we trying to do it all?

That’s a good question, and one that Greg challenged as well. I want to come to that, but first the Deep Web issue needs to be dealt with.

As I understand it, the problem for Google is that many useful web resources are stored in ways that exclude it — in databases, behind paywalls, or by using robots.txt files. That may be a problem on the public web, but it shouldn’t be in the enterprise context. By definition, an properly set up enterprise search engine is able to get access to anything that the user can see. If there is material in a subscription service like Westlaw or Lexis Nexis, then searches can be federated so that the result set includes links into those services as well as a firm’s own know-how. Alternatively, a firm or search provider can make special arrangements to index content through a paywall. There simply should not be a Deep Web problem in the enterprise context.

But what of the main issue — by storing too much, we lose our ability to find what is important? I think Greg and Mary are right to challenge the “store everything” model. There is much that is truly ephemeral — the e-mail that simply says “Thanks” or the doodles from that boring meeting. The problem with those, though is not that keep them, but that we created them in the first place. If the meeting was that boring, should the doodler not have gone and done something else instead? Isn’t there a better way of showing appreciation than sending an e-mail (especially if it was a reply-to-all)? I think that is the bit that is broken. Some other things are ephemeral even though they do need to be captured formally. Once an expenses claim has been paid, and the taxman is satisfied, there is little need to keep the claim forms available for searching. (Although there may be other reasons why they should not be discarded completely.)

However, I am still concerned that we cannot know what will be useful in the future, or why it might have a use. At the heart of an organisation like a law firm there are two strands of information/knowledge. The first is a body of technical material. Some of this is universally available (even if not comprehensible) — statutes, cases, codes, textbooks, journal articles: documents created externally that we all have to understand. Some is specific to the firm — standard documents, briefing notes, drafting guides: our internal know-how. I think this is the material that Greg and Mary are concerned with. And they are right that we should be critical about the potential immensity of these resources. Does that new journal article say anything new? Is that textbook worth the space that it takes on our shelves? Is our know-how really unique to us, or is it just a reflection of market practice? These are all crucial questions. However, almost by definition, as soon as we fix this material in some form it is of mainly historical interest — it is dying information. The older it gets, the less value it will have for our practice and our clients.

The other strand is intangible, amorphous, constantly shifting. It is the living knowledge embodied in our people, their relationships with each other and our clients, and their reactions to formal information. That changing body is not just responsible for the knowledge of the firm, but its direction and focus. At any time, it is the people and their connections that actually define the firm and its strategic preoccupations. In particular, what our clients want will drive our future knowledge needs. If we can predict what our future clients commercial concerns and drivers will be, then we can confidently know what we should store, and what to discard. I don’t think I can do that. As a result, we need to retain access to more than might seem useful today.

Patrick Lambe catches this tension neatly in his post “The War Between Awareness and Memory.” I looked at that in my last post (five weeks ago — August really isn’t conducive to blogging). As I was writing this one, I recalled words I last read thirty years ago. This is how John Dos Passos caught the same mood in the closing words of the eponymous prose poem that opens the single volume edition of his great novel U.S.A.

It was not in the long walks through jostling crowds at night that he was less alone, or in the training camp at Allentown, …

but in his mother’s words telling about longago, in his father’s telling about when I was a boy, in the kidding stories of uncles, …

it was in the speech that clung to the ears, the link that tingled in the blood; U.S.A.

U.S.A. is the slice of a continent. U.S.A. is a group of holding companies, some aggregations of trade unions, a set of laws bound in calf, a radio network, a chain of moving picture theatres, a column of stock quotations rubbed out and written in by a Western Union boy on a blackboard, a publiclibrary full of old newspapers and dogeared historybooks with protests scrawled on the margins in pencil. U.S.A. is the world’s greatest rivervalley finged with mountains and hills. U.S.A. is a set of bigmouthed officials with too many bankaccounts. U.S.A. is a lot of men buried in their uniforms at Arlington Cemetery. U.S.A. is the letters at the end of an address when you are away from home.But mostly U.S.A. is the speech of the people.

Sometimes questions about the “laws bound in calf” or “dogeared historybooks” are less important, maybe even a distraction. The real life and future knowledge of the firm is the speech of the people. That cannot be reconstructed. We need to be aware of all the ways in which we can preserve and retain access to it, for use when a client comes up with a new conundrum for us to help them resolve.


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 28 other followers

Recent micro-blog posts

Interesting stuff...

Bookmark and Share

When…

September 2009
M T W T F S S
« Jul   Oct »
 123456
78910111213
14151617181920
21222324252627
282930  

Follow

Get every new post delivered to your Inbox.

Join 28 other followers