Measuring performance and setting priorities

Today is a leap day, a quadrennial adjustment of the calendar on which tradition dictates that women may ask men to marry them. (Of course, this is just a convention — nothing in reality prevents either gender from popping the question.) In that spirit, I want to take a look at a convention in law firms that could do with being upset — recording time.

The sun is setting, tide incomingThis isn’t simply a plea to move away from time-based billing: that topic has been done to death by other commentators. In any case, I think clients are increasingly resistant to the idea to the extent that it has almost become the charging model of last resort rather than the starting point. No, I am more concerned about a figure that looms large in the daily life of almost every private practice lawyer: the annual chargeable hours target.

Even firms that have adopted alternative ways of billing clients still cling to timesheets. Their lawyers are expected to account for every six minutes of every day. (Even holidays — many time recording systems need to be told that a lawyer is not working.) Many of the same firms set annual targets for their lawyers — they must record a certain number of hours per year. I understand why this is important. Even when the link between time spent and the price of legal services is broken, firms still need to know what their input costs are in order to know whether the work is profitable.

I think the cost of this knowledge is too great. Although they are potentially useful to firms, these targets have a pernicious effect on lawyers’ behaviour, law firm culture, and client service.

Behaviour and culture

Although the point of recording time is to give the firm an idea of how much effort is being expended on client work (and other activities, so long as non-chargeable time is recorded too), many firms also use targets to inform other things, such as eligibility for bonus payments, suitability for promotion, and so on. As a result, lawyers prioritise meeting their annual targets above all else. Any work that cannot be attributed to a client file is therefore given a lower priority. Unless the firm explicitly elevates the priority of non-client work, perhaps by allowing work on specific internal projects to be counted towards the lawyer’s annual target, lawyers will naturally be reluctant to contribute to activities such as knowledge management.

Law firm culture is, in part, an accumulation of the way people commonly behave. As such, habitual prioritisation of chargeable work in preference to certain types of non-chargeable activities will naturally cause those activities to be considered less worthy. The personal financial preferences of time-recording lawyers become cultural norms. Worse than that, however, the act of recording time has come to signify importance within many firms. That is where the fee-earner/non-fee earner distinction arises. Firms that claim to deprecate the use of the term ‘non-fee earner’ will only succeed in changing their culture by removing fee-earners from the stage. If nobody records time, the most obvious distinction between lawyers and business services professionals is eradicated.

A fruit-based digression: customer focus

In my last blog post, I linked to and quoted from a fascinating piece by Horace Dediu on the way Apple appears to measure performance, and how that affects (and is affected by) the priorities the company sets. Dediu points to a difference between the way most businesses measure performance and the way Apple appear to do it. He argues that the norm is to depend heavily on easy to measure financial data:

These “financial” measures of success are considered prudent and optimized for return on equity (also known as the maximization of shareholder returns).

Unfortunately, Dediu argues, financial metrics prioritise some forms of information over others, to the extent that the wrong decisions might be taken in consequence:

The mass phenomenon of measuring the wrong thing because it’s the easiest to measure is called “financialization”. Financialization is the process by which finance and finances (rather than creation) determine company, individual and society’s priorities. It comes about from an abundance of data that leads to fixation on what is observable to the detriment of awareness of hazards or obstacles or alternatives. This phenomenon is more likely when the speed of change increases and decision cycles shorten.

By contrast with companies wedded to the need to maximise shareholder returns, Dediu notes that Apple puts the customer, rather than the company, at the centre of its decision-making processes. In doing so, it aims to make the best product it can for customers. The challenge is to assess how successful it is in doing that.

The idea that the purpose of the firm is to create and maintain customers is not new but it is relatively rarely practiced. The reason is that the data is harder to obtain. The data that comes from sales is crisp and concrete. The data about customers is muddy and soft. In a world where spreadsheets are used as weapons, the crisper the data, the better the ammunition.

Optimizing around customer acquisition rather than equity returns leads to a new set of metrics. What would these metrics look like for Apple?

The company publicly offers three separate sets of quantity of customers and quality of customers.

In terms of quantity we have:

  1. Number of iTunes accounts
  2. Number of iCloud accounts
  3. Number of active devices

In terms of quality of customers we have:

  1. Average selling prices for devices (as a proxy for willingness to pay)
  2. Customer satisfaction (as a proxy for loyalty)
  3. Services and accessory revenues (post-sales and recurring value)

Dediu then shows graphically what Apple’s performance over the last ten years looks like against those metrics. (Generally good.)

Client focus for law firms

There is no reason why law firms should not also consider themselves as client-focused. In fact, there is a good case for arguing that they should be more client-focused than a company manufacturing quasi-commodity technology products and services. Law firms’ dependence on basic financial metrics suggests, however, that they struggle to measure how well they perform from a client perspective.

Firms’ persistent emphasis on time recording may be the most significant factor working against proper client focus. In prioritising measurement of the time taken to perform tasks, rather than finding a way to assess their importance for clients, firms choose to elevate quantity over quality. Even when a firm has a good system for assessing client satisfaction, it is rare for that information to be integrated into decisions about individual lawyers’ progression or bonus. (And I suspect good systems for assessing client satisfaction are vanishingly rare in themselves.)

The leap-year challenge for law firms, then, is to consider what good qualitative measures of client service they might have or be able to generate, and to work out how those metrics could be used to supplant the outdated conventional measures of firm performance. In doing so, they should find improvements in lawyers’ openness to involvement in important practice support and development activities, in the firm’s culture, and in the quality and creativity of service provided to clients. I can’t promise that those improvements will lead to Apple-scale profitability, but other examples across the corporate world suggest that better performance flows more often from a customer focus than from a shareholder focus. In addition, firms would no longer need to invest in time-recording technology and the panoply of enforcement tools and processes that flow from them.

 

A little data (and emotion) about musical experience

One of the novelties of our connected world is the amount of data that can be collected. This may be a worry when the collector is unknown or untrusted, but it can also be an opportunity if one has access to the data oneself — especially when one might have no other record available. At the very least, it gives an insight into the way larger datasets might be used.

I signed up to last.fm ten years ago today. From that moment, almost every time I listened to a piece of the music, the fact was recorded by the service (a function they call ‘scribbling’). There are some gaps: I didn’t use it assiduously in the first few months, and the service itself didn’t timestamp scrobbles until later in 2005 (my first timestamped scrabble was on 18 December 2005, but music played before that is still included in the overall statistics). Apart from that, any music played on my iPod or in iTunes was captured. Also, as other services and devices became available (such as the iPhone and Spotify), they also sent data to last.fm. However, I have no record of CDs played other than through iTunes, nor of music heard on YouTube.

lastfm

So, I can say with a degree of certainty that in the last ten years, I have listened to over 134,000 tracks performed by over 5,000 different artists. I can also see which tracks, artists and albums I have heard most frequently.

There are some other things that I can do with the data, thanks to various tools  developed using the last.fm API. All this is very satisfying for the side of me that treasures useless information and therefore does quite well at general knowledge quizzes. I can even compare myself with others, and I am sure that information about patterns of listening could be useful to the music industry more generally. (That is one inference that can be drawn from the purchase of last.fm by CBS in May 2007.)

But, on reflection now, all this just leaves me cold.

Last.fm cannot tell you why I was listening to La Traviata, Leonard Cohen and Dave Brubeck in the week ending 21 May 2006, any more than it can say why this week last year resounded to Goldfrapp, Maria Callas and Sidney Bechet. Nor can I. There might be an interesting story there, but it cannot be told without additional prompts (such as might be found in my emails or notebook, and possibly not even then).

That is the problem. The real story — real knowledge — is as much emotional as analytical. And data cannot give access to the emotional truth. Given enough data, we might be able to see that something happened at a particular moment in time, and even what else was going on, but it cannot tell the truth of that moment.

This is the real challenge for data analytics (whether of ‘big data’ or otherwise). The analysis itself may be flawed because of the necessary exclusion of unmanageable information (the human factor). Even if perfect, the way a piece of analysis is received is also unpredictable (another human factor). Some people may react badly to what the data appears to be telling them. Some may react well, but act inappropriately.

I think this imposes a burden on those engaged in data collection and analysis to work carefully on understanding its limitations and its potential impact. I have read many excited articles and heard many breathless presentations about the power of data to make our lives better. I have rarely heard anyone refer to the corresponding responsibility to be sure that things won’t be made worse. That requires emotional intelligence, which is a purely human capability.

Data on its own will solve nothing.