Spray and Pray – The Base Coat

I kicked off a discussion about the hit and hope nature of employee engagement surveys. In summary it’s about why ask so many questions? Are our business leaders that far out of touch that they need to quiz us to that depth?

Jonathan Wilson offered some useful views which I share with you here:

Detailed social surveying produces at least three emotionally satisfying illusions. First is the illusion of activity. Just investing in the survey and getting everyone to complete it is a real activity that engages everyone and takes a lot of time. A survey of 50 questions will probably take a mean of 30-45 minutes to read, answer and submit. In a company of 100,000 people, if 80% respond that means it will takes about 28 person years, or more than £500,000 just to complete, before the costs of processing it.

The second is the illusion of precision. Few people read the small print that shows that any survey is a rough plus or minus estimate with distortions built in. Few surveys will claim retest reliability more than within 5%, 95% of the time. Because they include some percentages, which they choose to show to two or more decimal places, people are misled into thinking they are accurate to two decimal places. Can you measure your own attitude to two decimal places? So do you think they can measure thousands of peoples’ more accurately?

The illusion of precision leads to the illusions of tangibility, grasp and control. We think things we can measure are more ‘real’ than things we find harder to measure. We think we have a better grasp of them and can control them, or at least measure them again.

Other dangerous illusions include a mistaken belief in linearity, that the difference between 4 and 3 is the same as the difference between 5 and 4. On an attitudinal scale of 5, those two differences are huge, the difference between indifference, interest and passionate engagement.
I suggest that every manager should learn and be licensed in statistical literacy before being allowed to take or influence decisions based on survey responses – or anything else actually!

Statistics offer real insights, but rarely answers. Understanding them helps you see the questions better. The challenge is to meaningfully respond to the questions and the challenges they raise. What do you think?

I've had a terrible vision…

…it started when I was digesting the latest communication about the latest round of musical chairs where a few more chairs have been removed and the rest re-arranged in a neat “get a draw away from home” formation.

It had to do with shrinking pools – and the vision that flashed up was one I had seen on television in one of those fascinating nature programmes that both amaze and worry you.

There was a drought coming – the waters were receding, and in the deeper pools the shallower channels that connected them to the main flow were approaching dangerously low levels – effectively cutting them off.

Some of the cleverer little fish spotted what was happening, put on their metaphoric pumps and with a deft flick of their tails “legged” it across the channel in the style of Indiana Jones. They made it to a deeper pool and eventually out to the main stream – scary, but surprisingly invigorating!

The older, more “experienced” fish expected to “sit” it out in their old pond despite the diminishing amount of water (and therefore oxygen) and determined to carry on regardless. They accepted that it was good for those upstarts to leave and make more room for the serious boys in the good old pond.

The upstarts would never make it out there, if we lie still and don’t breathe too heavily we can stay here until it gets better and the new water comes in again and then we can carry on from where we were.

Guess what – the water levels continued to drop, the channels to the rest of the world dried up completely cutting them off, and the big fish got into more and more difficulty. They could no longer get out, nor could they all fit into the reducing water and had to fight it out amongst themselves to see who could get to the bottom of the pond fastest (and stay there) and therefore, literally, survive. The ones who couldn’t get to the bottom and stay there keeping their heads down out of sight, away from the sun’s heat, were slowly enduring an agonising death as they suffocated, which in turn was poisoning the pool and affecting everything in it, even those hiding out at the bottom.

Do we need to put our pumps on and get our little feet out there in the main stream and leave the overcrowded, big, but shrinking, pond?

Are we being poisoned by the stagnation in the big pond?

Perhaps I need to stop eating cheese just before I go to bed…