HR professionals should base decisions on the best available evidence. A beautiful study by Greer K. Gosnell, John A. List, and Robert D. Metcalfe, with the help of Virgin Atlantic, provides solid facts on a difficult topic: the productivity of highly skilled workers. What makes this study so impressive is that it is based on objective measures and well-designed control groups so we can have a high degree of confidence in the findings.
How the study was done
The study looked at the productivity of commercial airline captains. The productivity metrics were fuel use, time delays, and safety.
They tested four management practices:
Monitoring Performance. Captains got detailed reports on their performance.
Providing Performance Feedback. Captains were told how often they hit the targeted level of the performance metrics.
Setting Personalised Targets. Captains were encouraged to achieve personalized targets
Offering Prosocial incentives for Performance.Captains were told that if they achieved their targets the company would donate to a charity.
In each case, there was an experimental group and a control group. The researchers gathered data from 40,000 flights over a 27-month period for 335 captains.
What they learned
One might think that a motivated and skilled professional in a tightly regulated industry would already be operating at their best. If that were the case, these management practices wouldn’t have an impact. Let’s see what happened!
Here are the results of the study:
Result 1. The performance of captains improved considerably upon announcement of monitoring performance metrics.
Result 2. Providing captains with information on recent performance moderately improved their fuel efficiency.
Result 3. The inclusion of personalized targets significantly increases the captains’ performance.
Result 4. Adding a charitable component to the personalized targets intervention did not lead to better results than providing targets alone.
Result 5. The effects of the management practices attenuate or disappear after the practices areemoved.
Result 6. Largely due to fuel savings from the monitoring effect, the airline saved about $6 million throughout the eight-month experimental period.
Yes, but will it work here?
One of the questions we need to ask in assessing the relevance of research is how close the context of the research is to the context of our own situation. You probably have highly skilled workers in your company but it’s unlikely they’re pilots. You probably have a complex operation, but you are probably not an airline. Does it make sense to generalize these findings to your situation?
I believe you can generalize for a couple of reasons. One is that the apparent core causal mechanism that providing data on performance improves performance makes a lot of sense, and that causalechanism will apply in many situations. The other reason is that in the absence of other evidence you have to go with what you’ve got. Remember, the rule in evidence-based management is “the best available evidence” not “perfect evidence”. This is a good finding, use it.
What to tell your mangers
Managers may believe one or more of the following:
What HR says are good people management practices don’t really have enough of an impact to be worth spending time on.
The skilled and motivated professionals they have on their team don’t need to be managed.
Simple, obvious management interventions won’t make a difference, they need something sophisticated.
This study shows those beliefs are, at least in the case of airline pilots, false. Managers may not notice the impact of simple good people management practices if they are not doing a scientific experiment because the improvements are hidden in the noise of all the other things going on. The lesson is that the disciplined practices of good management pay off and that it’s a mistake to accept excuses that they are not really necessary. It would have been a reasonable guess that the simple management practices used in this study wouldn’t make a difference with these highly trained professional airline captains—and yet the evidence shows they do matter. That’s a finding worth remembering.
Where to find the full article
You can find the working paper here
Thanks to Lorenzo Galli of ScienceForWork.com for bringing this to my attention.