Scott Berkun

The data death spiral

Whenever data is misused as the only means for making decisions, a death spiral begins. The lust for data overwhelms all sensibilities. Cowardly decision makers howl in glee at reams of unnecessary data, while bright people sit handcuffed to ugly slidedecks and mediocre ideas. Decision makers forget their brains and wait for numbers, fueling an organizational addiction to unnecessary and distracting data.

Data is good: it dispels ignorance and raises questions, but it has its place. Data doesn’t make decisions or come up with good ideas: people do. As soon as you discount people in the name of data, the downward spiral begins.

You know you’re in a data death spiral if:

1. Confusion over what the hard part is. If you are working on anything challenging, data will only define the landscape. It won’t dictate a particular strategy: good data has different interpretations. If you present data to justify a decision you are emphasizing some data over others: which is an act of creativity and opinion, not fact and figure. Opinion is everywhere and is most dangerous when it’s hiding behind spreadsheets.
2. No opinions are considered without data. Opinions are good if they come from smart thoughtful experts. If you are in a world where you, as an expert, can’t make obvious improvements without 10 pages of supporting material, guess what happens? Nothing happens. People spend all their time defending the obvious and the scale of work, and the energy to improve, drops dramatically.
3. Creatives are hiding in the corner. (Good) Designers and other creative people know how to make things. Lots of things. They experiment and explore. When used properly they are a piston of progress. But if the landscape is data and analysis obsessed, creative types are relegated to refinement work, the least leveraged use of their talents. The team gets a fraction of their possible value.
4. No one questions data. Not all data is the same. To collapse data down into “70% of our customers prefer green” requires several layers of simplification of the truth. Which customers? What alternatives were they shown (were there just 5 kinds of green)? How many customers were there? If no one every questions the data and understands its nuances the gates are open for people to throw more misinterpreted data at each other. You need data quality not just data volume.

How death spirals start

The spiral begins with ignorance. Leaders confuse the collection of research with thought. People who throw more data (not better) at problems are rewarded and others follow. Soon no idea can be suggested without a data armory. Meetings are data battlegrounds. Or worse, data massacres. When someone says “Morale is low. People are crying in the halls. We must do something.” another says “but where is your data?” and the conversation ends.

As the spiral bottoms out, people are proud of numbers and slide decks: not the results.

Steps to preventing death spirals

1. Separate good data from bad. Good data self describes its biases. A smart person uses data with consideration for what things the data can’t tell you, and they include this understanding in their story. Encourage people to play devil’s advocate on their data and make data inquiry part of the discussion any time data is presented. Diffuse data obsession.
2. Data and ideas are partners. Make sure developments happen both ways: ideas that are pulled out from data, and ideas that come from minds, with supporting data found later. At its best it’s an itterative process: you have ideas, you research data, you refine ideas, you research more, etc.
3. Instincts matter. The Gladwell book Blink explores how our instincts serve us: are the instincts of your group serving you? Layers of process and data bury our hearts. Some people have better instincts about certain decisions and you want to harness that talent, not dilute it. It can be fine to say “Fred has the best perspective on this. It’s his decision.” And allow Fred to define how much or how little research is needed.
4. Let go of the fear. Many people collect data to defend their choices should things go wrong. When bad things happen they point to the data and say “See! We did the right thing!” despite the results. This kind of paranoid, self-protective thinking weighs on a team: instead of ensuring success, people are protecting their asses. Most of us can smell it when a leader is in this mode and it puts the entire project on its heels. Insurance is for the birds: A good manager earns the trust of his team and lets them know that even in failure, he’ll take responsibility for what was done.
5. Kill contrived group think meetings. Data obsession and consensus-driven teams go hand in hand. It’s on big, cowardly teams with leaders who demand everyone agree before decisions are made that people show up armed to the teeth with handouts and marathon presentations. Not everyone needs to agree to everything. Define who is in the best position to decide and give them the power. If necessary, pick a handful of people who disagree to go off and fight it out in private, reporting back when a decision has been made.
6. People are better than data. Much of a person’s talent is difficult to capture in numbers. Their experiences, instincts and passions all factor into what makes them good at what they do. As a manager, I want to make sure I have channels open to these assets, channels that don’t require the same kind of data in order for me to listen. If I shut these things out, I’m denying myself the benefits of true expertise. If I’m working on something hard, and with tight deadlines, the more good decisions I can have my team make without days of research, the better off our team will be.

(See also How to lie with statistics and Chapter 8, how to make good decisions, from the Making Things Happen)

• Posted Tuesday, November 1st, 2005 at 10:22 am


• in Management, Software/Web development | 4 Comments » |