The pharmaceutical industry has something called Eroom’s Law (which is ‘Moore’s Law’ spelled backwards). It’s the observation that the number of drugs discovered per billion dollars in research has dropped by half every nine years since 1950.

This is astonishing, because the entire science of biochemistry has developed since 1950. Every step of the drug discovery pipeline has become more efficient, some by orders of magnitude, and yet overall the process is eighty times less cost-effective.

A chain-smoking chemist injecting random things into mice is provably a better research investment than a genomics data center.

You might think of some reasons why this is happening. Maybe all the easy drugs were found first, or the regulatory environment is much stricter than it used to be.

But these excuses don’t hold up to scrutiny. In the worst case, they might have blunted the impact of the breakthroughs, slowed the rate of improvement. But how did things get eighty times worse?

This has been a bitter pill to swallow for the pharmacological industry. They bought in to the idea of big data very early on.

The growing fear is that the data-driven approach is inherently a poor fit for life science. In the world of computers, we learn to avoid certain kinds of complexity, because they make our systems impossible to reason about.

But Nature is full of self-modifying, interlocking systems, with interdependent variables you can’t isolate. In these vast data spaces, directed iterative search performs better than any amount of data mining.

My contention is that many of you doing data analysis on the real world will run into similar obstacles, hopefully not at the same cost as pharmacology.

The ultimate self-modifying, adaptive system is any system that involves people. In other words, the kind of thing most of you are trying to model. Once you’re dealing with human behavior, models go out the window, because people will react to what you do.

In Soviet times, there was the old anecdote about a nail factory. In the first year of the Five-Year Plan, they were evaluated by how many nails they could produce, so they made hundreds of millions of uselessly tiny nails.

Seeing their mistake, the following year the planners decided to evaluate them by weight, so they just made a single giant nail.