A principle I developed some time ago is to always be very careful what one measures, because that is naturally what people will optimize, and if that isn’t what one wants optimized, then don’t measure it.
Measure writers by the volume of text they produce and soon they will get very wordy. Measure academics by how much they publish and the world will soon be awash in trivial and inconsequential papers (as in fact it is). Measure an airline by how often they leave the gate on time and they will cut all sorts of corners to optimize that measure. Rank employees by how much overtime they put in and soon overtime will go up, though productivity most likely will go down. Whatever the measure is, people will soon learn how to game it to their advantage.
The No Child Left Behind initiative is an excellent example of this principle. Schools are measured on the performance of their students on tests, and so not surprisingly most of the school year is now focused on teaching to the tests. But of course only simple facts and skills can be measured on the sorts of standardized tests that can be given to masses of students, so the more important aspects of education – creativity and curiosity and logical thinking skills and passion for the subjects – are neglected in favor of cramming facts into the students.
The problem is that it is usually hard to find a way to accurately measure the things that really matter - product quality, productivity, creativity, innovation. But increasingly bureaucracies are looking for “objective” measurements, so they turn to things they can measure or count or rank easily. And usually these are not really the things one wants to optimize.
So if you want something optimized, measure it. If that isn’t what you intend to optimize, then don’t measure it.
(In some future post I will reveal Godwin’s first law, proposed years ago by my father)
Posts
