2020-06-14 - measuring KPIs... up to a point

recently at work we had a lot of discussions around KPIs for projects: what shall be measured, how and what to do with the results?

i'm all for measuring – i.e. how can you tell you have improved sth if you cannot measure values a priori and a posteriori? i'm however far more careful when talking about setting fixed thresholds – i'm in a “metrics are there to help your brain, not to replace it” club, here.

the problem is that the moment you start drawing consequences from raw numbers ppl tend to optimize for what they are being rewarded/punished for. this often leads straight to “creative booking” class of problems and thus defeats the whole purpose of measuring in a first place. i came up with an analogy to Heisenberg's uncertainty principle here – the more you depend on a metric, the less it will reflect reality.

none of the best-tested (i mean automated tests) projects i have worked on had any thresholds on test coverage. all of them however did measure test coverage.

yes – measure. but also do diversify and do not over-relay on the numbers alone. that's my call.