Estimates and mental debt

I'm reading the excellent Peopleware and one thing that stood out is their survey data on estimates: just the act of estimating a software project drops productivity by ~20-50%! This sounds bonkers - why would simply estimating change productivity? It is easy to write it off as a sloppy experiment (survey, self reported data, correlation ≠ causality).

I also recently viewed Kathy Sierra's keynote on Making Badass Developers. In it she references the well known experiment: one group memorizes a 2-digit number, another a 7-digit number. Afterward, they're offered fruit or cake. The 7-digit group overwhelmingly chooses cake. The experiment has been done in many variations and the thesis is that mental resources are easily depleted so subsequent decision making is poorer for it.

I think this ties in nicely. Having that estimate or deadline hanging over your head zaps some of your cognitive resources. It shouldn't mean that much but remember that just memorizing 7 numbers had an effect on decisions. Add to that all the small tasks that tend to follow once you estimate: delivering status updates, answering stakeholder questions, keeping Jira and OKR tickets up to date… it all adds up, eating into both uninterrupted time and mental energy.

I also read a thought provoking blogpost Seeing like a software company by Sean Godecke recently (a riff on "Seeing like a state"). In an aside he mentions how managers think estimates are given based on desired quality, but in practice the quality ends up driven by the estimate. If time pressure arises quality is cut to hit estimates and over time this lower quality means more mental resource drain on developers that operate this code. Tech debt becomes mental debt.

The evidence here isn't perfect, but I think the connections make sense, and it fits my experience. If we take it seriously the implications are big. As managers we must:

• Move away from estimates where possible. If teams are working on "the most important thing" does estimates give us much? Especially given how fallible they are.
• We must protect our teams mental resources by removing interruptions and letting them build in quality.
• We must prioritize learning so engineers can move more skills into the effortless zone where they cost less cognitive energy.

I know this is all a tradeoff - sacrificing legibility at the altar of productivity. But I think it is one we should make where we can, not just for productivity but also for the joy that comes from delivering quality products.

Further reading: Fruit salad, chocolate cake, cognitive control, and poverty - a deeper dive into the cognitive load research and its broader implications.