Beat the Curse of Knowledge by Explaining "Why"

I once watched an intern deftly answer an offhand question faster and more thoroughly than a room full of big shots and tech veterans.

What was the question?

“What is the best way to understand if the feedback I give is useful?”

To be clear, these experts provided a few high-level principles which, when summed up, are definitely important enough to remember: over the long-term, the best indicator of feedback quality is how many people ask you for feedback. But somehow, it lacked the air of concreteness that would make it actionable.

On my team, we had four engineers plus our summer intern, Sofia (probably not her real name). When the same question was asked of our team, as happened before, the engineers struggled to give concrete answers. Sofia, on the other hand, replied very concisely:

“As someone who gets feedback often, it’s most helpful when you explain ‘why.’”

Then she dropped the mic and walked. OK, I’m kidding about that last bit, but her answer contains a goldmine of practical wisdom that is somehow almost opaque to experts. I’m going to take the liberty to refine Sofia’s statement down into something a little more punchy:

“I don’t care until I know why.”

Why is it so hard for experts to communicate “why” to the novices in our field? Furthermore, why is it so hard for experts to experience the pain of not being told why? Interns are novices in their field. They are otherwise knowledgeable people who are also experts in being non-experts.

“Novices perceive concrete details as concrete details. Experts perceive concrete details as symbols of patterns and insights that they have learned through years of experience.” - Chip & Dan Heath

Patterns allow experts to understand the reason for the various concrete details in a broad system. It’s not just that Sofia sees a function parameter where I see inversion of control to facilitate both testing and modularity; I can no longer understand what it means to not understand. The Curse of Knowledge is the cognitive bias that prevents experts from remembering what it’s like not to know something and is the primary inhibitor in their ability to teach novices.

A dumpster fire is just a collection of burning trashbags

While The Curse is unavoidable, we can mitigate its effects by decomposing complex behaviors into core concepts. Explain how each concept works. Explain why it is important. Peel back the curtain on other complex behaviors and show where else these concepts show up.

Think back to mathematical proofs. Writing a proof is a way for a nerdy math teacher to flex on a bunch of super cool teenagers and a time-tested method for reducing a complex mathematical concept into irreducible axioms. The axioms are self-evident; the proof shows how they work together to form something more complex.

“The job of the teacher is to arrange victories for the students” - Quintilian

How small is too small? How simple is too simple? Decompose broad ideas into nuggets just granular enough that they can be understood and converted to small victories. If those around you aren’t able to make an immediate practical application from your nuggets of knowledge, then they are either irrelevant or still to complex.

A novice is aware of what confuses them but doesn’t know what options exist as solutions. Experts are aware of the solutions but don’t know what might be confusing. Like the menu at a restaurant, large decomposing ideas provides a menu of solutions to bridge the gap between experts and novices.

Overall there are three types of concepts to decompose.

First, learn how to break down behaviors.

I could tell Sofia that I code a lot on the weekends, formed an LLC, read a lot of technical and non-technical books, and even wrote a sudoku solver when I got bored on vacation. I could tell her that I attribute my most professional success to behaviors like these and recommend that she also engage in the same. But it would only be a diabolical form of reverse cargo-cultism if I don’t explain “why.”

Without conscious effort, it’s hard for someone who reads a lot to understand why I would need to make a value proposition for reading. It’s just good. But to make a meaningful case, I have to break down the seemingly simple act of reading so that I can adequately explain all of the moving parts, how and why they work, and how she might also benefit.

Reading lets you borrow others’ mistakes so that you don’t have to waste time and money making them yourself. I have over a billion dollars of mistakes sitting on my bookshelf; thankfully, none of them are mine. I will never have a billion dollars at my discretion to spend on screwing stuff up. But reading is only one of many ways to implement this principle. Her cup of tea might be a literal cup of tea while chatting with an expert who has “seen things.”

Reading exposes you to handy names for common patterns that serve a sort of shorthand when communicating with other experts. For example, the “validated learning cycle” might not mean much to Sofia, but those three words are absolutely dripping with meaning to anyone else who has read The Lean Startup. She may not enjoy reading, but the right podcasts are also a fantastic way to learn the names of patterns or problems that are frequently encountered in the wild.

Reading books by reputable authors and publishers provide a source of timeless principles that outlast the fads that grace tech twitter or the front page of Hacker News. But peer-reviewed research by academics is an additional source of such vetted knowledged.

The Curse of Knowledge naively tells Sofia: “Read books.” But the thoughtful expert reverse-engineers the seemingly simple behavior of reading books into its axiomatic values and recommends that she learns from the mistakes of others, speaks using ubiquitous language, and familiarizes herself with timeless patterns. The curse says: “Do the thing.” The thoughtful expert breaks down why.

Secondly, you can break down technology.

While reviewing her code, I could point to the coding guidelines and tell Sofia to pass an interface as a constructor parameter instead of constructing a database connection directly in a class method. But that guideline exists because of the Dependency Inversion Principle. We can further decompose that principle by showing how it interacts with other principles, such as testing in isolation and overall modularity.

But it’s essential to break it down, not dumb it down. If you were to sit down and mind map all of the implications of the DI pattern, you wouldn’t be able to explain everything in a single code review session. So don’t. Instead focus on a single thing, such as testability, and explore a few facets of that. Take a moment to show her a few good examples of clean, testable classes, and then show a bad example. Demonstrate the one small reason that her code would be better, and then move on.

Over time, the principles become patterns for Sofia. Inversion of control seems to pop up as the “why” behind a lot of code review feedback. Given enough time, she will be the one who implements it without thinking, and the vicious cycle of the Curse of Knowledge starts anew.

Lastly, you can break down legacy artifacts of old company code.

There are minefields that lurk deep within legacy codebases that only “lifers” can navigate safely. There are weird dead things that only exist because no one has the guts to touch them. The strategy here is to call attention to specific points of the architecture deviate from industry norms. Pinpoint the best practices went out of vogue but remain in the code.

Explain why the dumpster fire is the way it is one small trash bag at a time.

This approach to shattering metal monoliths provides a way to tune your communication to avoid the Curse of Knowlege. If you are working with a bootcamper, you will likely need to work with very small nuggets. With a mid-level engineer who is new to the team, you’ll be able to communicate effectively with larger concepts. Decomposing allows for mentoring at multiple levels, even concurrently, by motivating novices at multiple levels by focusing on different axioms.

We fixed the world's problems with feature flags

For the better part of a year, Hugh (probably not his real name) and the team had been working feverishly on the pilot for a full rebuild of our main customer-facing product. I had been hired about six months into the rebuild to inject much needed architectural direction into the project. I knew that there wasn’t any solitary nugget of expertise that could revive the project. But on the other hand, as the expert hired with a mandate, I also knew that I couldn’t just slap down a full architectural roadmap and watch perfection immediately materialize. I knew that Hugh couldn’t absorb every nuance all at once, but he could definitely internalize just one.

For the next six months, if you didn’t hear me talk about feature flags, then I would question whether or not you were even alive.

Here’s the thing, there were a lot of other meaningful concepts that I introduced as well: build automation, Typescript, Redux, transpilation, linters, CSS preprocessors, dependency injection, server-side rendering, functional programming, the adaptor pattern, and many more. They were all important, but I couldn’t just say “do all this stuff” without saying “do all this stuff because of all of these reasons.” Instead, I sliced off a single concept (feature flags) and explored every facet of it.

The effect was profound. Suddenly, feature flags seemed to be a solution for anything. Want to release to internal users for testing? Feature flags. Want to ship a pilot? Feature flags. Want to hide a very specific type of data from just one customer for legal reasons. Feature flags. Want to improve the quality of coffee in the break room? OK, not quite everything, but you get the point. Hugh and the team slowly started to internalize the concept and began to think of parts of the code in terms of features that could be turned on or off rather than commits that could be merged or rolled back.

“So the pilot wasn’t perfect. Who cares?”

However, the real payoff was the emotional implications that it had for the team. Hugh is incredibly driven and has high standards for himself, but the downside is that he would often internalize failure. By focusing on what we did with feature flags as a small win in the context of a larger struggle, he was able to refill his “emotional tank.” At one point, I flat out told him: “So the pilot wasn’t perfect. Who cares? I certainly don’t! We were able to progressively roll out the new site with minimal customer impact, and that’s awesome! We’ve never been able to do that in the history of the company! All of the people who matter have noticed and are more impressed with our agility than our perfection.”

And I wasn’t just being dramatic, they did notice. During the quarterly all-hands meeting, our CEO described how by leveraging feature flags, we allowed the whole company to move more quickly and nimbly than ever. Focussing on feature flags allowed him to attribute success to effort rather than luck. Everyone forgot about the failure. The team got recognition for the success, and both Hugh and the CEO knew precisely why.

You can learn from my mistakes

I backed into this methodology by accident, but you don’t have to.

What’s your big idea that you just can seem to get to take root? What expertise do you have that would benefit the rest of your team? For me, it started with the practice of peer code review. I had seen first hand (from both perspectives) how beneficial it was, but I couldn’t for the life of me get others around me to do it at all, let alone well.

I had recently been transferred to a new team and been tasked with improving the quality and velocity of the team. It was second nature to me to maintain code quality with peer code review. However, it was impossible to establish that best practice until I took the time to dig into WHY code review works.

They didn’t care until I told them why.

I knew the “what,” but I was punched in the face with the fact that first, I myself had to learn the “why.” Once I had broken it down in my own mind (and wrote this article), it was almost cheating to sneak various principles into everyday conversation.

Beat the Curse! Show new engineers why the small parts of big systems are worth it. Tee up small victories! Everyone will love you for it!

If you liked this article and would like to give my brain the good chemicals, mash a few of those response buttons below! It’s engagement from readers like you that got me nominated for TWO NOONIES! Thanks! You can now vote for me for Contributor of the Year in both Leadership and Personal Development.

Sourcse:

• Photos by Pixabay, Moritz Mentges, Kaleidico, and Louis Reed
• Made to Stick, Chip and Dan Heath
• Faculty Development Strategies for Overcoming the "Curse of Knowledge"; Froyd and Layne; 2008
• The Curse of Knowledge in Economic Settings: An Experimental Analysis; Camerer, Loewenstein, and Weber; 1989
• Decoding the Disciplines: a Model for Helping Students Learn Disciplinary Ways of Thinking; Middendorf, and Pace; 2004