Maps, Gold and Spices: Why Neglecting Documentation Can Shipwreck Your Project

What was the most valuable asset that the explorers of old brought home from their voyages? Gold and spices? Wrong. Maps.

Christopher Columbus would have never done his famous journey in 1492 without a map designed by his predecessors. A spy game to obtain Portuguese maps laid the foundation for the Dutch East India Company, arguably the most expensive company in human history. At its peak in 1637, it was worth 78 million Dutch guldens, an equivalent of $7.9 trillion in 2017 dollars. That is more than $10 trillion in 2024 money, more than Microsoft, Nvidia, and Apple combined. Palpable treasures from new lands were important in the short run, but from a longer perspective, it was the knowledge that forged real fortunes.

How come in today’s world of tech we tend to forget this? Chasing immediate success, we are often reluctant to commit valuable time and resources to producing and maintaining technical documentation. Speaking in 17th-century terms, we rush to grab gold and spices without plotting our maps which, in turn, could have led us to much more gold and spices. Are you skeptical? Ahoy, let’s take a closer look…

Wacky code

“As you no doubt remember, the hypnoid stasis of the neuronic patterns of the brain can be scanned by an extra-electromagnetic beam which –” “Come off it!” said Ard Vark impatiently. “What do you mean – as we no doubt remember?” How can we remember it when we never knew it? – This quote from Wacky World, a story by a great science fiction writer Edmond Hamilton, refers to Martians, not programmers. However, many people view developers as if they were from another planet – especially those who have only a vague idea of software development and its complexities. The fact is, that developers often assume that others know code just as well as they do and frequently consider technical documentation unnecessary. This mindset risks making the project as complex and incomprehensible to outsiders as a "hypnoid stasis," ultimately jeopardizing the project's potential success.

The reluctance to create documentation often stems from the same reasons people procrastinate in other areas: it requires significant time and financial investment. In other words, it’s often due to sheer laziness and a desire to save money, which aren’t easy obstacles to overcome. However, documentation is not just redundant information that's supposedly obvious to everyone; it contains critical details that can be indispensable. Often, the absence of documentation significantly complicates the detection and correction of errors, makes maintenance and updates more difficult, and increases the time needed to onboard new team members. While teams without documentation are stuck doing repetitive tasks, projects with well-structured documentation show high efficiency and reliability—this is a fact, not a mere opinion.

Yes, some programmers claim that the code they write is so clear and understandable that documentation is simply unnecessary. However, in reality, even the most perfect code can be confusing to others or lose its clarity over time. What seems clear today can become a puzzle tomorrow. For example, could you easily deal with a simple punch card from the '70s?

Bus factor, Burnout, and other fantastic beasts

Theory is good, but practice is more convincing. Here are some examples, based on true stories, with only the names of people and companies fictional. These brief case studies cover the most typical problems that arise due to poor technical documentation practices.

Story #1: Inability to Scale

The project "NoDocumentationPlease," initially a successful video streaming startup, faced serious problems when trying to scale due to poor technical documentation. When the team needed to expand, new employees couldn't fully comprehend their tasks, and no one could provide them with an adequate explanation. Without proper support and training, new hires quickly left. This not only slowed the project’s progress but also led to the loss of key talents, ultimately jeopardizing the project’s overall effectiveness and future. As a result, the streamers left the chat, and the project was shut down.

Story #2: Maintenance issues

The company "IKnowEverything" developed a cloud platform for data synchronization and storage. Initially, the project progressed quickly, but over time, its developers faced difficulty maintaining and updating the platform due to lack of clear and up-to-date technical documentation. This led to slower development, more bugs, and dissatisfied clients. Eventually, the company started losing its old customers, and new clients chose competitors with more stable and reliable solutions. Revenues decreased significantly while the cost of ineffective maintenance grew. Properly documenting the technical aspects from the start could have allowed them to scale successfully. However, it was not done in time. Consequently, the company couldn't overcome technical and financial challenges and was closed.

Story #3: Developer Burnout

The project "SmartestEver" faced severe issues because its main developer, Andrew, who handled just about everything, resigned after being overwhelmed by the team's numerous questions. Had "SmartestEver" had proper documentation, junior developers could have easily referred to the FAQ and solved routine issues. Instead, they bombarded Andrew with questions, and without him and the necessary documentation, the team just failed to go on and the project was shut down (press F for Andrew).

Story #4: Bus Factor

In the company "NoDocsNeeded," a promising software product was being developed by John, a key developer, who held all the knowledge but did not bother to document it. His managers didn’t bother to persuade him either. There came a day when John went on a business trip and just didn’t come back. Without documentation or understanding of the product’s architecture and logic, the remaining team members could basically do nothing. The project was frozen, and the money invested in it was wasted. The lesson is simple: documentation and knowledge distribution within a team are crucial to avoid dependence on a single person. By the way, they are still looking for John…

Story #5: No prophet welcome at Open source

Maria created her first open-source library but didn’t write any documentation for it. No one understood what the library did, and Maria decided she wouldn’t write any more libraries because to her it seemed pointless. Maria’s project ended before it even began and she decided to change her profession.

From cabin boy to captain

Ok, we got some theory and practice, now let us dive into research and statistics. Stack Overflow Developer Survey 2024 states that 84% of respondents use technical documentation to understand the functionality of technologies. However, even with documentation, they often can't find the answers they need, as shown by the next most popular resources: Stack Overflow (80%), Written Tutorials (68%), blogs (61%), and How-to videos (54%). Microsoft Research: The Daily Life of Software Developers found that, on average, developers spend 1-2% of their day (8-10 minutes) on documentation, and 10.3% of developers say outdated documentation forces them to spend too much time searching for answers on their own. The necessity of documentation is a major concern for the academic community as well. A simple Google Scholar search for <"documentation" AND "software"> yields over 4 million results, clearly indicating a vast number of scientific publications addressing the issue.

The main conclusions are astonishingly simple: #1 – Everybody needs documentation when it comes to understanding tech and/or other people’s work; but #2 – Few people bother writing and maintaining it; and consequently #3 – A lot of documentation is poorly written, out of date and generally useless. So what has to be done? Change your motivation at all levels.

A group of researchers from HAN University of Applied Sciences and the University of Groningen (both in the Netherlands) identified the most typical problems with technical documentation:

• Informal documentation that is often used by developers is hard to understand;

• Documentation is considered waste when it does not immediately contribute to the end product;

• Developer’s productivity is measured by the amount of working software only;

• Documentation is often out of sync with the actual software;

• Developers often keep just a short-term focus, especially in continuous software development environments.

  
  

Does any of this sound familiar? I can bet most of us bumped into most or even all of them at once in our routine daily work. And there is more to that than just procrastination or lack of resources. Some of these problems come from a lack of proper management, long-term planning, and, ultimately, of strategic vision. And here comes the difficult part, because it is not just up to us, developers to resolve. Some issues should be dealt with by managers, product stakeholders, or even company owners. That’s why it is crucial that proper views on technical ’t be just a nice accessory, but part of the entire company’s core values, shared by everyone from founders to junior developers.