How to Write Clean Code and Save Your Sanity

Writing code is a skill that anyone can acquire, but attaining the highest standards of clean code requires discipline and dedication. In this article, we will explore practical strategies and tools to help you elevate your Python code to impeccable levels. By gradually incorporating these best practices into your development lifecycle, you can ensure clean, bug-free code that stands the test of time.

Adhering to best practices not only maintains code cleanliness but also minimizes the risk of introducing bugs. While it may require an initial investment of time, following these practices ultimately reduces development effort in the long run. As a seasoned full-stack software engineer, I have consistently received accolades for upholding coding standards in the companies I have worked with.

To illustrate these best practices, we’ll consider a hypothetical project called Stack-Scraper. This project consists of a single-page website scraper that extracts questions and answers from (hypothetical) Stack Overflow. Please note that Stack-Scraper is not a functional project but serves as an example to demonstrate the ideas discussed here. The source code for this example can be found on GitHub.

I personally use VS Code as my go-to IDE, and so some of the tools and plugins mentioned here are for the same. Let’s dive into the best practices and tools I employ to ensure my Python code adheres to the highest standards.

Repository Structure

A thoughtfully designed repository structure serves as the foundation for clean code development. If the structure fails to meet the project’s requirements, developers may scatter code across different locations, resulting in a messy structure and decreased code re-usability. A carefully crafted repository structure plays a vital role in maintaining code organization and facilitating collaboration among developers.

It’s important to note that there’s no one-size-fits-all solution for repository structure. Each project may have unique requirements that warrant a specific organization scheme. However, examining a practical example can provide valuable insights. Let’s consider the repository structure of the Stack-Scraper project. Here are some key points to note:

• The Stack-Scraper repository employs clearly demarcated folders for different components, such as APIs (src/apis ) database models (src/db_wrappers ), domain-level constants (src/constants), pydantic models (src/domain_models), and scrapers (src/external_sources/scrappers) etc.. This segregation ensures a logical separation of code, enabling developers to locate and modify specific functionalities easily.
• The test files in the Stack-Scraper repository follow the same hierarchy as the main repository. This practice ensures that tests remain organized and aligned with the corresponding code. Consistent test organization simplifies test management and improves code testability.

Pre-commit

pre-commit is a framework that enables the execution of configurable checks or tasks on code changes prior to committing, providing a way to enforce code quality, formatting, and other project-specific requirements, thereby reducing potential issues and maintaining code consistency.

All you need to do is create a pre-commit-config.yaml file in your repository. You can install pre-commit by running the following commands.

pip install pre-commit
pre-commit install

Let’s examine the pre-commit-config.yaml of Stack-Scraper.

repos:
  - repo: https://github.com/ambv/black
    rev: 23.3.0
    hooks:
      - id: black
        args: [--config=./pyproject.toml]
        language_version: python3.11

  - repo: https://github.com/pycqa/flake8
    rev: 6.0.0
    hooks:
      - id: flake8
        args: [--config=./tox.ini]
        language_version: python3.11

  - repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
        args: ["--profile", "black", "--filter-files"]
        language_version: python3.11

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: requirements-txt-fixer
        language_version: python3.11
      - id: debug-statements
      - id: detect-aws-credentials
      - id: detect-private-key

Here, we have configured 7 hooks. You can add more hooks if you need but the above setup is more than enough to help you keep your code clean.

Type-hints

Python’s dynamic typing allows variables to be assigned without explicit type definitions, which can lead to convenience. However, this flexibility can pose challenges in maintaining code quality, especially in larger projects where type errors and inconsistencies may occur.

Consider the following example from the file: src/apis/question_no.py in Stack-Scraper. Can you identify if ques_no is an integer or a string?

@stackoverflow_blueprint.route("/stackoverflow/<ques_no>", methods=["GET"])
def get_question_answer(ques_no):
    # code to do stuff

Check out the same example now with type-hints. It clearly highlights that ques_no is expected to be an integer and get_question_answer returns a Response object.

@stackoverflow_blueprint.route("/stackoverflow/<ques_no>", methods=["GET"])
def get_question_answer(ques_no: int) -> Response:
    # code to do stuff

Doc-strings

Doc-strings are documentation strings added for each function to improve code readability. They provide additional information about the function’s purpose, parameters, and return values, making it easier for developers to understand and utilize the code effectively.

You can also use doc-strings to generate automated documentation for your code using a library like pdoc. Consider the following example from Stack-Scraper and the corresponding documentation generated using the pdoc library.

SonarLint

SonarLint is a code analysis tool that integrates with various IDEs (as a free plugin) and helps identify and fix code quality issues, security vulnerabilities, and bugs during the development process, enabling developers to write cleaner and more reliable code. This has been a must-install plugin for me for years now.

Pydantic

Pydantic is a Python library that provides runtime type checking and validation for data models. We don’t aim to make Python behave as C/C++ but there are certain use cases where it’s paramount to enforce type-checking. Examples: API inputs, methods in database wrapper to provide consistent in and out of data from database, etc.

Stack-Scraper demonstrates an implementation for the same. By leveraging the Pydantic model, StackOverflowQuestionAnswer as the output of the get_question_answer_by_ques_no method within the InMemoryDatabaseWrapper class, we establish a reliable means of retrieving questions from the database based on their unique identifiers. This design choice ensures that future changes to the database, including modifications to its schema, will not disrupt the application’s functionality as long as the method’s output adheres to the consistent model defined by pydantic.

Another demonstration is validating the inputs to the method upsert_question_answer in the same class as above to ensure that only allowed values go into the database.

Spell Checker

While it may initially appear counterintuitive, incorporating a spell checker into your code is a valuable practice that aligns with the other advice provided. The inclusion of a spell checker ensures code quality and readability, reinforcing the importance of attention to detail in maintaining clean and error-free code.

Firstly, it will help to avoid naming methods like upsert_qusetion_answer due to their unintuitive spelling, as it can lead to errors and confusion during usage. Notice the spelling mistake in qusetion.

Secondly, if you utilize an automatic documentation generator, it becomes essential to minimize spelling mistakes. Maintaining accurate and error-free documentation not only improves code understandability but also enhances the overall professionalism and credibility of the project.

Tests

Comprehensive testing is crucial for maintaining error-free and robust code, ensuring its adaptability to future changes and different developers. While aiming for 100% coverage is desirable, the focus should be on thoroughly testing the code rather than just the coverage number.

When writing test cases, it is important to consider all possible scenarios and edge cases that the code might encounter. This includes positive and negative testing, boundary testing, and error handling. An example test case for the get_question_answer function in the test/src/apis/test_question_no.py file can provide insights into designing effective test cases.

That’s all for this article on maintaining clean code practices and enhancing code quality. I hope you found the insights and strategies shared here valuable. Remember, implementing these best practices and incorporating them into your development habits one step at a time can make a significant difference in your code’s reliability and maintainability.

By prioritizing code cleanliness and adhering to these practices, you’ll witness your code soar to new heights. Enjoy the journey of writing impeccable code and reap the rewards of robust, error-free software development. Happy coding!

Additional Tips

Here I am listing a few more generic tips/comments that you can incorporate into your coding habits.

• Add comments wherever warranted in your code. If you make an assumption in the code, it’s better to highlight the same in the comments. Example: Line 78 in file src/external_sources/scrappers/stack_overflow.py .

  
  

• Try not to define constants inside functions/classes because, over time, you could end up having a bunch of these constants scattered across the code. Either define them in their respective constants file or in the respective module. Example: src/constants/api_constants.py .

  
  

• Define your own Exceptions rather than using the generic one. This helps in differentiating exceptions in code to respective segregation. Example: StackOverflowException in file src/external_sources/scrappers/stack_overflow.py represents exception raised by the Stack Overflow Scrapper.

  
  

• Follow OOPs for coding. Example: Inheritance by defining base class ExternalSource in file src/external_sources/external_source_base.py .

  
  

• Follow REST principles for API development.

  
  

• Do not add secrets to your repository. Instead, use an environment file or a Secrets Management Service like AWS Secrets Manager.

  
  

• Make your code environment agnostic. So, that it’s easy to maintain separate development, testing, staging and/or production environments. Example: src/config.py .

  
  

• Peg versions of the libraries that you install in requirements.txt file. It helps in avoiding application failure because of breaking upgrades to libraries.