Quality and test automation: don’t underestimate processes

Photo by Chunlea on Unsplash

True stories about quality and false solutions.

Cet article est également disponible en français.

Far be it from me to disavow the usefulness of test automation. I am a strong advocate of test automation. Yet, too often the topic is discussed from the wrong angle. Here are some examples!

We need help to get started with test automation!

A team seeks my advices as a Test Facilitator. They don’t control regressions. They come to me so that I can help them to define a test automation strategy.

What do I recommend to them?

After having discussed with them to understand the situation, my recommendations are mostly about processes:

• To be considered valid, a bug fix must come with an automated test which fails without the bug fix. (a unit test can be enough)
• They suffer from long-lived branches, taking way too much time before merging. Either they switch to another branching model, or they thoroughly review their current branching model and strictly follow it.
• Their releasing process is not clear and, most of all, they are not really assuming it. When do they check that there is no regression? How? They need to formalize their process and strictly follow it.

… and only when they will have taken care of these essential problems, will we start talking about test automation.

Automated tests are the answer to all our problems!

Ah! I have heard this sentence so many times…

Come on we really gotta automate these tests! All our problems will be gone after that.

Indeed, yes, automating these non-regression tests would be a wonderful idea. By the way, this is such a good idea that I’m wondering: why isn’t it done already?

Still I’m saddened to say you that, no, automating tests won’t magically solve all of your problems.

Even with a great non-regression test harness, you’ll have to keep challenging the product.

Test automation is awesome. But it’s no silver bullet. Test automation has not marked the demise of manual testing. Even worse, with test automation you are still required to think, to set up a strategy, to use the product.

Rigor is enough to fix most problems

Hell yes! How boring processes are. Nobody cares. Why are we bothering?

Ouch.

Yet processes — and following them! — are key to a well running team. As I like to put it:

Agile requires to be extremely rigorous.

Stop doing Agile under perfusion!_… And stop behaving like heroes!_hackernoon.com

And when I’m talking about processes, I am not expecting complex decision diagrams lost into the corporate intranet. No, processes can be simple, visual, understandable… Most of all, it must fit with how the team works.

Un process est un outil, pas une contrainte_Comment faire pour que vos process soient plus qu’une doc !_jp-lambert.me

Processes > Test Automation

Let’s keep it brief. People like talking to me about test automation because they think it is a silver bullet.

Unfortunately, it isn’t. Well, “unsurprisingly” would have been a better choice. You should know better by now: silver bullets simply don’t exist.

What’s a good automated test?

You’re not even close to know if you don’t already know what makes a good manual test.

Automating tests cost so much! Where should we start?

Well, why not starting with the most critical scenarios… I’m guessing you know them, right?

We’re scared of regressions! We have to automate tests.

Why are you scared of regressions?

If you don’t trust your non-regression test scenarios, automating them will do no good. Have you defined those test scenarios?

If you are not running the non-regression test scenarios by hand, then obviously regressions are not that a big deal. Are you sure you’re scared?

We don’t have time to automate the tests!

If you were wasting your time running the non-regression test scenarios by hand, you would find the time. Calculating the return on investment would be trivial and straightforward.

Thus, the situation doesn’t look that bad…

Overall, this is what happens:

People ask me guidance about quality and non-regression, people want to be told about test automation, my answer is about process and rigor.

Just start by reviewing thoroughly your processes, and follow them strictly. You can get incredibly far with just that! Take for instance this team that I have helped:

Our recipe for Daily Releases_It was upon a time a team that struggled to release…_jp-lambert.me

Liked this article? Show it!

Please clap 👏 and share the article! It is because of you that I put my heart and soul into writing.

And follow me on my blog to be notified when I publish new articles!

Thank you so much!