You Don't Have Time to Skip Test Automation

If you have a lot to do in a short amount of time, it can feel like test automation is one of the things you can afford to skip. After all, a good automation quality advocate (QA) is also a good manual tester — right? So why not skip it?

There are many reasons why you move faster by sticking with test automation. You do pay some upfront costs in terms of time to set up the test framework and integrate it with the build and deployment pipeline. Those costs are repaid with interest the longer the project continues.

Why is test automation important?

Would you rather push a button and launch a comprehensive regression test suite, or have the whole team spend a week doing manual testing? Test automation allows you to leave the unchanged parts of the app in the hands of tests, so you can focus on what's changed. 

A broken UI automation test can pinpoint where a problem is before a human can find it and provides the steps to reproduce the bug along with the proof that it's fixed. Reproducible bugs are fixable bugs and test automation logs provide you that reproducibility. That means more time for delivering new features.

Test automation allows a QA engineer to focus time on exploratory testing, which can uncover bugs that require some creativity and testing experience to find. The best strategy for a quality product is to have both exploratory and automated testing. Automated testing provides your safety net while exploratory testing exercises an application as a user would, including the malicious or inexperienced users.

One of the lesser-used aspects of test automation is an ongoing project-health metric. If a smoke test runs in five minutes for the first month of the project, then the same set of tests starts taking seven minutes, ten minutes or longer, you have an early warning sign that something has altered the performance and efficiency of your application. 

You can also use your test automation to perform a soak test to magnify what could be a small error into a large one you can fix. You start the test suite running on Friday afternoon and make sure it will repeat itself until Monday morning. Small errors repeated hundreds of times become bigger and more dramatic, which makes them easier to find. Load testing is prohibitively expensive to do manually, but an automated load test can simulate ten thousand users trying to order the same sandwich for delivery, without needing ten thousand people to actually do it.

Computers are great at doing the same thing over and over without making mistakes or getting bored. Humans are not as good at that, but much better at creative thinking, such as putting yourself in the shoes of a user. QA engineers are excellent at coming up with real-world scenarios you didn't think of when you were creating the application. 

For instance, what if the user is trying to create an account and they lose their network signal or their battery dies? What if the user has trouble with small buttons and text fields and needs more time to fill in a form than you gave them? Is the application easy to use or do you make the user jump through hoops to get simple tasks done? Have you thought about the latest security exploit or mobile operating system update? QA is a discipline that encourages curiosity across a broad range of topics.

The arguments against test automation usually fall into a few categories.

"But the test suite is so big it's useless!"

Part of the role of a QA engineer is both feeding and weeding the test suite. Every test must justify its existence. Tests that don't help your overall goal of delivering value to the client are a waste of time and effort. Break a large suite into chunks with tags and run each piece as necessary, you can run the whole suite overnight or at the weekend. Deleting tests is as important as adding them.

"But QA doesn't know how to write test code!"

The first step towards someone being a good test automation engineer is to work with their developers and see where the gaps are in what has been tested. You don't duplicate unit tests, you focus on the points where one module connects to another, where data is in transit and the nonfunctional requirements. While you need some code skills to write tests, those skills don't have to be flawless. We often work in cross-role pairs and each side learns from the other.

"But these tests are so flaky, they fail all the time!"

A failing test is trying to tell you something. If a test has suddenly become flaky, something in the application changed to make it that way. Finding out why could show you a problem you didn't know you had. If a test has always been flaky, explore why that is so. Reliability is part of quality. What the tests give you is data.

None of those objections outweigh the benefits of having a solid suite of automated tests. If you're starting from scratch, you can create a quick sanity test and expand out from there. If you're creating a brand-new project, you can bake in test automation from the start — it saves you time, effort and frustration. One team went from spending over 30 percent of their time on bugs in a project with no automation to spending less than 5 percent of their time on bug fixes on the following comparable project, where we put in automation on day one.

Paying the upfront cost of time and effort in test automation will pay off down the line in time saved, rework eliminated and overall quality.