The Future of Testing: Automation

Last week we talked about the Future of Agile Testing, and no discussion about Agile would be complete without discussing testing automation. In fact I would venture to say that automation is a key component enabling Agile to be successfully adopted by quality-conscious organizations. The following is based on a  recent interview with Softtek Qualty Assurance & Validation expert Laura Salazar.

First, a brief background on testing automation. There are three main approaches to automation:

Code-Driven testing: This approach, which uses testing frameworks such as xUnit and others and looks at test case execution to determine if various sections of code perform as expected, is the preferred testing method for agile software development teams where it’s also known as test-driven development (TDD). Unit tests are written to define functionality before the first line of code is created, and the testing actually evolves and become part of the coding process. If this type of continuous testing were manual it would significantly extend the time it currently takes agile teams to ship completed software, defeating the purpose of agile.

Graphical user interface (GUI) testing: Used to test applications with GUIs, testers record the actions of users and analyze these actions repeatedly. This is perfect for websites and consumer-facing apps.

API-driven testing: API testing is quickly gaining in popularity as an alternative to GUI-based automation testing because of the latter’s difficulty. Programmers or testers will write scripts that calls interfaces exposed by the application under test.

Short-Term and Long-Term Benefits of Automation Testing

Automation conjures images of the developer liberated from the drudgery of manual testing while the testing automation package, such as HP’s Quick Test Pro or Selenium, does its job. Although we’re not quite there yet, automation has brought substantial benefits to organizations hoping to deliver quality software more efficiently and on an accelerated timeline.

1. Near-term Benefits of Testing Automation: Testing automation provides developers a sort of “code safety net.” Because you can weave automated tests within your development lifecycle, you and your team can feel confident you won’t break the code you just wrote because you get instant feedback on changes. It also allows you to implement build verification tests, saving you from having to implement full-blown tests if the code you just wrote is faulty.
2. Long-term Benefits of Testing Automation: The near-term benefits of testing automation creates a cumulatively beneficial set of long-term benefits. The ability to gain instant feedback on software changes, test while you code, and implement time-saving build verification tests means that scrum teams can now maximize their velocity. The improvement in time utilization means time-to-market and development lifecycles have been reduced. Additionally, automation creates capacity for exploratory and ad-hoc testing and enables activities that can’t be performed cost-effectively by humans.

Wait, There are More Benefits!

However, there still exists the misconception that testing automation is primarily all about test cases. But because so much of it now leverages the cloud, automation has now become one of the principal enablers of continuous delivery and DevOps.

For example, testing automation is now used for test preparation, setup and configuration, test content development and execution, and non-functional testing. It can even be used for deployment since it simplifies hand-offs, collapses roles and eliminates error-prone and time-consuming work.

Test Automation: Not a Panacea

But testing automation is not a panacea either. It’s not possible (yet) to completely “outsource” all your testing to an automated process.

Automation is primarily implemented after development is completed because it focuses on the first-time discovery of defects and test coverage optimization – that’s why it should be embedded into each development sprint.

Keep expectations realistic and consider the following when implementing a testing automation regimen:

• Your Automation Use Case: Identify and evaluate the automation use case for your organization. As mentioned earlier, it doesn’t make sense to automate everything, since it will not find unexpected or unscripted bugs. That’s where ad hoc or exploratory testing comes in. Carefully select what to automate, target those core functionalities that can be accomplished during short periods and can withstand subsystem changes.
• Evaluate Automation Tools: Based upon your use cases, evaluate tools and choose the best fit for your scenario. Many ISVs claim they solve every problem, but it’s important to look under the hood and consider not only the cost, but the time you and your team will spend learning to use the tool as well as implementation time. Consider infrastructure, whether the software will be on-premise or in the cloud. If you’re evaluating cloud-based tools focus your evaluation on the product itself, then consider security and costs.
• Assess Your Team: Evaluate your team’s skillset and determine if they need additional training or you need to hire specialists, since your automation strategy requires higher technical expertise.

Conclusion

Testing automation is ultimately tool-dependent – and a tool is still a tool (no pun intended). They don’t have human capabilities - yet. Use automation to free up your teams in order to focus on ad-hoc and exploratory testing, which we will be covering in our next post.