This is from my series, The 5 Secrets to Automated Testing Success. I wrote this series in 2012 and am posting them again for you here.
The only thing I would add to this post, three years later, is that there are as many reasons for selecting a testing system as there are teams selecting them (probably more). If you want assistance, call us.
Selecting the proper tool for your team is one of the most important keys to successful test automation. There are many tools out there. Many vendors offer training and support options available for these tools.
Beaufort Fairmont has worked with many tools in the automation testing space, and we understand both their benefits and costs. We have the skills to sit down with your team and help make a non-biased decision with you. Further, if you pick a solution from the open-source space, we offer support & training for many of these tools.
The choice of an automation tool can be daunting. We’re here to help.
The main factors involved are:
· Maintenance of test cases
· Integration with Continuous Integration server
· Speed of writing tests
· Ability to refactor test cases into reusable code
· Integration with source code control
· Available Training
· Available Support
· Persisting results
· Visibility of results
· Available Metrics
One thing you don’t see listed is integration with an existing defect system. Many managers believe this is the most import feature a system can have. In fact, most of the reasons managers want this are invalid and harm the development model.
For instance, one manager thought it would great if failed test cases created defects in the defect system. Sounds great, right? But what happens when every test fails one night because of a small error a developer introduced? Your defect system is flooded with defects that don’t help anyone!
Is it helpful to know that the system broke last night? Absolutely. But the cost of adding hundreds or thousands of defects to a defect system is too high. Unless, of course, you want to make development look bad, and then you have other problems. See Secret #1.
Other teams think they want test cases persisted in a defect system in addition to the test system. This duplication generally leads to confusion. In my experience, there’s rarely a need for this.
Most of the time, features like these are requested by managers who need a way to report on their team’s metrics. There are other ways to accomplish this. Don’t duplicate test cases and flood defect systems with unnecessary noise.
In other words, these are not key features in testing systems.