Duplicate Automated Tests

Posted on July 10, 2018

Several times in the last few weeks I’ve had a conversation about duplicate test cases. It’s a familiar conversation to me. The other person asks whether it is ok to duplicate test cases or states adamantly it is not OK to have duplicate test cases. But which is right?

For the purposes of this conversation assume I’m talking about automated test scripts when I use the word “test case” or “test”.

When Duplication Is Unhelpful

You’ve likely seen where an UI test looks suspiciously like another test case. Maybe it has a slightly different name. It appears these two test cases do the same thing, but you can’t quite tell. Both tests use the same tool, are written at the same level and use similar data to verify functionality. They just look different. Worse yet, no one seems to know what the difference is or why there are two of them!

Delete these when you have reason to. If you can’t figure out why there are two of them after talking with others. Make sure to do your homework first and ask around about why these are there. We don’t want to overlook a small distinction that makes a huge difference in what the tests do. The amount of “homework” you need to do in determining if this is a duplicate is dependent on your company/org/dept/team and culture. You’re the expert in your workplace, not me.

Just like with functions or with libraries we use in coding, it is helpful to look for and using what others have already written. Don’t write what may be a duplicate test case just because you don’t understand at a glance what one is doing.

When you learn what they’re doing differently, take a moment and make it clear. Rename one. Change the behavior of the test slightly to make it more clear what the difference is. In the worst case, leave a comment referencing the other test and how this one differs. In short, don’t leave someone else the same mess you found. Leave the code better off than how you found it.

The Overlapping Case

Let’s say that you have a test case that allows a user to submit payment for a shopping cart on your retail ordering page. It is a UI test written in Selenium. You noticed that there is also a test case written to test whether the API works for purchasing the items in a cart. These tests “test the same thing”. Should you delete one of them? If you said yes, do you consider yourself open minded?

I’ve found a lot of developers would say “yes, delete one” and a lot of testers would say “no”. But often, I find the reasoning to non-exclusive. In other words, one argument doesn’t counter the other. A developer may say “yes” in an effort to avoid duplicate code and inefficiency. A tester may say “no” because of a desire to be thorough and mitigate risk.

Having two tests is not efficient, right?

When Duplication is Helpful

What do we learn when the UI test fails and the API test (that does “the same thing”) passes? It is likely there is an in the UI. We also have evidence that the issue is NOT in the API (the api test passed). Is it helpful to know where a defect is? Sure. Especially if you’re a developer. The two most time-intensive parts of programming (IME) are:

1. Figuring out what to build, and
2. Figuring out where a problem is in the code.

While testers may not have traditionally helped Developers locate defects, it certainly would be kind if we could.

Once developers experience this – test cases failing and passing leading them directly to a defect, it can be a career changing moment. For them to experience this, we as testers have to provide automated tests that run without human intervention and are completely trusted.

I’ll leave it as an exercise for you at home to figure out what we can learn in the other 3 cases: UI passes & API passes, UI passes & API fails, UI fails & API fails.

Testing is an exercise in efficacy, information gathering, communication, and risk mitigation. Should we require efficiency for all testing? Some times efficiency can be helpful in testing. Other times efficiency damages testing.

Polymorphism

One thing I love to do when using a test driver like Cucumber, RobotFramework, or Gauge is use the same test script to drive interaction with the application in different ways. To purposefully duplicate a testing activity with code. An example…

Let’s say that we have an endpoint that accepts input and posts it to a datasource (think database, or file or something). I’ve created a spec in Cucumber like this:

Given I am a user of the application
When I want to add a secondary contact detail to my profile
Then I can add my email address

I could write this test case using the UI with a tool like selenium or cypress or whatever. I could write this using a tool like RestAssured, calling an API endpoint to input the email address and verify. I could also call directly into the application code and go check the database, leaving all the api, UI, and infrastructure out of the loop. I would not consider this a unit test because it connects to a DB.

What I love about test drivers is that I can use the very same spec to implement a test in multiple ways. All 4 implementations of the test (and spec) above are valid tests. They each tell me different things:

1. Whether the end-to-end flow works,
2. Whether the POST and GET API work,
3. Whether the code works without the API (and network, client, servers, infrastructure, etc) and whether the proper db field gets updated.

All the implementations can be run from the same spec and arguably should be – so that we know each part of the application works as the Product Owner specified.

This is polymorphism. It means “multiple actions that look the same to an outsider.” I’m not going to explain more because it was explained so well in this webinar.

Finally

There are times when we need to eliminate duplicate test scripts. There are times when we don’t. There are times when duplicate test cases is actually a good thing. Finally, there are ways to implement the duplication that make sense for your context and ways that don’t. I hope this post helps illuminate some healthy motivations for duplicating test cases and some helpful techniques for doing so.