Over the last couple years, Angie Jones and I have facilitated a “Dominos of Automation Workshop” (check out the fun we had here, here, here and here) at several conferences. StarEast was the workshop’s swan song. It’s run its course.
At the beginning of the workshop, we ask individuals to pick an application under test (AUT) and create test ideas for it. We give participants 5 minutes to think up ideas of what to test.
For the workshop to work well, teams need a lot of ideas for tests. Each test idea is represented by a domino and the workshop helps participants think through test automation. Attendees are presented with the challenge of getting the most accurate information they can get from the AUT using automation and CI.
When we first did this workshop we asked teams to work together to come up with test ideas. What we found was that teams came up with few ideas. We iterated. Thinking the problem was that teams spent more time negotiating what the tests were, we made a change. We asked the next group to individually come up with test ideas. Afterwards, they’d come back together as teams. They’d discuss their ideas, de-duplicate them and make sure they had a good list.
What we found was that we still had many teams that would only have 6 or 7 test ideas! Think of this. In total attendees had 20 person-minutes of time devoted to coming up with ideas for tests. At 7 ideas that’s about 1 test idea every 3 minutes! That seems so slow to me!
Dollar$
The financially-minded among us look at this and realize that at local rates this is around $3 per test idea. Most products these days have millions or billions of permutations of operations. At $3 each, we very quickly cost more than a practical amount or money – just to think up ideas for tests.
In fact, the cost of the collection of ideas quickly becomes far more than the liability the test ideas intend to mitigate. That’s a big problem. (Pro-tip: don’t cost your employer more than you could save them.)
I noticed some other things during this segment of the workshop. I saw people looking up at the ceiling trying to think of ideas. I saw people pausing to think.
During the latest version of this workshop, I asked “was this difficult”? More than half the attendees said “yes”.
The Testing Activity
For me, testing is as much a creative activity as it is an analytical one. Learning about a system is mostly analytical for me. Figuring out how to get information from a system can be creative.
Practice
As you may know, I run Beaufort Fairmont. Business is a medium I’m enjoying creating within. There was a time when I believed an idea for a business was the most important part of building one. I remember discussing this with a friend. I wondered if I could get better at coming up with business ideas by practicing. I asked “why don’t we spend the next 5 minutes thinking of new business ideas.” He said, “I don’t think it works that way”. So… that exercise is one I’ve had to do on my own since. As it turns out, I was right. Practice helps. I’ve improved dramatically in my creativity in this and other areas with practice.
What if an exercise like this could help us get better at creating test ideas like it did for me with business ideas? What if this exercise is one of many we could be using to continue improving our testing skills?
If you’re open to getting better at creating test ideas, I want to help. I want to provide practices for you to extend your capabilities as testers and your reach into the billions of unique behaviors of a system.
A mini-workshop
In May I’ll do a mini-workshop on the topic of coming up with test ideas. Whether you’re an automation engineer, tester, developer, manager or something else, this session is for you. Here are a few things we’ll discuss together:
- How we approach creating test ideas
- The scope of a test idea
- The structure of a formal idea for test:
- Create a hypothesis
- Create a test for the hypothesis – happy path and control
- Create variations of the baseline test and use that information to determine if the hypothesis is correct.
- How to drop formality in creating test ideas
- Breadth-first vs depth-first across the application’s functional tree.
- Hueristics
Art
I’ve noticed creativity is a muddy pig. Catching it is difficult. With art, I think if you put me in a room with every conceivable medium and asked me to create whatever I wanted, I would likely be stuck trying to think through what to do. On the other hand, ifyou told me to create something with clay or with watercolors, I may be more able to create. Some of the most moving pieces of art I’ve experienced were great because of the constraints imposed on it – the medium, the subject, the style, the color palette, the texture, etc.
For me testing is much like this. In order to be creative, sometimes I need constraints. A process, a functional grouping, a persona, a level of the application, a component, an endpoint, a programing language, a test runner. Any or all of these things can help me. Changing any one of them can open doors.
Join me later this month for a mini-workshop on generating test ideas.
** Images courtesy unsplash.com
Thanks for sharing Paul 🙂