I was at TestBash a couple of weeks ago, and in the Lean Coffee meet-up one of the other testers at our table was telling us all about how they’re having to put huge amounts of resource into regular Regression Testing. I asked them whether they were finding enough issues in Regression Testing to justify it, and they responded “Lots. Every time”.

Which got me thinking – doesn’t that just mean that part of their process is really broken? Regression Testing is probably the least fun part of testing. The least interesting. The most stress related. The most labour intensive work for the lowest possible reward. It’s how we operated 15 years ago, when I first started testing. So why does it seem to be relatively unchanged in all that time, when so much else that we do has matured and grown?

Let’s think about it:

So there you are, you have a working Live Environment, and you have new work (features / stories / bug fixes) that you want to add to it. You have some kind of ‘new’ shiny process that you’re using to get your new features Live (Agile / Scrum / Kanban / Lean / or some variant thereof…). You’re doing all of the following:

Step 1: You’re agreeing the Acceptance Criteria before a single line of code is written.

Step 2: You’re telling the Developers exactly what you’re going to test.

Step 3: You’re deciding as a team at what level all the tests are going to be automated.

Step 4: You’re sitting with the Developers who are showing you all their automated tests. You’re suggesting more, which they are adding.

Step 5: You’re testing (often!) on the Dev machine.

Step 6: All of the existing automated tests are being run – and are, of course, all passing!

Step 7: You’re deploying to your Test Environment and checking all the basic functionality works (so no environment / deploy issues).

Step 8: You’re running Exploratory Testing sessions on your new functionality.

Step 9: You’re having your testing reviewed, and having new charters suggested.

Step 10: You’re adding additional automated tests, should you have thought of them.

So, you’re doing all of that.

And then you’re finding issues during Regression Testing?!!!!

Maybe it’s about time you stopped fixing the issues you find, and start fixing the reasons that they’re getting there?

Look at each issue that you raise – and identify how it didn’t get picked up earlier.

  • Do you not have the right automated tests (too few, the wrong level, just plain bad!)?
  • Are you not testing early enough, or not being a part of the development process until it’s too late?
  • Are your test environments out of kilter?
  • Are you not adequately using methods of protecting unready code (like feature toggles, etc)?

Spend the time closing that hole in your process – and soon you’ll be able to reduce the time that you spend Regression Testing – and get back to the good stuff…