Agile Testing: Test case management
July 12th, 2011
In a previous post on Agile testing, I looked at how Microsoft Test Manager and Twist from ThoughtWorks Studios allow QA and Devs to collaborate on creation of functional tests. In this post, I will examine how well each tool allows you manage your test cases.
Test Management
We all want large, comprehensive test suites because they allow us to change code safely. Without them, how would you know if that function you wrote two years ago still works?
Yet a large test suite is hard to maintain. Much like a large code base, it can easily get out of hand if not done carefully. And, much like code, it’s important to avoid duplication and promote reuse.
Consider the canonical example of login. Each new test case you write probably needs to log into the app first. Ideally, the steps required to log in should be made reusable across all test cases. Otherwise, you’re not only wasting your time writing the same steps over and over, but it’ll take forever to change in case login functionality changes.
Promoting Reuse
Not surprisingly, your test management tool has an important role to play in promoting reuse. It should allow you to structure tests in a way that lends itself well to reuse and it should have good discoverability. In other words, the tool should make it easy to find what’s already been created.
So, how do Test Manager and Twist compare when it comes to managing tests? Unfortunately, only one of them actually tries to promote reuse (Spoiler alert: it’s not Test Manager).
Sadly, this didn’t have to be the case because Test Manager does support the concept of reuse through shared steps:
The problem is in the way shared steps are implemented. To create a new test step, the user is presented with two choices: “Insert Step” and “Insert Shared Steps“.
This is bad for two reasons. First, the application seems to expect the user to know upfront what steps already exist as shared steps. Otherwise, why would would they choose the “Insert Shared Step” option?
Second, through this presentation, Test Manager creates a false equivalence between shared and non-shared steps. Instead of making it easy to reuse shared steps and hard to create new ones, it basically says “if you want to use a shared step, great. If you don’t, that’s fine too.” What this leads to in practice is very limited use of shared steps and lots of duplication.
Contrast this with Twist. When the user types in a new test step, Twist automatically suggests previously created steps which may be relevant:
Usually, the functions that stop working are the non-essential-for-survival functions that are just like the immune system leading to loss viagra no prescription deeprootsmag.org of the stomach cells. The physician will be checking for hardness of the prostate or for irregular shapes or bumps extending from the prostate – all cipla cialis india of which may indicate a problem with nerve function or blood supply problems. This results in an erection after a long process that occurs in the presence of this high quality cialis vs viagra ingredient ensures the best result in the continuous attempts to quit the habit by bringing a behavioral change and by adopting the nicotine alternatives. Although men rarely consider home and herbal remedies, they can order viagra viagra help boost their sexual performance by increasing the blood supply to the penile region.
In other words, the system actively forces the user to do the right thing by reusing a previously defined step.
Test Automation
Another topic relevant to test management is automation. Specifically, it matters how easy it is to identify manual steps and to automate them.
In Test Manager, there is a separate tab called “Associated Automation” which lets you associate a Coded UI Test to the entire test case:
Note that the Coded UI Tests must already exist and cannot be created directly within the tool. So, the automation has to either be done by someone else or the user needs to go to a different tool (Visual Studio), automate the test, and come back to Test Manager to link everything up.
On the other hand, Twist shows you any step that is not yet implemented in brown:
It takes a click to generate a Java fixture in which you can automate the step. That’s it.
Bottom Line
Unlike Twist, Test Manager is just not designed to promote test case reuse and automation. Whatever functionality it does have is too easy to ignore and/or too cumbersome to use.
You may also like:
Did you love / hate / were unmoved by this post?
Then show your support / disgust / indifference by following me on
Twitter!