Checklists for creating test cases

Problem: Test coverage is dictated by daily mental fitness and experience of the tester.

My experience tells me that the daily mental fitness of the tester often directly affect the test coverage. Another thing that will affect test coverage is the experience of the tester. That means worst thing that could happen to test coverage, is if the junior tester goes dancing all night before preparing the test for our most business critical functionality…?!

Solution: Get sources of inspiration to aid the tester in the right direction – Checklists can be a good start!

Well real life happens. But there is no need to worry! With a little inspiration in the form of a checklist you might just keep the show on the road even if the tester is inexperienced, tired, bored, stressed and/or annoyed.

It was a discussion with one of our developers that led me to look for a source of test case inspiration. His experience with test was limited and he needed a source of inspiration for doing unit testing. I came across Software Testing Help, that by the way is an excellent place to look for test inspiration. At STH I found this article called “180+ Sample Test Cases for Testing Web and Desktop Applications – Comprehensive Testing Checklist”

Here is what we use a test case checklist for:

• Unit testing – Developers can seek inspiration for their unit test cases.
• Manual test cases – Testers use this for headlines when doing test specifications.
• Exploratory test sessions – Most of the scenarios can be copy-pasted directly into a test charter.

So what are the pros:

• Lots of inspiration close at hand
• Better test coverage

What are the cons:

There is next to nothing related to business domain in the checklist, making is useless for business driven test cases. This means that this really does not fix lack of business related test cases, but none the less it is a good start.

 

 

Have a nice day & Happy testing!

/Nicolai

Microsoft Certification 70-497 Software Testing with Visual Studio 2012

Problem: Passing Microsoft Certification 70-497 Software Testing with Visual Studio 2012

There is many opinions about test certifications, and I am not going into pros and cons here. I have a few certifications myself, and common for those I have is that they helped me formalize my knowledge – Something that I can only recommend, no matter how you do it.

I have been working with Microsoft Test Manager (MTM) & Visual Studio for the past year, so now it is time to get the certification and formalize my knowledge about the tools in the MS toolbox.

 
Solution: Study for the exam and use real work as excises to anchor learning.  

Now that sounds easy enough, but especially the study part might require some explaining, as Microsoft Learning really does not offer much help on the preparation materials for 70-497 – Luckilly, googling seems to be the answer to everything, as I came across this great resource at Channel9.

This is how I prepared myself for the exam:

• I watched the Channel9 videos – Approximately 6 hours of video with great educational value, not to mention fun as the presenters are really great!
• I Browsed Microsoft Developer Network for instructions on how to use MTM & Visual Studio.
• I download the book Testing for Continuous Delivery with Visual Studio 2012 – Good source of information for putting theory into practice.
• I practiced all the topics in MTM, using the book as reference and the Channel9 video demos for inspiration. For exercising, I used one of our current projects as case and started creating test plans, settings and cases as detailed in the above materials. This took me all the way around the topics.

Tips that I found useful:

• Practice, practice, practice – In my book seeing is believing; so when you go do the exercises in MTM you will remember how it works at the exam.
• Know the relationship between plans, cases, requirements, configurations etc. This is central, so is the test lifecycle, and activities that must be executed in Test Manager and Visual Studio.

What I learned from completing the certification:

• MTM has several features that can accelerate manual testing – The Fast Forward feature is one that was new to me.
• There is a test suite type for everything – Previously I always just used Static or Requirement Suites, but it seems that Query based suites will move into my plans in the future.
• All the little details I missed when just learning on the job – I know those now, and will use them in the future when testing

 

Have a nice day & Happy testing!
/Nicolai

Input testing checklist

Problem: Input testing is not negative enough

One of our developers had an open book on his table. It had a catchy title for chapter 10: All Input Is Evil! The book is Writing Secure Code, Second Edition by Michael Howard  & David LeBlanc. I got rather curious and read the chapter, which was on the problems related to defensive coding versus input. It outlined two rules; “All input is evil until proven otherwise” and “data must be validated between untrusted and trusted environments”

That made me think – Why not make some kind of generic test case for input tests, that we could use repeatedly. From a test perspective, this case should be useable for manual and unit testing, not directly, but as a kind of checklist.

 

Solution: Equivalence Partitioning, boundary value analysis and positive/negative test cases for input testing.

We deal a lot in web-based applications that are driven by user input. Users are creative, some more than others, but at some point the user will violate all rules either by mistake or on purpose. Handling input like that needs to have been subject to test before our product leaves the code factory.

A generic test case should test for the following:

• Check validity – Input of logical values and illogical values, this is where positive and negative equivalence patitioning is extremely important.
• Check ability to handle all Unicode chars – Check UTS#18 Unicode Regular Expressions §1.2.1 for inspiration on the Equivalence Clases that needs to be tested. Remember country specific chars – I live in Denmark, we have letters like æøå & ÆØÅ to test for…
• Script injection – Input scripts, tags from SQL, HTML og XML.

Example using the above generic test checklist.

Consider the following input source: Namefield [2-28 char] a-z only






This field would require a minimum of 11 cases to cover the input (not including Script injection).

• Testing field length alone would take 6 cases: x<2, x=2, 2<x<28, x=28, x>28, NULL (example cases: x= 1, 2, 14, 28, 32, NULL)
• Testing alfa char [a-z] would take 5 cases: x in [a-z], x in [A-Z], x in [0-9], x in [spc. char], x=[space] (example cases: x=Eric, ERIC, 1234567890, !@#$%^&*()_+<>?,./;':"☺☻♥♦♣♠•◘○, [space])
• X cases for script injection (example cases: x=<ahref=www.google.com>, x=<br>).

That is many cases for a simple field, and most of these are easily forgotten when designing tests, that be unit-, automated- or manual test cases. I would like to encourage you to think about those negative Equivalence classes and boundary values, as they are 10 times more likely to hide a bug than the test cases that probe the positive side of things, simply because IT professionals forget that “All input is evil!”

 
Have a nice day & Happy testing!

/Nicolai