Input testing checklist

Problem: Input testing is not negative enough

One of our developers had an open book on his table. It had a catchy title for chapter 10: All Input Is Evil! The book is Writing Secure Code, Second Edition by Michael Howard  & David LeBlanc. I got rather curious and read the chapter, which was on the problems related to defensive coding versus input. It outlined two rules; “All input is evil until proven otherwise” and “data must be validated between untrusted and trusted environments”

That made me think – Why not make some kind of generic test case for input tests, that we could use repeatedly. From a test perspective, this case should be useable for manual and unit testing, not directly, but as a kind of checklist.

 

Solution: Equivalence Partitioning, boundary value analysis and positive/negative test cases for input testing.

We deal a lot in web-based applications that are driven by user input. Users are creative, some more than others, but at some point the user will violate all rules either by mistake or on purpose. Handling input like that needs to have been subject to test before our product leaves the code factory.

A generic test case should test for the following:

• Check validity – Input of logical values and illogical values, this is where positive and negative equivalence patitioning is extremely important.
• Check ability to handle all Unicode chars – Check UTS#18 Unicode Regular Expressions §1.2.1 for inspiration on the Equivalence Clases that needs to be tested. Remember country specific chars – I live in Denmark, we have letters like æøå & ÆØÅ to test for…
• Script injection – Input scripts, tags from SQL, HTML og XML.

Example using the above generic test checklist.

Consider the following input source: Namefield [2-28 char] a-z only






This field would require a minimum of 11 cases to cover the input (not including Script injection).

• Testing field length alone would take 6 cases: x<2, x=2, 2<x<28, x=28, x>28, NULL (example cases: x= 1, 2, 14, 28, 32, NULL)
• Testing alfa char [a-z] would take 5 cases: x in [a-z], x in [A-Z], x in [0-9], x in [spc. char], x=[space] (example cases: x=Eric, ERIC, 1234567890, !@#$%^&*()_+<>?,./;':"☺☻♥♦♣♠•◘○, [space])
• X cases for script injection (example cases: x=<ahref=www.google.com>, x=<br>).

That is many cases for a simple field, and most of these are easily forgotten when designing tests, that be unit-, automated- or manual test cases. I would like to encourage you to think about those negative Equivalence classes and boundary values, as they are 10 times more likely to hide a bug than the test cases that probe the positive side of things, simply because IT professionals forget that “All input is evil!”

 
Have a nice day & Happy testing!

/Nicolai