Obsessive testers?

We had quite a laugh here at the office the other day when looking at the pictures in this Blog post:

http://dunzo.net/30-infuriating-images-that-will-trigger-your-ocd

Obsessive or not, any testers looking at the pictures will recognize the feeling they get when testing and hunting for bugs. When the bug is close you get the feeling of “something-is-not-right-here!”

All of the pictures includes an element, which makes you wonder if this is right? Whenever I start asking myself that question while testing, then the hunt is on, because the bugs will be hiding here. The pictures are indeed situations similar to what the tester will encounter in a piece of software – in either the documentation, GUI, database or code.

I often start my test sessions with some exploratory testing. The reason for doing this to learn about the item under test, and to allow myself wonder is things are right. I would like to invite you to try this some time it is fun and rewarding.

Happy testing!

/Nicolai

Test Plan Reality Check

Problem: The test plan needs constant care in order for it to remain valid.

Plans are created as part of the startup of a project, and then often left alone during long periods of the project. As reality kicks in, the plan will become more and more invalid, leaving the reader alone in a world of false assumptions and inaccurate statements. “Plan the work, work the plan” is not an advisable strategy!

Solution: Put some reality checks into your plan and update the plan, as you wise up.

I came across this picture on LinkedIn, and realized that it serves as an excellent reminder that the test plan needs updating, just as any other plan. Looking at ‘Your plan’ vs. ‘Reality’ it becomes obvious that a reality check is needed every now and then.

Whenever reality bites, it will change the foundation of the plan either confirming some of your assumptions or invalidating them. It will be dependencies to other plans or deliveries that will rock the boat, suggesting that you will need to question your plan at least every time you (are supposed to) receive something from others. I have seen these points mentioned as quality gates in some organizations – No matter what you call the deliveries, you will require entry criterias (or similar) to communicate your expectations to the delivery organization and to check if the delivery meets the requirements.

When you find that the delivery does not match your expectation then update the plan, communicate risk/issues to stakeholders and take corrective action to ensure that the plan reflects what you know and where that will take you and the test project.

 

Remembering the project triangle (aka. Iron Triangle) I have some pointers where to look for changes in your plan:

 

Scope changes, check following areas of the plan:

• Test items
• Features to be tested
• Features not to be tested

 

Cost changes, check following areas of the plan:

• Item pass/fail criteria
• Suspension criteria and resumption requirements
• Test deliverables

 

Schedule changes, check following areas of the plan:

• Approach
• Responsibilities
• Staffing and training needs
• Schedule

I strongly advise that the plan is kept simple (see “KISS your test plan, but not goodbye”), and one of the reasons for this is to keep the administration workload at a minimum. In complex projects with loads of dependencies, just keeping a plan up to date can be quite a task.

Happy planning and testing!

/Nicolai

Hunting bugs in the jungle.

Problem: Some bugs are like ghosts – They are only seen rarely, but they can be quite a scare.

Some bugs are illusive, and (almost) impossible to recreate, but it does not mean that they are not there. It is most likely that these bugs are considered to be ‘not reproducible’, but they are lurking under the surface, just waiting to ruin your day - or the release you have been hacking on for the past weeks.



Solution: Team up, and gather all the information you can get – Then eliminate the possibilities.

We have recently been puzzled by a bug, one of those that require a lot of investigation to reproduce and document. The root cause was really simple, but pinpointing the origin and reproducing it was hard.

The problem was caused by faulty population of dropdown that dictated what data the user was working on in the system. The selection made by the user was supposed to be persisted in the session and saved in the database, to ensure that the selection stayed even if the user quit and reentered the application.

While executing the test we discovered that the selection changed – Apparently for no reason. It happened a couple of times and we started discussing the observation, and agreed that this was indeed a bug. It was a bug that we could not recreate, nor could we point at a single plausible root cause.

We did two things in order to start the hunt for the bug – First we recorded everything we knew about it in our bug-tracking tool, then we teamed up in order to make a bug-hunting crew. We asked the developer who wrote the code to assist with two things; First thing was to participate in the discussion on root cause theories, and the other was to enable all logging, and stand by for the next sighting of the bug.

We used the theories to guide the testing, as we performed the scenarios believed to lead to the reproduction of the bug. One by one, the theories were dismissed as root cause, until we had the bug cornered.

When we finally encountered the problem again we had lots of logs to look for and less possible root causes to check – That made it much easier to find the bugger and recreate it based on the information we had available.

It turned out that the session handling was not correctly set, making the population of the dropdown to take place before the value was fetched from the database. It only happened on the rare occasion when user was transferred from one instance in the cloud to another. This was the reason for the illusive nature of the bug.

Conclusion: Use root cause guessing/analysis as a guide and your peers to help you when hunting those illusive bugs.

Happy bug hunting!
/Nicolai