QA Engineer walks into a bar...

I came across this one a while back - A fun reminder on equivalence partitioning and test cases you can get from different equivalence classes.

Happy testing !

/Nicolai

Management Potential

I came across an interesting view on the (test) management discipline, during a course I attended this week. The discussion was on what it takes to be a manager, and this was quantified in the following formula:

Management Potential = P * E * L * M

P: Personality

E: Experience

L: Leadership

M: Management

The points that I find interesting about this way of perceiving the manager’s potential is:

Four key elements define the cornerstones of skills that are needed. They are all essential, so having no skill / ability to operate equals a management potential of zero. Good managers need not be masters of all, but can compensate for lack in one area by being skilled in another.

So what is the reasoning behind this discussion? Think about your own management potential, maybe you want to cultivate one area that you are weak in, or maybe you would like to strengthen a skillset to compensate for another? In my opinion, especially inexperienced managers can benefit from putting some thought into how they compensate for lack of experience (while building it).

Food for thought – Have a nice weekend & Happy testing!

/Nicolai

Guiding your estimates in the right direction

Problem: Estimating effort for (test-)activities is inaccurate

Estimation, or educated guessing is the foundation for many an activity related to management of a project. Given that estimation is not an exact science there will be inaccuracy in the numbers used for planning and resource allocation.

 

Solution: Apply general estimation rules and use a technique to guide your estimates.

In order to improve accuracy in your estimation you can rely on some general estimation rules and use a technique to guide your estimation.

General estimation rules:

·         A full-time resource will only be productive around 80% of the time.

·         Shared resources, working on multiple projects will have a lower productivity given that they will have to spend time when they do context switching.

·         There is a high degree of optimism in estimates, as most people have tendency of underestimating complexity and time consumption.

·         Base you estimates on multiple sources – Don’t just use own experience, pursue experience of others if possible.

·         Estimation of a task should be done by the person/team responsibility for doing the task.

·         Remember to include issue management, meetings and other supportive activities in the estimate.

·         Break estimates down if possible.

·         Make sure that assumptions, exceptions and limitations are clearly documented as part of the estimate

Techniques for estimating

·         Top-down estimation

·         Bottom-up estimation

·         Combined top-down & Bottom-up estimation

·         Comparative estimation

·         Parameter estimation

·         One-point estimation

·         3-point estimation

·         …

The point is that there are numerous techniques that allow you to do more or less scientific estimation – You need to choose the one that fits the purpose and tolerance of your project.

Happy testing!

/Nicolai

Supporting your principles of testing

Problem: Your principles of testing cannot stand alone.

Having spend the time defining the principles on which you want you test to run is not enough – It is definitely a good start, but the principles needs to be anchored in the project through activities and action that turns the principle into practice.

Solution: Walk the talk by supporting the principles with action.

One thing is to have a defined test principles, another thing is to support these in practice in the projects. I am going to have a look at some of the supportive actions related to the 7 Principles of Testing from the ISTQB syllabus, as these are commonly accepted.

1) Testing shows presence of defects, but cannot prove that there are no defects. This means that testing needs to be very structured to ensure that testing probes the right areas. Risk based testing can be an approach to testing the right areas, but no matter how you go about this there is two things I would recommend you to inject in the project to address this principle: A strategy that guides your test planning and priorities and expectation management that ensures that the  steering group, project mangers, external stakeholders etc are aware that test will not prove that there are no errors.  

2) Exhaustive testing is impossible and testing everything including all combinations of inputs and preconditions is not possible. Applying different test techniques such as boundary-value analysis, all-pairs testing and state transition tables aim to find the areas most likely to be defective. Use equivalence partitioning as a driver for designing the test to a level of detail that makes sense for the project.

3) Early testing. Remember the principles of verification and verification?  Apply them early and make a plan for when to do it – Some want the V-model to drive early test, others have different approaches, the one that is successful is the one that ensures that testing happens in a timely manner (read: as early as possible). Another thing that facilitates early test is to have a champion in the project from the start – Someone who advocates good quality already from the project planning and onwards.

4) Defect clustering. Does your risk driven approach consider technical complexity / risk? If not, then you might want to consider this in conjunction with some coverage analyses. Another approach is to do experience driven explorative testing, with the purpose of finding candidates for a thorough test – It is a shortcut to finding those troubled areas.

5) Pesticide paradox. Change your approach every now and then – Switch responsibility for functional areas in the test team. Invite new people in, try explorative test, arrange bug hunts, have a bug-off of testers vs. development or business vs. it-team. Point is that you need to refresh yourself now and then, and you might as well put in your plans and have some fun while doing so.

6) Testing is context depending. Base your strategy on the nature of the system and the error tolerance of the receiver. Don’t juse run all test cases just because you can – Run those that are related to the functions delivered.

7) Absence – of – errors fallacy. Invite those users in early, sooner than later, to gather feedback that ensures that you are moving towards ‘fit for purpose’. Involving users requires careful planning, and since you want early involvement this is something that you need to address ASAP in any project.

Again, my point is that you need to have actions that supports the principles, without this, you principles are worthless. Another point is that you need to drive all of the above aided by common sense. Do not invent a wheel unless you have a need for driving somewhere.

Happy testing!

/Nicolai

Risks against test start

Problem: Test is delayed already before it has started

A test may be delayed due to different reasons, causing the start of test to be delayed. Test not starting on planned time is due to events that violates the start criteria for the test activity.

Solution: Monitor and act on risks against your test start criteria.

In order to monitor the risks you need to know what they are – Most common risks against starting a test is related to delays in activities that are on critical path for your test. I have listed the most common problems I see:

• Missing test component or system.
• High error levels in test component or system.
• Testware not complete.
• Missing or wrong testdata.
• Too many (partial) deliveries for test

There are two ways of approaching these risks – reactive and proactive.

Reactive approach is easy, but introduces waste in your development life cycle:

1. Wait until test is delayed by one of the above.
2. Look for rootcause for dealy. 
3. Eliminate rootcause. 
4. Start test at first possible time. 

Proactive is also easy, but requires a higher degrees of advance planning than the reactive approach:

1. Do a risk identification workshop, listing things that will derail your test.
2. List actions that minimize likelihood and impact of identified risks.
3. Adopt actions in your test plans, and follow-up on these.
4. Do a test readiness review in due time before test starts.

To help you on your way you might want to consider the following actions to mitigate the mentioned risks:

• Missing test component or system.
• Likelihood is reduced through continuous integration, automation of deployment / build process, release management of test deliveries. A high degree of automation is preferred in order to ensure timeliness and correctness.
• High error levels in test component or system.
• Test as part of development and cooperation between developers and testers. Focus should be an early test, and on test of critical items that is a must-have in the following test phases. Furthermore a bug triage session on regular basis might help utilize resources on important (from a test perspective) bug-fixing, rather than random bugfixing.
• Testware not complete.
• Test planning is paramount. The plan needs to include all activities and the delivery these leads up to. Follow up on critical path, and make sure that you update the plan as you go along.
• Missing or wrong testdata.
• Just as there is a need for a test strategy you will need a test data management strategy. 
• Too many (partial) deliveries for test
• Release management will get you far – Know what is in the box, and bundle test to match the partial deliveries. Prioritize test cases based on the release content, to ensure that you do not run cases prematurely.

Happy testing!
/Nicolai

Gold plating = Cost injection

Problem: Gold plating of your product is expensive.

Ever experienced that features are added in your product without any adding any real value? If yes then you have probably been gold plating that delivery of yours. Gold plating is done with the best of intentions and most of the time is appreciated by the customer. However, there are many cases where it is not liked and the gold plating is backfires on your product.

Solution: Stick to implementing approved features.

Usually gold plating is introduced either by the project team or by a project manager, at no cost to the customer. Gold plating does not come cheap however! It can increase operation and maintenance costs significantly and will make a dent in quality. The reason is that it raises complexity, introduces features that are not traceable and operates on the outside of the feature approval process – In other words it is a risk that needs to be mitigated.

Although, gold plating sounds good to everyone, it is bad for the project team in the long run. Gold plating increases the development cost, raises the expectation of the customer by inflating the feature per hour / story-point of development. If you do another project for the same customer, he would again expect you to deliver a product with extra features.

Avoiding Gold Plating is important to stay in control of the product, and ensure that the development effort is spend on value-adding features. If you operate in an agile setup you should see gold plating as a violation of the product backlog setup and priorities. Avoiding gold plating requires discipline. It requires the team to ask themselves if they are adding unnecessary or over-engineered functionality into the application.

Start out by discussing gold plating as part of your retrospective, and the set some ground rules like:

• Never allow add any extra function or features to a story without approval.
• As a product managers and sales people should use the product log to add new features – not the sprint log.
• You must establish proper communication lines within the project team, ensuring that approval from customer is achieved.

Then do a little measurement on number of features per iteration vs. number of features in the accepted sprint log. You might have gold plated the solution, but make sure that the result can still be tested, and ultimately accepted by client, otherwise you just might end up paying a very high price for the golden plates.

Happy Testing!

/Nicolai

Your very best friend - The Service Delivery Manager

Working with test also means networking. Since most test management is about getting the product tested and shipped according to deadline a natural part of this networking is with the development organisation. Project managers, developers, testers, business partners and other SME's. They provide essential knowledge and resources to the test effort getting things prepared and executed.

This means we have a tendency to forget our most important body of knowledge - the "service organisation". They tend to take over service of the product at go live and then we forget them. Let's correct this mistake in the future.


First and foremost the service delivery manager should be included in the handover of the product prior to go-live. Essential knowledge from the test team must be embedded in the service organisation. Not as a long trivial list of open defects but as story about product quality, workarounds and other necessary information.



Secondly the service delivery manager sits on a treasure trove of knowledge about real user problems and user behaviour as well as technical problems faced by operations. This can be used by the test organisation for a number of tasks including estimation, prioritisation of test effort, general planning of test and as a source for planning test of "off-normal" situations.

The service delivery manager is as such an essential member of the test team. Not on a permament basis but ad hoc at the right time during the project the Service Delivery Manager is your best friend.
If the service delivery manager is not able to provide the input, he is at least the gateway to the organisation where the knowledge is present and available. Just kick in the door.