Fiction or reality? On a, further nice, kind of conference meeting, I did met Ebenezer Scrooge!
I didn't know that such Scrooge-type testers do exist!


Scrooge was fully focused on finding bugs, and if I say fully, I mean FULLY !
Scrooge was completely focused on his own world, collecting as much as bugs as possible, and enjoying it with a kind of scary laughter..... I must say his test results seems to be excellent, very fast in creating test cases, and he knows and uses more test techniques, I ever did. And he has, as we say it, a nose for finding bugs. But his eyes did spit fire when I suggested helping the developers to find the bugs earlier in the cycle. "Why should I destroy my own work" Scrooge replied..... I must confess, I didn't have a response right away. I know that most of the (good) testers enjoy finding bugs. But such a fanatic ego-centric type was new and a complete surprise for me. In fact, I think that Scrooge-type testers, especially this fanatic, are a disgrace for the test profession I love.

So I tried to convince him to change his attitude. Of course I didn't succeed the remaining 15 minutes we met. Probably he needs a visit from the ghosts of the past, present and future!


For the Scrooges among us
For the Scrooges in the test world I have a message: Finding bugs can deliver applause from your surrounding and it may look that your manager is pleased with the extra test hours (he can send a bigger invoice). But in the long term no one (beside you) is happy with a Scrooge attitude. The business users don't get their systems on time, the developers won't help you if you need them, and if the project manager becomes aware of this attitude, he will kick you out (so your manager can't send any invoice at all). So broaden your small ego centric world! Adopt Application Lifecycle Management [ALM] and find bugs as early as possible in a collaborative driven attitude.

For the non-Scrooges
How do we fight this irritating phenomenon? In my opinion, the best testers are the ones helping (actively) the developers to build better software. Luckily some of the big test gurus of this world preach the same opinion (see the entry measuring testers in the weblog of James Whittaker).

Wild thought
Can we make testers responsible for the quality of the software? For instance: Is it possible to award testers depending decreasing defect rates. I think a kind of Collaboration-bonus awarding mean time between failures and/or decreasing defect rates can work! Has someone a suggestion for the formulation of such a performance indicator or Collaboration-bonus?

I'll look for some like-minded friends and give it a try in the coming period. I’m sure this entry will be continued....
And again: Suggestions are very welcome!

Rob

Kuijt is a nice name. But it's quite difficult to pronounce (for non-Dutch speakers).

I've heard my name Kuijt as either "Kaut", "Kout", "Coyt", "Kite" or "Kooit"..... None of them is the correct pronunciation.... but don’t worry, I don’t care.

Nevertheless, if you ask me which one I like the most, than I should say that the (closest) phonetic pronunciation of Kuijt is: Kout
Listening to this mp3 may help...

We, professional testers, are proud that we can create good test cases from bad requirements.....That's a great achievement.......We can save projects much time by starting with the test process at an early stage in the life cycle........Or not? Who are we doing a favor?

Believe it or not, with test specification techniques like 'Data Combination Test' it is possible to create good test cases from incomplete and/or vague requirements. By combining the (bad) requirements with the knowledge of domain specialists in creative team sessions, it is possible to create great test cases, with even the choice to cover the risks with different test depths levels.

So, if the requirements team runs out of time (or don't understand their own job properly);
No problem;
Send the stuff;
We can start right away with the preparation of the test cases.

It is fun to do, it saves time and we do find many defects with this approach!

GREAT job! or NOT? ............Is deriving good test cases from bad requirements professionalism?

Let's look at the Project level
Besides us testers there are more parties trying to do their jobs on base of the requirements. For instance can the development team build the software? ....Yes, they can! Most teams are very experienced in making assumptions and interpretations, so bad requirements are not a problem.
OK, and the project managers? Can they do their job? Yes, they can...not an easy job, and sometimes a project don't make it or has some delays, but what the hack, that's how it works in the ICT!

GREAT job! or NOT?............What about the customers? Do we solve their problems?

Let's look at the Application Lifecycle Management [ALM] level
Try to answer this question: If the same bad requirements are sent to 10 different projects with the same assignment to make the needed Information System. What is the change that the projects will deliver an Information System that solves the problems of the customer?..............I think that, of course depending on the amount of communication with the customer during the projects, the changes are not very great, maybe some projects will, but most of them will need more than one release....

And that's NOT a GREAT job!
I know....we testers can't solve this problem.... but what, if we stop accepting Verbal Diarrhea as a test base....


What will happen if testers refuse to make test cases from bad requirements?

Or even....what happens if testers create, as a first step of the activity 'test case specification', for instance during the testability review, formal models like process flows or activity diagrams?
Problems like interpretations and/or assumptions rise instantly, and if these problems in the requirements can be solved before the development team start building the software, it would differ a lot!!

In this 'First Model then Test'(FMTT) approach , we can use the following set of models:

1. Process Flows and/or Activity Diagrams for sequential processes/activities,
2. CRUD matrices for database manipulations,
3. Pseudo Code for business rules and validations,
4. State Transition Diagrams for state dependent processes/activities.

Disadvantages of FMTT:

• Investment comes before profit: the testability review will take longer,
• You need to know more about models and modeling,
• It's different as we usually do,
• Performing the tests is less complex, less creative.

Advantages of 'First Model then Test' (FMTT):

• Finding Bugs As Early As Possible,
• The development and test process is less human dependant,
• Less interpretations and assumptions are needed during the building and testing,
• Less bugs to find (some will consider this as a disadvantage....),
• It's fun to collaborate with the other parties in Application Lifecycle Management [ALM],
• We testers help developers create higher quality systems.

Afraid to be bored by less complexity?
Try to make the next step: Model Based Development, Testing and/or Estimation!
I've already proved that generating test cases from models is possible (see previous articles on that subject).

Rob Kuijt

Application Lifecycle Management [ALM] should ensure that an organization experiences an improved "business as usual" in the event of the implementation of new and/or changed functionality. Other (older) industries can give assurance, so the ICT industry should follow (soon?). This article will, I hope, give some next steps towards adulthood, by giving [ALM] directions how to prevent costly or even lethal disasters caused by bugs.

Business Continuity has for [ALM] two viewpoints:

• [ALM] is part of the interdisciplinary concept, called Business Continuity Planning (BCP), used to create and validate a practiced logistical plan for how an organization will recover and restore partially or completely interrupted critical function(s) within a predetermined time after a disaster or extended disruption (see wikipedia for more information),
• [ALM] must give assurance that developing and/or changing applications won't create disasters or extended disruptions! (I won't explain what software bugs can accomplish...).

In this entry I will give some thoughts concerning the second viewpoint:
How to prevent a major disruption caused by the implementation of a new or changed application?

Of Course, the Acceptance Test is very important in our "battle" to prevent the worst to happen. But, as I stated earlier in my article "What's in the Box?", the Acceptance Test alone is not enough! Based on the Business Risks (Business Driven Test Management *) the Master Test strategy in [ALM] must contain at least the following three pillars:

1. Finding Bugs AEAP (As Early As Possibly)
2. Bug Prevention by Testing Requirements and Use Cases
3. Gathering and Analyzing data to do Business Continuity Predictions

*) Business Driven Test Management gives client grip on test process, uses client language, delivers appropriate test coverage on right spot and makes test results visible for client (see TMap.net for more information)

Pillar 1: Finding Bugs AEAP (As Early As Possibly)
Finding bugs as early as possible prevents changes in the software in the last phases of a project.
Recently I participated in an (early) evaluation of an organisation, where the strategy of finding bugs as early as possible was implemented for the complete software engineering department by choosing the Quality Levels, tuning the Test Coverage of the successive test levels and introducing Learning Cycles. They had geat results!
See two earlier articles on this topic:

• Testing in the Lifecycle [ALM]... a focus on test coverage,
• Finding Bugs AEAP (As Early As Possibly).

Problem solved? No, not yet, by diminishing the bugs, it became clear that the number of late change requests and/or wishes for extension of the functionality were disturbing the implementation of the applications. So it became obvious that the next pillar became more important...

Pillar 2: Bug Prevention by Testing Requirements and Use Cases.
If software development is based on inaccurate requirements, then despite well written code, the software will be unsatisfactory. No doubt that the users do want to change the application before it is implemented. Changing an application in the last stages of a project will generate huge risks.


Testing Requirements
Testing the requirements in the early stages of the project will minimize the changes on the application just before implementation. To give the testing of requirements a head start, I derived a checklist from the article "An Early Start to Testing: How to Test Requirements (Suzanne Robertson)"

1. Does each requirement have a quality measure that can be used to test whether any solution meets the requirement?
2. Does the specification contain a definition of the meaning of every essential subject matter term within the specification?
3. Is every reference to a defined term consistent with its definition?
4. Is the context of the requirements wide enough to cover everything we need to understand?
5. Have we asked the stakeholders about conscious, unconscious and undreamed of requirements? Can you show that a modeling effort has taken place to discover the unconscious requirements? Can you demonstrate that brainstorming or similar efforts taken place to find the undreamed of requirements?
6. Is every requirement in the specification relevant to this system?
7. Does the specification contain solutions posturing as requirements?
8. Is the stakeholder value defined for each requirement?
9. Is each requirement uniquely identifiable?
10. Is each requirement tagged to all parts of the system where it is used? For any change to requirements, can you identify all parts of the system where this change has an effect?

Testing Use Cases
After the requirements are tested they evolve in a functional model (for instance Use Cases) of the required application. To Test the Use Cases you can use the checklist I derived from the article: "Use Cases and Testing (Lee Copeland)"
1)Syntax Testing

• Complete:
  • Are all use case definition fields filled in? Do we really know what the words mean?
  • Are all of the steps required to implement the use case included?
  • Are all of the ways that things could go right identified and handled properly? Have all combinations been considered?
  • Are all of the ways that things could go wrong identified and handled properly? Have all combinations been considered?
• Correct:
  • Is the use case name the primary actor's goal expressed as an active verb phrase?
  • Is the use case described at the appropriate black box/white box level?
  • Are the preconditions mandatory? Can they be guaranteed by the system?
  • Does the failed end condition protect the interests of all the stakeholders?
  • Does the success end condition satisfy the interests of all the stakeholders?
  • Does the main success scenario run from the trigger to the delivery of the success end condition?
  • Is the sequence of action steps correct?
  • Is each step stated in the present tense with an active verb as a goal that moves the process forward?
  • Is it clear where and why alternate scenarios depart from the main scenario?
  • Are design decisions (GUI, Database, …) omitted from the use case?
  • Are the use case "generalization," "include," and "extend" relationships used to their fullest extent but used correctly?
• Consistent:
  • Can the system actually deliver the specified goals?

2) Domain Expert Testing

• Complete:
  • Are all actors identified? Can you identify a specific person who will play the role of each actor?
  • Is this everything that needs to be developed?
  • Are all external system trigger conditions handled?
  • Have all the words that suggest incompleteness ("some," "etc."…) been removed?
• Correct:
  • Is this what you really want? Is this all you really want? Is this more than you really want?
• Consistent:
  • When we build this system according to these use cases, will you be able to determine that we have succeeded?
  • Can the system described actually be built?

3) Traceability Testing

• Complete:
  • Do the use cases form a story that unfolds from highest to lowest levels?
  • Is there a context-setting, highest-level use case at the outermost design scope for each primary actor?
• Correct:
  • Are all the system's functional requirements reflected in the use cases?
  • Are all the information sources listed?
• Consistent:
  • Do the use cases define all the functionality within the scope of the system and nothing outside the scope?
  • Can we trace each use case back to its requirement(s)?
  • Can we trace each use case forward to its class, sequence, and/or state-transition diagrams?

Pillar 3: Gathering Data and Analyzing Trends to do Business Continuity Predictions
It is NOT possible to predict Business Continuity based on the testing process of the concerning project only. It is important to get a better foundation for the decision to implement a changed or new application. In the Rosario TAP we are doing some research for Gathering Data and Analyzing Trends to do Business Continuity Predictions from three viewpoints:

1. Fault Detection Trends
2. Change Control Trends (Changes in Requirements, Specifications, LOC’s,...)
3. Project Control Trends (Estimations, Budget, Overtime,...)

We’ve just started on this topic, so I will keep you posted in later entries.

Collaboration is a critical success factor in preventing a major disruption caused by the implementation of a new or changed application! All parties within [ALM] have to work together in creating good test coverage from the early until the last phases of the projects. I am sure that only when the Quality Levels, Learning Cycles and Metrics are in place, a good Business Continuity risk advice can be given to ensure that an organization experiences an improved "business as usual" in the event of the implementation of new and/or changed functionality.

Rob

Triggered by the article "Common Test Patterns and Reuse of Test Designs" on the MSDN Test Center, I realized that we are very good in reinventing wheels!
Last week I explained two different test teams the same solution for a test problem without referring to a Test Design Pattern at all...

So, let's make my life easier. There must be a giant collection of Test Patterns on the internet. If I find the right sites, all I have to do is referring to these Test Pattern Collections...
A few hours later I woke up:
OK, there are Test Patterns out there, but they are not easy to find.
My first tip: Don’t search for "Test Patterns", otherwise you may find some...


Better is to look for "Test Design Patterns". After some time I found some promising books, and, even better, I found some patterns.
I was hoping to find Test Design Patterns on a more abstract functional level, like "Search Engine Test Pattern", "Web Shop Test Pattern" or "Database Conversion Test Pattern". But nevertheless I am happy with the ones I found.

Because I didn't find a site with a kind of index mechanism on Test Design Patterns to refer to, I started one of my own. I spent my weekend on making two
Test Design Pattern indexes: Alphabetical and Grouped.

I'll keep on looking for more, and If I can find the time, I'll try to create some "How to Test"-articles on the more abstract test problems like testing search engines, web shops, database conversions and so on... May be these will grow to Test Design Patterns we all can use.

Rob