Model Based Testing is a powerful concept for testing information systems, from which, despite the increasing complexity, more quality and flexibility is expected. It is a methodology that fits well with companies with experience in structured design and testing. Model Based Testing will also adobe.lovers.com support and lower the threshold for companies who want to make a first step towards improving their test basis towards a better, faster and cheaper test process.

The core of Model Based Testing is the use of tools to first, create the actual design and secondly to compare the simulation of the process and predicted outcome of the test generation with the requirements at an earliest possible stage. Thus, the various "paths" through the procedures / processes including related business rules are tested and checked much earlier.
Subsequently, automatic test generation creates efficient and reliable high quality test cases which excel in compactness and low cost.


Model Based Testing helps to improve the overall development process: the impact of demands and wishes becomes clear in a fast manner, design errors are found at an early stage, debugging code caused by wrong assumptions and interpretations decreases and reuse of designs in new projects is easier.
In short, developers achieve greater results in less time and at lower cost.

The tools needed to support Model Based Testing are available and are developing rapidly based on the experience of cutting edge companies like Sogeti. It becomes increasingly easy for companies to make the choice and take advantage of Model Based Testing.

Rob Kuijt

Test Automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, test design, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. (Wikipedia)

Working with Model-Based Testing I get many questions on “Test Automation”. To be honest, around a year ago, I could not answer them at all. After a year playing with “test industrialization” my knowledge is better. But still, especially the timeline (“when was a specific type of test tooling introduced”) was a gap in my mind. An overview: “History of Test Automation” would be very practical.

On internet, I couldn’t find the complete Test Automation timeline I needed, so I started creating one…

Inspired by the powerpoint slides: “Generations of Test Automation” by dr Mark Utting, I made a slide show (kind of e-learning component) which gives an overview how the (automating of the) test process did evolve since the 1940's.
I created some nice graphics, added some definitions and expanded the presentation with more detailed slides and background information from TMap®, Wikipedia, James Bach, William E. Lewis, Gunasekaran Veerapillai and Marvin Niven (thanks guys!) and uploaded the slide show/e-learning component for future use…

Interested? See “History of Test Automation”

There are several tools that can help with the creation of test cases. So why don't we succeed in automating the Test Cases Design Process? In this short article I will explain that, if test collaborates with design, we can make huge progress on this topic!

4 Examples of Test Case Design
Let’s imagine a business process that is documented by the design team on one page. Test will create the test cases. In the first example the description is done in plain text. In that case it is not possible to automate the test, and also it is not possible to use a formal test specification technique:

Example 1: Test Design from plain text: many interpretations and assumptions...

Creating test cases from plain text may look easy, but when you ask 5 test engineers to create the test cases you get 5 different sets without any insight in the quality of the coverage.
Choosing the process flow diagram technique for describing the business process, will result in less interpretations and assumptions, therefore in much better test cases:

Example 2: Manual Test Design: Process Cycle Test (TMap®)

The advantage of formal Test Case Design techniques like Process Cycle Test (PCT) is that it can be automated! How? The first step of deriving test cases with PCT, is to identify the paths and path combinations within the process flow diagram. Step 2: Instead of manually combining those path combinations to test cases, the path combinations (joined with a short description) can be inserted in a Test Design Tool. Step 3: The test cases are generated. Comparing with manual test case design: much faster; less knowledge is needed, but you need a tool* (license):

Example 3: Using a Test Design Tool: much Faster!

Most of the time, designers make these process flow diagrams in modeling tools (like MsVisio, Protos, Aris, BiZZdesigner,..). A common feature of the modeling tools is exporting the models in XML-format. Model Based Testing tools* do read XML! So, let’s skip all manual steps and generate test cases directly from the process flow diagram:

Example 4: Model Based Test Case Design

Model Based Test Tools* can generate test cases within minutes. Of course, before using the test cases you must do a sanity check to confirm the tool understood the model correctly but nevertheless the time can’t be beaten manually.

Collaborate!
Test Case Design can be automated (I’m already working with these tooling on a daily basis).
But, besides getting the right tool(s), there is an important condition: the Test Base must have a minimum level of quality.
For instance: instead of describing business processes in plain text, they should be specified with the help of activity diagrams or process flow diagrams (and that is not (yet) common knowledge within the design processes). To get an automated Test Design process, let’s join the forces: Together, Design and Test can make projects much faster and cheaper!

*) Example of a Test Design/Model Based Testing Tool I personally often use: STaaS/COVER (Sogeti Netherlands)

Both ALM (Application Lifecycle Management) and MBT (Model Based Testing) are hot. I'm pretty busy running from customer to customer starting/supporting MBT proof of concepts and assignments. With some explanation all my contacts are convinced that the combination of ALM and MBT is THE roadmap for the future.

The advantages of ALM can already be implemented by choosing for the new generation of tools of Microsoft (VSTS2010) and IBM (JAZZ etc.) are presenting to strongly support the collaboration of the different roles in and around the ICT. Model Based Testing has not yet that same level of maturity. Most participants in the Model Based Testing world are very ambitious: They try to Generate Test Cases for direct automatic execution, based on test specific models.

I think this is a bridge to far for most companies using for instance mainly ERP-software. In the ERP-scope most of the testing is still done manually, and jumping to this ultimate form of Model Based Testing (Generate Test Cases for direct automatic execution) will be very tricky and/or costly. That's why we (Sogeti) are introducing also a basic form of Model Based Testing (STaaS/COVER), using the same models as the developers do.


"MBT: ...a bridge to far..."

MBT: Generate Test Cases for direct automatic execution
Does this mean that Generating automated tests is a dead end? NO! It is a very good approach for testing high risk functionalities, which are to risky or to complex to test manually.
Making test specific models is a costly business and top designers/test engineers are needed to make it work. So be sure to pinpoint these heavy coverage and tooling only on the high risk areas. AND!! Do not try to make the test specific models in the Testing Silo! Look for collaboration with the design department to get these test specific models in the same configuration mnagement as the models the devlopers use to build the software. Preferably in an "ALM Center Configuration Management".

MBT-basic: Using the same models as the developers do
On the other hand, when the ambition of Model Based Testing is lowered to only generating the test cases, it is a complete different story. Especially the generation of the (more global) logical test cases can be done from the same (functional and/or requirement) models the developers use as a bases for creating the software. For instance process flows (activity diagrams), decision tables and pseudo code are a perfect basis for generating (logical) test cases.
Enrichment of these models (in collaboration with the design department) will bring us to the next step: Generating the physical test cases. In that case the Test Cases are no longer Configuration Items (costly maintenance!) but Test Cases become Work Items (if desired: directly generated from the same models the developers use to build the software).

Rob Kuijt

For several years now, Model-Based Testing is mainly used within the (technical) development processes. In the Black Box testing community it was still "just a theory", and in my case: fun to play with. Today is different! I'm completely into Model-Based Testing nowadays. MBT turned, for Black Box testers, into serious business!

Wikipedia:
Model-Based Testing is software testing in which test cases are derived in whole or in part from a model that describes some (usually functional) aspects of the system under test (SUT).

For quite some years I'm working with a tool called COVER. Initially COVER was build to help lazy testers, like me, to avoid boring manual activities…and it worked! Using this tool I could easily amaze my surrounding with stunning Speed and Quality. Especially deriving test cases from formal specifications like pseudo code or activity diagrams was fun to do; instead of "solving a boring puzzle" time, it was done in seconds (as a matter of speaking). However, most of the testers in my surrounding didn't recognize the ease of this way of working (or were not lazy enough?) so COVER didn't have much users.

Today is different. The attention for Model-Based Testing is growing rapidly. Previously, when a project had to make test cases, it was accepted that it was done manually and, everybody knew, it should take quite some time.
Today the budgets are lowered...What to do?...Less productivity?
My suggestion: When you've less money to spent, be creative to get the same results!

First Model then Test!

Model-Based Testing is a nice example of creative thinking!
Within a few months the relatively unknown possibilities of Model-Based Testing tools became very popular for Black Box testers within Sogeti. Especially the collaboration-part (designers working together with testers) of Model-Based testing is an eye opener. Everybody knows that it is smart to find defects as early as possible, but it is very difficult to motivate teams to perform good design inspections. Now MBT tools like COVER can do a big part of those intensive inspections, because using the models for generating test cases will directly show the defects in those models in a very early phase of the project! And beside that, it generates a big part of the test cases!

So, when the Designers create models that COVER understands, projects will perform Better, Faster and Cheaper! ....and it works!
Some (early) successes from our proof-of-concepts and implementations:

• The amount of time needed for creating test cases dropped 50% (and sometimes more);
• The maintenance of the (regression) test cases became in average 70-80% cheaper;
• And maybe most important: We have proved to find defects much earlier than the project did manually!

Collaboration will change the world of testers!
Using MBT is in fact collaborating within the Application Lifecycle [ALM] supported by standardization and automation. Many defects, found in functional testing, have their origin in misinterpretations and assumptions of the requirements and/or specifications. The early usage (generating test cases) of formal models, just after or during the design phase, will create a much more solid base for software coding. For the testers this will mean less and less defects, so besides the generated test cases, the improved quality of the software will also fasten the test process.
(PS. When both software and test cases are generated from the same models, be sure that the test objective is clear!)

This industrialization can’t be stopped anymore. The time of creating most test cases by hand has had its peak. Of course, not all test cases can be generated, and also manual testing itself will never disappear, but handcrafted test cases will be less and less common practice. The work of Black Box testers will, in my opinion, shift in two directions: 1) Earlier in the life cycle: joined modeling and helping and supporting development to find defects as early as possible (Master Test Plan consultancy), or 2) the coming market End-to-End testing: the importance of complete chains of applications is growing rapidly. End-to-End Testing will become a separate specialism.

Examples
Back to Model-Based Testing: To give an idea what type of models can be used, and how test cases are generated, I've made a some slides (with examples) about the Sogeti MBT tool: COVER. Look for yourself if MBT will affect your work!

It did affect mine!

Rob Kuijt