Geeks With Blogs

News View Michael Stephenson's profile on BizTalk Blog Doc View Michael Stephenson's profile on LinkedIn
Michael Stephenson keeping your feet on premise while your heads in the cloud

I have decided to do a series of blog posts around the approach to testing I normally try to encourage on BizTalk projects where I am involved. 

My aim is to do some posts based around the following:

1. Development Testing

This will cover the following:

- Techniques to unit test various artifacts within a BizTalk solution including Schema, Maps, Custom Components (adapters, pipeline components)

- Functional testing with BizUnit

- Approaches to stub out external systems

2. Integration Testing

This will contain details of our approach to integrating our BizTalk solution with other external systems.

The above are the initial focus of the series.  In the future i intend to come back to this and discuss areas such as performance testing.

Testing Approach

My view on our approach to testing is based around the following pinciples:

- Catch defects as early as possible

Effective developer testing will catch bugs when they are easier and cheaper to fix.  If a solution is deployed and is being tested by a large test team, a high priority defect could potentially cost a lot to fix in terms of lost testing time.

- Be pragmatic

While we would normally aim for 100% code coverage, sometimes it is not practical to do this.  We must manage the balance between the risk of not quite covering all possible scenarios in developer testing with the cost of creating and maintaining the tests.  I will discuss this at various points during our tests.

- Tests should be repeatable and ran regularly

We want all of our tests to be easily repeatable and ran on a regular basis.  We want to automate as much as possible

- Learn from the .net world

A lot of the things we do in our approach to testing for BizTalk is taken from experiences of testing in .net projects.  Where possible we have used or adapted approaches and tools which are well proven on .net projects to help us deliver a quality BizTalk implementation

- Keep an eye on the community

There is a lot of good stuff in the community based around testing for BizTalk.  There are a number of tools which we use such as the BizTalk Orchestration Profiler, BizUnit Designer, BizUnit.  These tools are really effective and we choose to use them because as well as doing a good job any new members of the team are likely to have experience with them.

- Test code before it is put into BizTalk

As a practice I find the better you have tested any component before you use it within BizTalk the less problems you will have.  I have seen occasions where a developer has created say a helper class in C# which they then call from an orchestration.  They take ages to trackdown using HAT why their orchestration is failing and find it is a simple problem with the helper class.  To be honest I find the above frustrating, and I would expect the developer should test this class properly before it gets anywhere near being called from an orchestration.

Testing Challenges

Even though the approach we take to testing is intended to be pragmatic we have still encountered some challenges when using this on projects. 

Probably the biggest challenge is when someone (probably external to the development team) does not understand the benefits of the approach being taken.  One example of this was that in a meeting about a change request for a particular BizTalk implementation a couple of people at the meeting were surprised that what they expected to be a simple change which they thought would take less than 1 day was being quoted as 2 days for the work to be done.

The work itself was fairly straightforward but the change had a fundamental impact on a number of the tests of this particular process.  Basically our approach was to handle and map a file output from one system to a very complicated positional file format for another system.  We had a number of BizUnit tests which would push specific input through this process and then compare the output to an expected file to ensure the output was always exactly as expected.

The change would add new fields to the output file so the developer was going to have to change all of the expected files to have the right data for the new fields.  Because of the nature of the the output files this would require most of the effort.

In this particular case the detailed developer testing was a huge benefit because the only defect which we ever got relating to the mapping was because the specification of the file was incorrect, and not a code defect. 

In developer testing it is easy to produce the output file and make loose assumptions that the file is right, but it will not be until you integrate BizTalk with the external system you will pick up a lot of issues.  At this stage any defects have a higher cost as in addition to the fix time, they can also limit the test cases which can be performed by the testing team.

The lesson on this particular example is an extra day upfront can save a lot more in the long run.


Posted on Sunday, March 30, 2008 12:12 AM BizTalk , BizTalk Testing | Back to top

Comments on this post: BizTalk Testing Series - Introduction

# re: BizTalk Testing Series - Introduction
Requesting Gravatar...
Hi Michael,
Missed reading your blog for a while and just came across this series on testing. The material looks very interesting. I will start reading through in detail and post comments wherever i can. I was thinking of sharing my teams approach to testing as well and will trackback to your posts when i get that material together.

Left by Benjy on Apr 11, 2008 9:16 PM

# re: BizTalk Testing Series - Introduction
Requesting Gravatar...
Hi Michael,

I am actively into Biztalk Integration Solution testing. This blog seemed interesting to me. Please pass on if you have any additional resources on testing.

ALready I have been doing schema validation, rules verifircation, focus on persistent points in Orchestration, and end to end testing with different partners integrated.

Left by Bhagya on Jul 22, 2008 1:50 PM

# re: BizTalk Testing Series - Introduction
Requesting Gravatar...
Hi Michael,

I am getting good knowledge on testing by this blog.Please provide sample applications for Bizunit testing for SOAP web service scenarios

Left by Naresh V on Jul 22, 2012 11:29 PM

Your comment:
 (will show your gravatar)

Copyright © Michael Stephenson | Powered by: