NCache & BizTalk Follow up

Comments | Share

I was having in interesting discussion with Yossi Dahan following a recent SBUG meeting, as most user group sessions go the meeting was good but there was also some good discussion in the pub afterwards. Anyway we got onto the subject of caching and BizTalk and we discussed a few things I wrote ages ago about using NCache with BizTalk. As a follow up to this discussion I ran a few tests to get some rough figures to show some of the benefits of caching of data in this context.

The previous article I wrote talking through an implementation with NCache is discussed on the following link: http://geekswithblogs.net/michaelstephenson/archive/2008/09/21/125353.aspx

In taking this further I decided to come up with the following scenario. There would be 4 maps implemented which would all use the same basic schema containing a repeating element which would have its value converted from a system specific value on the left to a common id in the middle and then to another system specific id on the right. The below graphic shows this.

The transformation would be based on data in the Cross Referencing (XREF) tables within the BizTalk Management Database. I would implement 4 maps covering each of the following ways of doing the cross referencing.

1. A map would call helper functions which would read data from the tables and then cache it in the HTTP Runtime cache
2. A map that would call a helper function which would read data from the tables and then cache it within NCache as an out of process cache
3. A map that would use the out of the box Cross Referencing Functoids
4. A map that would use a helper class to call the BizTalk XRef stored procedures in the management database on a remote server (Note this is just to simulate a remote SQL Database as the functoids are fixed to use the database set in the registry).

In execution of the tests I would load up any caches before executing the map as I want to focus on the transformation times and not the loading of data.

To execute the tests I also used a MsTest as described in my BizTalk Testing Series which would just execute the map itself and remove the dependency on me needing to run things through a BizTalk pipeline which would make it difficult to work out the exact map execution time because other activity would be taking place which might affect the results.

Limitations

Before getting into the results, there are some limitations in my tests which you should be aware of.

1. I was using a single developer machine which has both BizTalk and SQL Server
2. The tests were done using BizTalk 2006 R2
3. The remote SQL Server for test 4 was just another BizTalk development machine on the network
4. I did not use velocity because I reused the code from a previous post and the point is to compare the approaches to caching and not the different caching providers

Results

The following table shows the results of these tests.

Notes:

• On the largest file with the database being hosted locally there were issues where I would overload the SQL Server. I think this was a combination of the BizTalk setup in addition to the test. I ignored this as the remote database test would more accurately represent the production BizTalk Cross Referencing setup
• The database related tests would have more unreliable latency as they would be affected by things such as network performance, other things running on the SQL Server, etc
• The test results above were approximations from a number of runs of the tests

Considerations

Based on these rough findings think when you use caching like this you should consider the following:

• If you want the fastest possible performance then the HTTP Cache which is commonly used in BizTalk development seems to offer this for in process caching
• When you use In Process caching you should be conscious of how much data you are actually caching and when objects are evicted from the cache. You don't want to cache too much data particularly for a BizTalk solution because this may cause Process Memory throttling.
• Out of process caching offers a number of benefits including:
  • You can deal with caching much larger amounts of data
  • You don't have to worry about lots of network hops like you would for database hits
  • The performance is much closer to the in process cache than the remote database
  • You only have 1 store of the data where as in process caching may mean the same data is stored in many places (eg: on each BizTalk Host instance)
• NCache has a configurable option on the setup for each cache which allows you to determine if the cache is in process of out of process. This could be very useful for allowing your code to be the same in its use of a cache but at runtime if can be configured to be the most optimal for the data you will be caching.