This is a post, describing an issue we recently had on the Azure Service Bus. 

Situation description

When a single appdomain is opening two (or more) service bus relay endpoints that have the same path (what is added after the servicebus.windows.net), but have a different namespace, the following exception is raised:

System.InvalidOperationException: The ChannelDispatcher at 'sb://repro2.servicebus.windows.net/ordersvc' with contract(s) 'ICustomerService' is unable to open its IChannelListener. ---> System.InvalidOperationException: A registration already exists for URI 'sb://repro2.servicebus.windows.net/ordersvc'.

The following is the configuration of my endpoints (not including the TokenProvider, etc, just the endpoints.

        <endpoint address="http://repro1.servicebus.windows.net/ordersvc"
            binding="basicHttpRelayBinding"
            bindingConfiguration="httpRelayBinding"
            contract="Demo1.Relay.ICustomerService"
            behaviorConfiguration="repro1Behavior" />
        <endpoint address="http://repro2.servicebus.windows.net/ordersvc"
            binding="basicHttpRelayBinding"
            bindingConfiguration="httpRelayBinding"
            contract="Demo1.Relay.ICustomerService"
            behaviorConfiguration="repro2Behavior" />

Why do we need the same services exposed on different namespaces?

Environment seperation

For Codit Integration Cloud, our azure-based integration platform, we are consuming some on-premises web services, exposed over the Azure Service Bus relay binding.  We do this for different services that make part of our runtime.  We also have a deployment cycle where we have different environments (a development, a test, a demo and off course the production environment(s)).

To isolate services, we make sure that every environment has a specific service bus namespace.  This allows for security segregation and to move services on premises, without touching other environments.  So this means that for our EdiFactParser service, we have the following endpoints (different names are being used here):

• sb://intcloud-dev.servicebus.windows.net/onprem/parse
• sb://intcloud-test.servicebus.windows.net/onprem/parse
• sb://intcloud-demo.servicebus.windows.net/onprem/parse
• sb://intcloud-prod.servicebus.windows.net/onprem/parse

As you can see the only difference in the above mentioned endpoints are the service bus namespaces.  As mentioned, these services are hosted on different servers on premises, but in some cases, we host them together.  (for example: dev & test are sharing the same local server).  In our cases, these services are hosted in BizTalk Server receive locations.

Disaster recovery

In the rare case where an Azure center would be out of the air, all service bus namespaces (hosted in that data center) are unavailable.  (There might be something like service bus federation in the future, but that’s not there yet).  To make sure that our on premises services are still available, we could expose these services on multiple service bus namespaces.  (hosted in different data centers).

In the case of a namespace being unavailable, the client that consumes the service could failover and try to consume the service over the second namespace.

How to work around this

This seems to be a bug in the Service Bus dll implementation, but we didn’t got a confirmation of this by product team yet.  (there might be a good reason for this, allthough I don’t see it).  Anyway, to workaround this, you need to open the different endpoints in separate service hosts.  The following describes how to do this.

In BizTalk Server

We initially had the issue on BizTalk Server, where we expose these services over the service bus.  Both receive locations were hosted in the same BizTalk host.  Just moving both locations to a different BizTalk host, solved the problem. 

In regular .NET applications

If you open the service host yourself, it as easy as creating two instances of the ServiceHost class and have them opening the endpoints seperately.

Just wanted to share this one

Sam Vanhoutte

SCOM (System Center Operations Manager) 2007 R2 comes with a very handy utility to create reports.
In this blogpost I will explain how you can create a detailed report concerning the CPU usage for a particular BizTalk Host Instance.

So let's get started!

The first thing you need to do is open the Operations Console.
Navigate to "Reporting > Microsoft Generic Report Library" in the left navigation bar. There you can double-click the report "Performance":

A new window will appear where you can enter some criteria. Here you can enter the timespan of the performance details you want to have. Let's say we want the stats starting from a week ago. To do this you need to modify the "From" date.

The next step you need to do is select the objects you want to visualize in your chart. You will notice a button named "Change" in the upper right corner. If you click on that button a window will appear where you can modify the settings of the chart.

To create a new chart, simply hit the button "New Chart". You can give the chart a meaningful name by setting this in the details pane.
After that you need to set the different series (these are the different objects inside your chart). Click on the button "New Series".

In the series, you can add a group/object. In our case this would be the particular server for which we want to see the CPU usage. Click on "Add Group.." and another window will appear. In the search pane, type a part of the computername and hit the search button. Select the computer you want to add.

Once that object is added, you still need to define the performance counter. You can define this in the "Rule" section. Click the browse button.
In the search pane type "CPU" and select the counter "BizTalk: CPU Usage Server Processes".

In the rule section you will see that you can choose a specific host instance (or all host instances) to which the rule will be applied. 

Now here's the tricky part. The UI only shows the Process name, not the actual host instance name. To know the actual host instance name you need to execute the steps below:

Go to the server that you have added in the object menu (so the server of which you want to see the performance details).
In the server, press start > Run > perfmon, this will open the built-in performance monitor of that server.
Navigate to "Monitoring Tools  > Performance Monitor" and right click in the window > "Add Counters".

Search for the counter "Process > ID Process" and select the process name of which you want to know the actual host instance name (in our case "BTSNTSvc").
Switch the current view to the "Report" view. In this view you can see the current "Process ID" of that host instance.

To link the Process ID to the host instance name, you have to execute following command in the cmd utility of windows:

TASKLIST /SVC /FI "IMAGENAME eq btsnt*"

This will give you a list of all the BizTalk host instances and their Process ID. Be aware that whenever you restart a host instance, the Process ID will change!
Now simply search for the Process ID that you retrieved from the performance monitor and you know which host instance you selected in the SCOM rule (in our case the host instance "CIPGW_ReceiveHost").

When everything is configured simply press the "Run" button at the top left of the window and SCOM will generate a nice chart. You can export the chart as pdf, excel,.. If you like to run this performance report again, you can save all your settings by clicking "File > Publish". This will allow you to save all the settings and run the report at a later time. The report will be available in the section "Authored Reports".

Here is a screenshot of such a report:

I had some trouble with linking the host instance name with the process name, so that's the main reason why I wrote this blog item.

I hope this saves you a lot of trouble :)

Many of us at Codit already installed the WCF LOB SDK multiple times. And in the process we have learned to know the caveats that come with the installation:
- 32 bit vs 64 bit
- Adapter Pack installation (again, 32 bit vs 64 bit)
- Enterprise application installation

Now since BizTalk 2010, there is a great wizard added to the installation screen if you click “Install Microsoft BizTalk Adapters”:

The screen shows up like this once you go through the entire installation:

Please note that the above screenshot is from a production server where I don’t have Visual Studio installed, so I could not install the developer tools (hence the warning)

It is a very welcome change, because before BizTalk 2010, people did not know exactly what they needed to install and in which order.
Now the screen gives you a very handy wizard-style interface.

Now recently I had a strange error: on a 64-bit machine I went through the installation.
I was sure that everything was installed correctly and in the right order!

I checked all machine.config files (for framework 2.0 and 4.0, both 32-bit and 64-bit) and all had the necessary entries added.
However, there was no way I could see/add the WCF LOB Adapters in the BizTalk console:

I went through a lot of online searching until I accidently stumbled upon the BizTalk modify wizard:

It happens that the WCF Adapter components in BizTalk were not installed, so the WCF adapters were not showing up for a good reason!

When I checked both checkboxes (WCF for BizTalk Runtime and Administration Tools And Monitoring), installed it on both nodes and re-opened the BizTalk consoles, I could add the WCF adapters I needed.

I hope this helps at least someone!

Recently, I encountered a strange problem at a customer. MSDN blogs warned us about the leap year date 2/29/2012 when using EDI in BizTalk Server 2006 R2 or 2009, as you can see on this blogpost. It was during the install of this hotfix on a BizTalk 2006 R2 server I was encountering the problem. I’ll explain the problem over here and show you the easy work around.

EDIT: Apparently Microsoft has solved this error. A more recent version of the hotfix download does autodetect Cumulative Update 3, without using a version number. The latest version of the hotfix can be found over here.

However, I will show you how I worked around the problem (because the newer version was not yet available at that time).

Prerequisites

MSDN blog is indicating that the install of the EDI leap year hotfix on BizTalk Server 2006 R2 requires some prerequisites.

Since BizTalk 2006 R2 Service Pack 1 was already installed in this particular case, I only still needed to execute step 2 and 3. Installing the Cumulative Update 3 gave me no problems. So far, so good.

Install of the leap year date hotfix itself

However, when installing the leap year hotfix, I encountered following error:

“The patch 861c5534-6cfa-4dcf-ba70-cbf01129b646 in the package Microsoft BizTalk Server 2006 R2 Hotfix [See KB article 2435900 for detail] cannot be applied.  The minimum installed version of Microsoft BizTalk Server 2006 must be 3.6.2222.12.  The installed version on this computer is 3.6.1404.0.”

This was quite strange to me, since MSDN blog was indicating that Service Pack 1 and Cumulative Update 3 should be sufficient as prerequisites to install the hotfix. A little investigation led to the following.

Checking the Registry Editor (HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server\3.0), showed us that the current product version is ‘3.6.1404.0’. 

This is just like the error indicated, while it expects a version of 3.6.2222.12 or higher. So I started searching the web to check if that Product Version Key was correct and related to BizTalk Server 2006 R2 SP1.

BizTalk Server 2006 R2 SP1 Product Version Key

On a Microsoft Technet wiki page I found following table with the all product version key’s correlated to the product name:

What you can see over there, is that the Product Version Key is not adapted when Service Pack 1 of BizTalk Server 2006 R2 is installed. This is not consistent if you compare it with the other version of BizTalk Server.

Strange thing, BizTalk 2006 R2 SP1 comes with Product Version Key 3.6.1404.0, while the install of the hotfix needs a version of 3.6.2222.0 or higher. Because the prerequisites for the hotfix (BizTalk 2006 R2 SP1 and CU3) were correctly installed and yet the Product Version Key seemed not to be right, I started thinking about a possible solution to fix this little error.

Workaround for the error

Since installing the hotfix for the leap year date is all about just running a ‘Setup’ and there’s also a Setup.xml file in the folder, I had a look over there.

As you can see in the print screen below, there indeed is a field ‘MinProductVersion’ indicating the version should be at least 3.6.2222.12. This is why we encountered the error.

So what about changing this value to a Product Version lower than 3.6.1404.0, the one that is installed on the server now? Let’s give it a try.

NOTE: make a copy of the original Setup.xml, in case the change would not be effective.

As you can see below, I changed that one value in the xml document to ‘MinProductVersion = “3.6.1.12” ‘.

Before executing the Setup again, and hoping it will not fail on the Product Version again, we need to find a way to check if the setup will be executed effectively.

In the Setup xml file we can see some assemblies that will be adjusted to a newer version when executing the hotfix. Therefore we can check the current file version of a specific assembly and see if the file version has been changed after the execution of the Setup.

For example, we can see in the setup xml file that ‘Microsoft.BizTalk.Edi.Shared.dll’ should have version 3.6.2230.12 after the execution.

A quick lookup shows us that the file version of that assembly is ‘3.6.2224.12’ right now.

If we now execute the Setup and check this file version of the assembly again afterwards, we should see it’s changed to ‘3.6.2230.12’ if the Setup was successful. Let’s try this!

I executed the Setup again and did not encounter the same error again, indicating the product version is too low. So that’s a good start. Now that the wizard indicates the execution has successfully ended, let’s have a look at the file version again.

And yes, the file version was changed to 3.6.2230.12 and therefore the assembly was updated. Together with the fact that our newly installed hotfix is shown in the ‘Installed programs and updates’-overview (as you can see below), we can conclude that our workaround for the error and therefore the installation of the hotfix itself has been successful!

Microsoft BizTalk Server 2010 ships with some assemblies that assist you with the administration and deployment of your BizTalk environment.  During the development of an automated deployment plan, we were able to perform most of the deployment tasks at a BizTalk application level, by using these assemblies:

Microsoft.BizTalk.ExplorerOM

-       Create/delete applications

-       Control receive locations

-       Control orchestrations

-       Control send ports

-       Control send port groups

-       Control application references

Microsoft.BizTalk.ApplicationDeployment

-       Add/remove resources

Microsoft.BizTalk.Deployment

-       Import/export bindings

Microsoft.BizTalk.Operations

-       Query for active service instances         

Remark that all of the used objects can be initialized by three parameters:

      -    SqlServerName

-    BizTalkMgmtDbName

-    ApplicationName

The object "BizTalk Group" is defined by the SqlServerName and the BizTalkMgmtDbName.  A "BizTalk Application" is defined by its BizTalk Group and the ApplicationName.  However, these assemblies have totally no notion of the object "BizTalk Server".  They only support operations at a database level (BizTalk Group), but some deployment tasks need to be executed on a BizTalk Server level:

- Install/uninstall assemblies to the Global Assembly Cache

- Control host instances

For both actions we needed to be able to dynamically determine which BizTalk Servers are part of the already defined BizTalk Group. After some research, we found two options to implement this logic:

Windows Management Instrumentation

WMI has some support for BizTalk via the WMI namespace “root\\MicrosoftBizTalkServer”.  We are able to loop though all configured host instances, by executing a query on the MSBTS_HostInstance class.  As each host instance runs on a BizTalk Server, it’s easy to implement some logic that retrieves all BizTalk Servers.

private List<string> GetBizTalkServers()
{
     List<string> btsServers = new List<string>();

     EnumerationOptions wmiEnumerationOptions = new EnumerationOptions { ReturnImmediately = false };
     ObjectQuery wmiQuery = new ObjectQuery("SELECT * FROM MSBTS_HostInstance");
     using (ManagementObjectSearcher wmiSearcher = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", wmiQuery.QueryString, wmiEnumerationOptions))
     {
          ManagementObjectCollection hostInstanceCollection = wmiSearcher.Get();
          foreach (ManagementObject hostInstance in hostInstanceCollection.Cast<ManagementObject>())
          {
                string btsServer = hostInstance["Name"].ToString().Split(' ').Last();
                if (btsServers.Contains(btsServer) == false)
                {
                     btsServers.Add(btsServer);
                }
          }
      } 

      return btsServers;
}

Custom SQL Server Query

Another option is to retrieve the information directly from the BizTalkMgmtDb.  The table adm_Server actually contains a list of all BizTalk Servers that are part of the BizTalk Group.  So this simple .NET code is able to return us the needed information:

private List<string> GetBizTalkServers()
{
     List<string> btsServers = new List<string>();

     SqlConnection sqlConnection = new SqlConnection(String.Format(CultureInfo.CurrentCulture, "Server={0};Database={1};Integrated Security=SSPI;", SqlServerName, MgmtDatabase));
     SqlCommand sqlCommand = new SqlCommand("SELECT Name FROM [dbo].[adm_Server]", sqlConnection);

     sqlConnection.Open();

     SqlDataReader sqlDataReader = sqlCommand.ExecuteReader();

     while (sqlDataReader.Read())
     {
          btsServers.Add(sqlDataReader["Name"].ToString());
     }

     sqlConnection.Close();

     return btsServers;
}

Conclusion

It's pretty easy to retrieve a list of all BizTalk Servers that are part of a BizTalk Group.  A limitation of the WMI approach is that this will only work when it's executed on one of the BizTalk Servers.  So I prefer the SQL approach, because this will also work for remote execution.

Toon Vanhoutte

Recently, I had a project where full side-by-side deployment was required.  In a first test phase of the automated deployment, we used MSBuild together with the Codeplex MSBuild Extension Pack. We tried deploying assemblies side-by-side, but BizTalk threw errors when trying to deploy a new version of our Blog.Codit.BizTalk.Pipelines assembly:

Failed to add resource(s). Change requests failed for some resources. BizTalkAssemblyResourceManager failed to complete end type change request. SQL exception: "Violation of UNIQUE KEY constraint 'UQ__bts_pipe__DD425CAE7908F585'. Cannot insert duplicate key in object 'dbo.bts_pipeline'. The duplicate key value is (D6C53CD9-8466-4520-A85B-7BD887A57455)." 

I received this error when trying to deploy from Visual Studio, from MSBuild (using BTSTask.exe) and also when trying to deploy it manually via the Admin Console (Add Resource).  So the error is actually independent from the deployment source.  As BizTalk returned a SQL exception, I decided to run SQL Profiler and detected this call:

exec btm_CreatePipeline 
   @PipelineID='D6C53CD9-8466-4520-A85B-7BD887A57455',
   @Category=1,
   @Name=N'Blog.Codit.BizTalk.Pipelines.rcvSalesOrder',
   @FullyQualifiedName=N'Blog.Codit.BizTalk.Pipelines.rcvSalesOrder, Blog.Codit.BizTalk.Pipelines, Version=1.2.0.0, Culture=neutral, PublicKeyToken=xxx',
   @IsStreaming=0,
   @AssemblyID=265,
   @Release=2,
   @ID=@p8output

So the deployment logic tries to create a new entry in the bts_pipeline table, but it uses exactly the same (unique) ID of the pipeline of the older version.  This entry was already present in the bts_pipeline table: 

D6C53CD9-8466-4520-A85B-7BD887A57455 Blog.Codit.BizTalk.Pipelines.rcvSalesOrder, Blog.Codit.BizTalk.Pipelines, Version=1.1.0.0.  

We opened a case at Microsoft Support for this issue.  They forwarded the issue to the BizTalk Product Team and we received this answer:

The pipeline GUID is stored in the rcvSalesOrder.btp.cs  file. The GUID gets regenerated when either of the following occurs:
 
- The pipeline is saved from within Visual Studio.
- When REBUILD the BizTalk project.

As we executed a BUILD, instead of a REBUILD, a small change to our MSBuild script solved the issue:

<MSBuild Projects="@(ProjectsToBuild)" Targets="Rebuild" />

This behavior is rather inconsistent, because it only occurs when your assembly contains a pipeline object. 

Big thanks to Microsoft for the fast and accurate support.

Toon Vanhoutte

Recently a customer of ours asked us if it was possible to include versioning information into the title of the installed MSI.

These MSI files can be generated by right-clicking a BizTalk application and selecting "Export" and then "MSI file":

This generates the MSI file on disk.

Problem however is that the MSI file which gets generated always has version 1.0.0.0 assigned to it.
We can always use Microsoft's Orca tool to manually update the properties of the file, but this is very cumbersome to say the least.

I went on to search if there was an option to pass some parameters to BTSTask.exe.
This tool allows you to perform the "Export MSI" via the command line, allowing you to script it.
Unfortunately this tool does not allow you to set the version information at all. 

Luckily, BizTalk comes with a .NET API that allows you to update the properties of the MSI installer package.
For BizTalk 2010 this is the Microsoft.BizTalk.MSIManager assembly.

using Microsoft.BizTalk.ApplicationDeployment.MSIManager.WindowsInstaller;

This assembly allows you to query/update the MSI database structure contained within the MSI file.

Just provide the following code to update the name of the application for example:
msiPath = full path to your MSI
applicationName = your custom application name that will appear in your "Programs and Features" list under Windows.

static public void UpdateMSIApplicationName(string msiPath, string applicationName)
        {
            Database db = Installer.OpenDatabase(msiPath, DatabaseOpenMode.Transact);
            string query = @"UPDATE Property SET Property.Value = '" + applicationName + "' WHERE Property.Property = 'ProductName'";
            Microsoft.BizTalk.ApplicationDeployment.MSIManager.WindowsInstaller.View vw = db.OpenView(query);

            vw.Execute();
            vw.Close();

            db.Commit();
            db.Close();
            db.Dispose();
        }

You can also update your Manufacturer information to customize the directory where the MSI gets installed. By default this is "Generated by Biztalk".
Use this query to perform the update then:

string query = @"UPDATE Property SET Property.Value = '" + manufacturer + "' WHERE Property.Property = 'Manufacturer'";

I also tried updating the version information, but for some reason this will insert some minor errors during installation having to do with the registry.
I believe this has to do with the registry keys that are installed by default within the BizTalk MSI version. 

Incorporating all of this into a properly automated build/deployment scenario will allow you to publish your MSI files with customized versioning.

Good luck.

This blog post contains some guidelines for consuming web services, with a focus on exception handling.  We will both handle pure messaging scenario and web service calls from within an orchestration.  I also make a distinguishment between asynchronous and synchronous flows.  Synchronous means that the flow needs to return a message to the original caller.  So in case of troubles, a SoapFault needs to be returned to the calling application.  In an asynchronous flow, you just need to make sure that your web service call is resumable. 

Messaging Scenario – Asynchronous

When an error occurs during the web service call, the default behavior of BizTalk retry 3 times and will suspend the request message on the BizTalk Send Port and this instance will be resumable.  This is the perfect behavior for this asynchronous process.

Messaging Scenario - Synchronous

If you are in a synchronous process, a suspended Send Port will lead to a timeout on your Receive Port.  In this scenario you need to enable "Propagate Fault Message" on the Send Port and a SoapFault will be returned to the original caller.  Also think about the Retry Count and Retry Interval settings, because a bad configuration can also lead to timeouts.

Orchestration - Asynchronous

When you consume a web service from within an orchestration, it's getting more complex.  If the consumed web service throws an exception, BizTalk creates by default 2 suspended service instances.  Resuming them does not always have the expected behavior.

-  The Orchestration is suspended.  A resume will not change anything to the state of the Orchestration.
-  The Send Port is suspended.  A resume will resume the web service call as expected, but the Orchestration will not be able to continue processing afterwards.

How can we now achieve a resumable web service call?

First of all enable "Propagate Fault Message" on the Send Port.

Also modify the calling orchestration:

-  You need to create a loop.  This loop continues as long as the Boolean bSuccess equals false.

-  Modify the logical Orchestration Send Port and add a Fault Message.  To keep it general, you can choose the SoapFault (which is part of the Microsoft.BizTalk.GlobalPropertySchemas).

-  Add an exception handler to the web service call. Catch the Fault Message that was returned.  Add a suspend shape for human intervention. Note that it’s a good practice to also add an extra exception handler which catches a System.Exception.

-  If the web service call fails: set bSuccess == false in the exception handler.  If the call succeeds, set bSuccess == true, so the orchestration will exit the loop.

The result should look like this:

Now we've created a resumable web service call from within an orchestration.  When testing the orchestration in a failure scenario, you will see that there are still two suspended service instances:

-  The Orchestration is suspended.  This is the one that we've made resumable.

-  The Send Port is suspended.  To avoid this second service instance to be suspended, you can enable "Routing for Failed Messages".  This will generate an ErrorReport, instead of suspending the Send Port.  Subscribe on the ErrorReport with the filter ErrorReport.ErrorType == “FailedMessage”.

Orchestration - Synchronous

In this scenario, you will also need to catch the SoapFault (or other exception) and make sure that you don’t end up with suspended send port instances. See “Orchestration – Asynchronous”

The extra functionality that you’ll need to add is returning a SoapFault to the original caller.

- Add a Fault Message to the Request/Response receive port of your orchestration.

- You can choose to return a typed Fault or, to keep it general, a SoapFault (which is part of the Microsoft.BizTalk.GlobalPropertySchemas).

- Make sure you return the catched SoapFault (or the created SoapFault in case of other exception) in case of errors.

- Remark that you’ll need to add a decide shape to determine you’re in a success/failure scenario. Otherwise, the compiler will fail (complaining about unconstructed messages).

Again, think about the Retry Count and Retry Interval settings on the send port.

Toon Vanhoutte

Recently, we've encountered some problems with SCOM 2007 R2 monitoring of a BizTalk 2010 environment. The SQL Agent Job "Backup BizTalk Server (BizTalkMgmtDb)" failed, but we didn’t receive any SCOM alert about that.  The first reflex was to look at the BizTalk Management Pack.

BizTalk Server Management Pack

The BizTalk Server Management Pack contains a rule, called “CRITICAL ERROR: A SQL Server agent job failed - Backup BizTalk Server”, which is responsible for alerting us in case of troubles.  The rule is disabled by default, but we had already enabled the rule with an override.

Why didn’t we receive an alert?  We had a closer look at the default rule and discovered that this is the way of identifying the backup job failure:

-    It’s a rule that subscribes on the event log [TypeID="Windows!Microsoft.Windows.EventProvider"]
-    It looks for event log entries with ID = 208 [EventDisplayNumber equals 208] in the Application log
-    The computer name is set to $Target/Property[Type="Microsoft.BizTalk.Server.2010.ServerRole"]/ComputerName$

When the error occurred, we indeed discovered event log entries with ID 208 on the SQL server.  But this rule is subscribing on the event log of the BizTalk server.  Conclusion: the default rule will only work on a single box installation (SQL server and BizTalk server have the same event log).

To solve this issue, we’ve created a custom rule, based on the default one.  The custom rule had exactly the same configuration, except for the computer name that we’ve changed to:
-    “$Target/Property[Type="Microsoft.BizTalk.Server.2010.BizTalkGroup"]/MgmtDbServerName$”

Now the rule was subscribing on the event log of the SQL Server (running the ManagementDb), and we received an alert when the backup job failed.  This worked fine, until we deployed this rule against a multi-server BizTalk environment, containing a SQL Server cluster.  Suddenly we’ve received SCOM errors, complaining that the SCOM Agent couldn’t access the event log of the MgmtDbServer.  The cause of this is that the BizTalk Management Pack actually discovers the virtual SQL Server Cluster name.  As this is not a physical server name, it’s normal that the SCOM Agent can’t access the event log.

To overcome this issue, we could have created a new custom rule, which uses the discovered MgmtDbServer and performs a query on the [msdb].[dbo].[sysjobhistory] system table.  The sysjob tables contain all needed information to detect SQL Agent Job failures.  As this is too complex, we decided to have a look at the SQL Server Management Pack.

SQL Server Management Pack

As it is actually a SQL process that fails, the SQL Server Management Pack should be responsible for alerting us.  But why didn’t that happen?  There are actually two reasons for this:

-    The discovery of SQL Agent Jobs is disabled by default.
-    The alerting for SQL Agent Job Failures is disabled by default.

We enabled the discovery and alerting, tested the solution and everything went smooth.

Conclusion

SQL Agent Jobs should be monitored by the SQL Server Management Pack.  This Management Pack should always be installed in a BizTalk environment, as SQL Server is the core of BizTalk.  Please, keep in mind that the alerting of SQL Agent Jobs is disabled by default.

It’s actually a strange approach of Microsoft to try to include the monitoring of the SQL Agent Jobs in the BizTalk Server Management Pack.  Certainly because the default implementation only works for a single box installation.  On MSDN you can find a vague description on this subject.

Toon Vanhoutte

The BizTalk Sever SDK comes with an interesting Pipeline component sample that allows executing Xsl mapping files in a pipeline.

This functionality can be of interest when the receive port or send port mappings are not executed at the desired time! Receive port mappings are executed after the pipeline execution, send port mappings are executed before the pipeline execution. Thanks to this pipeline component you get control over the exact time when a transformation gets executed.

Although this functionality can be very useful the code of this sample contains some problems that need fixing before using this in production environments.

Let’s take a look at the business end of the sample, the TransformMessage method (you can find the original sample code in this location: <BizTalk installation directory>\SDK\Samples\Pipelines\XslTransformComponent):

private Stream TransformMessage(Stream stm)
{
	 MemoryStream ms = null;
	 string validXsltPath = null;
 
	 try 
	 {
  		// Get the full path to the Xslt file
  		validXsltPath = GetValidXsltPath(xsltPath);
  
  		// Load transform
  		XslTransform transform = new XslTransform();
  		transform.Load(validXsltPath);
    
  		//Load Xml stream in XmlDocument.
  		XmlDocument doc = new XmlDocument();
  		doc.Load(stm);
    
  		//Create memory stream to hold transformed data.
  		ms = new MemoryStream();
   
  		//Preform transform
  		transform.Transform(doc, null, ms, null);
  		ms.Seek(0, SeekOrigin.Begin);
 	}
	catch(Exception e) 
 	{
  		System.Diagnostics.Trace.WriteLine(e.Message);
  		System.Diagnostics.Trace.WriteLine(e.StackTrace);
  		throw e;
	}

 	return ms;
}

The signature of the method accepts a Stream as input parameter and returns a Stream as result. Perfect to keep everything streaming…

Then a XslTransform object is created. At first, this might look like a good idea, but XslTransform will load the message into memory internally! For smaller messages this will not cause any issues but bigger messages will cause a System.OutOfMemoryException…

To get rid of this problem I replaced the XslTransform class with the BTSXslTransform class. This class uses the BizTalk transformation engine. The BizTalk transformation engine will only load small messages into memory and will use disk space if the message size reaches a certain threshold.

The following line instantiates the BizTalk mapper engine (add a reference to Microsoft.XLANGs.BaseType)

BTSXslTransform trans = new BTSXslTransform();

Use the following line to execute the transform:

trans.ScalableTransform(inputReader, null, vs, new XmlUrlResolver(), false)

 
The following problem we encounter in the original sample is this section:

XmlDocument doc = new XmlDocument();
doc.Load(stm);

This section reads the stream into memory again via an XmlDocument.
I replace this section with this line:

XmlTextReader inputReader = new XmlTextReader(stm);

Instead of feeding the message to the transform method as one memory chunk, I present it a stream that can be pulled by the BTSXslTransform object.

There is one last thing to change in this sample to make it suitable for large messages:

ms = new MemoryStream();

A MemoryStream is used to store the result of the transform operation. Unfortunately a MemoryStream uses memory to store the stream. This makes it unusable for large messages.

Luckily BizTalk Server comes with a stream class that has the same functionality as MemoryStream but one that uses disk space to store large streams. This class is the VirtualStream class (add a reference to Microsoft.BizTalk.Streaming).

Instantiate a VirtualStream object to hold the result of the transformation:

vs = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);

The AutoOverFlowToDisk instructs this stream to use disk space for large messages.

The changed TransformMessage function now looks like this:

private Stream TransformMessage(Stream stm)
{
 	VirtualStream vs = null;
	string validXsltPath = null;
  
 	try 
 	{
  		// Get the full path to the Xslt file
  		validXsltPath = GetValidXsltPath(xsltPath);
  		XmlTextReader stylesheet = new XmlTextReader(validXsltPath);

		// Load transform
		BTSXslTransform trans = new BTSXslTransform();
		trans.Load(stylesheet);

		XmlTextReader inputReader = new XmlTextReader(stm);
                    
  		//Create memory stream to hold transformed data.
  		vs = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
                
  		//Preform transform
		trans.ScalableTransform(inputReader, null, vs, new    XmlUrlResolver(), false);
    
  		vs.Seek(0, SeekOrigin.Begin);
 	} 
 	catch(Exception e) 
 	{
  		System.Diagnostics.Trace.WriteLine(e.Message);
  		System.Diagnostics.Trace.WriteLine(e.StackTrace);
  		throw e;
 	}
  	return vs;
}

When this code is compiled into a pipeline component, it allows executing large transformation exactly where you want it in the BizTalk pipeline infrastructure. This component will keep memory consumption flat, even when processing very large Xml files.

The source code of this article can be downloaded from this post as attachement.

XslTransform.zip (30,62 kb)

Peter Borremans