April 10, 2014 at 3:48 PM

Recently I wrote a post about advanced orchestration monitoring for BizTalk Server using System Center Operations Manager (SCOM).

 

There, I wrote about some of the shortages in the default monitor. In less critical environments the default monitoring for suspended orchestrations will be sufficient in most cases. I bumped into the same issue a couple of times now and I would like to share it with you, hoping it saves you some troubleshooting. For us, Codit Managed Services, it's very important to receive the right alerts when instances get suspended.  

 

This blog post concerns the default monitor for suspended orchestrations and the alerts it generates.
I often hear following question: "My orchestration is suspended for 4 hours now and still I didn't receive any alerts about it?!". At first I also did not find an explanation for this. Let's take a deeper look at this monitor.

 

If you have currently suspended orchestrations on your environment you should see them in the SCOM console having a critical or a warning state:

 

If you open the Health Explorer for this Orchestration you can see some history concerning the health state:

 

Now the question rises: "Why am I not receiving an alert even while my orchestration has a warning state.... ?".

If we take a look at the properties of the monitor at first sight everything seems to be correctly enabled:

I also missed it several times, but then I took a look at the first property. "Alert on State".

When you check the possible options of the property you will see 2 options. Choose the second one! Default a warning state will trigger no alerts!

 

The warning limit (the last property) has a default value of 10, so more then 10 orchestrations of the same type should be suspended before an alert is triggered.... If you change the "Alert on State" property you will always receive alerts when your environment contains a suspended orchestration instance, no matter how many instances.

 

But remember, as soon as the monitor is in a critical state you will no longer receive an alert when new instances get suspended! If you want an alert per instance, check out my previous post about advanced orchestration monitoring.

Posted in: BizTalk | Monitoring

Tags: , ,


March 28, 2014 at 3:45 PM

Recently, I needed to send a message from BizTalk to an external WCF service with Windows Authentication.

For easy testing, I created a Windows console application that will act as the client. I used basicHttpBinding with the security-mode set to TransportCredentialOnly. In the transport-node I chose for Windows as clientCredentialType.

The web.config file looks like this:

<basicHttpBinding>
	<binding name="DemoWebService_Binding"  textEncoding="utf-16">
		<security mode="TransportCredentialOnly">
			<transport clientCredentialType="Windows" />
		</security>
	</binding>
</basicHttpBinding>

 Before sending the test message, I needed to authenticate myself and insert the Windows Credentials:

proxy.ClientCredentials.Windows.ClientCredential.Domain = "xxx";
proxy.ClientCredentials.Windows.ClientCredential.UserName = "xxx";
proxy.ClientCredentials.Windows.ClientCredential.Password = "xxx";

This works, so now to get it right in BizTalk!

Problem:

Nothing special, just a Send Port, WCF-Custom with basicHttpBinding as Binding Type and the same binding configuration as in the console application:

 

I thought I just needed to add the credentials to the Credentials-tab in BizTalk to be able to do proper authentication.

Unfortunately, this does not work!

Apparently, when "Windows" is chosen as clientCredentialType, the Credentials-tab is ignored and the credentials of the Host-Instance running the Send Port are used instead.

Solution:

After some searching, I found the answer thanks to Patrick Wellink's blog post on the Axon Olympos blog: http://axonolympus.nl/?page_id=186&post_id=1852&cat_id=6.

 

The credentials of the Host Instance can't be right, because the web-service is from an external party.

To use the Windows Credentials, a custom Endpoint Behaviour has to be created.

So I've created a new Class Library in Visual Studio with a class that inherits from both BehaviorExtensionElement and IEndpointBehavior:

public class WindowsCredentialsBehaviour : BehaviorExtensionElement, IEndpointBehavior

For extensibility, the class needs to have these public properties:

[ConfigurationProperty("Username", DefaultValue = "xxx")]
public string Username
{
	get { return (string)base["Username"]; }
	set { base["Username"] = value; }
}

[ConfigurationProperty("Password", DefaultValue = "xxx")]
public string Password
{
	get { return (string)base["Password"]; }
	set { base["Password"] = value; }
}

[ConfigurationProperty("Domain", DefaultValue = "xxx")]
public string Domain
{
	get { return (string)base["Domain"]; }
	set { base["Domain"] = value; }
}

In the function "AddBindingParameters", I've added this piece of code that sets the Windows Credentials:

public void AddBindingParameters(ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
{
	if (bindingParameters != null)
	{
		SecurityCredentialsManager manager = bindingParameters.Find<ClientCredentials>();

		var cc = endpoint.Behaviors.Find<ClientCredentials>();
		cc.UserName.UserName = this.Domain + @"\" + this.Username;
		cc.UserName.Password = this.Password;
		cc.Windows.ClientCredential.UserName = this.Username;
		cc.Windows.ClientCredential.Password = this.Password;
		cc.Windows.ClientCredential.Domain = this.Domain;

		if (manager == null)
			bindingParameters.Add(this);
	}
	else
	{
		throw new ArgumentNullException("bindingParameters cannot be null.");
	}
}

Now after building and putting the assembly in the GAC, we need to let BizTalk know that it can use this custom Endpoint Behavior:


We need to add this line below to the behaviorExtensions (system.serviceModel - extensions) in the machine.config (32-bit and 64-bit):

<add name="WindowsCredentialsBehaviour" type="BizTalk.WCF.WindowsCredentials.WindowsCredentialsBehaviour, BizTalk.WCF.WindowsCredentials, Version=1.0.0.0, Culture=neutral, PublicKeyToken=1de22c2808f4ac2e" />

Restart the host instance that runs the send port and you will be able to select the custom EndpointBehavior:

 


February 14, 2014 at 4:35 PM

When using SCOM as a monitoring tool in a BizTalk environment you will find some shortages from time to time in the BizTalk Server management pack.

We already solved many of these shortages in our own Codit BizTalk management pack (containing custom rules, overrides,…).

An example of such a shortage is the default monitoring of suspended orchestrations. The out of the box management pack contains a monitor called “Orchestration Suspended Instances Availability Monitor”.

Assuming you know the difference between alerts and monitors, the biggest disadvantage of this monitor is its nature: being a monitor. When an orchestration gets suspended for some reason, you will receive an alert. However when other orchestrations of the same type are getting suspended, and the previous instance has not been terminated/resumed yet, no new alert(s) will be fired.

It can be fatal in a production environment if the support team doesn’t receive an alert. Also in a scenario where an orchestration gets suspended by code for some seconds (waiting for another orchestration to finish first, sequencing,…) a false alert could be triggered.

 

We created an alert that will notify us every time an orchestration gets suspended and stays suspended longer than a certain amount of time.

 

This is how to create such an alert:

 

Create a new rule to execute a script every 15 minutes:

image

 

image

 

image

 

image

 image

Script for this rule:

' ---------------------------------------------------------
' SQL Database Query Check
' ---------------------------------------------------------
' Param 0: The SQL connection string for the server 
' Param 1: The Database to use
' Param 2: SQL Query
' Author:  Brecht Vancauwenberghe
' Date:    05-02-2014
' ---------------------------------------------------------
Option Explicit

Sub Main()

    Dim oAPI, strServer, strDatabase, iThresholdHours, objBag, strErrDescription, objArgs, I, Param

    Const EVENT_TYPE_ERROR = 1
    Const EVENT_TYPE_WARNING = 2
    Const EVENT_TYPE_INFORMATION = 4

      ' Initialize SCOM Script object
      Set oAPI = CreateObject("MOM.ScriptAPI")

      ' Write Parameters to eventlog
      ' Enable for debugging.
      Set objArgs = Wscript.Arguments
      For I = 0 to objArgs.Count -1
          Param = objArgs(I)
           'strErrDescription = strErrDescription & ", " & Param
      Next
          'call oAPI.LogScriptEvent("SQL Database Query Check.vbs", 1313, EVENT_TYPE_INFORMATION, strErrDescription)     

      If WScript.Arguments.Count = 3 then

      
            ' Retrieve parameters
            strServer = CStr(WScript.Arguments(0))
            strDatabase = CStr(WScript.Arguments(1))

                  'Connect to the database
                  Dim cnADOConnection 
                  Set cnADOConnection = CreateObject("ADODB.Connection") 
                  cnADOConnection.Provider = "sqloledb" 
                  cnADOConnection.ConnectionTimeout = 60
                  Dim ConnString
                  ConnString = "Server=" & strServer & ";Database=" & strDatabase & ";Integrated Security=SSPI" 
                  cnADOConnection.Open ConnString
                  
                  'Connection established, now run the code
                  Dim rst 
                  Set rst = cnADOConnection.Execute(CStr(WScript.Arguments(2)))

                  ' should be just one record
                  ' oResults.MoveFirst

                  'Set objBag = oAPI.CreateTypedPropertyBag(1)
                  'Call objBag.AddValue("Count", CInt(oResults(0)))
                  
                  'oAPI.AddItem(objBag)

			Do While Not rst.EOF
			Call oAPI.LogScriptEvent("check_suspended_orchestrations.vbs",12223, EVENT_TYPE_Error, "An orchestration is suspended with following error :" & rst.fields.item(0) & " and is active for longer than 5 minutes. Check the BizTalk environment!") 
			rst.MoveNext 
			Loop
                  cnADOConnection.Close
                  
            'return the property bag objects
            'Call oAPI.ReturnItems

      End If 
            
End Sub

Call Main()

Parameters for this rule:

$Target/Property[Type="Microsoft.BizTalk.Server.2010.BizTalkGroup"]/MgmtDbServerName$ BizTalkDTADb "DECLARE @msgbox nvarchar (500)Set @msgbox = (select top(1) DBserverName from [BizTalkMgmtDb].[dbo].[adm_MessageBox] with (nolock))exec('SELECT * FROM [BizTalkDTADb].[dbo].[dtav_ServiceFacts] with (nolock) INNER JOIN [' + @msgbox + '].BizTalkMsgBoxDb.dbo.InstancesSuspended as msgboxdbinstancesSuspended ON [BizTalkDTADb].[dbo].[dtav_ServiceFacts].[ServiceInstance/InstanceId] = msgboxdbinstancesSuspended.uidInstanceID where [Service/Type] = ''Orchestration'' and datediff(minute,[ServiceInstance/StartTime],getutcdate()) between 5 and 15')"

The VBscript fires following SQL query defined as a parameter on the MSGBOX and DTA:

DECLARE @msgbox nvarchar (500)Set @msgbox = (select top(1) DBserverName from [BizTalkMgmtDb].[dbo].[adm_MessageBox] with(nolock))exec('SELECT * FROM [BizTalkDTADb].[dbo].[dtav_ServiceFacts] with(nolock) INNER JOIN [' + @msgbox + '].BizTalkMsgBoxDb.dbo.InstancesSuspended as msgboxdbinstancesSuspended  ON [BizTalkDTADb].[dbo].[dtav_ServiceFacts].[ServiceInstance/InstanceId] = msgboxdbinstancesSuspended.uidInstanceID  where [Service/Type] = ''Orchestration'' and datediff(minute,[ServiceInstance/StartTime],getutcdate()) between 5 and 15')

This query will return the type of the suspended orchestration,…

Another example is a query only using the MSGBOX:

SELECT nvcErrorDescription 
FROM [BizTalkMsgBoxDb].[dbo].[InstancesSuspended] with (nolock) 
JOIN [BizTalkMsgBoxDb].[dbo].[ServiceClasses] as serviceclasses 
ON uidClassID = serviceclasses.uidServiceClassID 
where serviceclasses.nvcName = 'Orchestration' and datediff(minute,dtSuspendTimeStamp,getutcdate()) between 5 and 15

 

This will return the error description why the orchestration got suspended. You can choose the query you prefer, or do much more with it.

Remark, for the MSGBOX example will need to use following parameters (query is executed on the MSGBOX):

$Target/Property[Type="Microsoft.BizTalk.Server.2010.BizTalkRuntimeRole"]/MsgBoxDbServerName$ BizTalkMsgBoxDb "SELECT nvcErrorDescription FROM [BizTalkMsgBoxDb].[dbo].[InstancesSuspended] with(nolock) JOIN [BizTalkMsgBoxDb].[dbo].[ServiceClasses] as serviceclasses ON uidClassID = serviceclasses.uidServiceClassID where serviceclasses.nvcName = 'Orchestration' and datediff(minute,dtSuspendTimeStamp,getutcdate()) between 5 and 15"

Also, when using the MSGBOX example don’t forget to change your Alert target to BizTalk Run-Time Role:

image

Now you need to create a second rule to check for this eventID in the eventlog so an alert is triggered per suspended orchestration. We have one rule that retrieves all custom alerts:

image

You can find lots of examples to do this on the internet: http://technet.microsoft.com/en-us/library/bb309568.aspx

This is the line in the script that writes to the eventlog, you will need to check for the eventid you write in the script. You can also write some result information of your query in the alert:

Call oAPI.LogScriptEvent("check_suspended_orchestrations.vbs",12223, EVENT_TYPE_Error, "An orchestration is suspended with following error :" & rst.fields.item(0) & " and is active for longer than 5 minutes. Check the BizTalk environment!")

 

This is the result, an emailed alert per suspended orchestration containing the error:

Alert: Check Eventlog for Custom SCOM Rules [Check alert content for more details]

Source: BizTalkMgmtDb.BELANSQLBTSPRDS\BTS

Path: BELANBTSPRD1.Ghent.corp.mycompany.com

Last modified by: System

Last modified time: 06/02/2014 15:48:52

Alert description: Event Description: check_suspended_orchestrations.vbs : An orchestration is suspended with following error :Uncaught exception (see the 'inner exception' below) has suspended an instance of service 'Mycompany.MyApplication.Ghent.Processes.CreateTimeOutAlerts(8b2addb3-0531-9078-786b-a96d77224319)'.

The service instance will remain suspended until administratively resumed or terminated.

If resumed the instance will continue from its last persisted state and may re-throw the same unexpected exception.

InstanceId: ee15bbad-e4a8-40fd-ad16-46e3cc6f1f81

Shape name: Save Alert

ShapeId: 2652d27b-61fc-47b9-84ad-eca188d776bf

Exception thrown from: segment 1, progress 62 Inner exception: Exception in SaveAlert

Exception type: ApplicationException

Source: MyCompany.MyApplication.Helpers

Target Site: Void Add(Int32, System.String, System.String) The following is a stack trace that identifies the location where the exception occured

at MyCompany.MyApplication.Helpers.AlertHelper.Add(Int32 stateId, String type, String status)

at MyCompany.MyApplication.Ghent.Processes.CreateTimeOutAlerts.segment1(StopCond and is active for longer than 5 minutes. Check the BizTalk environment!

 

This was tested on a SCOM 2012 SP1 environment using BizTalk Server 2010. We expect this to work on older/newer versions of SCOM and BizTalk, but at this moment it hasn’t been validated by Codit yet. Also note that you should be using an unsealed management pack.

Posted in: BizTalk | Monitoring

Tags: , ,


December 3, 2013 at 4:00 PM

For an environment I recently installed at the customer, we experienced some weird behavior with the BizTalkMsgBoxDb.
After some time (or whenever a failover happened) the BizTalkMsgBoxDb entered in recovery pending mode.
 

image

This had an impact on the entire environment, making the BizTalk environment unavailable.

 

 

The Problem

After some investigation on my side, I passed this problem to the infrastructure engineer to investigate the root cause of this problem.

The infrastructure team discovered that this was a problem related to the shared disk setup on the cluster environment.

 

The environment at the customer was setup in the following manner:

One virtual cluster with two virtual servers (each on a different VMware ESX version), with a shared cluster disk (VMware VMFS).

image

The actual problem with this setup is that it is NOT supported by Microsoft and actually does not work properly. It will work for some time until a failover happens or the passive node checks the status of the cluster disk. When one of those 2 actions happened on the cluster environment, the BizTalkMsgBoxDb entered ‘Recovery Pending’ mode.

 

For above configuration the VMware KB 1037959 states:

Supported in Cluster in a Box (CIB) configurations only. For more information, see the Considerations for Shared Storage Clustering section in this article.

 

So the above configuration would only work if the cluster of virtual machines is setup on a single host with the same ESX/ESXi and connected to the same storage.
If we would apply the above configuration (Cluster in a Box), this would only protect us against failures at operating system and application level, but not against hardware failures, so this option was out of the question.

 

The Solution

The solution to the problem described above was rather simple: move from an unsupported environment to a supported environment.

This was done by replacing the VMFS protocol with the RDM (Raw Device Mapping) protocol.

image

 

RDM is a mapping file in a separate VMFS volume that acts as a proxy for a raw physical storage device. The RDM allows a virtual machine to directly access and use the storage device. The RDM contains metadata for managing and redirecting disk access to the physical device.

The file gives you some of the advantages of direct access to a physical device while keeping some advantages of a virtual disk in VMFS. As a result, it merges VMFS manageability with raw device access.

 

More information on supported configurations for Microsoft Clustering on VMware vSphere can be found there: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1037959

 

Cheers,

Glenn Colpaert


November 25, 2013 at 9:21 AM

I'm just back to Belgium after the BizTalk Summit 2013 held  in Seattle on November 21st/22nd.  With this post I would like to share what we learned during those two days.

 

Day #1 - One integration stack for the future

In the long term Microsoft is moving to an unified integration platform that provides consistency across the products and across multiple services. A symmetric platform between cloud and on-premise.

 

BizTalk Services

Microsoft announced the general availability of Windows Azure Biztalk services as mentioned by Scott Guthrie.

With the GA some portal enhancements were delivered. For example: the possibility to manage multiple services with a single LiveID and some UI modifications.  EDI is now available in all editions of WABS and very soon the EDIFACT support will be added.

 

Basically, the things Microsoft will be investing on the coming 12 months for BizTalk Services are :

  • EAI :  Adapter extensibility,  Reading from queues/topics and derived type support in transformations.
  • B2B : custom code in bridges
  • Management : integration with AD for endpoint security, scheduled backup and restore.
  • BPM : This is a significant step of the Microsoft cloud integration strategy.
    • The .net  Workflow Foundation will be the workflow engine
    • Unified tool experience for BAM, Rule designer and Process Modeler
    •  BPMN is something they have committed to
    • Out of the box activities and templates
    • Extensibility : where partners will add value to the platform. Custom activities will be then available through the Windows Azure store.

 

The audience had a special interest about BPM but not much details were disclosed yet.

 

BizTalk Server 2013 R2

 

  •  This version has been announced for H2 of 2014
  •  They plan to release a minor version (R2) every second year, focusing on replatforming and minor updates, where they want to have major releases every other year
  • This version will contain the following
    •  JSON support for REST adapter
    • More authorization support for Service Bus adapter
    • Replatforming (VS2013, SQL2014, Windows2012R2)
    • Healthcare adapter improvements

 

 

Day #2

The agenda was divided in two tracks. One mainly focused to deep dive sessions about WABS and Service bus, the other dedicated to partner's products and best practices.
I attended - and personally appreciated - the sessions delivered by Ziv Rafalovich and Sam Vanhoutte; especially for the practical aspects, the similarities, the tips and hints.

Ziv explained, among the others things,  the details of the Service Bus event based programming model, the SAS authentication and the Service Bus partitioned entities.

Sam showed different integration scenarios as Identity Integration (claim based authorization), Data integration (SQL Data Sync), Application integration (Service Bus & WABS) and Network Integration (VPNs).

 

 

 Note : All the sessions were recorded and are available on the BizTalk Summit 2013 page on Channel9!

 

 
Here are some pictures of the event:
 

Day#1 : Scott Guthrie on Delivering Modern Enterprise Applications

 

 

Day#1 : HCA went from 0 BTS (18 months ago) to 36 (today). They process 26 million messages a day.

 

 

 

Day#2 : Sam Vanhoutte talking about Integration patterns

 

 

Posted in: BizTalk | WABS

Tags: , ,


November 21, 2013 at 4:00 PM

According to MSDN, the WCF Adapter copies SOAP headers to the ‘InboundHeaders’ context properties:
“The WCF adapters copy custom SOAP headers and standard SOAP headers in the inbound messages to the WCF.InboundHeaders property.”

This is true in most cases, but as I learned during the past days, this doesn’t work as described in all cases.

Let’s try with a very basic scenario – A client application sends a custom header with each call:

using (var scope = new OperationContextScope(proxy.InnerChannel)) 
{
    var header = MessageHeader.CreateHeader("peterheader", "http://peter.com", "myvalue", false);
                        
    OperationContext.Current.OutgoingMessageHeaders.Add(header);


    BTSServcieHttp.Request req = new BTSServcieHttp.Request();
    req.Param1 = "1";

    var resp = proxy.GetInfo(req);

    MessageBox.Show(resp.Result);
}

In BizTalk the incoming messages are tracked to monitor the content of 'InboundHeaders' as it is set by the WCF adapter.
A call from the client with the code shown above results in the following content for 'InboundHeaders':

<headers>
	<a:Action>GetInfo</a:Action>
	<peterheader xmlns="http://peter.com">myvalue</peterheader>
	<a:MessageID>urn:uuid:bb6d87cc-90b5-4ea7-8995-c5cd9060f74b</a:MessageID>
	<ActivityId CorrelationId="507d4c75-9d64-49f5-a095-f227437251fd">bbb9f44e-65b5-4d95-b238-493f4b54b0b3</ActivityId>
	<a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo>
	<a:To>replyaddress</a:To>
</headers>

*** Note: For the sake of readability, I removed all namespaces and security headers from the 'headers' section.

The behavior above is exactly as expected, the WCF adapter writes our header to the InboundHeaders property.

Let's slightly change our basic scenario to this:

using (var scope = new OperationContextScope(proxy.InnerChannel)) 
{
    var header = MessageHeader.CreateHeader("peterheader", "http://peter.com", "myvalue", false, "myActor");
                        
    OperationContext.Current.OutgoingMessageHeaders.Add(header);


    BTSServcieHttp.Request req = new BTSServcieHttp.Request();
    req.Param1 = "1";

    var resp = proxy.GetInfo(req);

    MessageBox.Show(resp.Result);
} 

The only thing I added was an extra parameter to the 'CreateHeader' method. This added parameter represents the 'Actor' for the SOAP header.

This is the content of 'InboundHeaders' for the call with an Actor set:

<headers>
	<a:Action>GetInfo</a:Action>
	<a:MessageID>urn:uuid:5c4c50f8-76dd-4bf9-83dc-7af7cc9af164</a:MessageID>
	<ActivityId CorrelationId="15cf87f2-6bb0-4b5e-aefb-a644d55836b7">f548f1b7-bc29-4102-a986-2f9dacb92202</ActivityId>
	<a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo>
	<a:To>replyaddress</a:To>
</headers>

As you can see, the custom header is now not written to the 'InboundHeaders' property.
This is not exactly what is documented on MSDN about 'InboundHeaders'. All standard and custom headers should be written to 'InboundHeaders'. Just adding an 'Actor' to our header will break this behavior.

This observation made me very curious to know why exactly this is happening.
To find the exact reason, I started digging in the WCF Adapter by decompiling the 'Microsoft.BizTalk.Adapter.Wcf.Runtime' assembly.

In this assembly I located the 'CopyHeadersToContext' method. This method is responsible to copy the SOAP headers to the 'InboundHeaders' property.
This is the 'CopyHeadersToContext' method implementation:

private static void CopyHeadersToContext(Message wcfMessage, IBaseMessageContext btsMessageContext)
{
    StringBuilder stringBuilder = new StringBuilder();
    stringBuilder.Append("<headers>");
    foreach (MessageHeaderInfo messageHeaderInfo in wcfMessage.Headers)
    {
        int header = wcfMessage.Headers.FindHeader(messageHeaderInfo.Name, messageHeaderInfo.Namespace);
        if (header >= 0)
        {
            using (XmlReader xmlReader = (XmlReader)wcfMessage.Headers.GetReaderAtHeader(header))
            {
                string str = xmlReader.ReadOuterXml();
                btsMessageContext.Write(messageHeaderInfo.Name, messageHeaderInfo.Namespace, (object)str);
                stringBuilder.Append(str);
            }
        }
    }
    stringBuilder.Append("</headers>");
    string str1 = ((object)stringBuilder).ToString();
    btsMessageContext.Write(WcfMarshaller.inboundHeadersProp.Name.Name, WcfMarshaller.inboundHeadersProp.Name.Namespace, (object)str1);
}

As you can see in the code above, the WCF Adapter loops over the wcfMessage.Headers collection and adds them to a stringbuilder. This stringbuilder is finally written to the 'InboundHeaders' property.
The key line here is the wcfMessage.Header.FindHeaders() method call. Only headers that return a non negative result are added to the 'InboundHeaders'.

Let's have a look at the implementation of FindHeader:

public int FindHeader(string name, string ns)
{
    //lines removed for readability
    return this.FindNonAddressingHeader(name, ns, this.version.Envelope.UltimateDestinationActorValues);
}

FindHeaders calls FindNonAddressingHeader. That method is implemented like this:

private int FindNonAddressingHeader(string name, string ns, string[] actors)
{
    int num = -1;
    for (int i = 0; i < this.headerCount; i++)
    {
        if (this.headers[i].HeaderKind == MessageHeaders.HeaderKind.Unknown)
        {
            MessageHeaderInfo headerInfo = this.headers[i].HeaderInfo;
            if (headerInfo.Name == name && headerInfo.Namespace == ns)
            {
                for (int j = 0; j < actors.Length; j++)
                {
                    if (actors[j] == headerInfo.Actor)
                    {
                        if (num >= 0)
                        {
                            if (actors.Length == 1)
                            {
                                throw DiagnosticUtility.ExceptionUtility.ThrowHelperError(new MessageHeaderException(SR.GetString("MultipleMessageHeadersWithActor", new object[]
                                                                                                                                {
                                                                                                                                                name,
                                                                                                                                                ns,
                                                                                                                                                actors[0]
                                                                                                                                }), name, ns, true));
                            }
                            throw DiagnosticUtility.ExceptionUtility.ThrowHelperError(new MessageHeaderException(SR.GetString("MultipleMessageHeaders", new object[]
                                                                                                                {
                                                                                                                                name,
                                                                                                                                ns
                                                                                                                }), name, ns, true));
                        }
                        else
                        {
                            num = i;
                        }
                    }
                }
            }
        }
    }
    return num;
}

What this function basically does is 'finding' a header based on its name, namespace AND actor! The actor of the namespace is compared with a collection of actors that is passed into the function as a parameter (string[] actors). Only headers that have an actor matching the actors collection will give a positive result.

So.. the next step is finding out what this collection of actors contains...
The parameter that was passed in by the 'FindHeader' method was 'this.version.Envelope.UltimateDestinationActorValues'.
The question is what is the content of that property. The only location where the content of that property is assigned is in the constructor of the EnvelopeVersion:

private EnvelopeVersion(string ultimateReceiverActor, string nextDestinationActorValue, string ns, XmlDictionaryString dictionaryNs, string actor, XmlDictionaryString dictionaryActor, string toStringFormat, string senderFaultName, string receiverFaultName)
{
   //code remove for readability
        this.ultimateDestinationActorValues = new string[]
        {
            "",
            ultimateReceiverActor,
            nextDestinationActorValue
        };
    }
}

Here we find the reason for what is happening in our two basic scenarios. The actors collection that is passed to the FindHeader function contains three strings: "", ultimateReceiverActor, nextDestinationActorValue.

Because "" is in this collection, having no actor in the header will make sure it is passed to 'InboundHeaders'.
But when an actor is set, and it doesn't matches the ultimateReceiverActor or nextDestinationActorValue, then the 'FindNonAddressingHeader' method return -1! This results in a lost SOAP header.

*** Note: ultimateReceiverActor and nexDestinationActorValue are assigned a fixed value by the parameter less constructor of MessageVersion.

Conclusion

After digging into the implementation of the WCF Adapter, we can perfectly explain why SOAP headers containing an actor are not written to 'InboundHeaders'. However, this strange behavior doesn't seem to be like intended. I would love to get Microsoft feedback on this issue.

Peter Borremans

Posted in: .NET | BizTalk | WCF

Tags: , , ,


November 13, 2013 at 4:10 PM

For one of our BizTalk Projects we integrate with SharePoint to track messages and make them available to the users.

All messages processed in our application share a common set of context properties that need to be available as column values. And, apart from that, we want to place the messages in different folders in the document library to configure security settings per folder.

In order to do this we just subscribe on our messages published to the MessageBox. Then we configure the library folder, column values and properties in a custom pipeline component as seen below. 

Configuring the destination folder

In our first attempt to set the correct destination we overwrote the ‘OutboundTransportLocation’ in the System Properties namespace but it turns out this property is completely ignored.

To update the destination you actually need to overwrite the ‘AdapterConfig’ property in the ‘http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties’ namespace.

The value of the property is an XML string that looks like the one below:

 

 

 To update the folder you write to, get the WssLocation node (This is the path relative to the Site), overwrite it and write the property back to the context :

adapterconfig = (string)pInMsg.Context.Read("AdapterConfig", "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties");
Xdocument xConfig = XDocument.Parse(adapterconfig);
var xWss = xConfig.Descendants("WssLocation").Single();
xWss.Value = string.Format("{0}/{1}", xWss.Value, sFolder);
adapterconfig = xConfig.Root.ToString(SaveOptions.DisableFormatting);
InMsg.Context.Write("AdapterConfig", "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", adapterconfig);

 

 

Configuring the content

To set the values for the different columns you need to overwrite the ‘ConfigPropertiesXml’ property in the

‘http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties’ namespace.

The value of the property is an XML string that looks like the one below:

 

 

All the properties we want to publish in SharePoint share the same namespace. We get them using the following extension method:

public Dictionary<string, string> GetPropertiesByNamespce(this IBaseMessageContext context, string popertyNamespace)
{
    Dictionary<string, string> dProps = new Dictionary<string, string>();
    string pName;
    string pNamespace;
    for (int i = 0; i < context.CountProperties; i++)
    {
        context.ReadAt(i, out pName, out pNamespace);
        if (pNamespace == popertyNamespace)
        {
            Object oVal = context.Read(pName, pNamespace);
            if (oVal != null)
            {
                string sVal = Convert.ToString(oVal);
                if (sVal != string.Empty)
                    dProps.Add(pName, context.Read(pName, pNamespace).ToString());
            }
        }
    }
    return dProps;
}

 

We construct the adapterconfig property value using the method above, just make sure not to use more than the maximum (16) number of properties here:

private static string CreateWSSContextValueString(Dictionary<string, string> configProperties)
{
    int i = 0;
    XElement xConfigProps = new XElement(WSSContextName);
    foreach (string pName in configProperties.Keys)
    {
        i++;
        xConfigProps.Add(new XElement("PropertyName" + i.ToString(), pName));
        xConfigProps.Add(new XElement("PropertySource" + i.ToString(), configProperties[pName]));
    }
    return xConfigProps.ToString(SaveOptions.DisableFormatting);
}

and write this value to the context:

InMsg.Context.Write("ConfigPropertiesXml", "http://schemas.microsoft.com/BizTalk/2006/WindowsSharePointServices-properties", sConfigProperties);

 

Conclusion

As a result all Messages end up in the following library folders:

clip_image001

 

 Inside the folder:

clip_image003

 

With the solution in place and with the capabilities of the platform, SharePoint turned out to be an interesting place to publish tracing information.

Security in SharePoint lets us show the information to the correct users in a format that is easy to understand and it gives us the possibility to present the data in other ways (i.e. Matching incoming and outgoing data) to provide extra business value.

 


October 16, 2013 at 4:00 PM

Sometimes you need to create one mappings for several different message types. Like for instance if you have a message body that reoccurs in every kind of message. Or you need to add an extra node to a message. Recently i needed to remove the namespaces and the prefixes of the incoming messages. I had 10 types of messages and i wanted to create one mapping to do this.

 

XLANGs Any type

To do this, I used the Any message type as an input schema and output schema for my mapping. You can find this in the XLANGs.BaseType assembly.

 

Add Reference

First add a reference from your project to the Microsoft.XLANGs.BaseTypes.dll

image

 

BizTalk Mapper

When you create a new mapping, you can select the Any type in the BizTalk Type picker as input and output schema.

image

The BizTalk Mapper should look like this.

image

 

 

 

Then I used a custom xslt to do the mapping i wanted to do, in my case, the removal of the namespace by copying the local name off every node instead of copying the namespace and namespace prefix along with it.

 

Usage of the mapping

You can’t use a transform shape inside a construct shape if you don’t have a message of the Any type already . For this reason, I use a dynamic mapping inside a message assignment shape because the message type of my input is an XmlDocument.

 

varMapName = Helper.GetMapNameByMessage(msgIncomming);

// Get the map type based on the map name.
varMapType = System.Type.GetType(varMapName);

// Perform the mapping.
transform(msgOutput) = varMapType(msgIncomming);

 

Deployment

Make sure you have following assembly in your BizTalk Application Resources:

image

 

 

 

 

Posted in: BizTalk | Mapping | Schemas

Tags: , ,


October 3, 2013 at 4:58 PM

I have given quite some sessions and presentations recently on Windows Azure BizTalk Services.  We had the opportunity to work together with the product team on WABS as a strategic Launch Partner.  I had a lot of discussions and questions on the technology and a lot of these questions were focused on a comparison with BizTalk Server…

Therefore, I decided to create this blog post, I created a comparison table, much like the comparison tables you can see on consumer web sites (for mobile phones, computers, etc…)

If you need more information or have some feedback, don’t hesitate to contact me.

In the mean time, I also came across the following wiki on msdn, that does a good overview too: click to open

Connectivity & adapters

Some adapters are not applicable in cloud services (File, for example), where others are.  BizTalk Services has a lot of adapters not available.  Custom adapters can only be written through the outbound WCF bindings.

  BizTalk Server BizTalk Services
File Yes No (cloud only)
FTP Yes Yes
SFTP Yes Yes
SOAP Web services Yes Yes
RESTful services Yes, no JSON Yes, no JSON
HTTP Yes HTTP
Email Yes Through custom WCF binding
Outbound only
SQL Yes Through adapter service
SAP Yes Through adapter service
Siebel Yes Through adapter service
Oracle DB Yes Through adapter service
Oracle Apps Yes Through adapter service
SharePoint Yes Through custom WCF binding
Outbound only
MQSeries Yes No
Service Bus Yes Outbound only (Relay + Messaging)
Azure Blob storage Through custom adapter Yes

Core messaging capabilities

The biggest difference here is the routing pattern that is totally different between both products.  More can be read in an earlier post: Windows Azure Bridges & Message Itineraries: an architectural insight.  If you want durable messaging, Service Bus queues/topics are the answer, but the biggest problem is that WABS cannot have subscriptions or queues as sources for bridges.

 

BizTalk Server

BizTalk Services

Durable messaging

Yes, MessageBox

Only through service bus and custom polling

Volatile messaging

No, always persistence

Yes

One-to-many routing

Yes

No, first match routing. Only through service bus topics

Property promotion

Yes

Yes

Retry mechanism

Yes

No

Alternative routing

Yes

No

Failed Message Routing

Yes

No

 

Message processing

We have good feature parity here on these items.  The main thing missing would be JSON support.  For that, we have written a custom component already that supports JSON.

 

BizTalk Server

BizTalk Services

Transformation

Yes

Yes

Schema Validation

Yes

Yes

Property promotion

Yes

Yes

Flat file processing

Yes

Yes

XML processing

Yes

Yes

JSON processing

No

No

Binary processing / routing

Yes

No, maybe in passthrough (to be checked)

EDIFACT processing

Yes

Coming soon

X12 processing

Yes

Yes

Management & deployment experience

In my opinion, this is where the biggest challenge lies for WABS.  Administration and management is really not what we are used with BizTalk Server.  Configuration is very difficult (no binding file concept) and endpoint management is also not that easy to do.

 

BizTalk Server

BizTalk Services

Endpoint management

Yes

Only powershell

View message tracking

Yes

Basic

Deployment portal

Yes

Only powershell, Visual Studio

Auditing of operations

Partial

Partial

Health dashboard

Yes

No

Isolation of processes

Yes (hosts)

No, single tenant

Hierarchy of artifacts

Yes (applications)

No

Trading partner management & EDI

The TPM portal of WABS is really very nice and much friendlier than the BizTalk admin console of BizTalk.  The biggest issue with EDI is the fact that there is no possibility to extend and customize the EDI bridge…

 

BizTalk Server

BizTalk Services

Trading partner management

Yes (Biztalk Admin)

Yes (TPM Portal)

EDI extensibility

Yes

No (only with bridge as destination)

Extensibility

Luckily the product team did good efforts to add extensibility and the usability of custom bridge components should still be evolved well.

 

BizTalk Server

BizTalk Services

Custom pipeline components

Yes

Yes

Custom functoids

Yes

Yes

Custom adapters

Yes

Only outbound WCF

Custom business rules components

Yes

No

Custom workflow components

Yes

No

Security

 

BizTalk Server

BizTalk Services

Role based security in management portal

Yes

No

Endpoint security

Extensive support

Only ACS & FTP security

Added value services

This is really where BizTalk Server leads, compared to WABS.  And for most real solutions, these services are often needed.

 

BizTalk Server

BizTalk Services

Workflow

Yes (orchestrations)

No

Business Rules

Yes

No

Business Activity Monitoring

Yes

No

RFID

Yes

No

UDDI

Yes

No

Posted in: Azure | BizTalk | WABS

Tags: , ,


September 27, 2013 at 3:50 PM

On september 19th, our very own Sam Vanhoutte (CTO Codit) gave a presentation at CloudBurst, the Swedish Windows Azure Group Developer conference. In his session, Sam elaborated on connecting the cloud with your local applications. The session has been recorded and made available on your Codit blog (see below) and on the Microsoft's Channel 9 website.

Content of this session 

In new scenarios, in which cloud is getting used, integration becomes very important. The Windows Azure platform provides a lot of different capabilities and services to make a secure link between your local systems and the Windows Azure services or machines. During this session, you will discover what the different technologies are and in what these are best applicable. You will learn more about the following technologies:

  • Connectivity on messaging level: Service Bus Messaging
  • Connectivity on service level: Service Bus Relay
  • Connectivity on data level: SQL Data Sync 
  • Connectivity on network level: Windows Azure Virtual Networking 
  • Connectivity on security level: Active Directory integration
 
 
Enjoy the video!
 
 

Channel 9 page: http://channel9.msdn.com/Events/Cloud-Burst/Cloud-Burst-2013/Connecting-the-Cloud-with-your-local-applications

 

 

 

Posted in: Azure | Service Bus

Tags: , ,