A few days ago I received the following error while restarting the BizTalk host instances on one of the BizTalk servers here after an update.

I found the following in the event log:

I was pretty shocked about this and immediately tried with other host instances on the same machine.

All failed to start with the same error in the event log and a popup error which doesn’t reveal much:

I first thought there was an issue with an underlying service (ESSO or SQL) but all of them were happily running and working fine it seemed.

Finally this forum post gave me a glimpse on what might be wrong. Someone mentions:
“Check out the configuration file for the 64-bit to ensure there are no custom entries. Let me know.”

Indeed, I had adapted the BTSNTSvc.exe.config and BTSNTSvc64.exe.config files with some custom appSettings entries.

The entry I added was something like this:

    <appSettings>
        <add key="Link" value="http://server/site/page.aspx?page=1&Id={0}"/>
    </appSettings>

I renamed the file to have an xml extension to see if it would validate in IE and it did. So it seemed to be valid XML.

Knowing that the ampersand (&) sign can cause some issues in XML I removed it and saved the file.
Indeed, the host instances started without an issue now…

So I escaped the character like this:

<appSettings>
<add key="Link" value="http://server/site/page.aspx?page=1&amp;Id={0}"/>
</appSettings>

And now the hosts have no issue starting up again!

I hope this helps someone, because I did not find a blog post anywhere about this fairly common issue it seems.
Leave a message if it does!

For a recent project I had to work with Windows Server AppFabric Caching. One of the requirements was to retrieve the CacheStatistics from the AppFabric cache.

To retrieve these statistics you have to use PowerShell by using following commands:

Offcourse we needed this data in our .NET Application and not in PowerShell.
We needed to call the PowerShell cmdlets from our .NET Application to be able to retrieve that data. This is how we did it.

First of all add a reference to the following assemblies:

The next step is to initialize the PowerShell RunSpace and initialize all the necessary values and objects as shown below:

When the pipeline is created we can start sending commands; Always start your command sequence with the "Use-CacheCluster" command.
In the code below you can see how we actually retrieved the CacheStatistics from the AppFabric Cache.

For a full list of the available AppFabric Caching PowerShell cmdlets please see following links:

http://msdn.microsoft.com/en-us/library/hh475806
http://msdn.microsoft.com/en-us/library/hh851388

Enjoy!

Glenn

With the introduction of Linq in .NET 3.5, the task of querying data became a lot simpler. In BizTalk, we often work with xml messages, and linq also supports querying over xml documents using the XDocument class.

In a pipeline component, we can pass the incoming message stream into an XDocument. This gives us all the flexibility of LINQ, but with the consequence of having to load the complete message in-memory. When working with large messages, this is probably not the best option, so we will need an alternative solution. 

In this blogpost we will focus on reading incoming messages with linq. When you want to compose/create a new xml message using linq in a streaming manner, there’s a straigthforward way to do that. Instead of creating XElements, you will want to work with XStreamingElements. More information about this topic can be found on the XStreamingElement msdn page. 

For illustration purposes, we need a large xml file. I’ve used the AdventureWorks database, selected all available customers, and converted the resultset to xml via XML AUTO. This resulted in a fairly large xml file containing around 18k customers. To be sure, I’ve duplicated the dataset a few times so I have an xml file of 130k customers, with a file size of 134MB.

A small excerpt: 

<Customers>
 <Customer CustomerID="11377" Title="Mr." FirstName="David" MiddleName="R." LastName="Robinett" Phone="238-555-0100" EmailAddress="david22@adventure-works.com" EmailPromotion="1" AddressType="Home" AddressLine1="Pappelallee 6667" City="Solingen" StateProvinceName="Nordrhein-Westfalen" PostalCode="42651" CountryRegionName="Germany"> 
  <Demographics> 
    <IndividualSurvey xmlns="http://schemas.microsoft.com/sqlserver/2004/07/adventure-works/IndividualSurvey"> 
      <TotalPurchaseYTD>83.97</TotalPurchaseYTD> 
      <DateFirstPurchase>2003-09-01Z</DateFirstPurchase> 
      <BirthDate>1961-02-23Z</BirthDate> 
      <MaritalStatus>M</MaritalStatus> 
      <YearlyIncome>25001-50000</YearlyIncome> 
      <Gender>M</Gender> 
      <TotalChildren>4</TotalChildren> 
      <NumberChildrenAtHome>0</NumberChildrenAtHome> 
      <Education>Graduate Degree</Education> 
      <Occupation>Clerical</Occupation> 
      <HomeOwnerFlag>1</HomeOwnerFlag> 
      <NumberCarsOwned>0</NumberCarsOwned> 
      <CommuteDistance>0-1 Miles</CommuteDistance> 
    </IndividualSurvey> 
  </Demographics> 
</Customer> 
<Customer CustomerID="11913" Title="Ms." FirstName="Rebecca" MiddleName="A." LastName="Robinson" Phone="648-555-0100" EmailAddress="rebecca3@adventure-works.com" EmailPromotion="0" AddressType="Home" AddressLine1="1861 Chinquapin Ct" City="Seaford" StateProvinceName="Victoria" PostalCode="3198" CountryRegionName="Australia">
...

The pipeline component 

In our simple scenario, we want to promote the average purchase amount of all customers residing in one of a select list of countries. We want to do this in a simple pipeline component, using linq.

The list of countries is configurable as a comma-seperated property:

private string _countries; 
public string Countries 
{ 
    get 
    { 
        return _countries; 
    } 
    set 
    { 
        _countries = value; 
    } 
}

Other than that, it’s a basic pipeline component, with most of its logic located in the Execute method. 

Parsing using XDocument

The easiest way to use linq functionality is to just use the XDocument class. We could implement it like this. 

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pContext, Microsoft.BizTalk.Message.Interop.IBaseMessage pInMsg) 

{ 
    // Get the message stream   
    Stream bodyStream = pInMsg.BodyPart.GetOriginalDataStream();

    // If the stream can't seek, wrap it in a seekable stream 
    // so we can reset the position to 0 again 
    if (!bodyStream.CanSeek) 
    { 
        bodyStream = new ReadOnlySeekableStream(bodyStream); 
        pContext.ResourceTracker.AddResource(bodyStream); 
    } 
    // Read stream into an XDocument 
    XmlReader reader = XmlReader.Create(bodyStream); 
    XDocument document = XDocument.Load(reader);

    XNamespace surveyNS = "http://schemas.microsoft.com/sqlserver/2004/07/adventure-works/IndividualSurvey";

    // Get the average purchase amount for the selected countries 
    string[] selectedCountries = Countries.Split(';'); 
    decimal averagePurchaseAmount = document.Root.Elements("Customer").Where(el => selectedCountries.Contains(el.Attribute("CountryRegionName").Value)).Average(el => Convert.ToDecimal(el.Element("Demographics").Element(surveyNS + "IndividualSurvey").Element(surveyNS + "TotalPurchaseYTD").Value));

    // Promote the result 
    pInMsg.Context.Promote("AveragePurchaseAmount", "http://schemas.codit.eu/blog/2012/06", averagePurchaseAmount);

    bodyStream.Position = 0; 
    pInMsg.BodyPart.Data = bodyStream;

    return pInMsg; 
}

The linq query selects all customers, checks to see if the country is one of the specified countries, and then calculates the average based on the Demographics/IndividualSurvey/TotalPurchaseYTD element. At the end, we promote the resulting average, and reset the stream position to 0, so other pipeline components can use the stream again.

When testing, we can see the promotion is done correctly, and the pipeline component works. However, if we launch perfmon to analyze the memory usage, we can see the issue:

The memory usage goes up dramatically each time our pipeline is called (3 times).We top at 385MB consumed, and this is for 1 message going through. Once multiple or larger files are coming in, we might run into memory problems.

Parsing using a custom axis method

An alternative solution is to use a so called custom axis method. This method is responsible for returning a collection of elements (typically XElements) by using yield return. This provides deferred execution, which enables us to stream over the elements.

In the previous example we used this to load the message:

XmlReader reader = XmlReader.Create(bodyStream); 

XDocument document = XDocument.Load(reader);

and used our query like

document.Root.Elements("Customer").Where( … )

We will need to create an axis method for getting all the customers. We can then use Linq queries on this method. Here’s the implementation of our  custom axis method:

static IEnumerable<XElement> StreamElements(Stream stream, string nodeName) 

{ 
    using (XmlReader reader = XmlReader.Create(stream)) 
    { 
        while (reader.Read()) 
        { 
            if ((reader.NodeType == XmlNodeType.Element) && reader.Name == nodeName) 
            { 
                XElement element = (XElement)XElement.ReadFrom(reader); 
                yield return element; 
            } 
        } 
    } 
}

While reading, we check if the name of the current node is equal to the one we specified in our method description. If that is the case, we read the full node and yield return this.

Next to adding this axis method, we will need to modify our pipeline component to utilize the method instead of the using an XDocument:

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pContext, Microsoft.BizTalk.Message.Interop.IBaseMessage pInMsg) 

{ 
    // Get the message stream   
    Stream bodyStream = pInMsg.BodyPart.GetOriginalDataStream();

    // If the stream can't seek, wrap it in a seekable stream 
    // so we can reset the position to 0 again 
    if (!bodyStream.CanSeek) 
    { 
        bodyStream = new ReadOnlySeekableStream(bodyStream); 
        pContext.ResourceTracker.AddResource(bodyStream); 
    } 
    XNamespace surveyNS = "http://schemas.microsoft.com/sqlserver/2004/07/adventure-works/IndividualSurvey";

    // Get the average purchase amount for the selected countries 
    string[] selectedCountries = Countries.Split(';'); 
    decimal averagePurchaseAmount = StreamElements(bodyStream, "Customer").Where(el => selectedCountries.Contains(el.Attribute("CountryRegionName").Value)).Average(el => Convert.ToDecimal(el.Element("Demographics").Element(surveyNS + "IndividualSurvey").Element(surveyNS + "TotalPurchaseYTD").Value));

    // Promote the result 
    pInMsg.Context.Promote("AveragePurchaseAmount", "http://schemas.codit.eu/blog/2012/06", averagePurchaseAmount);

    bodyStream.Position = 0; 
    pInMsg.BodyPart.Data = bodyStream;

    return pInMsg; 
}

We are no longer using an XDocument to load the stream, instead we forward this stream to our custom axis method so it can read and return XElements in a streaming way.

If we redeploy our pipeline, we can see the result in perfmon again (same scale as previous graph):

I have dropped the same file 3 times again, using the new pipeline component. It’s hard to notice when the files are dropped, but we can see the maximum memory used is 3MB (down from 385MB).

Remarks & Conclusion

The difference between using our custom axis method and just using XDocument is clear, the streaming component does its job with a significantly smaller memory footprint.

But that doesn’t mean we can just always use an axis method. In case of a large database table dump, log file, or other file with a relatively flat structure, it’s a good match since we can iterate over each single node individually, and each node is small. Once the xml structure gets more complex, it can be very difficult or impossible to create an efficient axis method.

Next to that, some Linq operators require all elements to be in memory to do its job. For example, when using OrderBy(), the memory usage will jump up because it needs to compare the current element of the iterator against the other elements which were processed already.

As a final note, we are reading the complete message in both scenario’s, and then reset the position to 0 again. This has the consequence that the message is read at least twice (once by our pipeline, once by the messaging engine). If we want to implement our components in a true streaming manner, we would need to create our own custom stream class. The issue then arises on how to, and when to call our linq queries, since we are no longer in control of when reads are happening. If you're interested in this scenario, it might be a good idea to look into the IObservable<T> and IObserver<T> interfaces introduced in .NET 4. Together with Reactive Extensions a solution might be available, but that is out of the scope of this blog entry.

As you all know, the BizTalk XML Disassembler checks - by default - if a schema is deployed that matches the incoming message.  BizTalk does this via the BTS.MessageType, which is a combination of TargetNamespace#RootNode.  This BTS.MessageType must be unique in your BizTalk Group.  If two schemas are deployed with the same TargetNamespace#RootNode, the XML Disassembler will throw this error message:

• Cannot locate document specification because multiple schemas matched the message type

Lately we had an issue, related to this subject.  We had two schemas deployed, that varied only by case.  Below, you find a similar example.

Now the question is: will BizTalk treat these schemas as identical schemas or not? My first guess was that BizTalk would treat these schemas as unique because it’s related to XML, which is case-sensitive.  However, when processing one of the two messages, the XML Disassembler threw an error because multiple schemas matched the message type.

I investigated how BizTalk resolved the schema, so I turned on SQL Server Profiler.  There, I discovered this entry:

• exec bt_GetDocSpecInfoByMsgType @nvcMsgtype=N'http://Codit.BizTalk.Blog.REQUEST#REQUEST',@iPipelineAssemblyId=2,@iFlag=@p3

The most important query of the GetDocSpecInfoByMsgType stored procedure is this one:

Because my SQL Server Collation, Latin1_General_CI_AS, is case-insensitive, the query results into 2 matched schemas, so the XML Disassembler will throw an exception.

After some searching on MSDN I’ve found this remark:

• If you will be using two namespaces that vary only by case, the BizTalk database must be installed with a case-sensitive collation. Examples of case-sensitive collations include binary and non-binary collations with case-sensitivity enabled. If this is not done, schema resolution will fail because XML is case-sensitive.

Conclusion: Is the BTS.MessageType case-sensitive or case-insensitive? It depends on your SQL Server Collation.

Toon Vanhoutte

The announcement

Because of the launch of the new Office version (read more here), there was also some interesting news for everyone interested in the Microsoft middleware… It was obviously not mentioned during the launch (which was off course consumer oriented), but as part of the new release, there are some interesting infrastructure components that are shipped. The service bus and the workflow team have shipped new bits that are available on premises and will be used by the new SharePoint version. This blog post dives deeper in the bits of the Service Bus for Windows Server.  (read here)

Codit has been actively involved as a TAP (technology adoption program) customer for the Service Bus for Windows Server and we can say that we now have full symmetry for Codit Integration Cloud with Codit Integration Server. The exact same configuration and modules that are running in the cloud are also running on premises, giving our customers the choice to deploy locally, in the cloud or in a hybrid model. 

Symmetry on the Microsoft cloud platform

The past years, we’ve seen Microsoft building a lot of things on the Windows Azure platform that they already had on premises. Some examples:

In all of the above cases, Microsoft brought capabilities (they had running on the Windows Server platform for years) from on premises to their cloud platform. And they always tried to make things as symmetric as possible (especially on the development model). But after reading the recent announcements where they are bringing Azure web sites, virtual machines and the management capabilities on premises (read more here), we now also see the first move in the other direction.  And with the recent Office announcements, Microsoft has also released Service Bus for Windows server, a symmetric on premises version of the powerful Windows Azure Service Bus messaging. This post digs deeper in the details of this beta release, but this move shows that Microsoft wants to be the software company that provides software and services, running on the environment of your choice, in the business model of your choice. This will also give a lot of flexibility to ISV’s and companies with a very distributed environment.

One of the biggest unknowns at this time, is how Microsoft will keep symmetry on server. The release cadence of server products versus cloud based services is just too different. New features are being added to the various services all the time, and they also have to ship to the server installations over time. I’m curious to find out what the release cycle will be for these updates. 

What's in the release

The announced release is the first public beta for Service Bus on Windows Server.  It is called beta 2.  The release of Service Bus for Windows server only contains the messaging capabilities of the Windows Azure Service Bus. This means there are no relaying capabilities (nor an on premises ACS service) added to this version. If you want to learn more about queues & topics/subscriptions, don’t hesitate to read more here:

• Service Bus Queues (how to)
• Service Bus Topics (how to)  

Scenarios

This server product can be used in a variety of scenarios.

Durable messaging only scenarios 

If you require to exchange messages in a local, messaging only scenario, you can perfectly use Service Bus for Windows server to deliver messages between applications and services in a durable and reliable fashion.

Store & forward scenarios

The Service Bus for Windows server release contains the possibility to define ForwardTo subscriptions on topics, so that messages that match the subscription of these rules, automatically get forwarded to the defined messaging entity. At this point it’s not possible to configure a ForwardTo that points to a remote entity, but one can work around that by having a subscriber that listens to a local ForwardTo entity and sends these messages to a public entity. 

Distributed scenarios 

A lot of organizations have different business units or plants that need to be interconnected. In a lot of companies (often after mergers and acquisitions), the technology that is used in these different plants is different. Therefore, it could be good to have Service Bus being used as the gateway to exchange messages between the different units, while every unit can use it’s standard of choice (REST, SOAP, .NET, AMQP…) to connect to that gateway. 

Installation

The bits are available in the Web Platform Installer (Service Bus 1.0 beta and Workflow 1.0 beta).

Prerequisites

• Service Bus for Windows server requires Windows Server 2008 R2 SP1 (x64) or Windows Server 2012 (x64). It also installs on Windows 7.
• SQL Server is needed to host the different entities & configurations. SQL Server 2008 R2 SP1 and SQL Server 2012 are supported (also with their Express editions)
• Only Windows Authentication is supported, meaning Active Directory is required in a multi-machine scenario.
• The Windows Fabric gets installed on the Windows system.

Everything is described in detail in the installation guide. 

Topologies

There are two typical topologies. A third could also be used, when having a dedicated SQL with one Service Bus server.

Configuration

To configure the Service Bus for Windows server, we needed to use PowerShell in the private beta. The following commands create a new service bus farm with a single server.

$mycert=ConvertTo-SecureString -string <Password> -force –AsPlainText
New-SBFarm -FarmMgmtDBConnectionString "data source=(local);integrated security=true" –CertAutoGenerationKey $mycert
Add-SBHost -certautogenerationkey $mycert -FarmMgmtDBConnectionString "data source=(local); integrated security=true"
Get-SBFarmStatus (shows three services running: Service Bus Gateway, Service Bus Message Broker, FabricHostSvc)

Create a service bus namespace

Creating a namespace happens by using the following statement, providing a administrative user and the name of the namespace.

New-SBNamespace -Name CoditBlog -ManageUsers Administrator

As a result, you can see the following information:

Name                  : CoditBlog   
AddressingScheme      : Path   
CreatedTime           : 10/07/2012 15:21:03   
IssuerName            : CoditBlog   
IssuerUri             : CoditBlog   
ManageUsers           : {Administrator}   
Uri                   :   
ServiceIdentifier     :   
PrimarySymmetricKey   : vnAJ7rHp###REPLACE FOR SECURITY####MOvH8Yk=  
SecondarySymmetricKey :

New features

Check for matching subscriptions with the PreFiltering feature

Everyone who ever tried to build a BizTalk application has to be familiar with the following error description: “Could not find a matching subscription for the message”. This was a very common exception, indicating that a message was delivered to the BizTalk messaging agent, while no one was going to pick up that message. The fact that this error was thrown really made sure we would not lose messages in a black hole.

Looking at the publish & subscribe capabilities that came with Windows Azure Service Bus, this was one of the first things I missed. If you submit a message to a topic that does not have subscriptions that match, the message does not get deadlettered, the client does not get a warning or error code and the message is lost. While in certain scenarios this can be desired, a lot more scenarios would find this dangerous. The good thing is that now we get a choice with the Service Bus Server.

A new feature is available that takes care of this. This is PreFiltering. To enable this, you need to configure this on the TopicDescription. And notice one of the longer property names in the Service Bus object model

nsManager.CreateTopic(new TopicDescription { Path = "PrefilterTopic", EnableFilteringMessagesBeforePublishing = true });

If we now submit a message to this topic that does not have a matching subscription configured, we’re getting the following exception.

Type: Microsoft.ServiceBus.Messaging.NoMatchingSubscriptionException   
Message: There is no matching subscription found for the message with MessageId %1%..TrackingId:%2%,TimeStamp:%3%.

As you can see, we can only configure this on a topic. I would have preferred to get the choice to configure this on the sender level. So that, even when sending to the same topic, the client could decide if he wants to have routing failures checked or not.

The complete sample code for this article can be downloaded at the bottom of this page.

ForwardTo: auto forwarding of messages to another entity

In our scenarios, we sometimes have the need to do archiving or auditing on incoming messages across all topics in our namespace. Until now, we have done that by creating a MatchAllFilter (1=1) on every topic and by creating a consumer for every subscription there. That off course takes up more threads and also more messaging transactions. This new feature is solving this for us. The ForwardTo allows us to define an automatic forwarding (transactional in the same namespace) action to another entity.

It’s important to understand that the subscription with ForwardTo enabled only acts as a ‘passthrough’ subscription and that you cannot read from it. The message automatically gets forwarded to its destination queue or topic. If you want to read from it, you get the following exception:

Can not create a message receiver on an entity with auto-forwarding enabled.

To specify a ForwardTo on a subscription, you need to specify this in the SubscriptionDescription, by setting the ForwardTo string property to the path of the destination entity, as can be found in the following snippet:

nsManager.CreateTopic(new TopicDescription("ForwardToTopic");   
nsManager.CreateSubscription(new SubscriptionDescription("ForwardToTopic", "ForwardAll") {ForwardTo = "AuditQueue"}, new SqlFilter("1=1");

This feature comes in very handy for audit/tap scenarios and aggregated message handling.

Sending messages in batch

In service bus, we already had the EnableBatchedOperations to optimize ‘transmission performance’. With this release, we can now send messages in a single batch to a service bus entity. All of that in one transaction. The following code writes a batch of messages to a topic with a filter that accepts all messages with a number different from 8. Since PreFiltering is enabled (see above, no matching subscription), we can easily simulate an exception in the batch, by playing with the loop values. When we make the loop stop before number 8 is hit, all messages are successfully delivered, when we include number 8 (like in the sample), we get the NoMatchingSubscriptionException and not a single message is being delivered.

All of this is achieved by using the SendBatch method of the MessageSender class.

string topicName = "PrefilterBatchTopic";
if (!nsManager.TopicExists(topicName))
{
	nsManager.CreateTopic(new TopicDescription(topicName) { EnableFilteringMessagesBeforePublishing = true });
	var preFilterSubscription = nsManager.CreateSubscription(new SubscriptionDescription(topicName, "NoNumberEight"), new SqlFilter("MessageNumber<>8"));
}

var batchSender = msgFactory.CreateMessageSender(topicName);
// Create list of messages
List<BrokeredMessage> messageBatch = new List<BrokeredMessage>();
for (int i = 0; i < 11; i++)
{
	BrokeredMessage msg = new BrokeredMessage("Test for subscription " + i.ToString());
	msg.Properties.Add("MessageNumber", i);
	messageBatch.Add(msg);
}
batchSender.SendBatch(messageBatch);

Receiving messages in batch

Just like we can send messages in a batch, we can also receive them in a batch. For this, we need to use the ReceiveBatch method of the MessageReceiver class. There are three overloads for this:

msgReceiver.ReceiveBatch(int messageCount);
msgReceiver.ReceiveBatch(IEnumerable<long> sequenceNumbers);
msgReceiver.ReceiveBatch(messageCount, serverWaitTime);

The first overload gets a batch of messages with a maximum of the specified messageCount. If there are 7 messages on the queue and the provided messageCount is 10, the method will return immediately with a batch of all 7 messages. It won’t wait until there are 10 messages on the queue, obviously. The third overload is much the same, except that you can specify a TimeSpan to wait. If no messages arrive during this timespan, an empty Enumerable will be returned.

The second overload is used to get specific messages (for example, messages that have been deferred) from the entity.

The following code reads the messages that have been written in the previous sample.

MessageReceiver msgReceiver = msgFactory.CreateMessageReceiver("PrefilterBatchTopic/Subscriptions/NoNumberEight", ReceiveMode.ReceiveAndDelete);
var messagesReceived = msgReceiver.ReceiveBatch(10, TimeSpan.FromSeconds(3));
Console.WriteLine("Batch received: " + messagesReceived.Count());

Some new properties

The TopicDescription and QueueDescription have some new properties available, aside from the ones discussed above. 

• Authorization: contains all rules for authorization on the topic or queue. (not on SubscriptionDescription)
• AccessedAt, CreatedAt, UpdatedAt: these are all DateTime values that indicate the corresponding last actions for a queue. (In my private beta version, the AccessAt was still the DateTime.MinValue, however.)
• IsAnonymousAccessible: indicates if the queue has opened anonymous access (not on SubscriptionDescription)
• Status: the status of the queue (Active, Disabled, Restoring)
• UserMetadata: this is a property that one can use to custom metadata and so to a queue or topic.  

Conclusion

With all recent announcements around symmetry between the cloud and the data center, we can fairly say that Microsoft is the only vendor that is powerful on both levels and gives its customers the choice to run their software in the best way possible.  We truly believe in that vision and that's why we have taken the effort from the beginning of Integration Cloud to make the archticture flexible and modular to support this symmetry.

With Service Bus for Windows Server, we now have the same features available in a server only solution.  And this now also gives me the chance to develop against the service bus while being disconnected. 

Sam Vanhoutte

Updates

• Removed IIS from the topology diagrams.  While IIS will often be installed and used in combination with Service Bus, it's not a requirement.

The out-of-the-box BizTalk Server Operators group has limited access to the BizTalk environment. An extract from MSDN:

Members of the BizTalk Server Operators group can do the following:

• View service state and message flow
• Start or stop applications
• Start or stop orchestrations
• Start or stop send ports or send port groups
• Enable or disable receive locations
• Terminate and resume service instances

Members of the BizTalk Server Operators group can do the following:

• Modify the configuration for BizTalk Server
• View message context properties classified as Personally Identifiable Information (PII) or message bodies.
• Modify the course of message routing, such as removing or adding new subscriptions to the running system, including the ability to publish messages into the BizTalk Server runtime.

Lately, I had a request to elevate the permissions for BizTalk Operators, so they were able to see the tracked message bodies.  The content of a message is often needed for a decent troubleshooting.  Because BizTalk security is actually based on SQL Server security, it was pretty easy to implement this request.  It’s sufficient to give the database role “BTS_OPERATORS” additional EXECUTE rights on specific BizTalk stored procedures, which are related to the retrieval of BizTalk message bodies.  All details can be found in the script below:

USE BizTalkDTADb; 
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessage
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageFragments
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartByID
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedParts
		TO BTS_OPERATORS;
GO 

USE BizTalkMsgBoxDb;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessage
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageFragments
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartByID
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedParts
		TO BTS_OPERATORS;
GO 

USE BizTalkMgmtDb;
	GRANT EXECUTE ON OBJECT::dpl_MessageType_Part_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_MessageType_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_Operation_MsgType_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_SaveItem
		TO BTS_OPERATORS;
GO

By executing this SQL Server script, you can easily grant them the rights to view BizTalk message bodies which allows easy debugging or follow up in different scenarios without having to change memberships.

Update:
Please note that the above method is not supported by Microsoft, so be sure to know what you are doing!
Also note that database schemas and security may vary depending on the version of BizTalk you are using.

For a recent project I needed to create and modify the party settings for a specific party through code.

All the ‘basic’ settings like UNA/UNB/Validation/… where rather easy to configure and are pretty straight forward.

However when wanting to configure the UNG segment, we ran into some issues.

But first things first, let’s create a new Party

To modify the EDI properties through code you need to initialize/load a Partner profile as shown below

When you loaded/created the Partner profile it’s rather straight forward to modify the basic Party Settings.

The real problem started when we had to create a UNG segment trough code.
To create a UNG you need to create a PartnerEdifactReceiverGroupUpdatable object and add it to the EdifactReceiver as shown below:

The actual problem here is that the PartnerEdifactReceiverGroupUpdatable class has no constructors defined. Also extending the base class

or implementing the interface did not give us the desired result.

After some investigation we ran into the GetUninitializedObject method from the FormatterServices class (http://msdn.microsoft.com/en-us/library/system.runtime.serialization.formatterservices.getuninitializedobject.aspx).

This method will create a new object of a class without calling the constructor, that way we initialized a new instance of PartnerEdifactReceiverGroupUpdatable and managed to create the UNG segment as show below.

Enjoy!

Glenn Colpaert

After years of publishing BizTalk WCF-Services and countless hours spend configuring the services in the wizard I just recently found out the WCF-Publishing Wizard can be started with a configuration file provided.

Actually the configuration file is generated after you published your service for the first time.

The configuration file ‘WcfServiceDescription.xml” is located in the App_Data/Temp subfolder in the folder where the service has been published:

To start the wizard with a configuration file, fire up a command prompt (using Administrative privileges). Start the wizard with the /WcfServiceDescription option and provide the configuration file:

In the “Create WCF Service” page of the wizard select the correct option:

The page that describes your service will have all information from your previous configuration:

As always there is one caveat:

The target namespace of the service is reset to the default and needs to be configured:

This definitely took the pain out of updating my services …

Lately, I had the task to create a generic WCF netTcp endpoint within BizTalk. The purpose was to create a single BizTalk ESB endpoint for each front-end application. BizTalk has the task to route and transform the SOAP requests to the corresponding back-end services.

I already had experience with hosting such an endpoint in-process (with the Custom adapter), but now it was required to host the endpoint in IIS (with the Custom-Isolated adapter). The reason for this decision is the fact that IIS, in combination with Windows Server AppFabric, adds extra tooling and manageability.

Enable Windows Activation Services

• Add WAS as a Windows feature:

• Turn on WCF Non-HTTP Activation

• Start all WAS Windows Services. Change their startup type to Automatic. You can do this via Properties.

Create IIS endpoint

• Create a web service directory: C:\inetpub\wwwroot\BlogService

• Open IIS Manager and convert the folder to an application

• Choose an application pool which is configured with the BizTalk isolated host user for .NET Framework 4.0

• Enable netTcp for the Default Web Site, via the Advanced Settings:

• Enable netTcp for the BlogService application, via the Advanced Settings:

• Add GenericEsbEndpoint.svc to the web service directory

• GenericEsbEndpoint.svc tells IIS to use the ServiceHostFactory of the BizTalk WCF adapter:

           <%@ ServiceHost Language="c#" Factory="Microsoft.BizTalk.Adapter.Wcf.Runtime.CustomWebServiceHostFactory, Microsoft.BizTalk.Adapter.Wcf.Runtime, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" %>

• A web.config can be added, but all service behaviors (e.g. serviceDebug) can be configured on your BizTalk receive location too.

Create BizTalk receive location

• Create a two-way receive port

• Create a two-way receive location and configure with the Custom-Isolated Adapter

• Configure the adapter URI with the format /<IIS Application Name>/<Service.svc>. Don’t use the scheme “net.tcp://” here!

• Configure the netTcp binding. Disable transport security (for performance reasons).

• Enable the BizTalk receive location

• Now you can additionally set up send ports and filters, to route the submitted SOAP request to the correct back-end services. Routing can be based on multiple properties, but the SOAP Action (WCF.Action) seems most suitable. Also transformations must be added in some cases.

Update the client binding

• The client needs to be updated to send to the generic BizTalk endpoint, instead of directly to the back-end service. Update the client with the following address format: net.tcp://<machinename>:<tcpPortNumber>//<IIS Application Name>/<Service.svc>

<client>

<endpoint address="net.tcp://machineName:8888/BlogService/GenericEsbEndpoint.svc"

binding="netTcpBinding" bindingConfiguration="NetTcpBinding_ICoditService"

contract="PersonSvc.ICoditService" name="NetTcpBinding_ICoditService" />

</client>

• The tcpPortNumber can be configured in the Site Bindings of the Default Web Site

Recently, we wanted to create multiple WCF services that should communicate with each other through the netMsmqBinding binding and we wanted to use a clustered MSMQ service.
However we encountered some problems and wanted to share them with you.

Situation: We run MSMQ as a part of resource group on a cluster. The service is hosted in WAS and the application pool runs under a functional user.

1. Without Windows Integration

Note: In our WCF service the security mode is set to "None".

When our services ran and started to communicate with each other we did not get any messages  in the queue as we expected, actually we did not find any message on the clustered MSMQ but we saw in our tracing that the WCF service did not report any error to place a message on the queue.

After some investigation we found all test messages in the local System queue “Transactional dead-letter messages” with the report “Access denied”.

So our first thought was to check the MSMQ security on that particular queue: we started up mmc, connected to the clustered resource e.g. “MyclusteredResource” to check on the MSMQ security:

There we wanted to check the security on a queue (by right clicking on the queue) we noticed we did not have a properties option.

It seems that you cannot configure MSMQ security when connecting through the mmc console.
Microsoft pointed out that we should use an other tool to configure the security (mmcv.exe).

You can download mmcv.exe : http://www.microsoft.com/download/en/details.aspx?id=18189

You use it as follows on the active node (Where the MSMQ resource is running):  
"<drive>:\<directory>\mmcv -s <Network Name of the clustered MSMQ resource>"  

When you drill down to the MSMQ queues we finally got the properties option. So we went on to check the security.

Note: If you are running the mmcv.exe tool on the non-active node you will not find the Message Queueing option.

It seemed that Everyone had full control and yet we got the error message “Access denied”

After a bit of searching we found the following KB article: http://support.microsoft.com/default.aspx?scid=kb;EN-US;952570 and following blog http://blogs.msdn.com/b/tomholl/archive/2008/07/13/msmq-wcf-and-iis-getting-them-to-play-nice-part-2.aspx

This explained why we got the error message “ the message does not include the security header. Therefore, the message does not contain the user ID of the sender "

So we added the “ANONYMOUS LOGON “ to the users and gave him the appropriate rights to read / write on the queue.

After that we ran a test and all messages were delivered on the clustered MSMQ! 

In the next step I’ll show you how you can check and enable the Windows Integration on your server...

2. With Windows Integration

Through the Features option in the server manager (Server 2008) you need to check if Directory Service Integration is enabled. Be aware that you need to do this for each node.

Once this feature is installed and your resources went online, an MSMQ object should be created in the active directory record of your network resource hosting your MSMQ e.g. “MyclusterresourceYou” can check it by the following steps:

Open the Active directory Users and Computers.

Through the View otpion select "Users, groups and computers as containers"

Finally search for your network resource and check if it has a MSMQ object.

If you don’t find any MSMQ object it is possible that when Directory Service Integration is installed afterwards no MSMQ object is created (more info: http://support.microsoft.com/default.aspx?scid=kb;EN-US;935498)

Once that is fixed, for Windows Integrated security you only need to add your user in the MSMQ security with mmcv.exe

Change your WCF security mode to "Transport" and enable the msmqAuthentication mode so your WCF service is ready to use Windows Authentication.

When you get the following error while starting your WCF service: "Binding validation failed because the binding's MsmqAuthenticationMode property is set to WindowsDomain but MSMQ is installed with Active Directory integration disabled. The channel factory or service host cannot be opened" then you need to check the MSMQ object in the active directory (see previous steps).

Jonathan Maes