March 6, 2013 at 4:05 PM

In many scenarios, it could be required to have a send port with a dynamic behavior:

  • Send messages to a FILE location, containing the date of today
  • Send a notification mail to many different recipients
  • Dynamically change the URL of the backend web service, depending on the incoming request

 

Many developers would solve this by using dynamic send ports.  These ports allow you to configure all adapter properties, even the adapter type, at runtime.  I’m not a big fan of dynamic send ports, because it introduces some drawbacks:

  • Dynamic send ports are very difficult to troubleshoot (a hell for your operations team)
  • Dynamic send ports require your port configuration to be stored in a secure way (custom SSO application)
  • Dynamic send ports have a bigger performance hit (solved in BTS 2013)
  • Dynamic send ports don’t allow to specify an adapter handler (solved in BTS 2013)

 

That’s why I prefer using static send ports.  With a very easy trick, you can make static send ports partially dynamic.  Mostly it’s only the connectionstring, URI, path that must be changed at runtime.  This can be achieved in a custom pipeline component in the send pipeline:

 

 

This dynamic behavior works smooth for FILE and FTP adapters.  However, when using this mechanism on WCF send ports, there’s a caveat.  You will find out that the BTS.OutboundTransportLocation of the first message will be cached on the WCF send port.  So all subsequent messages will be sent to the same location.  This port configuration is cached until your send port configuration is modified or until your adapter send handler is restarted.  Luckily: there’s a way to disable this caching:

 

 

By setting the BTS.IsDynamicSend property to true, you force the WCF send adapter not to use the cache, but the runtime value instead.  More information on this setting can be found on MSDN.

Posted in: BizTalk | General | WCF

Tags: ,


January 14, 2013 at 3:16 PM

Lately I came across a weird exception while building a BizTalk project with Team Foundation Server.  Let’s have a look.

 

The error

Although the build succeeded locally, we got this error while building our BizTalk project with TFS:

FlatFileSchema.xsd.cs: The type or namespace name '<ContextPropertyName>' could not be found in the global namespace (are you missing an assembly reference?)

 

The troubleshooting

After some try/error, we discovered that the error only occurred when building a BizTalk project in TFS that contains a flat file schema, using property promotion.   The first reflex was to compare the installed software on our local development machine and the Team Foundation Server.

  • Local machine: Full BizTalk 2010 Installation
  • TFS server: BizTalk 2010 Build Component

 

 

The solution

The solution was to install also the Developer Tools and SDK on the TFS server.  Although MSDN states that the Build Component should be sufficient.  This extra installation adds “Developer Tools\Schema Editor Extensions” to the BizTalk folder.  This is apparently needed for building flat files schemas that use property promotion.

Posted in: BizTalk | General | Schemas | TFS

Tags: ,


November 14, 2012 at 4:43 PM

It’s a good practice to have some XML validation in your BizTalk integration flows.  It prevents you of sending mal-formatted messages to back-end systems and it also gives you the opportunity to validate incoming messages.  In the BizTalk toolbox, you have two standard pipeline components that can perform this XML validation.  Let’s have a look...

 

XML Disassembler (or XML Assembler)

The default configuration of the XML Disassembler will not perform any kind of XML validation on the message


If you want to enable XML validation on the XML Disassembler you need to set Validate document structure to True and explicitly set the Document schema(s) that need to be validated. Compared to the specified Documents schema(s):
- If the processed message has the same TargetNamespace#RootNodeName combination, XML validation will be performed

- If the processed message has another TargetNamespace#RootNodeName combination, an exception will be thrown (can’t recognize the data)

 

    image


XML Validator

The default configuration of the XML Validator will already perform XML validation. How? It retrieves the XML definition from the BizTalkMgmtDb, at run-time, using the TargetNamespace#RootNodeName combination.

-   If a matching schema is found, XML validation will be performed on the processed message

-   If no matching schema is found, the XML Validator will throw an exception (cannot retrieve document specification)


You can also explicitly specify at design-time the Document schema(s).  Compared to the specified Documents schema(s):

-   If the processed message has the same TargetNamespace#RootNodeName combination, XML validation will be performed

-   If the processed message has another TargetNamespace#RootNodeName combination, no XML validation will be performed


    image

 

Conclusion

You see a different behavior when comparing the components, especially when specifying Document schema(s). Depending on your needs, you can choose the appropriate setup.  I mostly opt for the combination of the XML Disassembler and XML Validator, both with their default configuration.  It has the advantage that you don’t need to configure anything at design-time.  They work together in this way:

-   The XML Disassembler is responsible for retrieving the BTS.MessageType, promoting context properties and debatching

-   The XML Validator will validate all messages, deciding at run-time which schema to use

Posted in: BizTalk | General | Schemas | XML

Tags:


October 30, 2012 at 3:01 PM

It’s a common practice to reuse schemas / types within your BizTalk solution.  Lately, I’ve encountered an issue regarding this.  Here you can find a simplified example and a solution to the problem.

 

Assume you have two reusable schemas (no dependencies between them):

-   http://schemas.codit.eu/common#Address

-   http://schemas.codit.eu/common#ContactInformation

 

Now you want to create a “Person’ schema in the http://schemas.codit.eu namespace, by reusing the previous schemas.

-   First you need to import the Address schema

-   Secondly you need to import also the ContactInformation schema.  This results in the error:

 
 The namespace of the schema you are trying to add clashes with one of the namespaces already declared.  Please choose another schema.

 image

 

Although, in XSD this is not a limitation.  So I’ve tried to work around the BizTalk Schema Editor and opened the XSD as an XML file.  I’ve manually added the XSD import of the ContractInformation schema:

 

image

 

After this workaround, I’ve opened the schema again with the BizTalk Schema Editor, without any errors.  It just seems to be a limitation of the BizTalk Schema Editor.

 

image

Posted in: BizTalk | General | Schemas

Tags:


September 7, 2012 at 3:01 PM

For a recent project I had to work with Windows Server AppFabric Caching. One of the requirements was to retrieve the CacheStatistics from the AppFabric cache.

To retrieve these statistics you have to use PowerShell by using following commands:

clip_image001

 

Offcourse we needed this data in our .NET Application and not in PowerShell.
We needed to call the PowerShell cmdlets from our .NET Application to be able to retrieve that data. This is how we did it.

First of all add a reference to the following assemblies:

clip_image001[6]

 

The next step is to initialize the PowerShell RunSpace and initialize all the necessary values and objects as shown below:

clip_image001[8]

 

When the pipeline is created we can start sending commands; Always start your command sequence with the "Use-CacheCluster" command.
In the code below you can see how we actually retrieved the CacheStatistics from the AppFabric Cache.

clip_image001[10]

 

For a full list of the available AppFabric Caching PowerShell cmdlets please see following links:

http://msdn.microsoft.com/en-us/library/hh475806
http://msdn.microsoft.com/en-us/library/hh851388

 

Enjoy!

 

Glenn


July 2, 2012 at 4:22 PM

The out-of-the-box BizTalk Server Operators group has limited access to the BizTalk environment. An extract from MSDN:


Members of the BizTalk Server Operators group can do the following:

  • View service state and message flow
  • Start or stop applications
  • Start or stop orchestrations
  • Start or stop send ports or send port groups
  • Enable or disable receive locations
  • Terminate and resume service instances

 

Members of the BizTalk Server Operators group can do the following:

  • Modify the configuration for BizTalk Server
  • View message context properties classified as Personally Identifiable Information (PII) or message bodies.
  • Modify the course of message routing, such as removing or adding new subscriptions to the running system, including the ability to publish messages into the BizTalk Server runtime.

 

Lately, I had a request to elevate the permissions for BizTalk Operators, so they were able to see the tracked message bodies.  The content of a message is often needed for a decent troubleshooting.  Because BizTalk security is actually based on SQL Server security, it was pretty easy to implement this request.  It’s sufficient to give the database role “BTS_OPERATORS” additional EXECUTE rights on specific BizTalk stored procedures, which are related to the retrieval of BizTalk message bodies.  All details can be found in the script below:

USE BizTalkDTADb; 
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessage
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageFragments
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartByID
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedParts
		TO BTS_OPERATORS;
GO 

USE BizTalkMsgBoxDb;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessage
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageFragments
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::bts_GetTrackedMessageParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadParts
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessageContext
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedMessages
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPart
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartByID
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartFragment
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedPartNames
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::ops_LoadTrackedParts
		TO BTS_OPERATORS;
GO 

USE BizTalkMgmtDb;
	GRANT EXECUTE ON OBJECT::dpl_MessageType_Part_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_MessageType_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_Operation_MsgType_Save
		TO BTS_OPERATORS;
	GRANT EXECUTE ON OBJECT::dpl_SaveItem
		TO BTS_OPERATORS;
GO

 

By executing this SQL Server script, you can easily grant them the rights to view BizTalk message bodies which allows easy debugging or follow up in different scenarios without having to change memberships.

 

Update:
Please note that the above method is not supported by Microsoft, so be sure to know what you are doing!
Also note that database schemas and security may vary depending on the version of BizTalk you are using.


June 22, 2012 at 2:40 PM

For a recent project I needed to create and modify the party settings for a specific party through code.

All the ‘basic’ settings like UNA/UNB/Validation/… where rather easy to configure and are pretty straight forward.

However when wanting to configure the UNG segment, we ran into some issues.

 

But first things first, let’s create a new Party

image

 

To modify the EDI properties through code you need to initialize/load a Partner profile as shown below

image

 

When you loaded/created the Partner profile it’s rather straight forward to modify the basic Party Settings.

image

 

The real problem started when we had to create a UNG segment trough code.
To create a UNG you need to create a PartnerEdifactReceiverGroupUpdatable object and add it to the EdifactReceiver as shown below:

image

 

The actual problem here is that the PartnerEdifactReceiverGroupUpdatable class has no constructors defined. Also extending the base class

or implementing the interface did not give us the desired result.

 

After some investigation we ran into the GetUninitializedObject method from the FormatterServices class (http://msdn.microsoft.com/en-us/library/system.runtime.serialization.formatterservices.getuninitializedobject.aspx).

This method will create a new object of a class without calling the constructor, that way we initialized a new instance of PartnerEdifactReceiverGroupUpdatable and managed to create the UNG segment as show below.

 

image

 

Enjoy!

 

Glenn Colpaert

Posted in: BizTalk | EDI/AS2 | General

Tags:


February 22, 2012 at 11:22 AM

Microsoft BizTalk Server 2010 ships with some assemblies that assist you with the administration and deployment of your BizTalk environment.  During the development of an automated deployment plan, we were able to perform most of the deployment tasks at a BizTalk application level, by using these assemblies:

 

Microsoft.BizTalk.ExplorerOM

 

-       Create/delete applications

-       Control receive locations

-       Control orchestrations

-       Control send ports

-       Control send port groups

-       Control application references

 

Microsoft.BizTalk.ApplicationDeployment

 

-       Add/remove resources

 

Microsoft.BizTalk.Deployment

 

-       Import/export bindings

 

Microsoft.BizTalk.Operations

 

-       Query for active service instances         

 

Remark that all of the used objects can be initialized by three parameters:

      -    SqlServerName

-    BizTalkMgmtDbName

-    ApplicationName

 

 

The object "BizTalk Group" is defined by the SqlServerName and the BizTalkMgmtDbName.  A "BizTalk Application" is defined by its BizTalk Group and the ApplicationName.  However, these assemblies have totally no notion of the object "BizTalk Server".  They only support operations at a database level (BizTalk Group), but some deployment tasks need to be executed on a BizTalk Server level:

 

- Install/uninstall assemblies to the Global Assembly Cache

- Control host instances

 

For both actions we needed to be able to dynamically determine which BizTalk Servers are part of the already defined BizTalk Group. After some research, we found two options to implement this logic:

 

Windows Management Instrumentation

 

WMI has some support for BizTalk via the WMI namespace “root\\MicrosoftBizTalkServer”.  We are able to loop though all configured host instances, by executing a query on the MSBTS_HostInstance class.  As each host instance runs on a BizTalk Server, it’s easy to implement some logic that retrieves all BizTalk Servers.

 

private List<string> GetBizTalkServers()
{
     List<string> btsServers = new List<string>();

     EnumerationOptions wmiEnumerationOptions = new EnumerationOptions { ReturnImmediately = false };
     ObjectQuery wmiQuery = new ObjectQuery("SELECT * FROM MSBTS_HostInstance");
     using (ManagementObjectSearcher wmiSearcher = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", wmiQuery.QueryString, wmiEnumerationOptions))
     {
          ManagementObjectCollection hostInstanceCollection = wmiSearcher.Get();
          foreach (ManagementObject hostInstance in hostInstanceCollection.Cast<ManagementObject>())
          {
                string btsServer = hostInstance["Name"].ToString().Split(' ').Last();
                if (btsServers.Contains(btsServer) == false)
                {
                     btsServers.Add(btsServer);
                }
          }
      } 

      return btsServers;
}

Custom SQL Server Query

 

Another option is to retrieve the information directly from the BizTalkMgmtDb.  The table adm_Server actually contains a list of all BizTalk Servers that are part of the BizTalk Group.  So this simple .NET code is able to return us the needed information:

 

 

private List<string> GetBizTalkServers()
{
     List<string> btsServers = new List<string>();

     SqlConnection sqlConnection = new SqlConnection(String.Format(CultureInfo.CurrentCulture, "Server={0};Database={1};Integrated Security=SSPI;", SqlServerName, MgmtDatabase));
     SqlCommand sqlCommand = new SqlCommand("SELECT Name FROM [dbo].[adm_Server]", sqlConnection);

     sqlConnection.Open();

     SqlDataReader sqlDataReader = sqlCommand.ExecuteReader();

     while (sqlDataReader.Read())
     {
          btsServers.Add(sqlDataReader["Name"].ToString());
     }

     sqlConnection.Close();

     return btsServers;
}

 

 

Conclusion

 

It's pretty easy to retrieve a list of all BizTalk Servers that are part of a BizTalk Group.  A limitation of the WMI approach is that this will only work when it's executed on one of the BizTalk Servers.  So I prefer the SQL approach, because this will also work for remote execution.

 

Toon Vanhoutte

Posted in: BizTalk | Cluster | Deployment | General | Infrastructure

Tags:


February 16, 2012 at 7:00 PM

Recently, I had a project where full side-by-side deployment was required.  In a first test phase of the automated deployment, we used MSBuild together with the Codeplex MSBuild Extension Pack. We tried deploying assemblies side-by-side, but BizTalk threw errors when trying to deploy a new version of our Blog.Codit.BizTalk.Pipelines assembly:

 

Failed to add resource(s). Change requests failed for some resources. BizTalkAssemblyResourceManager failed to complete end type change request. SQL exception: "Violation of UNIQUE KEY constraint 'UQ__bts_pipe__DD425CAE7908F585'. Cannot insert duplicate key in object 'dbo.bts_pipeline'. The duplicate key value is (D6C53CD9-8466-4520-A85B-7BD887A57455)." 

 

I received this error when trying to deploy from Visual Studio, from MSBuild (using BTSTask.exe) and also when trying to deploy it manually via the Admin Console (Add Resource).  So the error is actually independent from the deployment source.  As BizTalk returned a SQL exception, I decided to run SQL Profiler and detected this call:

 

exec btm_CreatePipeline 
   @PipelineID='D6C53CD9-8466-4520-A85B-7BD887A57455',
   @Category=1,
   @Name=N'Blog.Codit.BizTalk.Pipelines.rcvSalesOrder',
   @FullyQualifiedName=N'Blog.Codit.BizTalk.Pipelines.rcvSalesOrder, Blog.Codit.BizTalk.Pipelines, Version=1.2.0.0, Culture=neutral, PublicKeyToken=xxx',
   @IsStreaming=0,
   @AssemblyID=265,
   @Release=2,
   @ID=@p8output

 

So the deployment logic tries to create a new entry in the bts_pipeline table, but it uses exactly the same (unique) ID of the pipeline of the older version.  This entry was already present in the bts_pipeline table: 

D6C53CD9-8466-4520-A85B-7BD887A57455 Blog.Codit.BizTalk.Pipelines.rcvSalesOrder, Blog.Codit.BizTalk.Pipelines, Version=1.1.0.0.  

 

We opened a case at Microsoft Support for this issue.  They forwarded the issue to the BizTalk Product Team and we received this answer:

 

The pipeline GUID is stored in the rcvSalesOrder.btp.cs  file. The GUID gets regenerated when either of the following occurs:
 
- The pipeline is saved from within Visual Studio.
- When REBUILD the BizTalk project.

 

As we executed a BUILD, instead of a REBUILD, a small change to our MSBuild script solved the issue:

<MSBuild Projects="@(ProjectsToBuild)" Targets="Rebuild" />

 

This behavior is rather inconsistent, because it only occurs when your assembly contains a pipeline object. 

 

Big thanks to Microsoft for the fast and accurate support.

Toon Vanhoutte