June 18, 2013 at 3:54 PM

Windows Azure BizTalk Services is a cloud-based integration service that provides B2B and EAI capabilities for cloud and hybrid solutions.

The Windows Azure BizTalk Services are provided by Microsoft through the Windows Azure portal.

This blog post will focus on the differences between the Windows Azure BizTalk Services mapping tool and the BizTalk Server 2013 mapper.

 

 

Interface

The screenshot shows the interface of both mapping tools head to head. On the left hand side you see the BizTalk 2013 interface, on the right hand side you see the Windows Azure BizTalk Services mapping tool.

image

 

As you can see Microsoft did a pretty amazing job of bringing the same ‘design’ of BizTalk server into the Windows Azure BizTalk Services mapping tool. Off course the icons of the functions are slightly different, but generally the WABS mapper tool provides the same look and feel as the BizTalk mapper.

 

Functoids

Both the BizTalk 2013 mapper and the Windows Azure BizTalk Services mapping tool are providing a wide range of functoids. The 2 tables below are giving a head to head comparison of different functoids supported in both applications.

image

image

 

As you notice, Windows Azure BizTalk services comes with a wide range of functoids as we know them from BizTalk 2013. The only difference is that some functoids are grouped in WABS. ex: Arithmetic Expression, Logical Expression,…

 

Below you can find a list of functoids that are available in WABS but are not ‘yet’ available in BizTalk 2013. Especially the DateTimeReformat and the GetContextProperty could be nice additions to the BizTalk mapper.

  • DateTimeReformat
    • Use this to reformat a date-time value. The input value is parsed using the specified input format string, and the output value is formatted using the specified output format string.
  • GetContextProperty
    • Use this to access the value of a specified property from the property bag associated with the input data.
  • List Operations
    • There are a number of functoids to manage/create lists. You can create a list, add items, select unique groups, order, get specific items…
  • MapEach Loop
    • Use MapEach to loop over a repeating input. Each iteration will produce an instance of the target node.
  • ForEach Loop
    • Use ForEach to loop over a repeating input. Unlike MapEach this does not produce any target nodes. It can contain MapEach Loop(s) that will produce target nodes, to flatten source loops. It can also be used to add items to a list.

 

Testing

For testing your map Windows Azure BizTalk Services and BizTalk 2013 offer different options, below you can find them head to head. On the left hand side you see the Windows Azure BizTalk Services options, on the right hand side you see the BizTalk 2013 options.

image

All functionality that BizTalk offers is supported in Windows Azure BizTalk Services. The only thing that I’m missing in the Windows Azure BizTalk Services mapping tool is the ‘Generate Instance’ option, this allows you to quickly test your mapping with fictive data, rather than having to designate an input instance manually.

 

 

Extensibility

For extending the mapper Windows Azure BizTalk Services and BizTalk 2013 offer different options, below you can find them head to head. On the left hand side you see the Windows Azure BizTalk Services options, on the right hand side you see the BizTalk 2013 options.

image

The good thing about the Windows Azure BizTalk Services mapping tool is that we can specify our own custom XSLT. The original design of the Windows Azure BizTalk mapper did not include this function.

On the other hand, it currently looks impossible to provide a ‘Custom Extension XML’ (http://msdn.microsoft.com/en-us/library/aa547368.aspx). That means when a C# function is needed in our XSLT, we will need to call it through inline C#. At Codit, it’s a best practice NOT to use inline C#, because of known issues with Memory Leaks (http://support.microsoft.com/default.aspx?scid=kb%3bEN-US%3b316775).

 

Migrating maps from BizTalk to Windows Azure BizTalk Services

The Windows Azure BizTalk services SDK (link) comes with a tool (BTMMigrationTool.exe) that allows you to easily migrate your existing .btm (BizTalk map) files to .trfm (Transform) files.

 

This migration tool is a simple command line tool that takes 1 or 2 input parameters as shown below :

BtmToTrfmConvertor.exe <Full BTMFilePath> [optional]<OutputTrfm>

image

 

As an example of demonstrating the migration capabilities of the tool, I took a simple mapping and migrated it with the BTMMigrationTool.
You can see the result in the screenshots below. First one in the original .btm file of BizTalk. Second one is the.trfm generated by the BTMMigrationTool.

image

image

 

Conclusion

As you noticed in this blog, Microsoft did a nice job of bringing the same look and feel of BizTalk server into the Windows Azure BizTalk Services mapping tool. Most of the functionality the BizTalk mapper offers, is also supported in Windows Azure BizTalk Services.
The only major thing that is missing in my opinion is the option to provide a ‘Custom Extension XML’, with this option not available we are ‘forced’ to call inline C# functions in our XSLT.

But also BizTalk can learn from Windows Azure BizTalk Services, for example the ‘GetContextProperty’ functoid could be a nice addition to BizTalk.

If you have remarks or questions about this blog post, please do not hesitate to put them in the comments section.

 

Cheers,

Colpaert Glenn


June 10, 2013 at 3:46 PM

When working / playing with Windows Azure BizTalk Services, you will find out that there are some things that are not exposed through the portal.  For these things, PowerShell comes to the rescue.

This blog post lists some of the different powershell scripts that I have been using for some administrative tasks.

 

The full documenation can be found here: http://msdn.microsoft.com/en-us/library/windowsazure/dn232360.aspx

 

For every script, you need to import the PowerShell module to load all possibilities.  This can be done by the following command:

import-module "C:\Program Files\Windows Azure BizTalk Services SDK\Microsoft.BizTalk.Services.Powershell.dll"

 

Querying the status of an endpoint in WABS

Get-AzureBizTalkBridgeSource -AcsNamespace mynamespace -IssuerName owner -IssuerKey key -DeploymentUri https://mynamespace.biztalk.windows.net/default -BridgePath BridgeOrders

 

Recycling the BizTalk Service

This is something you often need to do, when redeploying custom code to the BizTalk Service. 

Restart-AzureBizTalkService -AcsNamespace mynamespace -IssuerName owner -IssuerKey key -DeploymentUri https://mynamespace.biztalk.windows.net/default

 

Purging the tracking data

If you want to purge your tracking data, you can use the following statement (please note the quotes around the connection strings!)

Clear-AzureBizTalkTrackingStore -Sql "Server=tcp:servername.database.windows.net,1433;Database=mynamespace_db;User ID=user@servername;Password=pass;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;" -AzureStore "DefaultEndpointsProtocol=https;AccountName=storagename;AccountKey=key" -Ndays 100

 

If you get the error: “Could not load file or assembly 'Microsoft.WindowsAzure.StorageClient, Version=1.7.0.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.”, you’ll need to get the 1.7 storage client and use that one (you can GAC it, or copy it to your powershell runtime).

To get it from Nuget, use the following command:

Install-Package WindowsAzure.Storage -Version 1.7.0.0

Posted in: Azure | BizTalk | BizTalk Services | PowerShell | WABS

Tags:


June 7, 2013 at 3:04 PM

Introduction

With the release of BizTalk server 2013, BizTalk Server is ready to run in the cloud in 2 ways:

- BizTalk Server in an IaaS model

- BizTalk Services in a PaaS model.

 

In this post I will explain step-by-step how-to setup a BizTalk environment in Windows Azure IaaS in an automated way.

The environment we will setup will consist of 3 servers:

- A domain controller

- A SQL Server hosting our BizTalk databases

- A BizTalk 2013 Server

Besides the servers, we will also create a Virtual Network in Windows Azure and we will connect the Virtual Network with our local (on-premise) laptop.

To automate the provisioning of our environment we will make use of PowerShell and Remote PowerShell into our machines in the cloud.

 

STEP 1 – Prepare PowerShell

To be able to perform our Windows Azure tasks with PowerShell, you will need to download and install Windows Azure PowerShell.

You can find Windows Azure PowerShell on this location: http://www.windowsazure.com/en-us/downloads/ => download and install.

 

After downloading Windows Azure Powershell, it doesn’t know yet about your Azure subscription and how to connect to it.

To get this right, you have to download your PublishSettings file from this location: https://windows.azure.com/download/publishprofile.aspx.

You will be prompted for the credentials of your Azure account. Download the PublishSettings Xml file. This file contains the details about your Azure account and subscriptions.

 

Next, launch Windows Azure PorwerShell as an administrator.

 

Edit and execute these PowerShell commands:

#Initial settings
Set-ExecutionPolicy Unrestricted

#Import publish settings
Import-AzurePublishSettingsFile '<PATH>\Azure.publishsettings'   

# SET AZURE SUBSCRIPTION
Set-AzureSubscription -SubscriptionName 'BizTalk Launch' -CurrentStorageAccount biztalkvms
Select-AzureSubscription -SubscriptionName 'BizTalk Launch'
Set-AzureSubscription -DefaultSubscription 'BizTalk Launch'

First we change the ExcecutionPolicy to ‘unrestricted’.

Then we import the PublishSettings file we just downloaded.

After importing the PublishSettings file, set the Azure Subscription. If you have more subscriptions available in your Azure account, this will tell PowerShell on with subscription to work, and what storage account will be used.

 

Make sure you have a storage account prepared to store the Virtual Machines disks on:

image

 

When step 1 is completed successfully, PowerShell is ready to create our Azure Virtual Machines.

 

STEP 2 – Create a Virtual Network

This step is not automated, since the creation of a Virtual Network will typically be done once.

Logon to the Azure Management Portal.

Create a new Virtual Network (custom create):

image

 

In the create Virtual Network wizard, choose a name for your Virtual Network and assign an Affinity Group:

image

Affinity groups allow you to group your Windows Azure services to optimize performance. All services within an affinity group will be located in the same data center. An affinity group is required in order to create a virtual network.

 

On the next screen, you can configure a DNS server for your network (I will leave it blank in my setup).

Later in the setup of this environment we will connect our on-premise laptop with the Virtual Network in the cloud. For this, we will need a Point-To-Site VPN. Enable ‘Configure Point-To-Site VPN’.

image

 

Azure Virtual Networks Connectivity with on-premise networks can be achieved in two ways:

- Point-To-Site VPN

Point-To-Site connectivity allows you to connect individual devices with your cloud network. Point-To-Site connections will even work from behind a firewall!

- Site-To-Site Connectivity

Site-To-Site connections will connect entire networks with one another.

A Site-To-Site connection can be setup in two ways:

- Hardware VPN: a hardware VPN device is necessary (Cisco, Juniper, …)

- Software VPN: Windows RRAS (Routing and Remote Access) is used as a software VPN

In both cases (hardware and software), you will need a public IP that is not behind NAT!

 

On the next screen, select the IP range that will be used for issuing IP’s to the remote devices that connect to you Virtual Network through the Point-To-Site connections:

image

 

Make sure that the IP range used for the clients does not conflict with the IP range of the Virtual Network itself.

 

On the last screen of the ‘Create Virtual Network’ wizard, choose the ip range of the Virtual Network in Windows Azure:

image

 

The IP range will be different than the one we chose on the Point-To-Site connectivity screen.

The Point-To-Site VPN also requires us to have Gateway. The wizard will not allow us to create the Virtual Network without a Gateway.

Click the ‘add gateway subnet’ button. A Gateway will be added. You can keep the proposed IP.

 

When you finish the ‘Create Virtual Network Wizard’, the Virtual Network will be created in Windows Azure:

image

 

When the Virtual Network is created, open its properties in the Windows Azure Management Console:

 

image

 

As you can see, Windows Azure reports an error ‘The Gateway was not created’.

The only problem is that Windows Azure does not generate the Gateway automatically. The fix is easy, click the ‘Create Gateway’ button at the bottom of the screen, and the Gateway will be created for you.

After a while, the Gateway will be ready and the properties of the Virtual Network should look like this:

image

 

The error ‘Certificate not set up’ will be handled in a later step of our setup.

 

STEP 3 – Create Domain controller

In this step, we will run a PowerShell script that will create our domain controller.

The script looks like this (you should edit variables to match your environment).

# CREATE BIZTALK DOMAIN CONTROLLER

#Variables
$dcname = 'btsdcDemoLive'
$AG = 'BizTalkAFLive'
$vnet = 'BizTalkVNLive'
$adminusername = 'codit'
$myDNS = New-AzureDNS -Name $dcname -IPAddress '127.0.0.1'
$vmname = $dcname
$image = 'a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201305.01-en.us-127GB.vhd'# old image'a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201302.01-en.us-30GB.vhd'
$service = $dcname
$restart = 'false'
$credential = Get-Credential

#Create Virtual Machine
$mydc = New-AzureVMConfig -name $vmname -InstanceSize 'ExtraSmall' -ImageName $image | Add-AzureProvisioningConfig -Windows -Password $password -AdminUsername $adminusername | Set-AzureSubnet -SubnetNames 'Subnet-1'
New-AzureVM -ServiceName $service -AffinityGroup $AG -VMs $mydc -DnsSettings $myDNS -VNetName $vnet -WaitForBoot
WaitStartVM $service $vmname $restart

# Get the uri for remote configuration
# Get credentials to connect to the remote machine
$cmduri = Get-AzureWinRMUri -ServiceName $service -Name $vmname

#Install the certificate for remote PowerShell
InstallWinRMCert $service $vmname
 
#enable ping (closed by default)
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
Import-Module NetSecurity
Set-NetFirewallRule -DisplayName "File and Printer Sharing (Echo Request - ICMPv4-In)" -enabled True
}

#Add Domain Controller Windows feature
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
$loglabel = $((get-date).ToString("yyyyMMddHHmmss"))
$logPath= "$env:TEMP\init_dcservervm_install_$loglabel.txt"
Import-Module -Name ServerManager
Install-WindowsFeature -Name AD-Domain-Services -IncludeManagementTools -LogPath $logPath
}

#Add DNS Windows Feature
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
$loglabel = $((get-date).ToString("yyyyMMddHHmmss"))
$logPath= "$env:TEMP\init_dcservervm_install_$loglabel.txt"
Import-Module -Name ServerManager
Install-WindowsFeature -Name DNS -IncludeManagementTools -LogPath $logPath
}

#Promote to domain controller
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
$loglabel = $((get-date).ToString("yyyyMMddHHmmss"))
$logPath= "$env:TEMP\init_dcservervm_install_$loglabel.txt"
$safemodepass = ConvertTo-SecureString <password> -asPlainText -Force
Import-Module ADDSDeployment
Install-ADDSForest `
	-InstallDns:$true `
	-CreateDnsDelegation:$false `
	-DatabasePath "C:\Windows\NTDS" `
	-SysvolPath "C:\Windows\SYSVOL" `
	-DomainName "BTS2013.com" `
	-DomainNetbiosName "BTS2013" `
	-NoRebootOnCompletion:$false `
	-Force:$true `
	-LogPath $logPath `
	-SafeModeAdministratorPassword $safemodepass
}

Let’s go over the steps executed in the script above:

 

Line 4 to 12: we set variables that will be used

Line 8: DNS server, the domain controller will use itself as DNS server

Line 13: Get-Credentials asks for credentials in an interactive way. This allows you to create scripts without passwords in clear text in the script.

image

Line 16 to 17: Here we create the Virtual Machine in Windows Azure based on an Image. We set the configuration of our Virtual Machine via the parameters.

The Virtual Machine Image can be either:

- A Windows Azure Gallery Image: Microsoft provided Images

- A self-created Image that you first upload to Windows Azure

- Third party Image

Line 18: The WaitStartVM is a function that waits for a VM to be completely ready to start receiving remote PowerShell commands. By default, PowerShell will not wait for the Virtual Machine to finish ‘provisioning’. PowerShell well continue as soon as the Virtual Machine is ‘Started’. This function will fix that.

The function looks like this:

function WaitStartVM($serviceName, $vmname, $restart)
{
	if($restart -eq 'false')
	{
	Start-AzureVM -ServiceName $serviceName -Name $vmname
	}
	else
	{
	Restart-AzureVM -ServiceName $serviceName -Name $vmname
	}
	do{
	$vm = Get-AzureVM -ServiceName $serviceName -Name $vmname
	$state = $vm.PowerState
	$instancestate = $vm.InstanceStatus
	write-host "Waiting for VM to start..."
	sleep 10
	} until ($state -eq 'Started' -and $instancestate -eq 'ReadyRole')
	write-host "VM Started!"
}

We will also use this function throughout the other scripts of our environment setup.

Line 22: We get the URI where our Virtual Machine will accept remote PowerShell commands.

Line 25: The InstallWinRMCert is a function that downloads and installs the certificate that is necessary to execute remote PowerShell commands on our Virtual Machine in a secure way. More info about this function can be found here.

The InstallWinRMCert function looks like this:

function InstallWinRMCert($serviceName, $vmname)
{
    $winRMCert = (Get-AzureVM -ServiceName $serviceName -Name $vmname | select -ExpandProperty vm).DefaultWinRMCertificateThumbprint
 
    $AzureX509cert = Get-AzureCertificate -ServiceName $serviceName -Thumbprint $winRMCert -ThumbprintAlgorithm sha1
 
    $certTempFile = [IO.Path]::GetTempFileName()
    Write-Host $certTempFile
    $AzureX509cert.Data | Out-File $certTempFile
 
    # Target The Cert That Needs To Be Imported
    $CertToImport = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 $certTempFile
 
    $store = New-Object System.Security.Cryptography.X509Certificates.X509Store "Root", "LocalMachine"
    $store.Certificates.Count
    $store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite)
    $store.Add($CertToImport)
    $store.Close()
 
    Remove-Item $certTempFile
}

We will also use this function throughout the other scripts of our environment setup.

 

Line 28 to 31: The invoke-command executes on a remote machine, in this case our Virtual Machine in Windows Azure. In this step we enable ICMP in the Windows Firewall. This will allow us to ping the machine in the Virtual Network.

Line 33 to 39: We add the ‘Active Directory’ Windows Feature to our Virtual Machine.

Line 41 to 47: We add the ‘DNS’ Windows Feature to our Virtual Machine.

Line 49 to 66: We promote our Virtual Machine to domain controller.

 

Domain Users and Groups

Because we are creating a domain that must host a BizTalk environment, we will also need to created the necessary users and groups in Active Directory.

Also this is scripted.

In the first script, I create the necessary domain users:

#Create Biztalk user accounts

# BiztalkAdmin, btssvc, btsisolated, entsso,
New-ADUser -SamAccountName "BiztalkAdmin" -Name "BiztalkAdmin" -DisplayName "Biztalk Administrator" -PasswordNeverExpires 1
Set-ADAccountPassword -Identity BiztalkAdmin -NewPassword (ConvertTo-SecureString -AsPlainText "azert@12345" -Force)

New-ADUser -SamAccountName "btssvc" -Name "btssvc" -DisplayName "Biztalk Service Account" -PasswordNeverExpires 1
Set-ADAccountPassword -Identity btssvc -NewPassword (ConvertTo-SecureString -AsPlainText "azert@12345" -Force)

New-ADUser -SamAccountName "btsisolated" -Name "btsisolated" -DisplayName "Biztalk Isolated Host Users" -PasswordNeverExpires 1
Set-ADAccountPassword -Identity btsisolated -NewPassword (ConvertTo-SecureString -AsPlainText "azert@12345" -Force)

New-ADUser -SamAccountName "entsso" -Name "entsso" -DisplayName "Enterprise Single Sign Service" -PasswordNeverExpires 1
Set-ADAccountPassword -Identity entsso -NewPassword (ConvertTo-SecureString -AsPlainText "azert@12345" -Force)

#Now Enable those accounts that you just created

Enable-ADAccount -Identity "BiztalkAdmin"
Enable-ADAccount -Identity "btssvc"
Enable-ADAccount -Identity "btsisolated"
Enable-ADAccount -Identity "entsso"

Next, we need to create the BizTalk domain groups:

# Create Biztalk Administrators Domain Group
New-ADGroup -Name "Biztalk Server Administrators" -SamAccountName BiztalkServerAdmins -GroupCategory Security -GroupScope Global -DisplayName "Biztalk Server Administrators" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are Biztalk Server Administrators"
# Create Biztalk Server Operators Group
New-ADGroup -Name "Biztalk Server Operators" -SamAccountName BiztalkServerOperators -GroupCategory Security -GroupScope Global -DisplayName "Biztalk Server Operators" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are Biztalk Server Operators"
# Create Biztalk Server Operators Group
New-ADGroup -Name "Biztalk B2B Server Operators" -SamAccountName BiztalkB2BServerOperators -GroupCategory Security -GroupScope Global -DisplayName "Biztalk B2B Server Operators" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are Biztalk B2B Server Administrators"
# Create SSO Administrators Domain Group
New-ADGroup -Name "SSO Administrators" -SamAccountName SSOAdmins -GroupCategory Security -GroupScope Global -DisplayName "SSO Administrators" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are SSO Administrators"
# Create SSO Affiliate Administrators Domain Group
New-ADGroup -Name "SSO Affiliate Administrators" -SamAccountName SSOAffiliateAdmins -GroupCategory Security -GroupScope Global -DisplayName "SSO Affiliate Administrators" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are SSO Affiliate Administrators"
# Create Biztalk Application Users Domain Group
New-ADGroup -Name "Biztalk Application Users" -SamAccountName BiztalkApplicationUsers -GroupCategory Security -GroupScope Global -DisplayName "Biztalk Application Users" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are Biztalk Application Users"
# Create Biztalk Administrators Domain Group
New-ADGroup -Name "Biztalk Isolated Host Users" -SamAccountName BiztalkIsolatedHostUsers -GroupCategory Security -GroupScope Global -DisplayName "Biztalk Isolated Host Users" -Path "CN=Users,DC=BTS2013,DC=Com" -Description "Members of this group are Biztalk Isolated Host Users"

Finally, the users we just created must be added to the correct groups:

#Add users to appropriate Groups

#BiztalkAdmin to Biztalk Server Administrators
$user = Get-ADUser -Identity "BiztalkAdmin"
$group = Get-ADGroup -Identity "BiztalkServerAdmins"
Add-ADGroupMember $group -Member $user

#BiztalkAdmin to Domain Admins
$user = Get-ADUser -Identity "BiztalkAdmin"
$group = Get-ADGroup -Identity "Domain Admins"
Add-ADGroupMember $group -Member $user

#BiztalkAdmin to SSO Administrators
$user = Get-ADUser -Identity "BiztalkAdmin"
$group = Get-ADGroup -Identity "SSOAdmins"
Add-ADGroupMember $group -Member $user


#BTSSVC to Biztalk Application Users
$user = Get-ADUser -Identity "btssvc"
$group = Get-ADGroup -Identity "BiztalkApplicationUsers"
Add-ADGroupMember $group -Member $user

#BTSSVC to Biztalk Application Users
$user = Get-ADUser -Identity "btsisolated"
$group = Get-ADGroup -Identity "BiztalkIsolatedHostUsers"
Add-ADGroupMember $group -Member $user


#entsso to entssoadmins and entssoaffiliateadmins
$user = Get-ADUser -Identity "entsso"
$group = Get-ADGroup -Identity "SSOAdmins"
Add-ADGroupMember $group -Member $user


#entsso to entssoadmins and entssoaffiliateadmins
$user = Get-ADUser -Identity "entsso"
$group = Get-ADGroup -Identity "SSOAffiliateAdmins"
Add-ADGroupMember $group -Member $user

 

The Virtual Machines node in the Windows Azure Management Portal should look like this:

image

 

You can open a Remote Desktop session on the created Virtual Machine. Click the ‘Connect’ button at the bottom of the screen.

Let’s also verify that our users and groups are created in Active Directory:

image

 

When the scripts of STEP 3 executed successfully, you will have an operational Active Directory Domain Controller in the Virtual Network created in STEP 2.

The domain is ready and all necessary users and group are created in Active Directory.

We can now continue with the setup of SQL and BizTalk.

STEP 4 – Create SQL Server

In this step, we will run a PowerShell script that will create our SQL Server Virtual Machine.

After analyzing the script of STEP 3, most things will be already familiar! (the explanation will not be repeated here)

The script looks like this (you should edit variables to match your environment).

$credential = Get-Credential
$sqlname = 'sqlbtsDemoLive'
$AG = 'BizTalkAFLive'
$vnet = 'BizTalkVNLive'
$adminusername = 'codit'
$mySQLDNS = New-AzureDNS -Name 'sqldnsLive' -IPAddress '10.0.0.4'
$vmname = $sqlname
$image = '2cdc6229df6344129ee553dd3499f0d3__BizTalk-Server-2013-Evaluation'
$sqlservice = $sqlname
$restart = 'false'
					
#Create the SQL Server Virtual Machine																																																																																																																$password = 'Codit*1234'
$mysql = New-AzureVMConfig -name $vmname -InstanceSize 'Small' -ImageName $image | Add-AzureProvisioningConfig -Windows -Password $password -AdminUsername $adminusername | Set-AzureSubnet -SubnetNames 'Subnet-1'
 New-AzureVM -ServiceName $sqlservice -AffinityGroup $AG -VMs $mysql -DnsSettings $mySQLDNS -VNetName $vnet
WaitStartVM $sqlservice $vmname $restart

#Get the remote PowerShell URI
$cmduri = Get-AzureWinRMUri -ServiceName $sqlservice -Name $vmname
InstallWinRMCert $sqlservice $vmname

#Add the SQL Server Virtual Machine to the domain we created in STEP 3
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
$domain = "BTS2013.com"
$password = <password> | ConvertTo-SecureString -asPlainText -Force
$username = "$BTS2013\Codit" 
$credential = New-Object System.Management.Automation.PSCredential($username,$password)
Add-Computer -DomainName $domain -Credential $credential
}
#Restart for domain join
$restart = 'true'
WaitStartVM $sqlservice $vmname $restart

#Set firewall rule and disable services we don't use (save resources)
#Enable MSDTC network access
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
Import-Module NetSecurity
Set-NetFirewallRule -DisplayName "File and Printer Sharing (Echo Request - ICMPv4-In)" -enabled True

Set-Service -Name 'ReportServer' -StartupType manual
Set-Service -Name 'MSSQLServerOLAPService' -StartupType manual
Set-Service -Name 'MsDtsServer110' -StartupType manual
Set-Service -Name 'SQLServerAgent' -StartupType automatic
Stop-Service -Name 'ReportServer'
Stop-Service -Name 'MSSQLServerOLAPService'
Stop-Service -Name 'MsDtsServer110'
Start-Service -Name 'SQLServerAgent'
#MSDTC settings!
Set-DtcNetworkSetting -DtcName Local -InboundTransactionsEnabled 1 -OutboundTransactionsEnabled 1 -RemoteAdministrationAccessEnabled 1 -RemoteClientAccessEnabled 1 -AuthenticationLevel NoAuth -Confirm:$false
}

#Open SQL Ports in the Windows Firewall (closed by default)
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
Import-Module NetSecurity
$GroupName = "BizTalk 2013"
$SQLPortLocal = 1433
$SQLBrowserPort = 1434 
$RSQL = New-NetFirewallRule -DisplayName "SQL Server Communication (TCP-In)" -Description "This rule opens the SQL Server communication port $($SQLLocalPort) to the IP Address $($WFEIPAddress) for SharePoint 2013 communication" -Direction Inbound -LocalPort $SQLPortLocal -Group $GroupName -Protocol TCP -Profile Domain -Action Allow -RemoteAddress Any
Write-Host "$($RSQL.DisplayName) created: " -nonewline; Write-Host -foregroundcolor green "$($RSQL.PrimaryStatus)"

#Rule for SQL Browser - UDP Port 1434 Inbound
$RSQLBROWSER = New-NetFirewallRule -DisplayName "SQL Server Browser (UDP-In)" -Description "This rule opens the SQL Server Browser UDP port $($SQLBrowserPort) to the IP Address $($WFEIPAddress)" -Direction Inbound -LocalPort $SQLBrowserPort -Group $GroupName -Protocol UDP -Profile Domain -Action Allow -RemoteAddress Any
Write-Host "$($RSQLBROWSER.DisplayName) created: " -nonewline; Write-Host -foregroundcolor green "$($RSQLBROWSER.PrimaryStatus)"
}

Let’s go over the steps executed in the script above (I will only explain new things):

Line 6: DNS Server: The DNS Server that will be used by SQL Server should be the IP address of the domain controller we set up in STEP 3.

Line 8: Note that even for a SQL Server Virtual Machine we are using the BizTalk Evaluation Gallery Image. The reason for this is, at the point of writing this post, there is no SQL Server Evaluation Image available. The BizTalk Evaluation Image also contains a SQL Server Evaluation.

Line 21 to 28: We add our newly created SQL Server Virtual Machine to the BTS2013 domain we created in STEP 3.

Line 30 to 21: After joining the domain, we have to reboot our SQL Server.

Line 35 to 49: We configure the services start-up types, and stop any service we don’t need on our SQL Server.

Line 51 to 62: We open Windows Firewall to allow BizTalk Server to connect with SQL Server. SQL Server ports are closed by default on the Windows Azure Gallery Image we are using.

When the script of STEP 4 executed successfully, you will have an operational SQL Server in the Virtual Network created in STEP 2.

 

STEP 5 – Create BizTalk Server

In this step, we will run a PowerShell script that will create our BizTalk Server 2013 Virtual Machine.

After analyzing the script of STEP 3 and 4, most things will be already familiar! (the explanation will not be repeated here)

The script looks like this (you should edit variables to match your environment).

#Variables
$btsname = 'bts2013DemoLive'
$AG = 'BizTalkAFLive'
$vnet = 'BizTalkVNLive'
$credential = Get-Credential
$adminusername = 'codit'
$mybtsDNS = New-AzureDNS -Name 'btsdnsLive' -IPAddress '10.0.0.4'
$vmname = $btsname
$image = '2cdc6229df6344129ee553dd3499f0d3__BizTalk-Server-2013-Evaluation'
$btsservice = $btsname
			
#Create the BizTalk Virtual Machine																																																																																																																																						$password = 'Codit*1234'
$restart = 'false'
$mybts = New-AzureVMConfig -name $vmname -InstanceSize 'Small' -ImageName $image | Add-AzureProvisioningConfig -Windows -Password $password -AdminUsername $adminusername | Set-AzureSubnet -SubnetNames 'Subnet-1'
 New-AzureVM -ServiceName $btsservice -AffinityGroup $AG -VMs $mybts -DnsSettings $mybtsDNS -VNetName $vnet
WaitStartVM $btsservice $vmname $restart

#Get URI for remote PowerShell
$cmduri = Get-AzureWinRMUri -ServiceName $btsservice -Name $vmname
InstallWinRMCert $btsservice $vmname

#Add Domain Controller Windows feature
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
$domain = "BTS2013.com"
$password = <password> | ConvertTo-SecureString -asPlainText -Force
$username = "$BTS2013\Codit" 
$credential = New-Object System.Management.Automation.PSCredential($username,$password)
Add-Computer -DomainName $domain -Credential $credential
}

#Restart for domain join
$restart = 'true'
WaitStartVM  $btsservice $vmname $true

#Set firewall rules, Stop all SQL Services and set to manual start-up type, enable network access for MSDTC
Invoke-Command -ConnectionUri $cmduri.ToString() -Credential $credential -ScriptBlock {
Import-Module NetSecurity
Set-NetFirewallRule -DisplayName "File and Printer Sharing (Echo Request - ICMPv4-In)" -enabled True

Set-Service -Name 'SQLWriter' -StartupType manual
Set-Service -Name 'ReportServer' -StartupType manual
Set-Service -Name 'MSSQLServerOLAPService' -StartupType manual
Set-Service -Name 'MSSQLServer' -StartupType manual
Set-Service -Name 'MSSQLFDLauncher' -StartupType manual
Set-Service -Name 'MsDtsServer110' -StartupType manual
Stop-Service -Name 'ReportServer'
Stop-Service -Name 'SQLWriter'
Stop-Service -Name 'MSSQLServerOLAPService'
Stop-Service -Name 'MSSQLFDLauncher'
Stop-Service -Name 'MsDtsServer110'
Stop-Service -Name 'MSSQLServer'

#MSDTC settings!
Set-DtcNetworkSetting -DtcName Local -InboundTransactionsEnabled 1 -OutboundTransactionsEnabled 1 -RemoteAdministrationAccessEnabled 1 -RemoteClientAccessEnabled 1 -AuthenticationLevel NoAuth -Confirm:$false
}

WaitStartVM  $btsservice $vmname $true

Line 40 to 51: I stop all services related to SQL Server. In STEP 4 we set up a separate machine for SQL Server, we don’t need SQL on the BizTalk machines. Also the start-up types are set to manual.

 

When the script of STEP 5 executed successfully, you will have an operational BizTalk Server 2013 in the Virtual Network created in STEP 2.

Let’s now verify that all our machines our up-and-running in Windows Azure.

Navigate to the ‘Virtual Machines’ node. In the screenshot below you can see that the domain controller, SQL Server and BizTalk Server are provisioned and running.

image

Click on the ‘Virtual Network’ node, and navigate to the network that was created prior in this post.

As you can see, our 3 new virtual machines are added in the ‘biztalkvnlive’ Virtual Network.

image

 

STEP 6 – Configure BizTalk Server

The BizTalk Server Gallery images in Windows Azure provide us with a new tool to configure BizTalk.

The Provisioning Tool allows you to configure a complete BizTalk group (multiple servers) at once. You run the Provisioning Tool on one server, all other servers will get their configuration via the tool!

Unfortunately, at this point in time, the Provisioning Tool does not support Active Directory groups yet, so we can’t use it in this situation.

 

To configure BizTalk in this case, I will run the BizTalk ‘Configuration.exe’ as a command line tool. The configuration is set by providing an Xml file as a parameter. The Xml file contains the BizTalk configuration.

To run the configuration, execute this command:

configuration /s c:\BTSConfig\btsconfigExport.xml /l c:\BTSConfig\log.txt

The first parameter is the configuration as Xml, the second parameter is the log file to use where possible errors are written to.

After executing the command above, BizTalk will start its configuration:

image

 

STEP 7 – Deploy BizTalk test application

STEP 7 will not be detailed in this blog post. Just create a small BizTalk application that is capable of picking up a file at a certain location and write it to an output folder.

STEP 8 – Network connectivity – Certificates

In this step, I will setup network connectivity between the Windows Azure Virtual Network and my local laptop.

We want this connection to be secure,  because of that, we will need to generate a public/private key pair to secure the connection.

First we will generate a self-signed root certificate. This certificate will be used to generate client certificates for every device that needs to connect to the cloud network.

Certificates can be generated with the ‘makecert’ tool.

To create the root certificate, execute this command:

makecert -sky exchange -r -n "CN=BizTalkVNLiveRoot" –pe -a sha1 -len 2048 -ss My

 

Next, I will generate a client certificate from the root certificate that is just created. The client certificate will serve to identify my client computer to the cloud. The root certificate will be used to validate my client certificate in the cloud.

To create a client certificate from the root certificate, execute this command:

makecert -n "CN=BizTalkVNLiveClient" -pe -sky exchange -m 96 -ss My -in "BizTalkVNLiveRoot" -is my -a sha1

Install the client certificate on your local computer by double clicking the ‘.pfx’ file. Follow the instructions.

Export the root certificate to a .cer file (from the Windows Certificate store).

 

Now we have to tell our Windows Azure Virtual Network what client certificates are trusted to connect to the Virtual Network.

To do this, browse to the Virtual Network that was created in STEP 2.

The follow screen should be visible:

image

 

Click the ‘Upload client certificate’ link. The caption of this link is not correct! We will NOT upload the client certificate. We upload the root certificate that is able to issue client certificates.

 

STEP 9 - Network connectivity – VPN package

After uploading the certificate in the previous step, there is no more error on our Virtual Network:

 

image

Next we will prepare our client laptop to connect to the Virtual Network in Windows Azure.

Download and install the Client VPN Package that matches your machines hardware architecture. The download link to the VPN package is displayed on the bottom right of the screenshot above.

When the Client VPN Package is installed, go to you network settings.

The Virtual Network will have a separate entry in the network list:

image

 

Right click the ‘BizTalkVNLive’ network and click ‘Connect’. You will now connect your laptop with the Virtual Network in Windows Azure.

 

STEP 10 – Verification

In this step, test network connectivity between your client laptop and the Virtual Network by ‘pinging’ BizTalk Server.

When this works, you can add a receive location to the test application that you created earlier. Let this receive location pick-up a file from your local laptop over the Point-To-Site Virtual Network that is connected.

 

 

Peter Borremans


June 3, 2013 at 11:56 PM

The need for custom code


Very early in the beginning of EAI & EDI services on Azure, we had some discussions with the product team around their architecture and the impact on features.  In the early stages, everything was designed so that they could deploy Bridges to a high density, multi-tenant scaled out infrastructure, where flows of different companies and tenants would be sharing the same machines.

The impact of that decision was that would need to trade off on customization.  Things like custom functoids, custom WCF bindings and especially custom pipeline components were not in scope, because in that setup it seemed very hard to foresee sandboxing mechanisms to isolate all custom code in separate containers.  (this is something that is often true for other Azure services, it seems).

Luckily enough, based on feedback from early adopters, they decided to support custom code after all.  This lead to an architecture change where every deployment gets deployed to a set of dedicated virtual machines.

 

In integration projects, it is very often that there is a need to make that little, specific tweak to a message or an envelope.  Because interoperability is not always interoperability Smile  For things like that, extensibility is the only way out.

This post describes how to develop custom bridge components and set properties on these.

 

Develop a custom bridge component


Create a new project

  • First thing to do is to create a new Class Library project and make sure it compiles as a 4.5 project (needed to get all references)
  • Then add a reference to the BizTalk Services dll (can be found in \Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft\Windows Azure BizTalk Services SDK\Microsoft.BizTalk.Services.dll)
  • The project has to be strong signed, before it can get uploaded to the Windows Azure BizTalk Service, so you have to foresee a strong name key.

Implement the right interface

The first thing to do is to create your specific class and implement the IMessageInspector interface.

This interface has an Execute method that should return a Task:

Here is my class to start with.

using Microsoft.BizTalk.Services; 
using System; 
using System.Threading.Tasks;

namespace CoditBlog.CustomCodeComponent 
{ 
    public class BlogCustomComponent : IMessageInspector 
    { 
        public Task Execute(IMessage message, IMessageInspectorContext context) 
        { 
            throw new NotImplementedException(); 
        } 
    } 
}

 

Implement your logic in the Execute method

In the Execute method, you can implement your logic (here I am just promoting a hardcoded message property, nothing fancy.

return Task.Factory.StartNew(() => 
{ 
    message.Promote("PropertyName", "PropertyValue"); 
});

Using the custom component in a bridge


Get the fully qualified name of your class

The current SDK does not foresee in a very user friendly way of configuring a custom Message Inspector (Microsoft knows they’ll have to change that Smile), because you need to type your fully qualified name of the inspector type in the bridge configuration.

There are two options to get the fully qualified name

Option 1: the hardcore way

You can type it yourself:

FullTypeName, AssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=PublicKeyToken

To get the public key token, you can use the sn –T assembly.dll (use this in External tools with –T $(TargetPath) as arguments

Option 2: use the tool that comes with the samples

  • Get the sample GetAssemblyQualifiedTypeName from here
  • Compile the GetAssemblyQualifiedTypeName project and copy the output file path (GetAssemblyQualifiedTypeName.exe)
  • Go to Visual Studio and select Tools - External Tools in the menu
  • Add a new tool and provide the following information
    • Title: Get FQN
    • Command: the file path your just copied
    • Arguments: $(TargetPath) $(TargetName) ClassName
    • User Output Windows: checked
    • Prompt for arguments: checked
      image
  • When your custom code project gets compiled, you can launch this tool and the only thing you need to do is change the ClassName to the class name of your custom inspector.  As a result, you’ll get the FQN in the output window.
    CoditBlog.CustomCodeComponent.BlogCustomComponent, CoditBlog.CustomCodeComponent, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6c7158deb8680b90

Reference the inspector from your bridge

In your bridge, you can configure the custom inspector on the stage of your choice.  In the property window of that stage, you can select the On Enter Inspector or On Exit Inspector, which get obviously executed at the beginning or the end of that stage.

Click the ellipsis and in the window, you can enter your FQN of the class type, like you can see in the following screenshot:

image

In this pop up, you can also specify custom property values (see later)

Deploy the bridge

Add a reference from your bridge project to your custom bridge component and redeploy it (after specifying the right deployment URL). 

Now you can go ahead and test it.

 

 

Important remarks around deploying custom bridge components


If you deploy custom code to the BizTalk Service, you need to make sure your custom dll’s are getting deployed.  There are two options here:

Deploy from Visual Studio

This is the most common option.  In your bridge project, add a reference to your custom code project and make sure the reference is set with Copy Local = true.  If you then deploy, all locally referenced dll’s will get deployed with the bridge.

When deploying a changed or new component, it is important to flag the refresh checkbox on the deployment window.

image

It will then take a while before the BizTalk Service ‘container’ gets recycled to pick up the refreshed bits.

Tip: when using a lot of dll’s that get referenced (as we do with our framework), it is advised to set the Copy Local property to false.  This will make sure that the referenced assemblies will not be re-uploaded with every change in your bridge

 

Deploy the custom components as artifacts through the portal

You can also upload the bridge components through the portal, by adding them as a resource. (click the upload button in the screenshot below)

image

 

The possibilities with custom bridge components


There are luckily more things one can do with custom bridge components.

Using pipeline properties

This is a very new feature (during the BizTalk Summit presentation, end of 2012, that feature was not even on the roadmap).  What is really needed is that for a lot of components, you need to be able to specify some properties (like the PropertyBag with BizTalk pipeline components).  As we all want, we can create generic components that can then get specifically customized with the properties in a specific bridge.

To allow pipeline properties on a bridge component, you need to create public string properties and attribute them with the PipelineProperty name.

Here I have two pipeline properties declared for the property name and value:

[PipelineProperty(Name = "Value")] 
public string PropertyValue { get; set; }

[PipelineProperty(Name = "Name")] 
public string PropertyName { get; set; }

I also updated the bridge configuration to provide the property configuration to my inspector.  This can be done by adding a new line for every property configuration with the PropertyName=Value pattern:

image

Important to know:

  • Pipeline properties have to be of type string!
  • The property configuration you specify is using the Name of the PipelineProperty attributes, not the public property name (to which it defaults)
  • It is not needed to provide values for all pipeline properties

 

Changing the message body

It is possible to change the body of a message in an inspector.  This can be done by accessing the Data property on the message.

The Data property is a Stream, so you work in the same way as we used to work with streaming pipeline components in BizTalk.  Update the Data property of the message with your custom stream that wraps the incoming original stream…

Here, I am just updating the message to a hardcoded value:

message.Data = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("<updated />"));

The body is of type ‘Microsoft.ApplicationServer.Integration.Common.VirtualFileStream’.

 

Writing custom logging events

it is possible to write custom logging events to your diagnostics table in the configured storage account.  If you use the Tracer from the IMessageInspectorContext, you can write events to the WADLogsTable. 

context.Tracer.TraceEvent(TraceEventType.Warning, "Value: {0}", PropertyValue);

 

Using general configuration values

A disadvantage that I experienced during the testing of WABS, was that it is not possible to have environment-wide configuration values.  Connection strings, etc

The only thing I could do to make it loose coupled, was to deploy an assembly and load it through reflection (to avoid references between projects).  That assembly then contained a class with a static value with the setting.  I hope this gets better in a future release.

 

More information can be found here: http://msdn.microsoft.com/en-us/library/windowsazure/dn232389.aspx

 

Sam Vanhoutte


June 3, 2013 at 4:20 PM

In this second blog post on Windows Azure BizTalk Services, we dive a little bit deeper in a concept that is very important for our Codit Integration Cloud project.  All our message processing and integration capabilities is running over bridges. 

This post is not a step-by-step demo walkthrough (Richard Seroter just did a nice one on his blog).  This article intends however to describe some patterns and some architectural explanation.

 

 

Itinerary routing mechanism


A message itinerary is a definition of a flow (it was called the flow designer in the past).  Let’s take the following example of a more complex itinerary:

image

The arrows specify the path that a message can follow.  It is a visual flow representation that links different components together. 

 

Persistence of itineraries

An itinerary does not have any persistence points and executes 100% in memory.  (example: If an FTP destination in an itinerary is unavailable, then there are no retries or whatsoever.  the consumer will get a 500 exception in that case!)

First match unicast routing

There is no one-to-many routing in itineraries.  Routing always happens on the first-matching path!  At every connection point, the message checks the first connection (based on the Sequence Number).  If the condition on that link matches, the message will be routed over that connection and all the other connections are ignored. 

The sequence number gets defined automatically, based on the sequence of adding the connections.

If you want to change the Sequence Number, you need to select the bridge and then you can configure the routing table with sequences in the Route Ordering Table:

image

Implement multi-cast routing

Multi-cast routing is routing a message, so that one message can be routed to multiple (or none) destinations.  If this needs to be implemented, then it is needed to use a service bus Topic with multiple subscriptions.  However, there is a problem, because at this moment, there is no service bus messaging listener implemented yet (like an FTP listener, but on a service bus queue or subscription).

No matching subscription

When a message is not able to arrive on an actual destination node, then the client will get a 500 error from the bridge and in the tracking data, there will be an error with the following description: “No Filter matched for the message.”.

 

Different bridge types


After we discussed the message itinerary, we can have a look at the bridges in detail.

image

As you can see in the screenshot above, there are different types of bridges:

  • XML One-Way bridge: this is a bridge that processes incoming messages and passes them on to the next matching connection.
  • XML Request-Reply bridge: this is a bridge that processes incoming messages and their corresponding response message.
  • PassThrough bridge: this is a bridge that does not look at the message details and can parse every type of message.
  • EDI bridge: unfortunately, this bridge is not available to the Visual Studio designer and is shielded by the TPM portal.

I would have loved to see a PassThrough Request-Reply bridge and a customizable EDI bridge.  These are not part of this release, but I’m sure they will come in a future release.  (that is the advantage of a cloud service – it’s much easier to add functionality for Microsoft now)

 

 

The different stages of a bridge


A bridge (much like a BizTalk pipeline) has different stages.  Depending on the type of bridge, one or more stages can be missing for that bridge.  The first thing to know is that you can easily disable a stage in the property window.  (that is something you typically do when disabling schema validation, for example).  Below, you can see a screenshot of a Request-Reply bridge.

image

Message types

(only in XML bridges)

The first step in an XML bridge is the section where all accepted message types are being defined.  This is done by adding schemas from a project to the request or response (in a Request-Reply bridge) direction.

Important to know:

  • You can add multiple schemas to this section.  The schema that matches the incoming message will be chosen.
  • The schemas you add have to be part of the current project and can not be added from a referenced project.  This is something that hopefully will change shortly.  (we want to reuse schemas and not redefine them in every single project)

image

Decode stage

(only in One-Way XML bridge)

The decode stage will be used if the incoming message is a flat file schema.  Incoming messages will then be serialized to XML, using the flat file schema schema.  This schema contains all the annotations that the Flat File parser will use to create XML out of the incoming flat file message.

Important to know:

  • You can also add flat file schemas in a One-Way bridge (not in an Request-Reply bridge!)  This makes it difficult to provide flat file parsing as a service.
  • The flat file schemas (and the Flat File Schema editor extension) are exactly the same as in BizTalk Server.  This makes it very easy to move flat file parsing between BizTalk Server and Services.

Validate stage

(only in XML bridges)

The validate stage validates the message at that time against the schema that is assigned to it.  The validation can be easily disabled, by disabling this stage (Is Enabled property).

On the validate step, you can specify how warnings should be handled.

Enrich stage

(available in all bridges)

This stage allows to configure property promotion.  All properties can be defined here and can be extracted from the incoming message or the context of the bridge.  There are different locations where this stage gets executed (before / after mapping , request / reply direction)
To configure the stage, the property definitions window should be used.

image

Property values can be read from the following sources:

SOAP Action, MessageId, To, ReplyTo
HTTP Here the name of an incoming HTTP header can be extracted and promoted to the property
FTP FileName, ServerAddress, Folder
SFTP FileName, ServerAddress, Folder
System The following properties can be retrieved from the system context: RequestId (Guid), MessageReceivedTime (UTC time), SourceName, SourceType (Http, Ftp…)
XPath Here an xpath statement (of the long type) should be entered and the schema has to be selected.  If the incoming message matches that schema, the lookup value will be done and promoted.  If the Xpath is not found, the property does not get promoted.
Lookup Here, it is possible to do a SQL Azure database lookup and promote the result of the query to a configured property.  All the database configuration settings are stored in the LookupProviderConfigurations.xml file.
The following screenshot shows the configuration settings for this:
image

 

Important to know:

  • If a source property does not exist (for example: Ftp FileName in a Http bridge), there is no error raised, the property just does not get promoted (which is good)
  • With the Lookup configuration, it is not possible to provide multiple input fields to a table (as we do with our framework)

Transform stage

(only in XML bridges)

The transform stage allows to configure one or more mappings that will be executed against the incoming message.  The mappings and mapper will be discussed in a later post.

Important to know:

  • You can only select a mapping that matches with the schema, configured for that direction of the bridge.  In the receive side, only mappings with a source schema that matches the request schema will be enabled for selection.  In the reply side, only mappings with a destination schema that matches the response schema are enabled for selection.
  • You cannot select multiple mappings with the same source schema in one transformation selection.  In that case you get the compilation warning: “Two or more maps selected in the Request Transform stage have the same source message type.”.

image

Send reply stage

(only in XML Request-Reply bridges)

In the Send reply stage, it is possible to read message properties and write them to the specific destination protocol (like Soap or HTTP headers).  Message properties can be written to Http or Soap headers of the response.

Important to know:

  • Only properties that have been declared in the Enrich stages are visible here.  If you have promoted a property in a custom bridge component, it is not visible here.  To promoting custom bridge properties, it is needed to define them in an enrich stage first (and fill them with the Tracking Id, for example). 
  • When the destination of an itinerary is a Service Bus messaging entity, the promoted properties are automatically being written on the BrokeredMessage, there is no specific action needed to do this.

image

Encode stage

(only in XML One-Way bridges)

In the Encode stage, the flat file schema will be interpreted to convert the XML document to the flat file structure, if the message matches a Flat File schema.

 

Expose a pipeline as a service


You cannot deploy a Request-Reply bridge without connecting it to a destination endpoint.  (or you would get the compilation warning: “Bridge '<bridge>' needs to route to at least one connected entity.”).  That means you need to route it to a TwoWay destination endpoint (like a TwoWay External service endpoint). 

But when you just want to create a ‘pipeline as a service’, for example to provide transformation or property promotion as a service endpoint, you don’t want to route to an external service endpoint, because this would cause increased latency.

Luckily enough there is the possibility to use custom WCF bindings and update the configuration of the TwoWay endpoint to that custom binding.  And when you use the EchoMessageBinding that comes with the Samples in the SDK, the request message just gets returned as the response message to the bridge. 

How to configure the EchoMessageBinding

  • Add a TwoWay external service endpoint destination to the designer canvas.
  • Give it a friendly entity name (for example: LoopbackEP).
  • Add a reference to your project to the EchoMessageBinding

Update the LoopbackEP.config configuration file as the following:

<?xml version="1.0" encoding="utf-8" ?> 
<configuration> 
  <system.serviceModel> 
    <extensions> 
      <bindingElementExtensions> 
        <add name="EchoMessageBindingElement" type="EchoMessageBinding.EchoMessageTransportElement, EchoMessageBinding, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d39b54ec599a4cea"/> 
      </bindingElementExtensions> 
    </extensions> 
    <bindings> 
      <customBinding> 
        <binding name="EchoMessageBindingElement" > 
          <EchoMessageBindingElement /> 
        </binding> 
      </customBinding> 
    </bindings> 
    <client> 
      <!-- The clear below makes sure any client endpoints defined in machine.config don't get added--> 
      <clear /> 
      <endpoint name="TwoWayExternalServiceEndpointReference1" address="http://TwoWayExternalServiceEndpointReference2" binding="basicHttpBinding" contract="System.ServiceModel.Routing.IRequestReplyRouter"/> 
      <endpoint name="LoopbackEP" address="http://EchoMessage" binding="customBinding" bindingConfiguration="EchoMessageBindingElement" 
        contract="System.ServiceModel.Routing.IRequestReplyRouter" /> 
    </client> 
  </system.serviceModel> 
</configuration>

Then make sure the destination is using the LoopbackEP endpoint and you’re all set.

 

Unfortunately, it is not possible to expose Flat File parsing as a service at this point in time, because there are no decode / encode stages in Request-Reply bridges.

Some things you might now know about bridges


You can have multiple bridges as source for an itinerary. 

Every bridge has a specific URI.  So this means that a Message Itinerary can have multiple public endpoint URI’s.

With the following example itinerary, there are 3 public HTTP endpoints that can be called separately from each other.

image

The following itinerary has multiple source bridges available.  Based on the source, other ‘pre processing’ logic can be defined.

image

You can have custom code in bridges

It is possible to configure custom components in bridges.  These are being described in a future blog post.  A component can be configured on a stage in a On Enter and On Exit event.

 

Sam Vanhoutte


June 3, 2013 at 4:10 PM

You may have noticed it.  There is a new Windows Azure Service released on the Windows Azure portal: Windows Azure BizTalk Services.  This blog posts helps you getting started.

It explains the different steps you need to take in order to get you up and ready to begin development.

The official tutorials are to be found online: here.

 

Get the SDK


To download the SDK, adapter service and other items, you can go to this link.  There you can download the following items for the Windows Azure BizTalk Services preview:

  • BizTalk Adapter Service x64
  • BizTalk Adapter Service x86
  • BTMMigrationTool (this is to migrate BizTalk mappings to the new mapper)
  • MicrosoftEdiXsdTemplates (all X12/EDI schemas)
  • WindowsAzureBizTalkServices SDK

Once you have everything installed, your local machine is ready for WABS development

 

Register for the Windows Azure BizTalk Services preview


You should go to the Azure account portal and log on with your Microsoft account, linked with the subscription that you want to get registered for WABS.  On the portal, select the Preview Features button and you can register for the BizTalk Services preview.  After a while (depending on the available preview slots), you can be allowed to the program.

image

 

Create a BizTalk Service


Once you are registered and allowed to the preview program, you can go to the management portal and create a BizTalk Service

It is not a trivial process, so here you have some tips and a step by step guide:

Step 1: go to the BizTalk Services tab

Click on the BizTalk Services tab and you will be able to click the ‘Create a BizTalk Service’ link.

image

Step 2: specify your service settings

In the first step of the wizard, you’ll have to specify some settings:

  • BizTalk Service name: the prefix to the .biztalk.windows.net that will be used for your service DNS
  • Edition: the specific edition you will chose.  Important! : the edition influences your costs, so be aware of this.  The details on the editions will be discussed later on.  Developer is the one you should use for all your PoC / play ground
  • Region: the data center where your services will be running
  • Tracking database: this is a database on your own server that will be used for all tracking information!  This database can be used for your own tracking queries.  You can add it on a new server, or an existing server.
    It is strongly advised to run the database in the same data center as your service!
  • Subscription: the subscription where you will deploy your service.

image

Step 3: Database settings

In the database settings, you can provide a new database, or link to an existing one.  Here you need to specify some settings, based on your selection.

image

Step 4: Access Control namespace and monitoring settings

Here it becomes tricky.  You need to specify an ACS namespace that will be used to create items in, by the Windows Azure BizTalk Service account. 

image

When creating a new namespace, you will have to do this through the Active Directory tab and create an ACS namespace there. 

  • To get the user and password settings, you need to click on the Manage button in the Access Control Namespace section of the Active Directory module in the Azure portal.  This will bring you to the ‘old’ ACS management portal where you can get your user.
  • This should be done by selecting the ‘Management Service’ link on the left.
  • Then either use the existing ManagementClient service account or create a new one
  • Click the service account you want to use and then click the Password link.
  • To get the password, you can click the Show Password button.
    image

The user name is the name of the service account.

Step 5: SSL Certificate

In the last step before creation, you need to upload the private key of an SSL certificate (pfx) that is linked with your DNS. 

You can change this later and use a self-signed cert for now, by executing the following in the developer command prompt:

makecert -pe -r -n "CN=<yourservicename>.biztalk.windows.net" -e "01/01/2015" -ss my

After doing this, you can export your PFX from your certificate store and upload it here.

 

Step 6: Be patient

Once you click create, it can take up to 30 minutes to have your service generated.

image

 

Step 7: Register the service on the ‘Silverlight’ portal

And just when you thought everything was finished, then you have to register your service in the BizTalk management portal.  This portal is a Silverlight portal that is there during the preview. 

Click on the Manage button and you will be taken to the Silverlight portal

image

On this portal, you have to specify three settings:

  • BizTalk Service: the name of the BizTalk Service you specified in the first step
  • Issuer name: the name of an ACS user that belongs to your ACS namepsace
  • Issuer secret: the shared secret key of that user

image

 

 

 

More blog posts coming soon


As a launch partner, we have been working with Windows Azure BizTalk Services for more than a year now and we have prepared a number of blog posts.  In the coming days there will be blog posts on the architecture and features of this brand new service.

Stay tuned!

Sam Vanhoutte


June 3, 2013 at 4:00 PM

Today, Windows Azure BizTalk Services was officially announced on TechEd US.  (more information can be found here: http://blogs.msdn.com/b/windowsazure/archive/2013/06/03/announcing-new-offers-and-services-on-windows-azure.aspx and on the BizTalk blog soon: http://blogs.msdn.com/b/biztalk_server_team_blog/)  From now on, a real preview is available through the Windows Azure portal of these new capabilities.

 

Codit is one of the few world wide launch partners for this service and we have been actively working together with the product team for more than a year now.  With our online integration platform: Codit Integration Cloud, we are actively extending Windows Azure BizTalk Services (WABS) and are adding value for our customers.  At this moment, we are already live on Windows Azure BizTalk Services.

 

Over the coming days, a set of blog posts will be posted to dive a little more deeper in these new services:

  • Windows Azure BizTalk Bridges
  • Building Custom pipeline components for Windows Azure BizTalk Bridges
  • Windows Azure BizTalk Adapter Service
  • Windows Azure BizTalk Mapper
  • Windows Azure BizTalk Trading partner management

Stay tuned!

This first post gives a high level overview of the different components that are part of this release.

 

Bridges & Message itineraries

One of the most important concepts are Bridges (mostly XML bridges).  These can best be compared with BizTalk pipelines.  Bridges are always deployed in a Message itinerary.  An itinerary can be activated through an bridge or through a configured sources (like an FTP/SFTP location).  A bridge can perform the typical messaging/pipeline logic that we know from BizTalk Server: transformation, parsing (flat file and EDI support), property promotion, validation…).

The output of a bridge can then be sent to anther bridge or a destination, like a service bus queue/topic or relay endpoint.

 

Windows Azure BizTalk adapter service

It is perfectly possible to combine BizTalk Services from the cloud with a BizTalk Server on premises.  But for a lot of scenarios or customers, it is well possible that there is no BizTalk Server installed locally.  If these customers however want to have the possibility to expose some of their on premises systems or LOB applications to a bridge, they can leverage the Windows Azure BizTalk Adapter service.

This IIS-hosted service exposes local endpoints from the WCF LOB adapter pack to the cloud, using the Azure Service Bus Relay bindings.  The nice thing is that it is perfectly integrated with the EAI bridges and the designer, so you can easily drop local endpoints to the canvas and generate their corresponding schemas in the project.

 

The following schema depicts the two biggest EAI parts of this release: a Bridge (cloud) with the Adapter Service (on premises)

image

 

 

Windows Azure BizTalk Services – Trading partner management

This it he full featured EDI and TPM portal that is part of this release.  This component allows to configure Trading partners with all their specific settings.  It is possible to configure everything through the portal and to process X12 and shortly EDIFACT messages, coming from your business partners.

The service support multiple protocols, including AS/2 and Bridge/Service Bus entities.

 

Expect more blog posts soon! (the next one on bridges in a few minutes)

 

Sam Vanhoutte

Posted in: Azure | BizTalk | BizTalk Services | Service Bus | WABS

Tags: ,


May 30, 2013 at 10:07 PM

Yesterday, Codit organized an event to give an overview on the new capabilities and scenarios that are made possible with BizTalk Server 2013.  With around 55 people showing up, from 6 different countries, I think we can call this event a success.

 

Basically, Microsoft is delivering the following capabilities around BizTalk this year:

  • Continued innovation in BizTalk Server: The new release of BizTalk Server 2013
  • Use the cloud benefits with existing applications: Support to deploy BizTalk Server in Windows Azure Virtual Machines
  • PaaS components for next generation applications: Windows Azure BizTalk Services
  • Cloud messaging platform: Windows Azure Service Bus

 

We used these four important pillars of the latest enhancements on the integration platform of Microsoft to build our sessions for our event.

 

Here you can find the take aways from the different sessions:

WP_000987

WP_000988

Toon talking about the new #msbts 2013 capabilities @coditcompany

 

 

Keynote (Rudy van Hoe, Business manager Server & Cloud, Microsoft Belgium)

Rudy explained the strategy and vision of Microsoft around hybrid scenarios and deployments.  In the session of Rudy, it became clear that even if a company is not looking into public cloud, the cloud patterns and paradigms should also be applied in the architecture of next generation applications.  It was also clear for everybody that the take up of hybrid solutions will even drive more integration now.  And especially mobile services can be a driver in this.

 

 

What’s new in BizTalk Server 2013 (Toon Vanhoutte, Integration Architect, Codit)

In this interesting session, Toon gave an overview of the history of BizTalk until now, but then really focused on the new features of BizTalk Server 2013.  He told about the new generation adapters for Windows Azure Service Bus and the REST capabilities that were introduced with this new release of BizTalk. 

The final live demo showed how a pizza delivery company was using SalesForce.com as their CRM system and had an on premises BizTalk 2013 that received the closed opportunities and updated the Dynamics AX system.  This specific demo leveraged the Service Bus relay capabilities and the new REST adapter.  Now we are still looking to find a pizza delivery company with an application setup like this Smile

 

 

Service & API management with Sentinet (Andrew Slivker, CTO, Nevatech)

We really wanted to have this session in our event, because we are convinced that Sentinet, the Service & API management tool of Nevatech is filling an important gap around SOA governance.  We have been talking with different customers about this product and it seems they immediately get the value of it. 

A lot of companies have web services (SOAP or REST) running allover the enterprise.  These services are often running on different platforms or technology stacks.  Andrew Slivker showed how Sentinet could make sure that all these different services could be secured, monitored, configured and virtualized in the same and consistent way, without giving in into performance.

more information can be received by contacting Codit.

 

Running BizTalk Server 2013 in Azure Virtual Machines (Peter Borremans, Integration Architect, Codit)

Peter started his session by double clicking two impressive PowerShell scripts.  After this, he gave an overview of the new deployment models, the licensing impact of running BizTalk in an Azure virtual machine and on the scripts he had written.  After these few slides, he showed that his scripts were successfully completed and that they had created a multi-server BizTalk environment on Windows Azure virtual machines, all connected to a domain controller in the cloud.  After this, Peter impressed the audience further by leveraging the Azure Virtual Networking capabilities to set up a point-to-site VPN connection between his newly created machine and his local laptop.  And by the end of his 40 minutes session, his Azure-hosted BizTalk environment was picking up files from his local laptop.

we plan to provide these details in a blog post, later in June

 

 

Maintaining chain integrity after a disaster recovery (Jasper Defesche, Integration Architect, Axon Olympus)

In this session, Jasper explained that it’s typically not just enough to foresee in backups and standard disaster recovery scenarios, but that there are much more things that should be taken into consideration, when planning for good recovery after a disaster.  It is important to consider these things from the beginning, by defining a good architecture and to consider these potential problems at every following step in the development process.

the white paper of Jasper can be downloaded on the website of Axon Olympus.

 

 

BizTalk Services in the cloud (Sam Vanhoutte, CTO, Codit)

Codit is world wide launch partner for the BizTalk Services on the Windows Azure platform.  For more than one year, Codit is working together with the engineering teams of Microsoft to get closer to the release of this new service in the cloud.  Codit is leveraging these services in the cloud integration platform: Codit Integration Cloud.  During the session, Sam gave an overview on the possible scenarios and the different capabilities in this service.  A comparison was made between BizTalk Server and Services and every feature was demonstrated on the live environment.  The details of this session cannot be shared at this time, because the session was under NDA.

we are working on a series of blog posts around this new service, that we will make available as soon as the service is announced.

 

Andrew Slivker of @Nevatechinc  talking about services & API governance with #sentinet on @coditcompany #msbts launch

Now @Peterborremans is now demonstrating #msbts running on #azure #IAAS at the @CoditCompany BizTalk launch

Last session of the day about @integrcloud Azure Servicebus and wasbs by @SamVanhoutte  @CoditCompany

 

If you want to learn more on these sessions, don’t hesitate to get in contact.  We are more than happy to provide more detailed information.

 

Sam Vanhoutte

Posted in: Azure | BizTalk

Tags:


May 21, 2013 at 4:00 PM

At a certain point in time when installing BizTalk, you will need to install some extra “components”. In the BizTalk installation wizard this is referred to as a prerequisite file.

This file can be downloaded automatically from within the wizard, but most likely you have some good reasons not to do that:

  • You’re installing more than 1 BizTalk server and/or environment at the same time
  • You do not want to wait for downloads to finish
  • Internet access is not available on BizTalk servers
  • etc …

There are a bunch of reasons why a seasoned BizTalk professional/administrator needs to have a backup copy of these files available just in case you might ever need them…

Below you can find a list of all BizTalk prerequisite files I have come across and where to find any further localized versions in case you might need them.

 

BizTalk 2013

The following list is taken from a blog post I came across here.

Platform Localization Link
Windows Server 2008 R2 (64 bit) EN Download
Windows Server 2012 (64 bit) EN Download
Windows 7 (32 bit) EN Download
Windows 8 (32 bit) EN Download
Windows 8 (64 bit) EN Download

 

BizTalk 2010

The following list is taken from the installation document for BizTalk 2010 found here.

Platform Localization Link
Windows Server 2008 (32 bit) EN Download
Windows Server 2008 (64 bit) EN Download
Windows Server 2008 R2 EN Download
Windows Vista (32 bit) EN Download
Windows Vista (64 bit) EN Download
Windows 7 (32 bit) EN Download
Windows 7 (64 bit) EN Download

 

BizTalk 2009

The following list is taken from the installation document for BizTalk 2009 found here.

Platform Localization Link
Windows XP (32 bit) EN Download
Windows XP (64 bit) EN Download
Windows Server 2003 (32 bit) EN Download
Windows Server 2003 (64 bit) EN Download
Windows Vista (32 bit) EN Download
Windows Vista (64 bit) EN Download
Windows Server 2008 (32 bit) EN Download
Windows Server 2008 (64 bit) EN Download

 

BizTalk 2006 R2

The following list is taken from the installation document for BizTalk 2006 found here.

Platform Localization Link
Windows XP (32 bit) EN Download
Windows XP (64 bit) EN Download
Windows Server 2003 (32 bit) EN Download
Windows Server 2003 (64 bit) EN Download
Windows Vista (32 bit) EN Download
Windows Vista (64 bit) EN Download

 

I hope this helps someone who needs the quick download!

 

[Updated 2013-05-21]: Added the BizTalk 2013 links to this post.

Posted in: BizTalk

Tags: ,


May 17, 2013 at 4:05 PM

 

BizTalk2013 logo

BizTalk 2013 is out, but this is obviously old news for most of you. With a lot of new features and support for on premises, or in the cloud, this new version seems exciting and maybe challenging…

To help you overcome any potential issues and to make sure you are ready for the new version, Codit (partner of the global Microsoft Integration Alliance "IMPACK") organizes the European edition of the global BizTalk 2013 launch in Belgium - preceded by Breeze in Australia and in parallel with Matricis in North-America.

What will I learn?

  • What is new in Microsoft BizTalk Server 2013? New features!
  • The new cloud capabilities of BizTalk 2013
  • BizTalk 2013 & its new adapters
  • Demonstration of the Nevatech SOA governance tool called Sentinet - a flexible and scalable solution for the SOA governance of your BizTalk (or Azure) integration environment
  • Exclusively discover Windows Azure BizTalk Services (WABS) - Hear from the experts: Codit is selected Microsoft launch partner of WABS and is one of the first partners worldwide to incorporate these innovative services as a core component in its solution.

Practical details

  • Date: Wednesday May 29th, from 8:30 AM to 16:30 PM (including lunch)
  • Location: Communicatieloft in Ghent, easy accessible via R4 (large parking) & walking distance from railway station Gent Sint-Pieters
  • Participation is free, but open for pre-registrants only
  • Language: English
  • Bring your business cards!

Speakers & Sessions

  • Keynote: Microsoft Cloud OS as the Foundation for your Hybrid Cloud
    • Rudy Van Hoe, Business Manager, Server and Cloud Platform at Microsoft
  • Windows Azure BizTalk Services; notes from the field
    • Sam Vanhoutte, CTO Codit
  • SOA Governance and Runtime Capabilities Enablement for BizTalk Server 2013
    • Andrew Slivker, CTO Nevatech
  • BizTalk Server 2013 in Windows Azure IaaS
    • Peter Borremans
  • Maintaining chain integrity after a system or disaster recovery
    • Jasper Defesche, Axon Olympus
  • New adapters in BizTalk 2013
    • Toon Vanhoutte

 

For more information and registration, visit this page on our website.

Hope to see you there!

Posted in: BizTalk | Events

Tags: ,