How the Azure ServiceBus EAI & EDI Labs SDK impacts Microsoft integration

December 20, 2011 at 6:51 AM

From private to public CTP

Last weekend, a new big step was set for companies in the Microsoft integration space, since the Microsoft Azure team released the first public CTP of the ServiceBus EAI & EDI Labs SDK.  This was formally known as Integration Services.  In september, we got a chance to play with a private CTP of this Azure component and since then we have provided a lot of feedback, some of which has already been taken up in the current release.  The current release seems much more stable and the installation experience was great.  (I installed the CTP on various machines, without a hickup)

It wasn’t surprising to see the BizTalk community picking up this CTP and start trying out various typical BizTalk scenarios in the cloud:

  • Rick Garibay discussed the various components in his blog post.
  • Kent Weare posted two articles (introduction & mapper) on his blog.
  • Mikael Håkansson showed the content based routing capabilities in this article.
  • Steef-Jan Wiggers added an overview TechNet wiki.
  • Harish Agarwal wrote an overview on EAI bridges.

Multi-tenancy & high-density

A huge difference for this model is that our pipelines, mappings and message flows are running on shared resources in the Windows Azure data center in some kind of high-density / multi-tenant container.  This introduces an entire new architecture and concept for isolation.  That also seems like the main reason why following capabilities are not yet possible or available:

  • It is not possible (yet) to develop custom pipeline components and have them running in the Azure runtime.
  • The mapper does not support (yet) custom XSLT or Scripting functoids.
  • Message flow does not contain workflow or custom endpoints.

This is feedback that the Microsoft team has received and is well aware of.  Things just get more complex and risky, when running custom applications or components in a shared environment.  You don’t want my memory leak causing a downgrade, or even failure, in your processes, right?  Based on feedback I gave on the connect web site, I really have the feeling these items will be available over time, so being patient will be important here.

The various components

This CTP contains a nice set of components:

Service Bus Connect

Service Bus Connect introduces an on premises service that exposes local LOB applications (like SAP, SQL, Oracle…) over the Service Bus Relay to the cloud.  This is a more lightweight service, compared with BizTalk and gives customers the ability to have a more lightweight and cheaper solution than buying BizTalk Server for the LOB connectivity.  It also make sure we no longer have to write custom services, or even worse: console applications to expose LOB adapter services over the Service Bus.

Service Bus Connect leverages the LOB Adapter SDK and the available adapters and exposes these endpoints over the Relay services of the Service Bus.  The local endpoints are hosted in Windows Server AppFabric and we can manage these endpoints in various ways:

  • In Visual Studio Server Explorer (ServiceBus Connect Servers).  This makes me think back about the BizTalkExplorer in Visual Studio that took 2 or 3 releases to disappear.  Admins still want MMC, in my opinion.
  • Over a management service, exposed as a WCF service.  One of the things that we want to do for our Codit  Integration Cloud, is exposing this management service over the service bus, so that we can maintain and operate the on premises services from our Cloud management portal.
  • Through powershell

 

EDI processing

It took Microsoft 5 releases of BizTalk Server, until they provided a full fledged EDI solution in BizTalk Server.  Therefore, it’s great to see these capabilities are already available in this release, because EDI is not disappearing and doesn’t seem to disappear any time soon, how much we all would like it to...  Support for X12 is available and also for AS/2 (the exchange protocol).  What is missing for us, Europeans, is EDIFACT.  Again, some patience will be rewarded over time.  What is very promising for migration or hybrid scenarios, is the fact that the provided EDI/X12 schemas look the same as those that were provided with BizTalk Server.  That will make migration and exchange of these messages from Cloud to On premises BizTalk much easier.

The EDI Trading Management portal allows to define partners and agreements, linked with various messaging entities on the service bus.  All of this is provided through a nice and friendly metro-style UI (what else).

I believe this is one of the most suitable scenarios for cloud integration.  B2B connectivity has been very expensive for years, because of the big EDI Value Added Networks and this is, after AS/2, a new step forward in making things cheaper and more manageable. 

EAI processing

One of the strengths and differentiators of BizTalk towards its competition was that it was one generic suite of components that could be used for both EAI, B2B and ESB solutions.  One products, one tool set, one deployment model.  That’s why it worried me a little to see that EDI and EAI are being seen as two totally different things in the release.  Looking a litle deeper shows that the EDI portal is mainly configuration of partners and endpoints, in order to link messages to the right Service Bus or messaging entities.  Still I think the same portal concepts can become valid for non-EDI integration, where companies just exchange messages in different formats (flat file, XML…) over various protocols (mail, file, FTP, AS/2).  I hope to see that we will be able to configure all of these connections in the same and consistent way.

Looking at the EAI capabilities, there are some nice new and refreshed items available:

  • Mapper: this looks like the BizTalk mapper, but is much different.  One of the biggest differences is that it does not create or support XSLT, or it does not support custom code. 
  • XML Bridges: these are the new pipelines.  A bridge is a head in front of the service bus that is exposed as an endpoint.  Various options are available and are discussed in the posts, mentioned above.  The biggest missing link here, is the extensibility with custom components.  At this moment, it doesn’t look possible to have (de)batching, custom validation, zipping and all these other things we do in BizTalk pipeline components.
    • Another thing that is interesting that a new concept of pub/sub is introduced here.  You can use filters in the message flows and those filters are applied in memory and the first matching filter will make sure the incoming request gets routed to the right endpoint.  The pub/sub here is in memory and is not durable.  Making it durable, requires to have it routed to either a queue or topic.  So, this routing model looks much similar to the WCF Routing service and less to the durable BizTalk or Service Bus topics.

 

Conclusion

This release is much more stable than the private CTP and looks promising.  To be able to deliver real world integration projects with this, there is still a lot of things needed towards customization and extensibility and towards management and monitoring.  The components are also targeted to real technical/developer oriented people.  With Codit Integration Cloud, we provide the level of abstraction, monitoring and extensibility, while using these EDI & EAI capabilities under the hood.

Sam Vanhoutte

Posted in: Azure | Service Bus | WCF

Tags: , , ,

Comments (1) -

Mick Badran
Mick Badran Australia
1/19/2012 3:51:49 AM #

Nice article Sam,

Point to note here is that the offering is simply a CTP. Put something out there and let us play with it.

It does appear a little disjoint in places with several sites need to be configured to make a solution work, but it's simply a CTP.

Some further details for you:
- internally (MS) there's several other Bridge types such as flat file and so on. These didn't make this release.

Pipelines/Bridges at this moment, are implemented on top of WF. They are currently reasonably restrictive as MS didn't want an entire integration solution implemented as 1 stage. We'll wait and see what transpires in this space.

It is reasonable to expect the Windows Server AppFabric to have identical functionality going forward.

So all in all, a great milestone in terms of functional offering here - and that's the context.

Keep up the good work,

Mick.









Reply

Pingbacks and trackbacks (1)+

Add comment

  Country flag

biuquote
  • Comment
  • Preview
Loading