April 12, 2012 at 1:42 PM

This is a post, describing an issue we recently had on the Azure Service Bus. 

 

Situation description

When a single appdomain is opening two (or more) service bus relay endpoints that have the same path (what is added after the servicebus.windows.net), but have a different namespace, the following exception is raised:

System.InvalidOperationException: The ChannelDispatcher at 'sb://repro2.servicebus.windows.net/ordersvc' with contract(s) 'ICustomerService' is unable to open its IChannelListener. ---> System.InvalidOperationException: A registration already exists for URI 'sb://repro2.servicebus.windows.net/ordersvc'.

The following is the configuration of my endpoints (not including the TokenProvider, etc, just the endpoints.

        <endpoint address="http://repro1.servicebus.windows.net/ordersvc"
            binding="basicHttpRelayBinding"
            bindingConfiguration="httpRelayBinding"
            contract="Demo1.Relay.ICustomerService"
            behaviorConfiguration="repro1Behavior" />
        <endpoint address="http://repro2.servicebus.windows.net/ordersvc"
            binding="basicHttpRelayBinding"
            bindingConfiguration="httpRelayBinding"
            contract="Demo1.Relay.ICustomerService"
            behaviorConfiguration="repro2Behavior" />

Why do we need the same services exposed on different namespaces?

Environment seperation

For Codit Integration Cloud, our azure-based integration platform, we are consuming some on-premises web services, exposed over the Azure Service Bus relay binding.  We do this for different services that make part of our runtime.  We also have a deployment cycle where we have different environments (a development, a test, a demo and off course the production environment(s)).

To isolate services, we make sure that every environment has a specific service bus namespace.  This allows for security segregation and to move services on premises, without touching other environments.  So this means that for our EdiFactParser service, we have the following endpoints (different names are being used here):

  • sb://intcloud-dev.servicebus.windows.net/onprem/parse
  • sb://intcloud-test.servicebus.windows.net/onprem/parse
  • sb://intcloud-demo.servicebus.windows.net/onprem/parse
  • sb://intcloud-prod.servicebus.windows.net/onprem/parse

As you can see the only difference in the above mentioned endpoints are the service bus namespaces.  As mentioned, these services are hosted on different servers on premises, but in some cases, we host them together.  (for example: dev & test are sharing the same local server).  In our cases, these services are hosted in BizTalk Server receive locations.

Disaster recovery

In the rare case where an Azure center would be out of the air, all service bus namespaces (hosted in that data center) are unavailable.  (There might be something like service bus federation in the future, but that’s not there yet).  To make sure that our on premises services are still available, we could expose these services on multiple service bus namespaces.  (hosted in different data centers).

In the case of a namespace being unavailable, the client that consumes the service could failover and try to consume the service over the second namespace.

How to work around this

This seems to be a bug in the Service Bus dll implementation, but we didn’t got a confirmation of this by product team yet.  (there might be a good reason for this, allthough I don’t see it).  Anyway, to workaround this, you need to open the different endpoints in separate service hosts.  The following describes how to do this.

In BizTalk Server

We initially had the issue on BizTalk Server, where we expose these services over the service bus.  Both receive locations were hosted in the same BizTalk host.  Just moving both locations to a different BizTalk host, solved the problem. 

In regular .NET applications

If you open the service host yourself, it as easy as creating two instances of the ServiceHost class and have them opening the endpoints seperately.

 

Just wanted to share this one

Sam Vanhoutte


February 22, 2012 at 10:10 PM

Recently, I was asked by Microsoft to give a presentation on how to integrate CRM online with on premises applications, on the Techdays in Belgium.  While hybrid integration was totally not new for me, the challenge here was to learn the extensibility capabilities and integration aspects of CRM online.  And I have to say, I was happily surprised with the extensibility framework that Dynamics CRM Online provides, especially with the Plugin mechanism.

All code of this article will be added with the second blog post of this series.

Extending CRM Online

There are different ways to extend CRM Online.  This is possible through forms customization, or by injecting Ajax/Silverlight controls.  However, these customizations are only applied on changes and events that happen through the front end.  Knowing that more and more actions of a CRM applications (especially data creation) are getting automated, I believe that customization should be injected in the actual ‘processing pipeline’ of CRM.  And that is where custom plugins come into the picture.

Getting started

Getting started is very straightforward.

  1. If you don’t have a CRM online account, it is very easy to create a trial account (which is only valid for 30 days).  Just browse to http://www.crmonline.com and register for a trial account, using your Windows Live Id (WLID).
  2. After this, you need to get the CRM SDK.  This one can be downloaded here.
  3. Once you have the SDK downloaded, you should build the PluginRegistration project, that can be found in %sdk%\tools\pluginregistration directory.  You will use this tool to inject plugins and create endpoints on your CRM system.

Writing custom plugins

Every plugin you create, needs to implement the IPlugin interface, which has one method; Execute.  These plugins should then be linked with the corresponding CRM event, through the PluginRegistrationTool.

using Microsoft.Xrm.Sdk;
 
namespace CRMPlugins
{      
  public class MyPlugin : IPlugin       
  {             
     public void Execute(IServiceProvider serviceProvider)             
     {    }  
   }
}

Implementing a ‘Base plugin’

Soon, I noticed that almost every plugin has some generic code that is applicable to almost all plugins.  For that, I created an abstract class BasePlugin, that every plugin will implement.  This BasePlugin takes care of a lot of common aspects:

  • Tracking: plugins can now easily track events and logging info, in case exceptions occur.
  • Configuration: plugins can receive external configuration, when linked with an event (example: connection strings, passwords, service bus settings…)  This configuration is then passed as a string to the plugin constructor, where one can parse it.
  • Entity filtering: some plugins should only be executed against specific entities (for example: only for invoices, only for salesorders…).  It is possible in the BasePlugin to filter out the entity, just by passing in the entity logical name with the constructor.
  • OrganizationService: the organization service allows to query and lookup entities that are configured on the CRM system.  This service is made available to every executing plugin.
  • Entity handling: reading and updating attributes on entities is made generic now.
  • Exception handling: all exceptions are caught and handled in a consistent way.

Implementing a plugin now just comes down to writing the specific logic and not bothering about the general logic that is always the same.  The only things left are the following:

  • Define the constructors: a constructor should exist with two string parameters as arguments.  These are the secured and unsecured configuration settings that get passed in from the plugin registration.  If you want to make sure the plugin only gets executed for certain entity types, you can easily pass in the entity type name to the base constructor, like here (only execute on invoice entities)
    public MyPlugin(string unsecure, string secure) : 
        base(unsecure, secure, "invoice") 
    { 
    } 
  • Implement the InternalExecute method: this is the abstract method that gets called by the BasePlugin and that is wrapped in the exception handling and is preceeded with the EntityType check.  This method will contain the actual logic of the plugin.
public override void InternalExecute(IServiceProvider serviceProvider, Microsoft.Xrm.Sdk.Entity entity)
{}

All the following plugins are using this Base Plugin class, in order to focus on the actual logic and abstract away all CRM specific logic.  (for that, you can easily find resources and blogs online).

Writing a plugin: sending a text message to the invoicee

As a first test, I created a plugin that sends a Text message, when an invoice gets created. For this, it looks up the phone number of the account, linked with that invoice. To send the plugin, we just use an existing online service (www.smsbox.be), that exposes the texting functionality through an HTTP endpoint.

Execution logic

The execution logic for the plugin is pasted below.  The following steps take place

  • First, the invoice number and amount are read from the Entity.  (the entity is of type “invoice”, because we have passed the “invoice” as an argument to the BasePlugin constructor).  To read these values, we call the base method ReadAttribute.
  • If the customerId attribute is available, we make an EntityReference, using the customerId value and we retrieve that account through the OrganizationService (provided by the BasePlugin).  We also specify (for performance reasons) that we only want to receive the telephone number attribute.
  • If that attribute is available, we send the Text message to the right phone number.  Full code is available in the provided zip file.
// Obtain the target entity from the input parmameters.
string invoiceNr = ReadAttribute(entity, "invoicenumber", "(unknown)");
string amount = ReadAttribute(entity, "totalamount", "0");
if (entity.Attributes.Contains("customerid"))
{
  EntityReference Customer = (EntityReference)entity["customerid"];
  if (Customer.LogicalName == "account")     
  {        
    Entity Account = OrganizationService.Retrieve("account", Customer.Id, new ColumnSet(new string[] { "address1_telephone1" }));         
    string phoneNr = ReadAttribute(Account, "address1_telephone1", null);         
    if (phoneNr != null)         
    {            
      SendSms(phoneNr, string.Format("Invoice {0} has been created for you with the amount of {1}", invoiceNr, amount), null);         
    }     
  }
}

Sending the SMS

For those who are interested, the code to do the HTTP post to the smsbox.be endpoint is pasted below:

private void SendSms(string phoneNumber, string text, ITracingService tracingService) 
{ 
  try   
  {    
    phoneNumber = phoneNumber.Replace("+", "");      
    string prefix = phoneNumber.Substring(0, 2);      
    string phoneNr = phoneNumber.Substring(2);     
    WebRequest request = WebRequest.Create(string.Format("http://www.smsbox.be/scripts/sendsms.php?login=####&pwd=####&prefix={0}&number={1}&message={2}", prefix, phoneNr, text));     
    request.Method = "POST";     
    var response = request.GetResponse();     
    response.Close();   
  } 
  catch (Exception ex)   
  {     
    if (tracingService != null)      
      tracingService.Trace(ex.Message);     
    throw;   
  } 
}

 

Registering the plugin with the PluginRegistration tool.

As indicated earlier, you need the PluginRegistrationTool to register the plugin on the CRM system and to link it with the corresponding events.  To do that, follow these steps:

  1. Open the PluginRegistrationTool
    • Enter the discovery URL of your CRM account.  This can be found in the CRM portal, in the resources section.
    • Enter the WLID that is linked with your CRM online subscription and click connect, to enter the password.
      image
  2. In the left pane, you now can see all CRM subscriptions that belong to your WLID.  Double clicking one of them will retrieve the entire customization profile.
    • Now you can add assemblies, plugins and endpoints and save them to your CRM instance.
      image
  3. Now it’s time to upload our assembly with the plugin.  For that, it’s as easy as clicking the Register new assembly button in the toolbar and selecting the plugin.
    • For CRM Online, the only Isolation Mode that is available, is the Sandbox mode, meaning you have reduced functionality.
    • The assembly should be deployed to the database, since the Disk or GAC is not available in CRM Online.
    • Once everything is configured, clicking the Register Selected Plugins will upload and register the plugin on the CRM Online system.
      image
  4. The plugin we have created, should be executed, when a new invoice is created.  To do that, we need to register a new step for the plugin. 
    • Select the plugin and click Register new step. This pops up a new window, where we need to provide the following information.
    • Message: this is the type of event.  In this case, we type Create (notice the intellisense, while typing)
    • Primary Entity: the logical name of the entity to which the event will be linked.  In this case, we select invoice. (case sensitive)
    • Then we can specify the execution order and security context in which the plugin should be executed.
    • The plugin can be executed at a specific time in the processing pipeline.  Pre-validation, Pre-operation and Post-operation.  In this case, we will execute the action, when the invoice has been saved in the system.  Therefore, we select the post-operation stage.
    • The execution mode defines if the user will be waiting during the exeuction of the pipeline (synchronous mode).  Asynchronous mode will increase the user experience, but when this action fails, it is not visible to the end user and only visible to the administrators through the CRM portal.
    • On the right, it is possible to specify configuration values that will be passed to the constructor of the plugin.  (we handle this in the BasePlugin)
      image
  5. That’s all that is needed to link our custom action with the right event on CRM Online.  Straight forward and easy, isn’t it?

CRM out of the box service bus connectivity

Dynamics CRM provides some out of the box capabilities to integrate events with the Azure Service Bus on different levels: Messaging and Relay.  For that you first need to configure the Access Control Service (ACS).  This is typically a complex step, but the registration tool makes this extremely easy and shows a perfect example of how Microsoft is using its own components and capabilities in a good way.

To do this, you need to register a new service bus endpoint through the PluginRegistrationTool.  (Register New Service Endpoint). 

  • Here we need to provide the following settings:
    • Now we need to specify our service bus namespace and the path to our queue/topic or service endpoint.
    • It’s also needed to specify the type of endpoint in the Contract drop down: Oneway, Queue, TwoWay, Rest.
      image
  • Secondly, we need to configure the ACS settings for this endpoint, so that the CRM online system has access to the service bus endpoint.
    • Clicking on Save & Configure ACS opens a new tool window, where we need to specify the management key and the issuer name for that key.
    • We also need to upload a certificate.  This public certificate key can be downloaded from the CRM online portal by clicking on Settings > Customizations > Developer Resources and Download Certificate.
      image
    • After clicking the Configure ACS button, we get a nice log, indicating what this wizard has done for us.  The highlights are pasted below:
      Trying to find out the ACS Version.
      ACS Version is: V2
      Creating ManagementService for codittest-sb
      Created RelyingParty with Name: testqueue, RealmName:
      http://codittest.servicebus.windows.net/testqueue, ID: 10004406
      Created RuleGroup with Name: Rule group for testqueue
      Assigned RuleGroup to RelyingParty
      Created Rule: sampleorgsendrule
      Created Rule: sampleownersendrule
      Created Rule: sampleownerlistenrule
      Created Rule: sampleownermanagerule
    • To test if the ACS was configured successfully, it is possible to click on the Save & Verify Authentication button.

Messaging capabilities

It is possible to use the Plugin Registration tool to register a service bus messaging endpoint.  This endpoint can then be of type queue, but it also works with topics (since both topics and queues are exposing the same REST interface).  On this endpoint, it is also possible to register a step, linked with an event.  When this event is fired, the entity will be serialized and written to the messaging endpoint.  From then on, it is possible to use the receiver functionality to receive these entities from the messaging endpoint. 

There is one downside, however.  No properties are being added to the BrokeredMessage, which makes it hard to use the publish/subscribe pattern through subscriptions that are using a routing filter.

Relay service capabilities

The Relay service capabilities can also be leveraged through these endpoints, but then it is required that the on premises service, exposed over the service bus relaying endpoint implements the specific contract (Microsoft.Xrm.Sdk.IServiceEndpointPlugin), as indicated in the SDK documentation.  A sample service can be found in the SDK (%SDK%\samplecode\cs\azure\onewaylistener).  This application exposes itself on the service bus endpoint through WsHttpRelayBinding.

Conclusion

Microsoft Dynamics CRM Online is truly a good and well-designed product that leverages the various capabilities and components that the Microsoft platform is offering.  It provides a very extensible plugin model and application concepts in order to configure and customize the application.  Next to that, it also provides some out of the box capabilities to integrate with the various Azure Service Bus capabilities, including relaying and messaging.

However, these out of the box integration capabilities have some limitations and my next post will show how to work around these limitations.

Sam Vanhoutte, Codit

Posted in: CRM | Azure | Service Bus

Tags:


December 20, 2011 at 6:51 AM

From private to public CTP

Last weekend, a new big step was set for companies in the Microsoft integration space, since the Microsoft Azure team released the first public CTP of the ServiceBus EAI & EDI Labs SDK.  This was formally known as Integration Services.  In september, we got a chance to play with a private CTP of this Azure component and since then we have provided a lot of feedback, some of which has already been taken up in the current release.  The current release seems much more stable and the installation experience was great.  (I installed the CTP on various machines, without a hickup)

It wasn’t surprising to see the BizTalk community picking up this CTP and start trying out various typical BizTalk scenarios in the cloud:

  • Rick Garibay discussed the various components in his blog post.
  • Kent Weare posted two articles (introduction & mapper) on his blog.
  • Mikael Håkansson showed the content based routing capabilities in this article.
  • Steef-Jan Wiggers added an overview TechNet wiki.
  • Harish Agarwal wrote an overview on EAI bridges.

Multi-tenancy & high-density

A huge difference for this model is that our pipelines, mappings and message flows are running on shared resources in the Windows Azure data center in some kind of high-density / multi-tenant container.  This introduces an entire new architecture and concept for isolation.  That also seems like the main reason why following capabilities are not yet possible or available:

  • It is not possible (yet) to develop custom pipeline components and have them running in the Azure runtime.
  • The mapper does not support (yet) custom XSLT or Scripting functoids.
  • Message flow does not contain workflow or custom endpoints.

This is feedback that the Microsoft team has received and is well aware of.  Things just get more complex and risky, when running custom applications or components in a shared environment.  You don’t want my memory leak causing a downgrade, or even failure, in your processes, right?  Based on feedback I gave on the connect web site, I really have the feeling these items will be available over time, so being patient will be important here.

The various components

This CTP contains a nice set of components:

Service Bus Connect

Service Bus Connect introduces an on premises service that exposes local LOB applications (like SAP, SQL, Oracle…) over the Service Bus Relay to the cloud.  This is a more lightweight service, compared with BizTalk and gives customers the ability to have a more lightweight and cheaper solution than buying BizTalk Server for the LOB connectivity.  It also make sure we no longer have to write custom services, or even worse: console applications to expose LOB adapter services over the Service Bus.

Service Bus Connect leverages the LOB Adapter SDK and the available adapters and exposes these endpoints over the Relay services of the Service Bus.  The local endpoints are hosted in Windows Server AppFabric and we can manage these endpoints in various ways:

  • In Visual Studio Server Explorer (ServiceBus Connect Servers).  This makes me think back about the BizTalkExplorer in Visual Studio that took 2 or 3 releases to disappear.  Admins still want MMC, in my opinion.
  • Over a management service, exposed as a WCF service.  One of the things that we want to do for our Codit  Integration Cloud, is exposing this management service over the service bus, so that we can maintain and operate the on premises services from our Cloud management portal.
  • Through powershell

EDI processing

It took Microsoft 5 releases of BizTalk Server, until they provided a full fledged EDI solution in BizTalk Server.  Therefore, it’s great to see these capabilities are already available in this release, because EDI is not disappearing and doesn’t seem to disappear any time soon, how much we all would like it to...  Support for X12 is available and also for AS/2 (the exchange protocol).  What is missing for us, Europeans, is EDIFACT.  Again, some patience will be rewarded over time.  What is very promising for migration or hybrid scenarios, is the fact that the provided EDI/X12 schemas look the same as those that were provided with BizTalk Server.  That will make migration and exchange of these messages from Cloud to On premises BizTalk much easier.

The EDI Trading Management portal allows to define partners and agreements, linked with various messaging entities on the service bus.  All of this is provided through a nice and friendly metro-style UI (what else).

I believe this is one of the most suitable scenarios for cloud integration.  B2B connectivity has been very expensive for years, because of the big EDI Value Added Networks and this is, after AS/2, a new step forward in making things cheaper and more manageable. 

EAI processing

One of the strengths and differentiators of BizTalk towards its competition was that it was one generic suite of components that could be used for both EAI, B2B and ESB solutions.  One products, one tool set, one deployment model.  That’s why it worried me a little to see that EDI and EAI are being seen as two totally different things in the release.  Looking a litle deeper shows that the EDI portal is mainly configuration of partners and endpoints, in order to link messages to the right Service Bus or messaging entities.  Still I think the same portal concepts can become valid for non-EDI integration, where companies just exchange messages in different formats (flat file, XML…) over various protocols (mail, file, FTP, AS/2).  I hope to see that we will be able to configure all of these connections in the same and consistent way.

Looking at the EAI capabilities, there are some nice new and refreshed items available:

  • Mapper: this looks like the BizTalk mapper, but is much different.  One of the biggest differences is that it does not create or support XSLT, or it does not support custom code. 
  • XML Bridges: these are the new pipelines.  A bridge is a head in front of the service bus that is exposed as an endpoint.  Various options are available and are discussed in the posts, mentioned above.  The biggest missing link here, is the extensibility with custom components.  At this moment, it doesn’t look possible to have (de)batching, custom validation, zipping and all these other things we do in BizTalk pipeline components.
    • Another thing that is interesting that a new concept of pub/sub is introduced here.  You can use filters in the message flows and those filters are applied in memory and the first matching filter will make sure the incoming request gets routed to the right endpoint.  The pub/sub here is in memory and is not durable.  Making it durable, requires to have it routed to either a queue or topic.  So, this routing model looks much similar to the WCF Routing service and less to the durable BizTalk or Service Bus topics.

Conclusion

This release is much more stable than the private CTP and looks promising.  To be able to deliver real world integration projects with this, there is still a lot of things needed towards customization and extensibility and towards management and monitoring.  The components are also targeted to real technical/developer oriented people.  With Codit Integration Cloud, we provide the level of abstraction, monitoring and extensibility, while using these EDI & EAI capabilities under the hood.

Sam Vanhoutte

Posted in: Azure | Service Bus | WCF | EDI/AS2

Tags:


October 27, 2011 at 7:29 AM

Great news coming from the Azure AppFabric Service Bus team.  As from today, they have updated the Service Bus to include load balancing in the Relay Service capabilities.  It will now be possible to have up to 25 listeners registered to the same service endpoint.  When a message is then sent to that endpoint, only one of these endpoints will get the service call, based on a random decision on the relay cloud infrastructure.

  • First, we got an e-mail with the announcement through our Azure agreement
  • After that, a tweet of Valery Mizonov (from the AppFabric CAT team) appeared: #ServiceBus relay load balancing was one of your top requirements, dear community. We are done. What’s the next big thing you need?

 

How to enable this functionality.

No update of SDK is needed, everything is included in the AppFabric 1.5 SDK.

The only thing that needs to be done, is change your code to manage the number of concurrent listeners per endpoint and set the limits right. I expect that each of these listeners will count for a billable connection, so the limits are important indeed.

Scale out scenarios

Before this load balancing

Before this capability, a lot of issues were encountered when opening service bus endpoints on multi-instance computers.  In that case, the AddressAlreadyInUseException was thrown.

  • When a Web or Worker role opened a relay connection, but had more than one instance (to be high available), the second instance that tried to open that same endpoint got an exception, because the other instance already registered on it.
  • With BizTalk Server multi-server groups, the receive location with the Relay Binding only could be opened on one host instance.

New scenarios possible

Now it becomes possible to have real scaled out infrastructure and logic on premises too.  Customers who have geo-clustered data centers with redundant applications and logic can now expose that same level of availability to the cloud.

A concrete example

An concrete example of how we can leverage this, is the following. For one of our cloud scenarios, we need to do a lot of flat file parsing.  This we do on an on-premises hosted BizTalk Server.  Therefore, we created a generic web service that accepts the flat file with a message identifier and returns the parsed XML.

Until now, only one server could expose this functionality over the service bus.  But, in case of outage in our on premises data center, we would’ve been offline with this capability, making our entire solution not functioning for that amount of time.  With the new load balancing feature, we just have to expose the same service on a BizTalk Server in another data center (of our hosting partner) and we’re high available just like that.

 

So we can conclude that with this new feature, the following things have been shown:

  • The service bus keeps on being improved to be a world class reliable hybrid connectivity bus.
  • It is great that this feature is introduced, without a new release of SDK being required.  That is the power of the cloud.
  • Most importantly: on premises services can now be scaled better to leverage cloud power on premises too.

 

Sam Vanhoutte


June 5, 2011 at 11:19 PM

In the recent CTP of Windows Azure AppFabric, we can see a lot of new rich-messaging features.  In my previous blog post, I blogged about the publish-subscribe features.  Another interesting feature is the use of sessions when sending messages to a queue.  And that’s what this post is about.

We will demonstrate how to send large messages in an atomic batch of chunks over a queue.

The new queues

The new AppFabric Service Bus queues are much richer in functionality and features than the V1-message buffers or the Windows Azure storage queues.  A list of the biggest differences:

  • Reliable & durable storage (no limit on the TTL-time to live).
  • The maximum size of a queue is 1GB. (100Mb in the CTP version).
  • A message can measure up to 256KB.
  • Different messaging API’s are available: REST, .NET client and WCF bindings.
  • Transactional support in sending messages to a queue.
  • De-duplication of messages.
  • Deferring of messages.

As you can see, the limit of a message size is 256Kb, which is much more than the storage queue limit, but can still be not enough in certain cases.  And that is where sessions come in the picture.

Session concept

The concept of sessions allows receiving messages that belong to a certain logical group all by the same receiver.  This is done by specifying a SessionId on a message.  Receivers can listen on a specific session, or can lock the session for their usage on a first come first served basis.

Sessions can also be used to implement Request-Reply patterns over a queue.  But more on that can be expected in a future blog post.

 

Creating a session-enabled queue

By default, queues are created session-less.  The usage of sessions can only be used on queues that have session-support enabled.  The following code extract shows how to create a queue with sessions enabled.  (notice the usage of the ServiceBusNamespaceClient object.  This is the object you’ll always use in administrative operations.

sbClient.CreateQueue("qName", new QueueDescription { RequiresSession = true });

In this way, the queue will now be created and be able to handle sessions.

 

Sending messages to the queue

The only specific thing that is needed to send messages in a session to a queue, is to define the SessionId property on the specific message.  In this case, I also write a value to the message properties to indicate that the last message of a session is being sent.

var message = BrokeredMessage.CreateMessage(msgContent);
message.SessionId = "MySessionId"; 
message.MessageId = "MyMessageId"; 
if (isLastMessage) { message.Properties["LastMessageInSession"] = true; }

 

Receiving messages from a session

The specific thing on receiving messages in a session, is to use a SessionReceiver.  This receiver will make sure that all messages it receives will belong to the same session.  It is possible to listen on a specific session (by passing in the session name) and to specify a session timeout.

QueueClient queueClient = msgFactory.CreateQueueClient("LargeFileQueue");
SessionReceiver sessionReceiver = queueClient.AcceptSessionReceiver();
while (sessionReceiver.TryReceive(TimeSpan.FromSeconds(10), out receivedMessage))
{
	Console.WriteLine("Message received in session " + receivedMessage.SessionId);
	if (receivedMessage.Properties.ContainsKey("LastMessageInSession"))
	{     
		Console.WriteLine("Last message of session received");
		break; 
	} 
}

In the above mentioned sample, I am looping until I receive the ‘last message in the session’.

The sample Sending large messages in chunks to a session

In the sample that I upload, I send large messages in chunks to a session-enable queue.  Using multiple receivers, I am guaranteed that each file will only be received by exactly one receiver.

 

Functionality

The sample contains a sender where you can specify a directory where all messages in that directory will be picked up and submitted (in parallel) to a session-enabled queue, if the checkbox is checked.  Otherwise they will be sent to a session-less queue. (to demonstrate that messages will be received in random order).

The receiver is a console app that listens for incoming messages based on the session.  You can startup multiple receivers, to indicate that each receiver will receive messages to its own session.

sessioncapture

These are the most important design steps I took:

 

Sending messages

  • In the sample, I am using a Parallel.ForEach to loop over a bunch of files and send them in parallel.  This way, I am sure that messages won’t arrive in sequence on the queue.
  • Each large message is being sent in a transaction to the queue. 
  • On every message, I use the file name as the session id.
  • On the last message in the batch, I write the ‘LastMessageInSession’ property to make this visible to the receiver.

Receiving messages

  • I am using the PeekLock receive method to make sure I only remove the messages from the queue, when the full session has been received.
  • I receive the messages in a TransactionScope.  This makes sure that the session on itself is being rollbacked, in case of an exception.

The code of this sample can be found here.

Conclusions

This post was a new example of another great messaging feature that comes with the AppFabric Service Bus Enhancements.  We explored the sessions to group messages to the same receiver.  This scenario in this post demonstrated that we can split large messages in smaller chunks but still are able to handle them in one atomic batch.

Sam Vanhoutte