The announcement
Because of the launch of the new Office version (read more here), there was also some interesting news for everyone interested in the Microsoft middleware… It was obviously not mentioned during the launch (which was off course consumer oriented), but as part of the new release, there are some interesting infrastructure components that are shipped. The service bus and the workflow team have shipped new bits that are available on premises and will be used by the new SharePoint version. This blog post dives deeper in the bits of the Service Bus for Windows Server. (read here)
Codit has been actively involved as a TAP (technology adoption program) customer for the Service Bus for Windows Server and we can say that we now have full symmetry for Codit Integration Cloud with Codit Integration Server. The exact same configuration and modules that are running in the cloud are also running on premises, giving our customers the choice to deploy locally, in the cloud or in a hybrid model.
Symmetry on the Microsoft cloud platform
The past years, we’ve seen Microsoft building a lot of things on the Windows Azure platform that they already had on premises. Some examples:
| Cloud |
On premises |
| Office 365 |
Sharepoint & Exchange |
| CRM Online |
CRM 2011 Server |
| SQL Azure |
SQL Server |
| Windows Azure |
Windows Server |
| AppFabric caching |
Windows Azure Storage Cache |
| … |
… |
In all of the above cases, Microsoft brought capabilities (they had running on the Windows Server platform for years) from on premises to their cloud platform. And they always tried to make things as symmetric as possible (especially on the development model). But after reading the recent announcements where they are bringing Azure web sites, virtual machines and the management capabilities on premises (read more here), we now also see the first move in the other direction. And with the recent Office announcements, Microsoft has also released Service Bus for Windows server, a symmetric on premises version of the powerful Windows Azure Service Bus messaging. This post digs deeper in the details of this beta release, but this move shows that Microsoft wants to be the software company that provides software and services, running on the environment of your choice, in the business model of your choice. This will also give a lot of flexibility to ISV’s and companies with a very distributed environment.
One of the biggest unknowns at this time, is how Microsoft will keep symmetry on server. The release cadence of server products versus cloud based services is just too different. New features are being added to the various services all the time, and they also have to ship to the server installations over time. I’m curious to find out what the release cycle will be for these updates.
What's in the release
The announced release is the first public beta for Service Bus on Windows Server. It is called beta 2. The release of Service Bus for Windows server only contains the messaging capabilities of the Windows Azure Service Bus. This means there are no relaying capabilities (nor an on premises ACS service) added to this version. If you want to learn more about queues & topics/subscriptions, don’t hesitate to read more here:
Scenarios
This server product can be used in a variety of scenarios.
Durable messaging only scenarios
If you require to exchange messages in a local, messaging only scenario, you can perfectly use Service Bus for Windows server to deliver messages between applications and services in a durable and reliable fashion.
Store & forward scenarios
The Service Bus for Windows server release contains the possibility to define ForwardTo subscriptions on topics, so that messages that match the subscription of these rules, automatically get forwarded to the defined messaging entity. At this point it’s not possible to configure a ForwardTo that points to a remote entity, but one can work around that by having a subscriber that listens to a local ForwardTo entity and sends these messages to a public entity.
Distributed scenarios
A lot of organizations have different business units or plants that need to be interconnected. In a lot of companies (often after mergers and acquisitions), the technology that is used in these different plants is different. Therefore, it could be good to have Service Bus being used as the gateway to exchange messages between the different units, while every unit can use it’s standard of choice (REST, SOAP, .NET, AMQP…) to connect to that gateway.
Installation
The bits are available in the Web Platform Installer (Service Bus 1.0 beta and Workflow 1.0 beta).
Prerequisites
- Service Bus for Windows server requires Windows Server 2008 R2 SP1 (x64) or Windows Server 2012 (x64). It also installs on Windows 7.
- SQL Server is needed to host the different entities & configurations. SQL Server 2008 R2 SP1 and SQL Server 2012 are supported (also with their Express editions)
- Only Windows Authentication is supported, meaning Active Directory is required in a multi-machine scenario.
- The Windows Fabric gets installed on the Windows system.
Everything is described in detail in the installation guide.
Topologies
There are two typical topologies. A third could also be used, when having a dedicated SQL with one Service Bus server.
|
Single box
This is perfect for development purposes and light weight installations. Having everything in a single box, still allows to use virtualization to get better availability.
|
Farm for high availability
To get high availability, it is needed to have at least 3 service bus instances, due to the new concepts of the Windows Fabric. This is important, when making estimates or system topologies.
|
Configuration
To configure the Service Bus for Windows server, we needed to use PowerShell in the private beta. The following commands create a new service bus farm with a single server.
$mycert=ConvertTo-SecureString -string <Password> -force –AsPlainText
New-SBFarm -FarmMgmtDBConnectionString "data source=(local);integrated security=true" –CertAutoGenerationKey $mycert
Add-SBHost -certautogenerationkey $mycert -FarmMgmtDBConnectionString "data source=(local); integrated security=true"
Get-SBFarmStatus (shows three services running: Service Bus Gateway, Service Bus Message Broker, FabricHostSvc)
Create a service bus namespace
Creating a namespace happens by using the following statement, providing a administrative user and the name of the namespace.
New-SBNamespace -Name CoditBlog -ManageUsers Administrator
As a result, you can see the following information:
Name : CoditBlog
AddressingScheme : Path
CreatedTime : 10/07/2012 15:21:03
IssuerName : CoditBlog
IssuerUri : CoditBlog
ManageUsers : {Administrator}
Uri :
ServiceIdentifier :
PrimarySymmetricKey : vnAJ7rHp###REPLACE FOR SECURITY####MOvH8Yk=
SecondarySymmetricKey :
New features
Check for matching subscriptions with the PreFiltering feature
Everyone who ever tried to build a BizTalk application has to be familiar with the following error description: “Could not find a matching subscription for the message”. This was a very common exception, indicating that a message was delivered to the BizTalk messaging agent, while no one was going to pick up that message. The fact that this error was thrown really made sure we would not lose messages in a black hole.
Looking at the publish & subscribe capabilities that came with Windows Azure Service Bus, this was one of the first things I missed. If you submit a message to a topic that does not have subscriptions that match, the message does not get deadlettered, the client does not get a warning or error code and the message is lost. While in certain scenarios this can be desired, a lot more scenarios would find this dangerous. The good thing is that now we get a choice with the Service Bus Server.
A new feature is available that takes care of this. This is PreFiltering. To enable this, you need to configure this on the TopicDescription. And notice one of the longer property names in the Service Bus object model
nsManager.CreateTopic(new TopicDescription { Path = "PrefilterTopic", EnableFilteringMessagesBeforePublishing = true });
If we now submit a message to this topic that does not have a matching subscription configured, we’re getting the following exception.
Type: Microsoft.ServiceBus.Messaging.NoMatchingSubscriptionException
Message: There is no matching subscription found for the message with MessageId %1%..TrackingId:%2%,TimeStamp:%3%.
As you can see, we can only configure this on a topic. I would have preferred to get the choice to configure this on the sender level. So that, even when sending to the same topic, the client could decide if he wants to have routing failures checked or not.
The complete sample code for this article can be downloaded at the bottom of this page.
ForwardTo: auto forwarding of messages to another entity
In our scenarios, we sometimes have the need to do archiving or auditing on incoming messages across all topics in our namespace. Until now, we have done that by creating a MatchAllFilter (1=1) on every topic and by creating a consumer for every subscription there. That off course takes up more threads and also more messaging transactions. This new feature is solving this for us. The ForwardTo allows us to define an automatic forwarding (transactional in the same namespace) action to another entity.
It’s important to understand that the subscription with ForwardTo enabled only acts as a ‘passthrough’ subscription and that you cannot read from it. The message automatically gets forwarded to its destination queue or topic. If you want to read from it, you get the following exception:
Can not create a message receiver on an entity with auto-forwarding enabled.
To specify a ForwardTo on a subscription, you need to specify this in the SubscriptionDescription, by setting the ForwardTo string property to the path of the destination entity, as can be found in the following snippet:
nsManager.CreateTopic(new TopicDescription("ForwardToTopic");
nsManager.CreateSubscription(new SubscriptionDescription("ForwardToTopic", "ForwardAll") {ForwardTo = "AuditQueue"}, new SqlFilter("1=1");
This feature comes in very handy for audit/tap scenarios and aggregated message handling.
Sending messages in batch
In service bus, we already had the EnableBatchedOperations to optimize ‘transmission performance’. With this release, we can now send messages in a single batch to a service bus entity. All of that in one transaction. The following code writes a batch of messages to a topic with a filter that accepts all messages with a number different from 8. Since PreFiltering is enabled (see above, no matching subscription), we can easily simulate an exception in the batch, by playing with the loop values. When we make the loop stop before number 8 is hit, all messages are successfully delivered, when we include number 8 (like in the sample), we get the NoMatchingSubscriptionException and not a single message is being delivered.
All of this is achieved by using the SendBatch method of the MessageSender class.
string topicName = "PrefilterBatchTopic";
if (!nsManager.TopicExists(topicName))
{
nsManager.CreateTopic(new TopicDescription(topicName) { EnableFilteringMessagesBeforePublishing = true });
var preFilterSubscription = nsManager.CreateSubscription(new SubscriptionDescription(topicName, "NoNumberEight"), new SqlFilter("MessageNumber<>8"));
}
var batchSender = msgFactory.CreateMessageSender(topicName);
// Create list of messages
List<BrokeredMessage> messageBatch = new List<BrokeredMessage>();
for (int i = 0; i < 11; i++)
{
BrokeredMessage msg = new BrokeredMessage("Test for subscription " + i.ToString());
msg.Properties.Add("MessageNumber", i);
messageBatch.Add(msg);
}
batchSender.SendBatch(messageBatch);
Receiving messages in batch
Just like we can send messages in a batch, we can also receive them in a batch. For this, we need to use the ReceiveBatch method of the MessageReceiver class. There are three overloads for this:
msgReceiver.ReceiveBatch(int messageCount);
msgReceiver.ReceiveBatch(IEnumerable<long> sequenceNumbers);
msgReceiver.ReceiveBatch(messageCount, serverWaitTime);
The first overload gets a batch of messages with a maximum of the specified messageCount. If there are 7 messages on the queue and the provided messageCount is 10, the method will return immediately with a batch of all 7 messages. It won’t wait until there are 10 messages on the queue, obviously. The third overload is much the same, except that you can specify a TimeSpan to wait. If no messages arrive during this timespan, an empty Enumerable will be returned.
The second overload is used to get specific messages (for example, messages that have been deferred) from the entity.
The following code reads the messages that have been written in the previous sample.
MessageReceiver msgReceiver = msgFactory.CreateMessageReceiver("PrefilterBatchTopic/Subscriptions/NoNumberEight", ReceiveMode.ReceiveAndDelete);
var messagesReceived = msgReceiver.ReceiveBatch(10, TimeSpan.FromSeconds(3));
Console.WriteLine("Batch received: " + messagesReceived.Count());
Some new properties
The TopicDescription and QueueDescription have some new properties available, aside from the ones discussed above.
- Authorization: contains all rules for authorization on the topic or queue. (not on SubscriptionDescription)
- AccessedAt, CreatedAt, UpdatedAt: these are all DateTime values that indicate the corresponding last actions for a queue. (In my private beta version, the AccessAt was still the DateTime.MinValue, however.)
- IsAnonymousAccessible: indicates if the queue has opened anonymous access (not on SubscriptionDescription)
- Status: the status of the queue (Active, Disabled, Restoring)
- UserMetadata: this is a property that one can use to custom metadata and so to a queue or topic.
Conclusion
With all recent announcements around symmetry between the cloud and the data center, we can fairly say that Microsoft is the only vendor that is powerful on both levels and gives its customers the choice to run their software in the best way possible. We truly believe in that vision and that's why we have taken the effort from the beginning of Integration Cloud to make the archticture flexible and modular to support this symmetry.
With Service Bus for Windows Server, we now have the same features available in a server only solution. And this now also gives me the chance to develop against the service bus while being disconnected.
Sam Vanhoutte
Updates
- Removed IIS from the topology diagrams. While IIS will often be installed and used in combination with Service Bus, it's not a requirement.
7c402bee-0dfc-4285-a830-2bc7ea9ed260|8|5.0