September 13, 2010 at 11:50 PM

Since I am coming from the BizTalk world, I feel very comfortable in implementing Workflows, using the Microsoft Workflow 4.0 runtime.  The engine comes with a lot of similar activities and capabilities as the XLANG engine provides us for more than 6 years already. 

One of the typical patterns that involve long-running processes is correlation and more in specific; convoys.  Each time different messages or calls have to be related together in order to achieve something, we are using convoys. Convoy processing is a typical design pattern.

There are two types of convoys, which I will demonstrate both:

  • Sequential convoy - Items that are in a order, one after another (often messages of the same type)
  • Parallel convoy - Items that are received in any order, but that all need to be received, before processing continues

The main difference in the two types of convoys is the order of receipt of the items.

This blog post shows two scenarios in an online travel website, where users can rate hotels and where hotels can be added, by consuming workflow services from any mash-up application.

The source code of my example can be found here.

Correlation

The concept of correlation is when various messages or calls need to be linked to the same instance of a long-running workflow.  These calls are typically related, based on content in the message, or based on context of the call (endpoint, for example).

An example would be where a workflow creates a user task on sharepoint and waits until the task is completed.  Multiple instances of that workflow can run in the same time and tasks can be completed in a different order than they were created.  Therefore, we need to make sure that the task completion event is sent to the correct instance of the workflow that created the task.  This is called correlation.

Setting up correlation in workflow is done by defining a variable of type System.ServiceModel.Activities.CorrelationHandle This variable needs to be scoped to make sure it is available for all receive and send activities that need to participate in the correlation.

Correlations are initialized or followed on the messaging activities.

Sequential convoy

The first scenario is a workflow service that processes the user rates that is given for a certain hotel.  A requirement for the web site is that the rating for a hotel should only be published when at least 5 user ratings have been received for that specific hotel.  This is needed to prevent influences of one specific rating.

This is why we implement a sequential convoy.

 

Receive web service operation and correlation

For this, we have a sequential workflow that exposes a web operation: SubmitHotelReview.  This operation accepts a message of type HotelReview (a datacontract class with some basic parameters).   All calls to this workflow service will be correlated to the same instance, based on the hotel ID. 

The most important settings of the Receive activity are shown in the following table.

Correlations  
CorelatesOn Xpath query key 1 : sm:body()/xg0:HotelReview/xg0:HotelId
CorrelatesWith hotelCorrelationHandle
CorrelationInitializers no specific changes here (only the request/response handle is initialized here)
Misc  
OperationName SubmitHotelReview
ServiceContractName {http://blog.codit.eu/workflow/convoys/}IHotelReviewService
CanCreateInstance Checked (True)

A common mistake is to also specify an entry in the CorrelationInitializers property to initialize the content-based correlation handle (hotel id).  But doing this, would result in the following exception: An instance key of value {Guid}' already exists. This could be because there are multiple MessageQuerySets defined that evaluate to the same CorrelationKey.  The reason for this is that the correlationkey would be initialized twice, which results in this exception.

Looping condition for sequential convoy

To have the sequential convoy implemented, we need to correlate all receives together and therefore we can add a loop, using the DoWhile activity.  Here we just add the receive activity in the 'Do-body’ and define a condition, making sure we continue processing (publishing the hotel rates), once five reviews have been received.

The entire workflow service is displayed in the following screenshot. (click to enlarge)

Sequential

 

Parallel convoy

The second scenario is a workflow service that adds new hotel information to the web site.  Data for these hotels is coming from two different sources.  One consumer of our workflow service will pas in the general hotel information, where another consumer application will provide pricing information for a hotel.  The sequence of these events is unknown and can differ from one hotel to the other. 

This is why we implement a parallel convoy.

 

Receive web service operations and correlation

For this, we have a sequential workflow that exposes two web operations: SubmitGeneralInformation and SubmitPricingInformation.  Both receive activities correlate on the same correlation handle that is correlating on the HotelId.

The settings of the receive activities are similar to the settings listed in the sequential convoy sample.

Parallel activity for multiple receives

To make sure we can accept both pricing and general information, we add the receives in a parallel shape, which will make sure we will only continue the flow if all branches have been successfully completed (and thus all messages have been received).

The entire workflow service is displayed in the following screenshot. (click to enlarge)

image

 

Conclusion

Implementing convoys in Workflow 4.0 is rather straightforward and does not require very complex tricks or configuration.  The magic is all in the correlation handling.

In comparison with BizTalk, I really like the sequential convoy implementation, since we now only have the need for one single Receive activity, where in BizTalk the initializing and the following receives need to be two different shapes.

Something I curious about is if there would also be the concept of zombie messages that are typical in the convoy scenarios.  (when a message is being received at the exact same time as the looping condition of the sequential convoy completes.)

Sam Vanhoutte, CODit


September 10, 2010 at 9:17 PM

I was building a test solution for my next blog post and I started to create a new workflow service in Visual Studio.  But apparently the designer came up with an exception for me:

Workflow Designer encountered problems with your document
Please check the document for invalid content, namespaces, references or reference loops.

'\' is an unexpected token. The expected token is ';'. Line 3, position 99.

After opening the XAML view, I noticed the full path of the xamlx file was added to the xml in the sad:XamlDebuggerXmlReader.FileName attribute of the sequence (main element).  And there it appeared that the fileName was not well-encoded, resulting in invalid XML, because of the usage of '&' sign.  (I added my blog tests in a folder with name R&D …)

The solution to this issue was to manually escape the R&D with R&D string.

Off course, one could argue why the ‘source code’ or ‘model’ of a Workflow service needs to maintain a reference to a physical file on a developers machine (in source safe, everyone on the same project can check out to different locations…)

Just wanted to share this one.

Posted in: AppFabric | Workflow

Tags: , ,


June 2, 2010 at 4:00 PM

Introduction

As described in the following wiki white paper on the social technet site (http://social.technet.microsoft.com/wiki/contents/articles/biztalk-appfabric-better-together.aspx) there is a nice new feature that gets installed through the released beta of BizTalk 2010.  Two capabilities of BizTalk Server are made available to WF 4.0!  This is about the BizTalk mapper and the Line of Business adapters (like SQL, Oracle, SAP…).

These features are also described in the blog post of Richard Seroter: post of Richard Seroter, (http://seroter.wordpress.com/2010/05/24/using-the-new-biztalk-mapper-shape-in-a-windows-workflow-service/). 

This is a first article in a series of three that is about these features.  Three articles will be posted in the coming days:

1.       Using the BizTalk mapper in Workflow

2.       Using the LOB Adapter functionality in Workflows

3.       Performance impact of the mapper shape, compared to code mapping.

The BizTalk mapper used in workflow services.

It happens a lot that data structures that are exposed through web services are different from the contracts and object types that need to be consumed by that service. 

During one of the demos I gave lately, I showed how an AppFabric workflow service was used to compose and consume three different other web services.  Allthough the customer really liked the pattern and the performance, he felt it was a bit too technical and too much code was involved to instantiate the web service requests and objects for the different services.  They were right (I needed to use too much assign shapes and different variables).

Luckily, we have a good answer to this at this moment, through the BizTalk mapper workflow activity.  It makes these workflows more model-driven and less code-oriented.

Installing the activities

It is important to know that the activities are only available in the Workflow designer toolbox, when installing the BizTalk LOB Adapter pack!  They are not part of the BizTalk 2010 installable and it looks like the installer of the adapter pack is checking if BizTalk 2010 is available to see if he should make the activities available.

Sample workflow service

The scenario I’ll be working out in this post is the exposing of a service that will call another web service.  This service will implement some specific logic and will also hide and abstract the actual names and contracts of the 'backend-service’.  The functionality we provide through this service is a loanrequest, where a consumer can request a loan and gets response with a result. 

The exposed contracts  

[DataContract]
    public class LoanRequest
    {
        [DataMember]
        public string CustomerName { get; set; }
        [DataMember]
        public long Amount { get; set; }
        [DataMember]
        public int Duration { get; set; }
        [DataMember]
        public LoanPart[] Parts { get; set; }
    }
    [DataContract]
    public class LoanPart
    {
        [DataMember]
        public LoanPartType PartType { get; set; }
        [DataMember]
        public long Amount { get; set; }
    }
    [DataContract]
    public enum LoanPartType
    {
        [EnumMember]
        Standard = 0,
        [EnumMember]
        Budget = 1,
        [EnumMember]
        Green = 2
    }
    [DataContract]
    public class LoanResponse
    {
        [DataMember]
        public double Intrest { get; set; }
        [DataMember]
        public double Variance { get; set; }
        [DataMember]
        public bool Approved { get; set; }
        [DataMember]
        public string Description { get; set; }
    }

 

Designing and beginning the workflow service

1.       After adding a new workflow service, I changed the action and namespace on the ReceiveRequest activity. 

2.       I also defined two variables: loanRequest and loanResponse that apply to their corresponding DataContract message types.  These are linked to the ReceiveRequest and SendResponse activities.

Adding a service reference to the ‘backend service’.

1.       Right-clicking the project and selecting Add Service Reference, pops up the dialog where I am linking to the WSDL of my web service.
 

2.       After a rebuild of this project, a new custom activity is added to my toolbox that can be used to call the web service operation.  If my web service would have had multiple operations, multiple activities would have been added, one for each operation.  They totally hide the implementation of doing this web service call.

 

3.       Now, two new variables get defined that represent the request and response to the backend service.  (notice the uppercased host-like names J)
 

4.       Dragging the LOANSIM activity to the designer canvas allows us to link the variables to the Activity Parameters of this web service operation call.

 

Use the BizTalk mapper activity to map request and response variables

1.       The Mapper activity is available in the BizTalk category in the Workflow Toolbox.  I just dragged two instances of this activity to the canvas, just before and after the LOANSIM activity.

2.       Once the mapper is dragged to the canvas, the input and output data types need to be configured, using the type picker dropdown.  This will be used to define the BizTalk map inputs and outputs.

 

3.       After this, the correct variables need to be linked and configured to their corresponding parameter.

 

 

4.       Clicking the Edit button on the activity, allows us to automatically generate a new BizTalk map that will use these types or we can reuse an existing map.

5.       Creating a new map, results in a new Biztalk mapper being opened.  Various schemas are being generated and added to the project in a specific sub folder.  The nice new BizTalk mapper can be used to make the mapping between both objects.  If required, custom XSLT functionality can also be used for this.

 


6.       The above logic has been done for both the request and the response messages.

 

 

Resulting workflow

The resulting workflow looks like the following:

 

Conclusion

The AppFabric + BizTalk better together story is a compelling offering that brings the BizTalk features closer to the AppFabric and Workflow runtime.  Considering that WF will be more and more important in the future of BizTalk and AppFabric, I believe this is a good step forward and we might see more of these things happen in the future.

Does this make AppFabric a free BizTalk Server?  No, it can be used to implement various scenarios that were done in BizTalk previously, but it still does not provide the rich management, deployment capabilities.  We still cannot accept flat file messages or EDI.  There is no pub/sub or direct binding.  Yet?

 

Sam Vanhoutte,-


December 3, 2009 at 2:02 PM

Last PDC, a very interesting session was given about the future and vision of BizTalk Server.  The content of this session can be downloaded on the following location: http://microsoftpdc.com/Sessions/SVR15

Posted in: BizTalk | Workflow

Tags: ,