Since I am coming from the BizTalk world, I feel very comfortable in implementing Workflows, using the Microsoft Workflow 4.0 runtime.  The engine comes with a lot of similar activities and capabilities as the XLANG engine provides us for more than 6 years already. 

One of the typical patterns that involve long-running processes is correlation and more in specific; convoys.  Each time different messages or calls have to be related together in order to achieve something, we are using convoys. Convoy processing is a typical design pattern.

There are two types of convoys, which I will demonstrate both:

• Sequential convoy - Items that are in a order, one after another (often messages of the same type)
• Parallel convoy - Items that are received in any order, but that all need to be received, before processing continues

The main difference in the two types of convoys is the order of receipt of the items.

This blog post shows two scenarios in an online travel website, where users can rate hotels and where hotels can be added, by consuming workflow services from any mash-up application.

The source code of my example can be found here.

Correlation

The concept of correlation is when various messages or calls need to be linked to the same instance of a long-running workflow.  These calls are typically related, based on content in the message, or based on context of the call (endpoint, for example).

An example would be where a workflow creates a user task on sharepoint and waits until the task is completed.  Multiple instances of that workflow can run in the same time and tasks can be completed in a different order than they were created.  Therefore, we need to make sure that the task completion event is sent to the correct instance of the workflow that created the task.  This is called correlation.

Setting up correlation in workflow is done by defining a variable of type System.ServiceModel.Activities.CorrelationHandle This variable needs to be scoped to make sure it is available for all receive and send activities that need to participate in the correlation.

Correlations are initialized or followed on the messaging activities.

Sequential convoy

The first scenario is a workflow service that processes the user rates that is given for a certain hotel.  A requirement for the web site is that the rating for a hotel should only be published when at least 5 user ratings have been received for that specific hotel.  This is needed to prevent influences of one specific rating.

This is why we implement a sequential convoy.

Receive web service operation and correlation

For this, we have a sequential workflow that exposes a web operation: SubmitHotelReview.  This operation accepts a message of type HotelReview (a datacontract class with some basic parameters).   All calls to this workflow service will be correlated to the same instance, based on the hotel ID. 

The most important settings of the Receive activity are shown in the following table.

A common mistake is to also specify an entry in the CorrelationInitializers property to initialize the content-based correlation handle (hotel id).  But doing this, would result in the following exception: An instance key of value {Guid}' already exists. This could be because there are multiple MessageQuerySets defined that evaluate to the same CorrelationKey.  The reason for this is that the correlationkey would be initialized twice, which results in this exception.

Looping condition for sequential convoy

To have the sequential convoy implemented, we need to correlate all receives together and therefore we can add a loop, using the DoWhile activity.  Here we just add the receive activity in the 'Do-body’ and define a condition, making sure we continue processing (publishing the hotel rates), once five reviews have been received.

The entire workflow service is displayed in the following screenshot. (click to enlarge)

Parallel convoy

The second scenario is a workflow service that adds new hotel information to the web site.  Data for these hotels is coming from two different sources.  One consumer of our workflow service will pas in the general hotel information, where another consumer application will provide pricing information for a hotel.  The sequence of these events is unknown and can differ from one hotel to the other. 

This is why we implement a parallel convoy.

Receive web service operations and correlation

For this, we have a sequential workflow that exposes two web operations: SubmitGeneralInformation and SubmitPricingInformation.  Both receive activities correlate on the same correlation handle that is correlating on the HotelId.

The settings of the receive activities are similar to the settings listed in the sequential convoy sample.

Parallel activity for multiple receives

To make sure we can accept both pricing and general information, we add the receives in a parallel shape, which will make sure we will only continue the flow if all branches have been successfully completed (and thus all messages have been received).

The entire workflow service is displayed in the following screenshot. (click to enlarge)

Conclusion

Implementing convoys in Workflow 4.0 is rather straightforward and does not require very complex tricks or configuration.  The magic is all in the correlation handling.

In comparison with BizTalk, I really like the sequential convoy implementation, since we now only have the need for one single Receive activity, where in BizTalk the initializing and the following receives need to be two different shapes.

Something I curious about is if there would also be the concept of zombie messages that are typical in the convoy scenarios.  (when a message is being received at the exact same time as the looping condition of the sequential convoy completes.)

Sam Vanhoutte, CODit

I was building a test solution for my next blog post and I started to create a new workflow service in Visual Studio.  But apparently the designer came up with an exception for me:

Workflow Designer encountered problems with your document
Please check the document for invalid content, namespaces, references or reference loops.

'\' is an unexpected token. The expected token is ';'. Line 3, position 99.

After opening the XAML view, I noticed the full path of the xamlx file was added to the xml in the sad:XamlDebuggerXmlReader.FileName attribute of the sequence (main element).  And there it appeared that the fileName was not well-encoded, resulting in invalid XML, because of the usage of '&' sign.  (I added my blog tests in a folder with name R&D …)

The solution to this issue was to manually escape the R&D with R&D string.

Off course, one could argue why the ‘source code’ or ‘model’ of a Workflow service needs to maintain a reference to a physical file on a developers machine (in source safe, everyone on the same project can check out to different locations…)

Just wanted to share this one.

Some time ago, with a client using BizTalk 2006 and Covast Accelerator 2006, we suddenly had the following issue popping up in their event log:

There was a failure executing the send pipeline: "Covast.BizTalk.Pipeline.EDI.CovastEDISendPipeline, Covast.BizTalk.Pipeline.EDI.Default, Version=5.0.0.0, Culture=neutral, PublicKeyToken=68bef120a46b49ad" Source: "Covast EDI Assembler" Send Port: "sp_MySendPort" URI: "D:\BTS2006\DumpLocation\%MessageID%.edi" Reason: Message translation failed: (2217) Unknown error (-2217)

The source of the problem was Covast.
Somehow, it was no longer able to perform valid assembling and disassembling (both were failing).

After a lot of digging and research I finally came to the "Covast EDI Accelerator 2006 Reports" screens, hidden deep within the BizTalk Administration Console HAT :-)

When you double click the interchange which was in error, you saw the following error:

The format of the document has not been configured

Finally, this led me to the source of the problem.

Someone at the client fiddled around with one particular ASC file trying to add some extra segments to the schema to accompany a change in one of their flows. No harm done, but unfortunately when re-generating the XSD, he also regenerated the SYNTAX schema and deployed.

The result was that there were now 2 SYNTAX schemas deployed within BizTalk and Covast doesn't know how to handle that.
Reverting back to the old version of the flow fixed the error and after that we just removed the SYNTAX schema from the project, rebuilt and deployed and now it worked.

Reason for posting this is that at the time, neither search engine could come up with a decent answer, so hopefully someone benefits from this post.

Let us know in any case!

Pieter