September 13, 2010 at 11:50 PM

Since I am coming from the BizTalk world, I feel very comfortable in implementing Workflows, using the Microsoft Workflow 4.0 runtime.  The engine comes with a lot of similar activities and capabilities as the XLANG engine provides us for more than 6 years already. 

One of the typical patterns that involve long-running processes is correlation and more in specific; convoys.  Each time different messages or calls have to be related together in order to achieve something, we are using convoys. Convoy processing is a typical design pattern.

There are two types of convoys, which I will demonstrate both:

  • Sequential convoy - Items that are in a order, one after another (often messages of the same type)
  • Parallel convoy - Items that are received in any order, but that all need to be received, before processing continues

The main difference in the two types of convoys is the order of receipt of the items.

This blog post shows two scenarios in an online travel website, where users can rate hotels and where hotels can be added, by consuming workflow services from any mash-up application.

The source code of my example can be found here.

Correlation

The concept of correlation is when various messages or calls need to be linked to the same instance of a long-running workflow.  These calls are typically related, based on content in the message, or based on context of the call (endpoint, for example).

An example would be where a workflow creates a user task on sharepoint and waits until the task is completed.  Multiple instances of that workflow can run in the same time and tasks can be completed in a different order than they were created.  Therefore, we need to make sure that the task completion event is sent to the correct instance of the workflow that created the task.  This is called correlation.

Setting up correlation in workflow is done by defining a variable of type System.ServiceModel.Activities.CorrelationHandle This variable needs to be scoped to make sure it is available for all receive and send activities that need to participate in the correlation.

Correlations are initialized or followed on the messaging activities.

Sequential convoy

The first scenario is a workflow service that processes the user rates that is given for a certain hotel.  A requirement for the web site is that the rating for a hotel should only be published when at least 5 user ratings have been received for that specific hotel.  This is needed to prevent influences of one specific rating.

This is why we implement a sequential convoy.

 

Receive web service operation and correlation

For this, we have a sequential workflow that exposes a web operation: SubmitHotelReview.  This operation accepts a message of type HotelReview (a datacontract class with some basic parameters).   All calls to this workflow service will be correlated to the same instance, based on the hotel ID. 

The most important settings of the Receive activity are shown in the following table.

Correlations  
CorelatesOn Xpath query key 1 : sm:body()/xg0:HotelReview/xg0:HotelId
CorrelatesWith hotelCorrelationHandle
CorrelationInitializers no specific changes here (only the request/response handle is initialized here)
Misc  
OperationName SubmitHotelReview
ServiceContractName {http://blog.codit.eu/workflow/convoys/}IHotelReviewService
CanCreateInstance Checked (True)

A common mistake is to also specify an entry in the CorrelationInitializers property to initialize the content-based correlation handle (hotel id).  But doing this, would result in the following exception: An instance key of value {Guid}' already exists. This could be because there are multiple MessageQuerySets defined that evaluate to the same CorrelationKey.  The reason for this is that the correlationkey would be initialized twice, which results in this exception.

Looping condition for sequential convoy

To have the sequential convoy implemented, we need to correlate all receives together and therefore we can add a loop, using the DoWhile activity.  Here we just add the receive activity in the 'Do-body’ and define a condition, making sure we continue processing (publishing the hotel rates), once five reviews have been received.

The entire workflow service is displayed in the following screenshot. (click to enlarge)

Sequential

 

Parallel convoy

The second scenario is a workflow service that adds new hotel information to the web site.  Data for these hotels is coming from two different sources.  One consumer of our workflow service will pas in the general hotel information, where another consumer application will provide pricing information for a hotel.  The sequence of these events is unknown and can differ from one hotel to the other. 

This is why we implement a parallel convoy.

 

Receive web service operations and correlation

For this, we have a sequential workflow that exposes two web operations: SubmitGeneralInformation and SubmitPricingInformation.  Both receive activities correlate on the same correlation handle that is correlating on the HotelId.

The settings of the receive activities are similar to the settings listed in the sequential convoy sample.

Parallel activity for multiple receives

To make sure we can accept both pricing and general information, we add the receives in a parallel shape, which will make sure we will only continue the flow if all branches have been successfully completed (and thus all messages have been received).

The entire workflow service is displayed in the following screenshot. (click to enlarge)

image

 

Conclusion

Implementing convoys in Workflow 4.0 is rather straightforward and does not require very complex tricks or configuration.  The magic is all in the correlation handling.

In comparison with BizTalk, I really like the sequential convoy implementation, since we now only have the need for one single Receive activity, where in BizTalk the initializing and the following receives need to be two different shapes.

Something I curious about is if there would also be the concept of zombie messages that are typical in the convoy scenarios.  (when a message is being received at the exact same time as the looping condition of the sequential convoy completes.)

Sam Vanhoutte, CODit


September 10, 2010 at 9:17 PM

I was building a test solution for my next blog post and I started to create a new workflow service in Visual Studio.  But apparently the designer came up with an exception for me:

Workflow Designer encountered problems with your document
Please check the document for invalid content, namespaces, references or reference loops.

'\' is an unexpected token. The expected token is ';'. Line 3, position 99.

After opening the XAML view, I noticed the full path of the xamlx file was added to the xml in the sad:XamlDebuggerXmlReader.FileName attribute of the sequence (main element).  And there it appeared that the fileName was not well-encoded, resulting in invalid XML, because of the usage of '&' sign.  (I added my blog tests in a folder with name R&D …)

The solution to this issue was to manually escape the R&D with R&D string.

Off course, one could argue why the ‘source code’ or ‘model’ of a Workflow service needs to maintain a reference to a physical file on a developers machine (in source safe, everyone on the same project can check out to different locations…)

Just wanted to share this one.

Posted in: AppFabric | Workflow

Tags: , ,


June 17, 2010 at 9:37 AM

Since the latest release of the Windows Azure Development Kit (June), Azure provides support for .NET 4 applications, which is a real important step from the point of adoption, I believe.

 

The first thing we wanted to try was to host a XAMLX Workflow Service in a web role on Azure.

Example workflow service

I created a standard Workflow Service that accepted two parameters, on  one operation (Multiply) and returns the result of the multiplication to the client.  This service was called Calc.xamlx.

 

These are the steps I followed to make my service available.  I added the exceptions, because that’s typically what users will search for J

 

Enable visualization of errors

 

Standard behavior of web roles, is to hide the exception for the users, browsing to a web page.  Therefore, it is advised to add the following in the system.web section of the web.config:

 

<customErrorsmode="Off"/>

 

Configure handler for Workflow Services

 

The typical error one would get, when just adding the Workflow Service to your web role and trying to browse it, is the following:

 

The requested url '/calc.xamlx' hosts a XAML document with root element type 'System.ServiceModel.Activities.WorkflowService'; but no http handler is configured for this root element type. Please check the configuration file and make sure that the 'system.xaml.hosting/httpHandlers' section exists and a http handler is configured for root element type 'System.ServiceModel.Activities.WorkflowService'.

 

We need to specify the correct HTTP handler that needs to be used for XAMLX files.  Therefore, we link the workflow services and activities to the correct Workflow Service ServiceModel handler.

 

To solve this, the following needs to be added to the web.config.

 

<configuration>
       <configSections>
             <sectionGroupname="system.xaml.hosting"type="System.Xaml.Hosting.Configuration.XamlHostingSectionGroup, System.Xaml.Hosting, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
                   <sectionname="httpHandlers"type="System.Xaml.Hosting.Configuration.XamlHostingSection, System.Xaml.Hosting, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
             </sectionGroup>
       </configSections>
       <!-- Removed other sections for clarity -–>
       <system.xaml.hosting>
             <httpHandlers>
                    <addxamlRootElementType="System.ServiceModel.Activities.WorkflowService, System.ServiceModel.Activities, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"httpHandlerType="System.ServiceModel.Activities.Activation.ServiceModelActivitiesActivationHandlerAsync, System.ServiceModel.Activation, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
                    <addxamlRootElementType="System.Activities.Activity, System.Activities, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"httpHandlerType="System.ServiceModel.Activities.Activation.ServiceModelActivitiesActivationHandlerAsync, System.ServiceModel.Activation, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
             </httpHandlers>
       </system.xaml.hosting>
</configuration>

 


Enabling metadata exposure of the workflow service

 

To enable consumers of our workflow service to generate their proxy classes, we want to expose the WSDL metadata of the service, by providing the following configuration section in the web.config.

 

Notice the simplicity of configuration, compared to WCF3.5.  Making use of the default behaviors, allows us to only specify what we want to override.

 

<system.serviceModel>
       <serviceHostingEnvironment multipleSiteBindingsEnabled="true" >
             <serviceActivations>
                    <addrelativeAddress="~/Calc.xamlx" service="Calc.xamlx"  factory="System.ServiceModel.Activities.Activation.WorkflowServiceHostFactory"/>
  
          </serviceActivations>
       </serviceHostingEnvironment>
       <behaviors>
             <serviceBehaviors>
                    <behavior>
                           <serviceMetadatahttpGetEnabled="true"/>
                    </behavior>
             </serviceBehaviors>
       </behaviors>
</system.serviceModel>

 

Testing and publishing

After successfully testing locally in the Azure Development Fabric, I uploaded and deployed the package, using the new Tools in Visual Studio to my Azure test account.

Sam Vanhoutte, CODit

Posted in: AppFabric | Azure | WCF | Workflow

Tags: ,


June 4, 2010 at 6:04 AM

 

Yesterday’s post showed the usage of the new BizTalk mapper activity in Workflow Services and how it made these workflows more declarative and drastically reduced the amount of assignment code that was typically required without this new feature.

Another interesting BizTalk feature that is made available in AppFabric workflows is the LOB (Line of Business) connectivity.  This functionality gets installed with the new BizTalk WCF LOB Adapter pack.

This post will show how this feature is used in the Workflow designer.  I will be using the SQL LOB adapter, because that’s the easiest to demonstrate.  (No SAP or Oracle test system available at this time).  I continue on the workflow that was created in the previous post.

Metadata generation

1.       Right-clicking the workflow project, will show the Add Adapter Service Reference… menu item. 
 

 

2.       This shows the familiar Add Adapter Service Reference dialog box.  (Don’t you all agree this tool still looks too much of ‘Form1/Button1’ app? J).  After selecting the sqlBinding and providing the ServerName and DataBaseName, the connection was made.

3.       This showed all the available metadata for this connection.  It is possible to select stored procedures or tables that will be used in the process.

 

4.       Since the LoanRequest needs to be inserted in the database table, the Client contract type was specified and the correct table (LoanRequests) and operation (Insert) were selected.

 

Using the LOB activity.

 

1.       After clicking OK and performing a rebuild on the project, a new custom activity was added to the Workflow Designer toolbox.  Each operation (Insert, Update …) that was selected results in a specific activity that is part of a category with the database object name.

2.       Dragging this activity on the canvas, shows that the following parameters can be specified on the shape.

 

3.       I only declared the variable ‘saveLoanResult’ that represents the records that should be added to the database.   This variable was linked with the Rows parameter of the InsertActivity shape and is an Array of LoanRequest objects.

4.       The InsertResult parameter would return the result of the database activity, but this parameter is optional, so I did not link it with a variable here.

 

5.       The BizTalk mapper activity was added right before the InsertActivity shape.  Here, I defined a mapping between the loanRequest and the databaseInsert.

6.       Compiling the project and executing the workflow, resulted in a new record in my database table.
 

 

 

Conclusion

The power of connectivity and adapter is now made available to the AppFabric platform in a standardized and user-friendly way.  It is now also easy to connect to various LOB systems, databases and host systems (using the BizTalk Adapters for Host Systems). 

What is not clear at this point is how the licensing will work for these AppFabric features.  But more information about that can probably be expected by the time of the public release.

Sam Vanhoutte

 


June 2, 2010 at 4:00 PM

Introduction

As described in the following wiki white paper on the social technet site (http://social.technet.microsoft.com/wiki/contents/articles/biztalk-appfabric-better-together.aspx) there is a nice new feature that gets installed through the released beta of BizTalk 2010.  Two capabilities of BizTalk Server are made available to WF 4.0!  This is about the BizTalk mapper and the Line of Business adapters (like SQL, Oracle, SAP…).

These features are also described in the blog post of Richard Seroter: post of Richard Seroter, (http://seroter.wordpress.com/2010/05/24/using-the-new-biztalk-mapper-shape-in-a-windows-workflow-service/). 

This is a first article in a series of three that is about these features.  Three articles will be posted in the coming days:

1.       Using the BizTalk mapper in Workflow

2.       Using the LOB Adapter functionality in Workflows

3.       Performance impact of the mapper shape, compared to code mapping.

The BizTalk mapper used in workflow services.

It happens a lot that data structures that are exposed through web services are different from the contracts and object types that need to be consumed by that service. 

During one of the demos I gave lately, I showed how an AppFabric workflow service was used to compose and consume three different other web services.  Allthough the customer really liked the pattern and the performance, he felt it was a bit too technical and too much code was involved to instantiate the web service requests and objects for the different services.  They were right (I needed to use too much assign shapes and different variables).

Luckily, we have a good answer to this at this moment, through the BizTalk mapper workflow activity.  It makes these workflows more model-driven and less code-oriented.

Installing the activities

It is important to know that the activities are only available in the Workflow designer toolbox, when installing the BizTalk LOB Adapter pack!  They are not part of the BizTalk 2010 installable and it looks like the installer of the adapter pack is checking if BizTalk 2010 is available to see if he should make the activities available.

Sample workflow service

The scenario I’ll be working out in this post is the exposing of a service that will call another web service.  This service will implement some specific logic and will also hide and abstract the actual names and contracts of the 'backend-service’.  The functionality we provide through this service is a loanrequest, where a consumer can request a loan and gets response with a result. 

The exposed contracts  

[DataContract]
    public class LoanRequest
    {
        [DataMember]
        public string CustomerName { get; set; }
        [DataMember]
        public long Amount { get; set; }
        [DataMember]
        public int Duration { get; set; }
        [DataMember]
        public LoanPart[] Parts { get; set; }
    }
    [DataContract]
    public class LoanPart
    {
        [DataMember]
        public LoanPartType PartType { get; set; }
        [DataMember]
        public long Amount { get; set; }
    }
    [DataContract]
    public enum LoanPartType
    {
        [EnumMember]
        Standard = 0,
        [EnumMember]
        Budget = 1,
        [EnumMember]
        Green = 2
    }
    [DataContract]
    public class LoanResponse
    {
        [DataMember]
        public double Intrest { get; set; }
        [DataMember]
        public double Variance { get; set; }
        [DataMember]
        public bool Approved { get; set; }
        [DataMember]
        public string Description { get; set; }
    }

 

Designing and beginning the workflow service

1.       After adding a new workflow service, I changed the action and namespace on the ReceiveRequest activity. 

2.       I also defined two variables: loanRequest and loanResponse that apply to their corresponding DataContract message types.  These are linked to the ReceiveRequest and SendResponse activities.

Adding a service reference to the ‘backend service’.

1.       Right-clicking the project and selecting Add Service Reference, pops up the dialog where I am linking to the WSDL of my web service.
 

2.       After a rebuild of this project, a new custom activity is added to my toolbox that can be used to call the web service operation.  If my web service would have had multiple operations, multiple activities would have been added, one for each operation.  They totally hide the implementation of doing this web service call.

 

3.       Now, two new variables get defined that represent the request and response to the backend service.  (notice the uppercased host-like names J)
 

4.       Dragging the LOANSIM activity to the designer canvas allows us to link the variables to the Activity Parameters of this web service operation call.

 

Use the BizTalk mapper activity to map request and response variables

1.       The Mapper activity is available in the BizTalk category in the Workflow Toolbox.  I just dragged two instances of this activity to the canvas, just before and after the LOANSIM activity.

2.       Once the mapper is dragged to the canvas, the input and output data types need to be configured, using the type picker dropdown.  This will be used to define the BizTalk map inputs and outputs.

 

3.       After this, the correct variables need to be linked and configured to their corresponding parameter.

 

 

4.       Clicking the Edit button on the activity, allows us to automatically generate a new BizTalk map that will use these types or we can reuse an existing map.

5.       Creating a new map, results in a new Biztalk mapper being opened.  Various schemas are being generated and added to the project in a specific sub folder.  The nice new BizTalk mapper can be used to make the mapping between both objects.  If required, custom XSLT functionality can also be used for this.

 


6.       The above logic has been done for both the request and the response messages.

 

 

Resulting workflow

The resulting workflow looks like the following:

 

Conclusion

The AppFabric + BizTalk better together story is a compelling offering that brings the BizTalk features closer to the AppFabric and Workflow runtime.  Considering that WF will be more and more important in the future of BizTalk and AppFabric, I believe this is a good step forward and we might see more of these things happen in the future.

Does this make AppFabric a free BizTalk Server?  No, it can be used to implement various scenarios that were done in BizTalk previously, but it still does not provide the rich management, deployment capabilities.  We still cannot accept flat file messages or EDI.  There is no pub/sub or direct binding.  Yet?

 

Sam Vanhoutte,-


April 28, 2010 at 5:18 PM

We gladly inform you that the "Beta 2 Refresh for Windows Server AppFabric" has been released on april 26th 2010.

This build supports the recently released .NET Framework 4 and Visual Studio 2010 RTM versions.

For more information we refer to http://blogs.iis.net/appfabric/archive/2010/04/26/windows-server-appfabric-beta-2-refresh-for-visual-studio-2010-net-4-rtm.aspx

Are you interested to extend your knowledge of the AppFabric technologyjQuery15204462349978275597_1383041730524 Then sign up to attend "CODit Deep Dive Training into AppFabric" (http://www.CODit.eu/DeepDiveIntoAppFabric) training which will be presented by Jon Flanders.

Posted in: AppFabric

Tags: ,


April 7, 2010 at 2:50 PM

CODit invited 'the legendary' Jon Flanders to Belgium for a deep-dive AppFabric and .Net 4.0 workflow training.  This training is open for customers and partners all over EMEA.  The training has limited seats available and registrations are on a 'first come / first served' schedule.

This training will be a level-300 training and will be organized on June 15-18, 2010.

This is the agenda of the 4 days training:

Day 1

  • Introduction to Windows Server AppFabric Monitoring and Hosting
  • Introduction to Windows Workflow Foundation 4.0
  • Building Declarative Services using WF 4.0 and WCF

Day 2

  • Hosting Services with Windows Server AppFabric
  • Configuring Windows Server AppFabric Services
  • Windows Server AppFabric Management

Day 3

  • Monitoring Services using Windows Server AppFabric
  • Making your services robust and manageable
  • Advanced Workflow Topics

Day 4

  • Introduction to Windows Server AppFabric Caching
  • What's new in WCF 4.0
  • Introduction to Windows Azure AppFabric

For more details and subscription, please visit the training page: http://www.codit.eu/deepdiveintoappfabric

Posted in: AppFabric

Tags: ,