Tracking self hosted WorkflowApplication events

There are a numerous examples out there showing how to configure workflow instance tracking most if not all assume that the workflows are hosted in and exposed via WorkflowServiceHost. Tracking profiles can be configured using xml in the configuration file when using this mechanism for hosting your workflow service.

<system.serviceModel>
    <tracking>  
      <trackingProfile name="Sample Tracking Profile">
        <workflow activityDefinitionId="*">
          <workflowInstanceQueries>
            <workflowInstanceQuery>
              <states>
                <state name="Started"/>
                <state name="Completed"/>
              </states>
            </workflowInstanceQuery>
          </workflowInstanceQueries>
        </workflow>
      </trackingProfile>        
    </profiles>
  </tracking>
</system.serviceModel>

So what if you are not hosting in a WorkflowServiceHost but hosting WorkflowApplication objects in your own host process what are the options.

The first option is to define the tracking profile in the application configuration file and then load the tracking profile using code similar to the GetProfile method here. One of the problems with this approach in my mind stems from the fact that the XML definition is defined below the system.serviceModel so you are required to define a system.serviceModel element in your application configuration file regardless of whether or not you are using WCF. This in my mind was not the best approach to take for configuration of tracking profiles. An alternate would have been to create a separate element that the system.service model configuration could reference maybe by name.

The second option is to specify the tracking profile configuration in code and apply the tracking profile to a tracking participant more on this later in this post. First lets take a look at tracking profiles.

Tracking Profiles

A tracking profile determines which tracking records get processed by a tracking participant. There numerous tracking records that are generated by a given workflow instance during its execution. WF4 tracking has a number of TrackingQuery derived classes that are used to create subscriptions to WF4 workflow tracking records. The following is a list of the tracking query types and there purpose:

• WorkflowInstanceQueryUsed to track workflow instance related events. For example Started,Completed,Aborted events.
• CustomTrackingQueryUse this to create your own custom tracking query in combination with custom tracking records.
• ActivityScheduledQueryUsed to track various events related to activities. Includes events to indicate that a particular activity was scheduled, started or completed.
• FaultPropagationQueryYou can this query to detect errors that are raised within activities in the workflow.
• CancelRequestedQueryEvery time the workflow tries to cancel one of its children a CancelRequestedRecord is written.
• BookmarkResumptionQueryUsed to track bookmark resumption events.

Tracing WorkflowApplication

For the application that I am delivering I do not need the flexibility defining TrackingQuery types in a configuration file. I want to log all events and apply filtering when viewing the events. The following is an example method to apply the tracking profile to a tracking participant:

private static void ConfigureTrackingParticipent(TrackingParticipant trackingParticipant, WorkflowApplication workflowApplication)
{
    var trackingProfile = new TrackingProfile()
    {
        ActivityDefinitionId = "*",
        ImplementationVisibility = ImplementationVisibility.All,
        Name = "TrackingProfile"
    };
    var workflowInstanceQuery = new WorkflowInstanceQuery();
    workflowInstanceQuery.States.Add("*");


    var activityScheduledQuery = new ActivityScheduledQuery();
    activityScheduledQuery.ActivityName = "*";
    activityScheduledQuery.ChildActivityName = "*";


    var faultPropagationQuery = new FaultPropagationQuery();
    faultPropagationQuery.FaultHandlerActivityName = "*";
    faultPropagationQuery.FaultSourceActivityName = "*";


    var cancelRequestedQuery = new CancelRequestedQuery();
    cancelRequestedQuery.ActivityName = "*";
    cancelRequestedQuery.ChildActivityName = "*";


    var bookmarkResumptionQuery = new BookmarkResumptionQuery();
    bookmarkResumptionQuery.Name = "*";


    trackingProfile.Queries.Add(workflowInstanceQuery);
    trackingParticipant.TrackingProfile = trackingProfile;
    workflowApplication.Extensions.Add(trackingParticipant);
}

Note the “*” is a wild card which allow you to subscribe to all of the relevant tracking records for a given tracking query type. For example activityScheduledQuery.ActivityName = "*"; subscribes to all activities. Which is exactly what I want as workflows are dynamically loaded into my hosting process so I do not know up front what activities are defined in the workflow.

The tracking participant is then conditionally applied to a given WorkflowApplication, in my case I am using a EtwTrackingParticipant participant but this could be a custom tracking participant.

if (WorkflowManagerSettings.Current.EnableETWTracing)
{
    var etwTrackingParticipant = new EtwTrackingParticipant();
    etwTrackingParticipant.ApplicationReference = "WorkflowEngine";
    ConfigureTrackingParticipent(etwTrackingParticipant, workflowApplication);
}

The output from the tracking can be viewed  with the system event viewer. Expand the Applications and Services Logs node then expand the Microsoft, Windows, and Application Server-Applications nodes

Enterprise Library WCF Validation Performance

Following on from my previous post on WCF Request Validation Performance I thought I would share some of the performance test comparisons between the Enterprise Library WCF Validation and performing manual validation of service request objects.
It should be noted that the aim of this testing was not to be an exhaustive test of the relative performance of each approach but to simply get an order of magnitude comparison between the two options.
The test comprises of a host that exposes two services:

• EntLibValidationService – A service that is configured with Enterprise Library WCF validation
• ManualValidationService – A service that is performing manual validation

A client console application which has a number of test cases configured for each of the services:

1. A null argument passed to a service request
2. A property of a request object out of range
3. A request containing a list of objects
4. A request containing a list of derived request objects

The tests are run 1000 times in a loop and timed the average is then calculated. The following are the results executed on my desktop which has the following specification:

Processor: Intel(R) Xeon(R) CPU E5606  @ 2.13GHz (4 CPUs), ~2.1GHz
Memory: 3072MB RAM

	Test Case	Run 1	Run 2	Run 3	Average
EntLIb	1	7.6 ms	7.7 ms	7.7 ms	7.6 ms
	2	7.5 ms	7.7 ms	7.7 ms	7.6 ms
	3	9.1 ms	9.4 ms	9.7 ms	9.4 ms
	4	9.8 ms	10 ms	10 ms	9.9 ms
Manual	1	9.6 ms	9.8 ms	9.7 ms	9.7 ms
	2	9.2 ms	9.5 ms	9.5 ms	9.4 ms
	3	9.3 ms	9.5 ms	9.5 ms	9.4 ms
	4	9.9 ms	9.9 ms	9.8 ms	9.8 ms

From the above results it can be seen that there is not much overhead in using the Enterprise Library for WCF request validation. In the case of argument or property validation, Case 1 and Case 2, it can be seen that it actually performs better than manual validation.

For test Case 3 and 4 the numbers do not reflect an accurate comparison of manual validation due to the fact that the Enterprise Library validator performs validation on all of the objects in the collection for each call therefore it has actually performed better than the manual validation.

I suspect that most of the overhead of the request’s are serialising and de serialising the faults generated from both validation mechanisms.

WCF Request Validation

On a recent project that required the implementation of a WCF service, the purpose of which was to act as an interface for a third party to query and submit entities to our system over the internet, as usual with such integration the requirement to provide validation of the content of requests submitted to the service prior to execution of the request.

This validation takes the usual form of guarding against null values, empty collections, values out of range etc. In the past I would have traditionally written validation logic within the service implementation. This validation logic would then throw a fault to inform the caller of the data violations.

The requirements

I decided on this particular project to evaluate the other options available for WCF service request validations. Some of the issues of the traditional mechanism of service side validation that I wished to address where the following:

1. Minimise the number lines of code for validation constructs
2. Separate validation logic from the business logic
3. Reduce the number and complexity of unit tests through exclusion of validation from unit testing
4. Remove the need for weak entity attribute names from code. For example the common pattern of manual validation shown below

public void ProcessSomething(Something thing)
{
    if(thing == null)
    {
        var dataFault = new DataFault("thing cannot be null");

        throw new FaultException<DataFault>(dataFault, "Request validation error");
    }
}

In the above code snippet if the parameter name is modified without modifying he fault message, they may be located in different parts of code, the information conveyed by the fault is misleading to the caller.

The Options

From the relatively quick research I found the following alternatives available for WCF request validation:

• Message schema validation, via message inspection, an example of such a strategy implementation is shown here 
• IParameterInspector this in conjunction with some metadata could be used as the basis of a mechanism to inspect parameters on each operation invoke of a service 
• Enterprise Library Validation Block this component provides a set of attributes or alternately application configuration metadata to define operation and data type validation rules

The Choice

When the three options are evaluated against the requirements outline above all of the options will address requirements 2, 3 and 4 at some level. Message schema validation and IParameterInspector are basically variations on a message inspection pattern and share the same disadvantages namely the amount of configuration plumbing code in the form of endpoint, service and operation behaviours and configuration elements that are required. As an exercise in understanding the internals of WCF and the extensiblity of the framework either implementing message schema validation or an IParameterInspector would be an interesting and educational task.

With the clock ticking on the project I decided that the Enterprise Library Validation Block was the most optimal solution give the time constraints and the requirements for what I wished to achive. The following where some of the specific advantages to me:

• It reduces the number of LOC that I need to write for validation logic, it contains a comprehensive list of built in validators
• It is straight forward to configure, I chose to use attribution on my service contracts for configuration

[ServiceContract]
[ValidationBehavior]
public interface IService
{
    [OperationContract]
    [FaultContract(typeof(ValidationFault))]
    void ProcessSomething(
        [NotNullValidator]
        Something thing);
}

• Simplifies business logic as there is no longer a requirement to checks that are irrelevent to the business logic for example null checking. For example the validation attributes can enforce minimal values for Strings

[DataContract]
public class Something
{
    [DataMember]
    [NotNullValidator]
    [StringLengthValidator(0,10)]
    public String Name
    {
        get;
        set;
    }
}

// business logic can simply check the field value
// the validation attributes will ensured that the field 
// is not null by the time the business logic is invoked 
if(derivedSomething.Name == "Widget")
{
    OrderNewWidget();
}

• The caller is returned a ValidationFault which contains a collection of ValidationDetail objects, which contains all of the validation violations for a given request
• Custom validators are easy to implement and apply to code using the same mechanisms as with the built in validation attributes

There is one minor cavet when using the validation block, when a data contract type references another data contract type and a validation error occurs for the contained type, the ValidationDetail.Key for the validation error is always the type name of the parent data contract type.

To elaborate further take the following example data contract type Something that contains a nested Widget data contract type. The service comprises of an operation to process an instance of a Something type

[DataContract]
public class Something
{
    [DataMember]
    [NotNullValidator]
    [ObjectValidator]
    [ValidatorComposition(CompositionType.And)]
    public Widget ChildWidget
    {
        get;
        set;
    }
}

[DataContract]
public class Widget
{
    [DataMember]
    [NotNullValidator]
    public String Name
    {
        get;
        set;
    }
}

[ServiceBehavior]
public class Service : IService
{
    public void ProcessSomething(Something thing)
    {
    }
}

When the client invokes the service with a null value for Widget.Name it will recieve a ValidationFault whos Tag property refers to the containging types argument name on the service operation.

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<s:Fault>
<faultcode xmlns="">s:Client</faultcode>
<faultstring xml:lang="en-IE" xmlns="">The creator of this fault did not specify a Reason.</faultstring>
<detail xmlns="">
<ValidationFault xmlns="http://www.microsoft.com/practices/EnterpriseLibrary/2007/01/wcf/validation" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Details xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Practices.EnterpriseLibrary.Validation.Integration.WCF">
<a:ValidationDetail>
<a:Key>Name</a:Key>
<a:Message>The value cannot be null.</a:Message>
<a:Tag>thing</a:Tag>
</a:ValidationDetail>
</Details>
</ValidationFault>
</detail>
</s:Fault>
</s:Body>
</s:Envelope>

If both the Something type and the Widget type contained a Name propery then the information conveyed by the ValidationFault would be ambigous.

My first post

I've been meaning to start a blog for an age now so I have decided that 2012 will be the year I start. I think I need to get moving as I only have 345 days left according to the Mayans