Wednesday 14 March 2012

Configurable OData Service - Part 2 – Data Type Generator

In the this post I am going to cover some of the implementation details of the Data Type generation component of the configurable OData service.
The primary purpose of this component is to dynamically generate data types based on some metadata. The metadata describes the properties of the types and how the type maps to the underlying data store. The metadata is defined by an end user via a WPF application and is stored in a database via an intermediary WCF service that manages the metadata.
Metadata metadata

Reflection Type Generation

Given metadata for a Type to be generated at runtime, take for example the following metadata definition:
EntityDescriptor entityMetadata = new EntityDescriptor()
{
    BaseType = typeof(IEntity),
    EntityName = "GeneratedEntity",
    EntityProperties = new List<EntityPropertyDescriptor>()
    {
        new EntityPropertyDescriptor()
        {
            ColumnName = "Id",
            IsIdentity = true,
            IsNullable = false,
            Name = "Identifier",
            Type = typeof(Guid)
        },
        new EntityPropertyDescriptor()
        {
            ColumnName = "Comment",
            IsIdentity = false,
            IsNullable = false,
            Name = "Comment",
            Type = typeof(String)
        },
TableName = "TestTable"     } };

The first step in generating the dynamic types is to create an instance of an AssemblyBuilder and a ModuleBuilder into which the Type, based on metadata will be emitted.
ModuleBuilder moduleBuilder;
AssemblyBuilder assemblyBuilder;
DynamicTypeGenerator.Instance.CreateDynamicAssembly("TestAssembly", out assemblyBuilder, out moduleBuilder);

Now we have an AssemblyBuilder instance and an instance of a ModuleBuilder which will be used in subsequent calls to the DynamicTypeGenerator instance to emit types into. For the purpose of this example only one Type is to be emitted into the Assembly.

Given the above entity metadata a call to the CreateLinqToSqlType method of the DynamicTypeGenerator instance passing the type metadata generates the underlying Type for the metadata description.

DynamicTypeGenerator.Instance.CreateLinqToSqlType("TestAssembly", assemblyBuilder, moduleBuilder, entityMetadata);

The CreateLinqToSqlType as expected generates a simple LinqToSql data type entity that has attributes applied to its properties to support the mapping of the Type to an underlying database. There is not anything extremely complicated about this method. It uses a number of the ReflectionEmit constructs to generate the Types and the ILGenerator class to emit IL into the get and setters of the Types properties.

Given the entityMetadata above the following is the generated C# code extracted using ilSpy

GeneratedType

Looking a bit closer at the GeneratedEntity Type above we see the following:


  • A Table attribute has been applied to the Type

  • Each of the Types properties has a Column attribute applied

  • The identity property has the IsPrimaryKey set to true

  • The generated Type  derives from an interface IEntity

IEntity

So what does this give us?


  • A type that can be used via LinqToSql to query, update and delete entities in the underlying data store

  • An abstraction from the underlying data stores table and column names. It is not a requirement for the table and generated type names to match. This also applies to the properties

  • A type that derives from a statically types known interface type called IEntity. This allows code that consumes these types to cast to a known defined type. Thus reducing the amount of reflection based referencing required to manipulate these generated types

What doesn't this give us?


  • There is no abstraction of the underlying types from the data store. So if the underlying column is of type nvarchar then the property of the generated entity must be a String. The generator does not support type coercion or conversion which can get messy. The WCF service that allows the definition of the types metadata ensures that this cannot occur

  • There is no abstraction from the underlying data stores foreign key relationships. So if a type is mapped to a table that requires a foreign key value to be specified which is not exposed via the generated type then new rows cannot be inserted

  • There is no support for nested relationships. i.e a Thing contains a list of Parts. For the use cases of the overall project this is not a key requirement and can be added with more elaborate metadata and a change to the type generator

So we now have one of the key building blocks of the configurable OData service. In the next post, Part 3 – Custom Data Service Provider, we will see how this fits into place.

Sunday 4 March 2012

WCF Data Services + Reflection + Custom Data Provider = Configurable OData Service

Part 1 Intro

I really enjoy a good technical challenge, in this day and age of highly evolved frameworks and tool kits it is good to get a set of requirements for which there is not an obvious out of the box solution for. One that requires a little bit more than a few LOC and a bit of configuration tweaking.

Background

The challenge which I was presented with, was in the context of a windows workflow 4 (WF4) project for which I am in the midst of designing and developing a number of components. The project is to integrate WF4 into the main product line. The aim is to enable end user business analysts or customer service consultants to design a set of workflow processes that are unique to each customer.
The main components of this solution are the following:
  • Workflow Designer – A WPF application to enable the design, management and deployment of workflow processes
  • Workflow Activities – A set of custom activities that encapsulate common business actions
  • A Workflow Engine – A host component to manage the execution, lifecycle and versioning of the various running workflows that a end user can design
  • A Domain Model Service – A service which exposes a consistent, version invariant view of the applications data model
Future blog entries will cover some of the other components. This is the first post in a series that will cover some of the background behind the Domain Model Service

Requirements

  • Enable Users Define Entities
    • The Domain Model Service should allow an end user to define a Domain Model entity via a designer invoking an API
  • Support CRUD on Entities
    • The Domain Model Service should support CRUD operations on all Entities that are defined
  • Enable Database Abstraction
    • Expose Entities in a manner that allows attribute names and entity names to differ from that of the underlying data store. This prevents implementation leaking out to the upper layers
  • Enable Versioning
    • A workflow instance can potentially execute indefinitely. It is therefore a requirement to maintain multiple versions of the Domain Model
  • Enable Query Of Entities
    • Allow an Entity to be queried by any combination of its attributes

The Solution

ComponentView

Metadata Store – This component is the store of all Domain Models and the entities that are defined for each of the Domain Models. The data within this store is managed by a different set of components, which are beyond the scope of this post.

WCF Service – This component provides the hosting and lifecycle management for the WCF Data Services (OData), which are defined within the metadata. This component exposes the OData services each of which exposes a version if the Domain Model at a different unique URL. It would have been possible to host each version of the Domain Model in its own process but this is not required for the initial implementation. The Domain Model services could be hosted in an own process if required in the future.

Data Type Generator – This component generates the set of entities for a given Domain Model through the use of reflection.

Domain Model Service – This component is the realisation of a concrete implementation of a Domain Model version. This is a WCF Data Services (OData) service that uses the Custom Data Service Provider as the source and sink of the Domain Model data that it exposes.

Custom Data Service Provider – This component provides the data and the mechanisms for querying an updating Domain Model objects for a given Domain Model Service instance.

In the following posts I will delve into more detail for each of the components.

Part 2 – Data Type Generator
Part 3 – Custom Data Service Provider
Part 4 – WCF Service Host
Part 5 - Put it all together