Having fun with code
  • Blog
  • Favorites
    • Favorite Books
    • Favorite Libraries
    • Favorite Tools
  • Utilities
    • SP 2010 Colors
  • Contact
  • Home /
  • .NET /
  • Service-based data adapter for LLBLGen entities using ServiceStack

Service-based data adapter for LLBLGen entities using ServiceStack

December 12, 2012 / Matt C. / .NET
This post is about leveraging the LLBLGen Pro and ServiceStack tools to allow users to use the existing LLBLGen Adapter API on the client to manage, manipulate, query and update entities on remote servers, across database platforms even (SQL, MySql, Oracle, etc…). This allows LLBLGen users to fully take advantage of the query API they are familiar with and write code manipulating entities as if they were on the server.

Intro

LLBLGen is a full-service ORM that produces a statically typed entity api to work with entities on top of relational databases (SQL, MySql, Oracle, DB2, etc…). ServiceStack is an open-source services framework that makes it easy to pass data between servers, applications, on and off the web. Put together, these 2 tools can be extremely powerful. LLBLGen makes working with databases a breeze, and ServiceStack makes building service-based interfaces and applications really simple. This post is about leveraging these 2 tools to allow users to use LLBLGen apis on the client to remotely manipulate entities (full CRUD) on your server(s), across database platforms even (SQL, MySql, Oracle, etc…). Let’s jump right in.

Use-case / Scenario

The use-case / scenario is the following:
  • Developers should be able to fetch, update, save, and delete entities from their desktop/laptop computers without ever logging onto any servers, and without having to learn a new API
  • The business is willing to set up one or more http service endpoints for developers to connect to as long as they can secure the service endpoints appropriately

Solution

Instead of subjecting you, the reader, to a lengthy overview of the implementation details to meet the use-case described above, I’ll just post the solution right now, and leave it up to you to read the implementation details if you’re interested. So, the solution? A single assembly that contains client-side and server-side logic for transmitting LLBLGen data across the wire using ServiceStack, allowing LLBLGen developers to use the exact same API they are already familiar with on the client! I’ve named the assembly: LLBLGen.DataServices.Contrib.dll, and the source is included as a download to this post. Use it, abuse it, refactor it, improve it. The download includes:
  • source code for the primary library
  • 2 host applications to simulate the server: a web application host, a console application which hosts an http listener
  • an NUnit test project, with sample usage code, which fires up an http listener to run CRUD and Unit of work tests
    • LLBLGen entity assemblies for the Northwind database, and SQL files to create a northwind database to run the tests

Usage

Basic CRUD:

LLBLGen developers are already familiar with the following code which allows them to interact with their entities and the database:
var customer = new CustomerEntity("CHOPS");
using (var adapter = new DataAccessAdapter())
{
    // fetch
    adapter.FetchEntity(customer);

    // modify and save
    customer.Address = "123 Mills Lane";
    adapter.SaveEntity(customer);

    // delete entity
    adapter.DeleteEntity(customer);
}
The code to do the same thing across the wire, in the new service-based paradigm looks identical:
var customer = new CustomerEntity("CHOPS");
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345"))
{
    // fetch
    adapter.FetchEntity(ref customer);

    // modify and save
    customer.Address = "123 Mills Lane";
    adapter.SaveEntity(customer);

    // delete entity
    adapter.DeleteEntity(customer);
}
As you can see, instead of using your DataAccessAdapter, you use a wrapper adapter called DataAccessAdapterServiceWrapper, and the method calls are all essentially the same (there are just a few methods where I had to introduce a “ref” parameter).

Transaction and Unit of Work:

For transaction support, at this time, you would use the Unit of Work paradigm, with a new class “UnitOfWorkWrapper” class, which works the same way as the UnitOfWork2 class:
var uow = new UnitOfWorkWrapper();

var category = new CategoryEntity{ CategoryName = newCategoryName };
uow.AddForSave(category);
uow.AddDeleteEntitiesDirectlyCall(typeof(CategoryEntity),
  new RelationPredicateBucket(CategoryFields.CategoryName == categoryNameToDelete));

using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345"))
{
    uow.Commit(adapter);
}

Security

LLBLGen has some validation and authorization hooks that can be leveraged server-side within the entity model to create some custom fine-grained access rules. In our use-case though, the requirement is to primarily secure the endpoint, and ServiceStack has all the hooks needed for that. The DataAccessAdapterServiceWrapper wraps a ServiceStack client. To add some basic authentication to your calls, use a custom method on the data adapter:
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "dev_db"))
{
    // authenticate with basic authentication
    adapter.SetCredentials(username, password);
}
Or, access the service client from ServiceStack and leverage all the hooks you need (see the authentication and autorization wiki page for ServiceStack), for example:
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "dev_db"))
{
    // authenticate with iis authentication
    adapter.ServiceClient.Credentials = new System.Net.NetworkCredential(username, password, domain);
}

One service interface for all your database:

You can use one service interface for any or all of your entities across databases, making it really easy to work with data, regardless of where the data lives: MySql, SQL, Oracle, DB2, etc… You can also use the same interface if you want to work on your dev, staging, prod databases as well, all you have to do is tell the server where to connect. Tell the server which connection to use with the constructor override, as follows:
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "dev_db"))
{
    // you are working against your dev database
}
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "staging_db"))
{
    // you are working against your staging database
}
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "Northwind"))
{
    // you are working against your Northwind database
}
using (var adapter = new DataAccessAdapterServiceWrapper("http://service_endpoint:12345", "BugsDbOnMySql"))
{
    // you are working against your Bugs database on MySql
}
Well, that’s it folks for the general overview, download the code if you want and take it for a spin. Read on for more implementation details where I’ll describe a little more how this is put together.

Code Download

Implementation Details

Server Implementation

Have a look at the code in the download for more details, but essentially the service side of things is textbook ServiceStack, so if you get familiar with that library, this will be nothing new to you. You can instantiate the server application host with something as simple as the following:
private static void Main(string[] args)
{
    var hostManager = new DataAccessAdapterServiceHostInitTeardown(port);

    // if you want to hook up an authentication / authorization provider, do it here before you initialize the host
    // see samples in the downloadable code, and/or visit the ServiceStack site

    hostManager.Init();
}
To tell the server which implementations are supported, add the following configuration to your app.config, or web.config, whichever applies:
  <configsections>
    <section type="LLBLGen.Contrib.DataServices.ConfigClasses.LLBLGenContribSection, LLBLGen.Contrib.DataServices" name="llblgen.contrib" />
  </configsections>
  <connectionstrings>
    <add name="NorthwindConnectionString" providername="System.Data.SqlClient" connectionstring="data source=localhost;initial catalog=northwind;integrated security=SSPI;persist security info=False;packet size=4096" />
    <add name="MyCustomOracleDB" ... />
    <add name="MyCustomMySqlDB" ... />
  </connectionstrings>
  <appsettings>
    <add value="2" key="CatalogNameUsageSetting" />
    <add value="2" key="SchemaNameUsageSetting" />
    <add value="0" key="ForceLLBLRouteAuthentication" />
  </appsettings>
  <llblgen.contrib>
    <adapterfactories defaultfactory="Northwind">
      <adapterfactory key="Northwind" connectionstringname="NorthwindConnectionString" adaptertype="Northwind.Data.DatabaseSpecific.DataAccessAdapter, Northwind.DataDBSpecific, Version=1.0.4712.34952, Culture=neutral, PublicKeyToken=null" />
      <adapterfactory key="MyOracleApp" connectionstringname="MyCustomOracleDB" adaptertype="My.Custom.Oracle.App.DatabaseSpecific.DataAccessAdapter, My.Custom.Oracle.App.DataDBSpecific, Version=1.0.4712.34952, Culture=neutral, PublicKeyToken=null" />
      <adapterfactory key="MySqlApp" connectionstringname="MyCustomMySqlDB" adaptertype="My.Custom.MySql.App.DatabaseSpecific.DataAccessAdapter, My.Custom.MySql.App.DataDBSpecific, Version=1.0.4712.34952, Culture=neutral, PublicKeyToken=null" />
    </adapterfactories>
  </llblgen.contrib>
Now, from the client, you can just use the appropriate adapter factory key as your 2nd parameter in the DataAccessAdapterServiceWrapper previously discussed. If no 2nd parameter is passed, the default factory in the configuration above is used.

How the data is passed between the client and the server

In order for this all to work, I considered several serialization options for passing data. I started by creating DTOs to mimick the PrefetchPaths, Predicate expression, and Relations, but I basically realized I was going down the road of re-creating complex object graphs and having to map these back and forth to the LLBLGen objects was simply more work than I was willing to do. So I basically just decided to use binary serialization and use ServiceStack's capability to directly compose and intercept, serialize and deserialize messages into and from the request and response streams. It turns out that using binary formatting and serialization did just the trick. The entire serialization and deserialization and stream parsing for all methods is done in one simple extension method:
public static class DataAccessAdapterServiceExtensions
{
    public static TResponse ExecuteRemoteDataAccessAdapterMethod<trequest  , TResult TResponse,>(
        this JsonServiceClient client, object objectToStreamInRequest)
        where TResult : IStreamWriter
        where TRequest : IRequiresRequestStream, IReturn<tresult>, new()
        where TResponse : class, IHasResponseStatus, new()
    {
        client.LocalHttpWebRequestFilter = httpReq =>
            {
                httpReq.AllowWriteStreamBuffering = false;
                httpReq.SendChunked = true;
                httpReq.ContentType = "multipart/form-data;";
                httpReq.Timeout = int.MaxValue;

                using (var m = new MemoryStream())
                {
                    LLBLSerializationHelper.SerializeToStream(m, objectToStreamInRequest);
                    m.WriteTo(httpReq.GetRequestStream());
                }
            };

        object response = null;
        client.LocalHttpWebResponseFilter = httpResp =>
            {
                using (var memoryStream = new MemoryStream())
                {
                    Stream responseStream = httpResp.GetResponseStream();
                    if (responseStream != null)
                    {
                        responseStream.CopyTo(memoryStream);
                        response = LLBLSerializationHelper.DeserializeFromStream(memoryStream);
                    }
                }
            };

        var request = new TRequest();
        try
        {
            client.Post(request);
        }
        catch (WebServiceException serviceException)
        {
            if (serviceException.ResponseStatus != null &&
                !string.IsNullOrEmpty(serviceException.ResponseStatus.ErrorCode))
            {
                var responseStatus = serviceException.ResponseStatus;
                throw new ApplicationException(string.Concat(responseStatus.ErrorCode, ": ", responseStatus.Message));
            }
            throw;
        }
        return response as TResponse;
    }
}
The "LLBLSerializationHelper.SerializeToStream" method just does binary serialization:
public static class LLBLSerializationHelper
{
    public static Stream SerializeToStream(Stream stream, object o)
    {
        if (stream == null)
            stream = new MemoryStream();
        var formatter = new BinaryFormatter();
        SerializationHelper.Optimization = SerializationOptimization.Fast;
        formatter.Serialize(stream, o);
        SerializationHelper.Optimization = SerializationOptimization.None;
        return stream;
    }

    public static object DeserializeFromStream(Stream stream)
    {
        var formatter = new BinaryFormatter();
        stream.Seek(0, SeekOrigin.Begin);
        SerializationHelper.Optimization = SerializationOptimization.Fast;
        object o = formatter.Deserialize(stream);
        SerializationHelper.Optimization = SerializationOptimization.None;
        return o;
    }
}
There's a little more to the code, but that's the essentials. If you have questions, ask in the comments and/or ping me directly. Take care! C#, llblgen, orm, servicestack, web api

9 comments on “Service-based data adapter for LLBLGen entities using ServiceStack”

  1. James Saunders says:
    August 27, 2013 at 3:54 pm

    How can I create a web services using your code that is not attached to an entity. I need to pass data to a web service call and have it process a command and update multiple entities using each repository.

    Thanks James

    • Matt C. says:
      August 27, 2013 at 4:37 pm

      Hi James, this post was about trying to create a facade layer that allows you to work with the DataAccessAdapter conventions remotely (off-server), so I’m not sure it applies too well to creating one-off web services of your own. If you’re trying to create some custom web services using ServiceStack to work with your LLBLGen entities, I’d recommend just creating a custom ServiceStack service of your own (see: https://github.com/ServiceStack/ServiceStack/wiki/Create-your-first-webservice), and then doing whatever you need to do on the server with your DTO data, and/or you might want to check out my post https://www.mattjcowan.com/funcoding/2013/03/10/rest-api-with-llblgen-and-servicestack/ which describes how to create a truly detached (client/server) service layer (as opposed to working with the DataAccessAdapter convention). Hope that helps a little.

  2. Matt C. says:
    March 3, 2013 at 10:38 pm

    Ok Rudy, thanks for the feedback. I’ll try to put a post together to address that scenario.

    • Matt C. says:
      March 10, 2013 at 11:04 pm

      Rudy, check out my new post for a fully RESTful API created with LLBLGen over your entities using ServiceStack as the service layer. Let me know what you think.

  3. Rudy Scott says:
    March 1, 2013 at 11:12 am

    I’d Be VERY interested in that. This is a direction I’m considering for my next project.

  4. scotru says:
    February 8, 2013 at 4:22 am

    Very interesting post. Am I correct in understanding that the client’s here are dependent on the LLBLGen runtime libraries? Have you tried to get this to work with say a MonoTouch or Monodroid client?

    • Matt C. says:
      February 11, 2013 at 3:20 pm

      You’re correct, this specific implementation does require the LLBLGen runtime libraries on the client. I’ve done zero testing at this point with llblgen on mono, something I’d like to do very soon. The implementation in this post is something I use in LINQPad to do CRUD on remote databases with (without having to login to the server). As a side-note, I started working on a set of LLBL templates to generate a REST API (with some basic hypermedia and discoverability enhancements) using ServiceStack on top of the LLBL RTF, and which is I think better suited for high-performance mobile dev and client apps (zero requirements on the client-side beside an http connection), I could potentially do a post on that if anybody finds that interesting.

  5. Frank Nezrick says:
    December 13, 2012 at 9:49 am

    Nice Job! As always!!!

    • Matt C. says:
      December 14, 2012 at 10:31 am

      Cool, glad you liked it.

Categories

  • .NET (20)
  • ASP.NET MVC (4)
  • JAMstack (2)
    • Headless CMS (2)
  • Miscellaneous (2)
  • Python (1)
  • REST (3)
  • SharePoint (8)
  • WordPress (1)

Tags

.net ado.net autofac binding C# chrome code generation command line console application csv dapper di entity framework integration ioc job scheduling engine json jsv learning llblgen load balancing micro-orm mycorp odata orm people editor people picker picker controls picker dialog plugins pmp python Quartz.NET rest saf service application servicestack sharepoint smo soap sql sqlalchemy tornado web server validation web api

Archives

  • May 2020 (2)
  • November 2013 (1)
  • June 2013 (1)
  • May 2013 (1)
  • March 2013 (1)
  • December 2012 (1)
  • November 2012 (1)
  • October 2012 (3)
  • September 2012 (2)
  • June 2012 (2)
  • May 2012 (1)
  • April 2012 (1)
  • February 2012 (2)
  • January 2012 (1)
  • December 2011 (2)
  • September 2011 (2)
(c) 2013 Having fun with code - "FunCoding" theme designed by mattjcowan using the Theme Blvd framework