Having fun with code
  • Blog
  • Favorites
    • Favorite Books
    • Favorite Libraries
    • Favorite Tools
  • Utilities
    • SP 2010 Colors
  • Contact
  • Home /
  • .NET /
  • Adding LLBLGen to the Dapper performance benchmark test project

Adding LLBLGen to the Dapper performance benchmark test project

February 24, 2012 / Matt C. / .NET
The Dapper project on GitHub has a performance/benchmark test project that compares a number of ORMs and data access strategies. While the project only tests a very simple fetch algorithm, it's a good starter indicator for performance. I thought I would add one of my favorite ORMs (LLBLGen) to the mix, and see how it performs. Code samples are included in this post. The Dapper project hosted on GitHub at https://github.com/SamSaffron/dapper-dot-net has a performance/benchmark test project that compares a number of ORMs and data access strategies. While the project only tests a very simple fetch algorithm, it’s a good starter indicator for performance. I thought I would add one of my favorite ORMs (LLBLGen) to the mix, and see how it performs. [box style="blue"]Post update: 5/3/2013 LLBLGen has recently published it's V4 release. In addition to this post, you may also be interested in these LLBLGen Pro v4 benchmarks.[/box]

The Tests

Here’s the code I added to the PerformanceTests.cs class file. Use this code, in conjunction with the download further down to perform the tests yourself if you're familiar with LLBLGen. Here are the LLBLGen Adapter Tests:
//LLBLGen Adapter
var adapter1 = new DataAccessAdapter(Program.connectionString, true, CatalogNameUsage.Clear, null);
adapter1.OpenConnection();
tests.Add(id => adapter1.FetchEntity(new PostEntity(id)), "LLBLGen Adapter (FetchEntity w/ KeepConnectionOpen)");

var adapter2 = new DataAccessAdapter(Program.connectionString, false, CatalogNameUsage.Clear, null);
adapter2.OpenConnection();
tests.Add(id => adapter2.FetchEntity(new PostEntity(id)), "LLBLGen Adapter (FetchEntity)");

var adapter3 = new DataAccessAdapter(Program.connectionString, true, CatalogNameUsage.Clear, null);
adapter3.OpenConnection();
tests.Add(id =>
  {
      var bucket = new RelationPredicateBucket();
      bucket.PredicateExpression.Add(PostFields.Id == id);
      var entities = new EntityCollection();
      adapter3.FetchEntityCollection(entities, bucket);
      entities.First();
  }, "LLBLGen Adapter (Predicate w/ KeepConnectionOpen)");

var adapter4 = new DataAccessAdapter(Program.connectionString, false, CatalogNameUsage.Clear, null);
adapter4.OpenConnection();
tests.Add(id =>
  {
      var bucket = new RelationPredicateBucket();
      bucket.PredicateExpression.Add(PostFields.Id == id);
      var entities = new EntityCollection();
      adapter4.FetchEntityCollection(entities, bucket);
      entities.First();
  }, "LLBLGen Adapter (Predicate)");
Here are the LLBLGen Self-Servicing Tests:
//LLBLGen SelfServicing
CommonDaoBase.ActualConnectionString = Program.connectionString;
tests.Add(id => (new LLBLGenSS.EntityClasses.PostEntity()).FetchUsingPK(id), "LLBLGen SelfServicing (UC Fetch)");

tests.Add(id => new LLBLGenSS.EntityClasses.PostEntity(id, (IPrefetchPath)null), "LLBLGen SelfServicing (PK in constructor)");

tests.Add(id =>
  {
      var entities = new LLBLGenSS.CollectionClasses.PostCollection();
      entities.GetMulti(LLBLGenSS.HelperClasses.PostFields.Id == id);
      entities.First();
  }, "LLBLGen SelfServicing (Predicate)");
var transactionManager = new LLBLGenSS.HelperClasses.Transaction(IsolationLevel.ReadUncommitted, "Test", Program.connectionString);
tests.Add(id =>
  {
      var entity = new LLBLGenSS.EntityClasses.PostEntity();
      transactionManager.Add(entity);
      entity.FetchUsingPK(id);
  }, "LLBLGen SelfServicing (UC Fetch in Transaction)");
Now let’s see the results.

The Results

image In plain text format:
Strategy Time
Dynamic Mapper Query (buffered) 57ms
Mapper Query (non-buffered) 58ms
Mapper Query (buffered) 59ms
Dynamic Mapper Query (non-buffered) 59ms
hand coded 59ms
PetaPoco (Fast) 63ms
Dapper.Cotrib 64ms
OrmLite QueryById 64ms
PetaPoco (Normal) 67ms
Dynamic Massive ORM Query 69ms
BLToolkit 95ms
Simple.Data 98ms
Linq 2 SQL Compiled 106ms
LLBLGen Adapter (FetchEntity w/ KeepConnectionOpen) 121ms
SubSonic Coding Horror 123ms
NHibernate Session.Get 124ms
LLBLGen SelfServicing (UC Fetch in Transaction) 126ms
LLBLGen Adapter (Predicate w/ KeepConnectionOpen) 127ms
NHibernate SQL 128ms
Entity framework CompiledQuery 135ms
NHibernate HQL 141ms
LLBLGen Adapter (FetchEntity) 145ms
LLBLGen SelfServicing (UC Fetch) 148ms
LLBLGen SelfServicing (PK in constructor) 150ms
LLBLGen Adapter (Predicate) 155ms
LLBLGen SelfServicing (Predicate) 164ms
NHibernate Criteria 175ms
Soma 186ms
Linq 2 SQL ExecuteQuery 240ms
Linq 2 SQL 737ms
NHibernate LINQ 769ms
Entity framework ESQL 812ms
Entity framework ExecuteStoreQuery 829ms
Entity framework 1063ms
Entity framework No Tracking 1065ms
SubSonic ActiveRecord.SingleOrDefault 4874ms

Download

To run the LLBLGen test, just include the 2 folders in the attached zip file into the Dapper project, and reference the LLBLGen assemblies

Project:

image

References:

image [button link="/funcoding/files/2012/02/LLBL.zip" color="primary" size="large" title="Download Code" icon_before="download"]Download Code[/button]

Conclusion

While LLBLGen is not a Micro-ORM, the LLBLGen Adapter and SelfServicing runtime frameworks rank up there at the top performance-wise. I’ve worked with Entity Framework extensively in the past few months, including Code-First, even though that doesn’t make me an expert, but none-the-less, LLBLGen is more attractive now than ever before. We’ll see how EF 5.0 turns out. Meanwhile, NHibernate, PetaPoco, Dapper, and LLBLGen all look like good options.

Separate study (updated on 4/20/2012)

I found the codebase I used a while back to compare basic CRUD operations between EF 4.1, Dapper and LLBGen 3.1. I took a quick stab at refactoring it a little with newer libraries and here are the findings. The results are in milliseconds, and are averages taken over 5 iterations of 1000, 2000, and 5000 user records. Code is provided as a download. I make a distinction in my tests between BATCH and ATOMIC transactions. A BATCH transaction is a transaction over the entire span of records (start_transaction foreach { do_record_update } commit_transaction); whereas, an ATOMIC transaction is a transaction on each individual record (foreach {  start_transaction do_record_update commit_transaction }).
read_atomic read_batch
insert update
delete
All frameworks “delete”, and “insert” about the same. Dapper clearly outperforms both LLBGen and EF in batch reading data out of a database (although, all frameworks retrieve over 5000 records in less than 90ms in my tests, which is probably good enough for most applications), unless you’re running a site like StackOverflow, in which case, you might as well switch to a search engine instead. LLBLGen is a near 2nd, EF takes 3rd place (with some concerns in the area of “atomic reads” and “updates”). Of course, I make assumptions in my tests, and the database in these tests is not indexed, so this isn’t the final say on the matter, take it as you will Smile. [button link="/funcoding/files/2012/04/misc_orm_perf_tests.zip" color="primary" size="large" title="Download Code" icon_before="download"]Separate Study Code[/button] ado.net, dapper, entity framework, llblgen, micro-orm

5 comments on “Adding LLBLGen to the Dapper performance benchmark test project”

  1. Matt C. says:
    April 20, 2012 at 8:30 pm

    Sam, I found the bit of code I wrote a while back. I refactored it a little, and updated the post.

  2. sam says:
    April 19, 2012 at 11:25 pm

    I tried to download the insert sample and could not. Can you ensure your insert/delete etc batch is both in a transaction and using the batch interface eg: cnn.Execute(“insert t values {@a)”, new[] { new {a = 1} , new {a =2}]}, transaction: trans)

  3. Dew Drop – February 24, 2012 (#1,272) | Alvin Ashcraft's Morning Dew says:
    February 24, 2012 at 10:33 am

    […] Adding LLBLGen to the Dapper performance benchmark test project (Matt Cowan) […]

  4. Frans Bouma says:
    February 24, 2012 at 9:55 am

    Nice 🙂

    Though I still think benchmarks on data-access code is a bit of a gamble… for example: we profile our code a lot, making everything that is a tiny bit slow faster etc. however we also know that the things which makes our framework slower than e.g. dapper or the other micro-ORMs is that during fetching or other db interactions, some features have to be called, simply because they’re supported in the framework. This makes things less ideal from the raw performance point of view (so it might look ‘less ideal’) but from the ‘features vs. performance’ point of view it’s a different picture.

    Nevertheless, code shouldn’t perform unnecessarily slow, so a user must be able to trust the developers who wrote the framework that they did everything they could do to make it perform as fast as possible. I’m glad the tests show we did.

    There’s always room for improvements though. But in general those optimizations aren’t free: always something has to pay the price, e.g. a feature is not supported anymore, or code is less maintainable etc.

    • Matt C. says:
      February 24, 2012 at 10:42 am

      I agree, well said, and these tests are not meant to be gospel, but they are indicative. I also agree that features should be included in product comparisons. It would be monumental however to test every feature combination across each ORM :-), especially when you take designers, modelers, code-generation templating, apis and more into consideration. There’s a range of acceptability in these things, in my opinion, unless one is building a system where raw speed is the primary measure. It’s nice in the end that LLBLGen didn’t sacrifice performance for the sake of features.

Categories

  • .NET (20)
  • ASP.NET MVC (4)
  • JAMstack (2)
    • Headless CMS (2)
  • Miscellaneous (2)
  • Python (1)
  • REST (3)
  • SharePoint (8)
  • WordPress (1)

Tags

.net ado.net autofac binding C# chrome code generation command line console application csv dapper di entity framework integration ioc job scheduling engine json jsv learning llblgen load balancing micro-orm mycorp odata orm people editor people picker picker controls picker dialog plugins pmp python Quartz.NET rest saf service application servicestack sharepoint smo soap sql sqlalchemy tornado web server validation web api

Archives

  • May 2020 (2)
  • November 2013 (1)
  • June 2013 (1)
  • May 2013 (1)
  • March 2013 (1)
  • December 2012 (1)
  • November 2012 (1)
  • October 2012 (3)
  • September 2012 (2)
  • June 2012 (2)
  • May 2012 (1)
  • April 2012 (1)
  • February 2012 (2)
  • January 2012 (1)
  • December 2011 (2)
  • September 2011 (2)
(c) 2013 Having fun with code - "FunCoding" theme designed by mattjcowan using the Theme Blvd framework