Tips

LINQ-To-SQL Improving Performance

Linq to SQL is great. I love it because it adds a simple abstraction layer that can greatly speed up building a data access layer.

If not used properly, LINQ to SQL can also create performance issues. Here are my general LINQ to SQL guidelines when I work in projects:


Use the “using” statement when working with a context

This is mostly a general C# programming guideline but there have been several times when I see programmers missing this step. Here is more information from MSDN.

The using statement allows the programmer to specify when objects that use resources should release them. The object provided to the using statement must implement the IDisposable interface. This interface provides the Dispose method, which should release the object’s resources.

using statement can be exited either when the end of the using statement is reached or if an exception is thrown and control leaves the statement block before the end of the statement.

Here is an example:

using (NorthwindDataContext context = new NorthwindDataContext())
{
  //do stuff here
}


Compiled queries

To query something with LINQ to SQL there are several “startup” procedures. This procedures are not too bad when queries are not used too often. If the same query is done several times, its heavy and it is the core of the product then it is VERY important to make it a compiled query.

I will not go into too many details about this because there are several posts about the subject:


Use multiple tiny contexts instead of big bulky ones

Contexts are meant to keep track of the objects in the database. By having small contexts with a single purpose then the burden of tracking is lessen and therefore there is less memory consumption.


Do not keep track of object changes in the database unless its needed

There are two good ways to improve the performance of queries that do not involve concurrency issues:

  1. Optimistic concurrency  = off
  2. Object tracking = off

For object tracking, is super easy to turn off:

context.ObjectTrackingEnabled = false;

Sadly, as  Rick Strahl covers in his blog there are a few things/issues to consider when turning off Object Tracking.


Combine Queries and custom expressions

Combining queries is a good idea when working with databases, just grab what you need and aggregate the data into a POCO model or anonymous type. Finally, if extreme fine control is needed, there is always custom expressions.

TeamCity Agent has unregistered (will upgrade)

Recently I was setting up Teamcity and the build agent kept going down. It was starting and stopping. Sometimes it was “starting…” for a long time.

After doing a bit of research I came across this. The agents would appear only for seconds under team city and then go down as inactive with the message “Agent has unregistered (will upgrade)”. The culprit was my antivirus. Apparently this is a common issue but I have not seen many people blog about it.

Sophos and SVN

If you use Sophos Antivirus you might be getting Tortoise SVN related erros. It took me a while to figure it out. The best way to fix it is to exclude your projects directory from Sophos active scan.

Windows Azure Tools for Microsoft Visual Studio 1.2 (June 2010)

I am very happy Microsoft updated the Azure Tools to have full integration with Visual Studio. It is extremely easy to set up, just a few clicks to create a certificate and upload it. The process is all guided and it took me around 30 seconds. After the certificate has been uploaded you can deploy from Visual Studio with a neat status bar.

There are some other neat features like IntelliTrace and others that are now available.

Here is a link to the latest release: Windows Azure Tools for Microsoft Visual Studio 1.2 (June 2010)

Complete control over XML in WCF

Sometimes it is needed to have complete control over how WCF manipulates the data being returned. By default, WCF serializes objects and returns them as XML. Sadly, there is not much control on how to create templates over how objects will be serialized (flat structure, hierarchical, etc). In many cases this does not matter. A few weeks ago I stumbled for the first time when I need 100% control over how the XML was formated and being sent.

To send custom formated XML use the message envelope and not the string datatype. If the envelope is not used, it will add extra meta content.

Here is how it can be done:

public Message GetMessage(string xml)
{
    XmlDocument x = new XmlDocument();

    //This is very important as it will VALIDATE the XML. Saved my butt a few times.
    x.LoadXml(xml);

    XmlElementBodyWriter writer = new XmlElementBodyWriter(x.DocumentElement);

    Message msg = Message.CreateMessage(MessageVersion.None,
                OperationContext.Current.OutgoingMessageHeaders.Action, writer);

    return msg;
}

public class XmlElementBodyWriter : BodyWriter
{
    XmlElement xmlElement;

    public XmlElementBodyWriter(XmlElement xmlElement)
                : base(true)
    {
         this.xmlElement = xmlElement;
    }

    protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
    {
         xmlElement.WriteTo(writer);
    }
 }

Also a little warning, the string passed in should be formated and already encoded in the format you want. This can make a huge difference specially when internationalization is involved.

Value Objects in F#

Value Objects are objects that can be shared across different parts of a program. This can have great benefits in performance. Because  the objects are shared it is very important for value objects to be immutable. If value objects are not immutable then any part of the program can change the values and all the other parts can do calculation with erroneous data.

One of my goals was to make immutable objects in F# to take advantage of parallelization and automatic compiler optimizations:


type City(Name:string, X: float, Y:float) =

member t.Name = Name

member t.X = X

member t.Y = Y

type NeighborCities(city1:string, city2:string) =

member x.fromCity = city1

member x.toCity = city2

Objects created in this manner in F# are immutable. Even when they are accessed in other projects outside their definition, their members are read-only. This was specially helpful on the distributed traveling salesman problem I built last year.

Testing F#

F# is very cool. By combining the regular iterative features of the other .NET languages with an Ocaml syntax variant, F# is extremely flexible and powerful.

Here is a simple loop:

 for i in 0..loopSize do

Here is another example for reading in contents from a file, making an object city and returning a list of cities:


let ReadFile (s:string) = match s with
    | null -> raise (ApplicationException "Please provide a non-empty string for the filename!!")
    | s ->
    let cityList = new ResizeArray<CityDef.City>() in
    let rec ReadFile (r:StreamReader) =
        if not r.EndOfStream then
            cityList.Add(ParseCityString(r.ReadLine()))
            ReadFile r
        r.Close()
    let r = new StreamReader(s) in
    ReadFile r
    cityList

F# has powerful features because it has a lot of high order functions on top of the usual map, filter and reduce. The extra high order functions (such as iter, fold, scan, skip, skipWhile, sum, take, takeWhile, truncate, unfold, windowed, zip, zip3, etc.) make it easy to compound complex tasks.

The next example is a little snippet that returns a dictionary distances between cities for a small distributed traveling salesman problem I did last year to test F# on a software engineering class.


let squared x = x*x

let CompareNeighbors = {
    new System.Collections.Generic.IEqualityComparer<CityDef.NeighborCities> with
    member x.Equals(a,b) =  (String.Compare(a.fromCity, b.fromCity) = 0 ) && ( String.Compare(a.toCity, b.toCity) = 0 )
    member x.GetHashCode(a) = a.toCity.GetHashCode() }

let Distance (distanceDic : Dictionary<CityDef.NeighborCities, double>, i: CityDef.City, j: CityDef.City ) =
    let x2 = squared (i.X - j.X)
    let y2 = squared ( i.Y - j.Y)
    let total = sqrt( x2  +  y2)
    let n = new CityDef.NeighborCities(city1 = i.Name, city2 = j.Name)
    distanceDic.Add(n, total)

type DistanceDictionary() =
    interface Infrastructure.IDistanceCalculator with
        member this.DistanceDictionary (genericListOfCities:ResizeArray<CityDef.City>)  =
            let dic = new Dictionary<CityDef.NeighborCities, double>(CompareNeighbors)
            Seq.iter(fun z -> Seq.iter(fun x->Distance(dic, z, x)) genericListOfCities) genericListOfCities
            dic

Also it is nice being able to use interfaces with F# in the simple format of “interface Infrastructure.IDistanceCalculator with”.

As a little tip, for applying something to a list that does not require a new collection, it is important to avoid using the higher order function map. Map should only be used if a new list with a different set of values is expected because it creates a new list. In this case, map was not necessary and iter was the choice.

Dynamic Azure Web Services URLs

I am currently building a lot of web services for Azure. Since URLs change a lot in development and deployment environments I needed to be able to have those services URLs updated dynamically. The easiest way I found is using handlers and:


HttpContext.Current.Request.Headers["Host"];

This way none of the URLs get hard-coded and deployment is extremely easy. This also brings a lot of neat options for Intercloud communication, but that is for another post.

Unitiy IOC

Basically, with Unity, I ended up writing something like this (simplified version) :

IUnityContainer container = new UnityContainer();

container.RegisterType<IRiskRepository, RiskRepository>();

string conn = ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString;

container.Configure<InjectedMembers>().ConfigureInjectionFor<RiskRepository>(new InjectionConstructor(conn));

UnityControllerFactory factory = new UnityControllerFactory(container);

ControllerBuilder.Current.SetControllerFactory(factory);

Unity turned out to work very well. I was able to save a lot of lines of code and it is greatly improving the testability of the dynamic modules I am using on the mock project architecture. Also, it was easy to set up. Its a good alternative to the other more popular IOC.

Namespaces be gone

If you do not need specific namespaces, remove them. Most of the time it is not a big deal in small projects but specially on ASP.NET websites projects it can make a difference. The main difference is that  the compilation time can be significantly reduced and it will speed up your IDE.

If the website contains a lot of namespaces and if the project has C# 3.0 + which enables the use of extension methods it can make a significant difference. By having a lot of namespaces and using extension methods it can make the autocomplete list huge!

One last thing to note that you wont see any particular speed up on the actual execution time of the project if you remove them, they will not affect performance at all once the application is compiled. Its just one of those practices that makes development a bit faster and nicer specially in old computers like the ones I use at work.