Wednesday, 15 December, 2004

Last friday I had to design a method that would stream the response off one web-server to a client connected to our web-server in real time. Theres a whole range of algorithms you can design to perform this function but building one that doesn't eat the CPU is actually quite challenging.

There's a number of considerations to be taken into account when you're designing code. The buffer you use to collect data from the socket might not fill completely when the read method is called. The key to making the code efficient is being able to dynamically resize this underfilled array which is a task that is meant to be "impossible" in C# by many accounts on the Internet. Of course, this is utter rubbish you just have to know where to look.

The method looks like this:

while(Response.IsClientConnected &&  numberOfBytesRead < contentLength)
{
        int numberOfBytesReadOnThisGo = dataStream.Read(buffer, 0, 512);
        numberOfBytesRead += numberOfBytesReadOnThisGo;
        
        
        if (numberOfBytesReadOnThisGo<512)
        {
                Array newArray = Array.CreateInstance(typeof(byte), numberOfBytesReadOnThisGo);
                Array.Copy(buffer,0, newArray,0,numberOfBytesReadOnThisGo);
                Response.BinaryWrite((byte[]) newArray);
        }
        else
        {
                Response.BinaryWrite(buffer);
        }
        
        Response.Flush();
}

The key to the efficency of this method is the creation of an Array instance and the ability to use the Copy method to copy the items from the larger array to the smaller one. If you ever need to resize an array that's how you do it in C#. It's funny, now that i'm programing on a much larger code base that I could ever have hoped for at my previous job the kind of things you learn aren't so much about how to program well - it's about how not to program badly. It's important to write well commented, well structured and easy to read code because five months down the line I probably wont have a clue how the thing I've written works and neither will the rest of the programming team. If someone wants to build on my work later on it's going to be cheaper for the company if that code is easy to follow.

Here's a nice little rule I've come up with for governing optimisations: only optimise a program when the cost of the time it takes a programmer to fully understand the optimisation is less than the cost that the absence of the optimisation would incur.

Simon.