Saturday, April 26, 2008

Infected by Scott Sigler

The book is a classic American sci-fi book, including small town mentality, a sports and US centric view, lots of government agencies, all working for the good of the citizens, keeping them all safe and ignorant and the warped morality that tells people they should destroy before they understand, just because they fear it.

That is something to be expected from an American author, though, and the book itself is not bad. It felt like it was inspired a lot by Stephen King's Dreamcatcher, which featured a similar personal dillema of alien infestation while the gov'ment was on the chase, but that one had more oompf. Of course, you can't compare aspiring Sigler to King, but then again, King's writing was never so great to me to begin with.

What I found really astounding is that a civilisation that uses biological machines to create a beach head on another planet would be so easily thwarted by a college athlete, a trigger happy black ops CIA agent and about a doctor and a half. Oh, and some Apache helicopters. What bothered me to no end is that I also felt this was a plausible scenario. I hope I am just stupidly influenced by similar literature, but would it really help to destroy the enemy before you get to at least understand it? What about the technology that was so easily recognisable as foreign and above Earth's current scientific level?

As a conclusion to both book and my own feelings: it was a nice read; not spectacular, but good enough to keep reading till the end. It is also available in podcast format and I myself have read it from a text file saved from a PDF that was gracefully provided free of charge by mr Sigler on the Escapepod podcast site.

Wednesday, April 23, 2008

Coma Light concert at Preoteasa in Bucharest

Coma is one of my favourite bands if not THE favourite. They are Romanians, I know them, they're cool guys. Recently they organised an unpluggish concert called Coma Light. This is one of the songs there, although I do encourage you to look at them all (YouTube links gracefully provided by Imig/Smallex under the video). Great job, guys! Most of the songs you played deserve an album of their own. Here is the video for Culori, high quality from MetalHead TV. from YouTube.



Links to all the songs in the concert:
Stai
Mai presus de cuvinte
Bizz
Culori
In mine in soapta
Hectic
Un loc sa ajung
Daddy
Morphine
Coboara-ma-n rai
Cine iubeste si lasa
Canta-mi povestea

Tuesday, April 22, 2008

Compressing UpdatePanel output

Update: I've posted the source and binaries for this control on Github. Free to use and change. Please comment on it.

This is the story of a control that shrinks the content sent from an UpdatePanel to down as 2% without using compression algorithms. I am willing to share the code with whoever wants it and I only ask in return to tell me if and where it went wrong so I can find a solution. Even if you are not interested in the control, the article describes a little about the inner workings of the ASP.Net Ajax mechanism.

You know the UpdatePanel, the control that updates its content asynchronously in ASP.Net, allowing you to easily transform a normal postback based application in a fully fledged Ajax app. Well, the only problem with that control is that you have to either put a lot of them on the page in order to update only what changes (making the site be also fast as you would expect from an Ajax application) but hard to maintain afterwards, or put a big UpdatePanel on the entire page (maybe in the MasterPage) and allow for large chunks of data to be passed back and forth and also other clear disadvantages, some detailed in this blog entry.

Not anymore! I have made a small class, in the form of a Response.Filter, that caches the previously rendered content and instructs the browser to do the same, then sends only a small fraction of the data from the server to the browser, mainly what has changed. There is still the issue of the speed it takes the browser to render the content, which is the same no matter what I do, like when rendering a huge table. It doesn't matter that I send only the changes in one cell, the browser must still render the huge grid. Also, if, for some reason, the update fails, I catch it and I send to the server that the updatepanel must be updated again, the old way.

Enough; let's talk code. I first had to tap into the content that was sent to the browser. That can only be done at Page render level (or PageAdapter, or Response.Filter and other things that can access the rendered content). So I did catch the rendered content in a filter, I recognized it as Ajax by its distinctive format, and I only processed the updatePanel type of token.

Here I had a few problems. First I replaced the updatePanel token with a scriptBlock token that changed the innerHTML of the update panel div element. It all seemed to work until I tested it a little. I discovered that the _updatePanel javascript method of the PageRequestManager object used by the normal ajax rendering on the browser was doing a few extra things, so I used that one instead of just replacing the innerHTML, resulting in a lower speed. But that didn't help either, because it failed when using validators. Even if I did replace the updatePanel token with a correct javascript block, it still got executed a bit later than it should have.

The only solution I had was to replace the _updatePanel method with my own. Itself having a small block of code that disposed some scriptblocks and some other stuff, then a plain innerHTML replace, I could not 'override' it, since it would change the innerHTML with some meaningless stuff (the thing I would send from the server), then I would parse and change the innerHTML again, resulting in bad performance, flickering, nonsense on screen, etc. So I just copy pasted the first part and added my own ApplyPatch function instead of the html replace code line.

Now, here I met another issue. The innerHTML property of an html element is not a simple string. It gets parsed immediately when set and it recreates when read, as explained in this previous article of mine. The only solution for that was create my own property of the update panel div element that remembers the string that was set. This solved a lot more problems, because it meant I could identify the content to be replaced by simple position markers rather than through regular expressions (as was my initial idea). That property would not get changed by custom local javascript either, so I was safe to use it.

About the regular expression engine in Javascript: it has no Singleline option. That means you can only change the content line by line. I could have used Steve Levithan's beautiful work, but with the solution found above, I would not need regular expressions at all.

The only other issue was with UpdatePanels inside UpdatePanels. I found out that in this case, only parent UpdatePanels are being rendered. That meant that the custom property I added to the child panel would disappear and break my code. Therefore I had to keep a tree of the updatepanels in the page and clear all the children cached content when the parents were being updated. So I did that, too.

What else? What if somehow the property would get deleted, changed, or something else happened, like someone decided to recreate the update panel div object or something like that? For that I made a little HttpHandler that would receive an UpdatePanel id and it would clear its cached content. Then, on return from the asynchronous call, the javascript would just push another update panel refresh using __doPostBack(updatePanelId,""). I don't particularily like this approach, since it could back fire with multiple UpdatePanels (as you know, only one postback at a time is supported), but I didn't find a better solution yet. Besides, this event should normally not happen.

So, the mechanism was all in place, all I had to do was make the actual patching mechanism, the one that would find the difference between previously rendered content and current content, then send only the changed part. First thing I did was remove the start and end of the strings that were identical. As you can imagine, that's the most common scenario: a change in the UpdatePanel means all the content up to the change remains unchanged and the same goes for the content after the change. But I was testing the filter with a large grid that would randomly change one cell to a random string. That meant two changes: the previous position and the last. Assuming the first change was in one of the starting cells and the last was in one of the cells at the end, then the compression would be meaningless. So I've googled for an algorithm that would give me the difference between two files/strings and I found Diff! Well, I knew about it so I actually googled for Diff :) It was in the Longest Common Substring algorithm category.

Anyway, the algorithm was nice, clear, explained, with code, perfect for what I wanted and completely useless, since it needed an array of m*n to get what I needed. It was slow, a complete memory hog and I couldn't possibly use an array of 500000x500000. I bet they were optimizations that covered this problem, but I was miserable so I just patched up my own messy algorithm. What would it do? It would randomly select a 100 characters long string from the current content and search for it in the previous content. If it found it, it would expand the selection and consider it a Reasonably Long Common Substring and work from then on recursively. If it didn't find it, it would search a few times other randomly chosen strings then give up. Well, actually is the same algorithm, made messy and with no extra memory requirements.

It worked far better than I had expected, even if it clearly could have used some optimizations. The code was clear enough in detriment of speed, but it still worked acceptably fast, with no noticeable overhead. After a few tweaks and fixes, the class was ready. I've tested it on a project we are working on, a big project, with some complex pages, it worked like a charm.

One particular page used a control I have made that allows for grid rows and columns to have children that can be shown/hidden at will. When collapsing a column (that means that every row gets some cells removed) the compressed size was still above 50% in up to 100 patch fragments. When colapsing a row, meaning some content from the middle of the grid would just vanish, the size went down to 2%! Of course, putting the ViewState into the session also helped. Gzip compression on the server would complement this nicely, shrinking the output even more.

So, I have demonstrated incredible compression of UpdatePanel content sent through the network with something as small and reusable as a response filter that can be added once in the master page. You could use it for customers that have network bandwidth issues or for sites that pay for sent out content. It would with sites made with one big UpdatePanel placed in the MasterPage as well :).

If you want to use it in your sites, please let me know how it performs and what problems you've encountered.

Saturday, April 19, 2008

innerHTML and the rearranged attributes

I have been working on an idea, one that assumed that if you remember the content sent to an UpdatePanel, you can send only the difference next time you update it. That would mean that you could use only one UpdatePanel, maybe in the MasterPage, and update only changes, wherever they'd be and no matter how small.

However, I also assumed that if I set the innerHTML property of an HTML element, the value will remain the same. In other words that elem.innerHTML=content; implied that elem.innerHTML and content are identical at the end of the operation. But they were not! Apparently the browser (each flavour in its own way) interprets the string that you feed it then creates the element tree structure. If you query the innerHTML property, it uses the node structure to recreate it.

So comparing the value that you've fed it to the actual value of innerHTML after the operation, you see quoted attribute values, altogether removed attributes when they have the default value, uppercased keywords, changed attribute order and so on and so on. FireFox only adds quotes as far as I see, but you never know what they'll do next.

On the bright side, now that my idea had been torn to shreds by the browser implementation, I now have an answer to all those stuck up web developers that consider innerHTML unclean or unstructured and criticize the browsers for not being able to render as fast when using DOM methods. The innerHTML property is like a web service. You feed it your changes in (almost) XML format and it applies it on the browser. Since you pretty much do the same when you use any form of web request, including Ajax, you cannot complain.

Thursday, April 17, 2008

Angelina in all her machine glory

This is a little YouTube video that shows a photoshoot of Angelina Jolie's when she was a teen. As you will see, she is a lovely looking girl, but that is not the reason I am posting this in my blog. The reason is the feeling that I am having watching this: dread. Here is a 16 years old girl, faking all the emotions in all the pictures that are made of her, with an ice cool professionalism, chewing gum when the film is being changed in the camera. You can imagine what kind of pictures will result from this shoot, when all of this is probably filmed in a small humid room with people watching TV at its more stupidest. Can you imagine trying to concentrate with that noise around? If you ever thought Angelina got to where she is because of her looks only, think again. This little human robot has good programming.


Angelina Jolie - Bikini modeling from 1992 (Photoshoot)

Tuesday, April 15, 2008

AjaxControlToolKit zIndex issue.

Update: On June 20th 2009, Codeplex notified me that the patch I did for the ACT has been applied. I haven't tested it yet, though. Get the latest source (not latest stable version) and you should be fine.

The Ajax Control Toolkit has a PopupExtender module that is used throughout the library in whichever controls need to show above other controls. I wanted to use the Calendar Extender in my web site, but the calendar appeared underneath other controls. I checked it out and it had a zIndex of 1000, which should have been enough. I took me an hour to realise that in the toolkit code zIndex was a property of the div element, not of the div style!

A download of the latest version from Feb 29 shows the problem is still there. The fix? go to the PopupExtender folder in the source code, open the PopupBehaviour.js file, search for a line that looks like this:
element.zIndex = 1000;
and replace it with
element.style.zIndex = 1000;
. Now it works!

The issue is already in the AjaxControlToolKit issue tracker, but it was not addressed yet.

Monday, April 14, 2008

Hot Kiss - Juliette & the Licks

No, you are not mistaken, the voice for this band is Juliette Lewis herself, a pretty known actress that usually interprets beautifully screwed up characters. No wonder her music is similar :) But when I first heard of the band I didn't even bother to listen to them. Not another acting celebrity trying to sing! But when I found out the vocal on Prodigy's Spitfire and Hot Ride songs was Juliette Lewis, I changed my mind. And I am glad I did. Here is the weird and nicely sung Hot Kiss.



Licks:
MySpace page for Juliette & the Licks
Wikipedia entry
Juliette & the Licks fan site

Using the SqlConnection InfoMessage to return variable amounts of data

A little used thingie on the SqlConnection object called the InfoMessage event fires whenever there were info messages (duh!) from the last query or stored procedure execution. That means errors, of course, but it also means warnings and simple prints! You get where I am going with this?

Instead of changing stored procedures, datalayers and code whenever I need to get some new information from SQL, I can just add some nicely formatted PRINT commands and get all the information I need! Here is some code:

using System;
using System.Collections.Generic;
using System.Data.SqlClient;
using System.Windows.Forms;

namespace SqlPrintCommands
{
public partial class Form2 : Form
{
public Dictionary<string, string> values;

public Form2()
{
InitializeComponent();
values = new Dictionary<string, string>();
}

private void button1_Click(object sender, EventArgs e)
{
SqlConnection connection =
new SqlConnection("[connectionString]");
connection.Open();
connection.InfoMessage += sc_InfoMessage;
SqlCommand comm =
new SqlCommand("pr_Test", connection);
comm.ExecuteNonQuery();
connection.Close();
string s = "";
foreach (KeyValuePair<string, string> pair in values)
{
s += string.Format("{0} : {1}\r\n",
pair.Key, pair.Value);
}
label1.Text = s;
}

private void sc_InfoMessage(object sender,
SqlInfoMessageEventArgs e)
{
string commandPrefix = "Eval: ";
foreach (SqlError err in e.Errors)
{
if ((err.Message ?? "").StartsWith(commandPrefix))
{
string command =
err.Message.Substring(commandPrefix.Length);
string[] cmd = command.Trim().Split('=');
string commandArgument = cmd[0];
string commandValue = cmd[1];
values[commandArgument] = commandValue;
}
}
}
}
}


In this scenario I have a simple form with a button and a label. I execute a pr_Test stored procedure and then I parse the messages it returns. If the messages are of the format
Eval: Name=Value
I store the keys and values in a Dictionary. Not the nicest code, but it's for demo purposes.

So, you want to know the count of whatever operation you executed? Add
PRINT 'Eval: RowCount='+cast(@@rowcount as varchar)
in your stored procedure. Pretty cool huh?

Unfortunately I haven't been able to send messages asynchronously, even if the connection was async and the running was async and the messages were generated with
RAISERROR('message',1,1) WITH NOWAIT
. BTW, who is the idiot that spelled RAISERROR with only one E? What's a Rror and why would I raise it?

Grendel by John Gardner

Don't go all "Oh no, not another Beowulf remake!" on me. This is a book that was written in 1971 by John Gardner, presenting the story of Beowulf through the eyes of Grendel. But it is not really the same story, just uses it as a scaffold for the philosophical ideas that he wanted to expose.
Grendel book cover

Structured into 12 chapters - each for a year in Grendel's life, each for a description of a philosophical current, each for an astrological sign - the book is not an easy one to understand, albeit pretty short. The language is modern and the wording is clear, but the underlying ideas need time and brain power to process, so don't read it in short bursts when you feel bored. Give it what it needs.

In the book, Grendel is not an animal monster, a thing with no thinking, quite the opposite. He is intelligent, articulate, philosophical, all these qualities being given to him at birth, not as a merit to anyone. He is hopelessly depressed and malevolent. He sees life and existence as meaningless, all the Universe a hollow illusion, a thing set to hurt him, set him apart, mock him. It is really easy to identify with him and to feel his feelings, while in the same time despise what he does and why he does it. Grendel is the part of us which we hate and which hates itself.

Enough, though, the book has bad parts as well. The occasional poem lyrics are meaningless in this book. The ending is confused and confusing. I would have liked a clearer ending, that's for sure. And also, it is hard to understand the book without at least knowing the Beowulf story and researching a bit from the Wikipedia article to find out what are the philosophical references hidden in each chapter. But then again, it was never a simple book, and the research (even if I haven't found time to do it) is worth it.

There was an animation film made in Australia in 1981 and featuring Peter Ustinov called Grendel Grendel Grendel which was based on the book, although I haven't been able to get my hands on it. It was partly musical as well, as expected in such a period, ugh!

If you are interested in finding out more about the meanings in the book and discussing about it, here is a link: The Grendel Board.

Wednesday, April 09, 2008

The DataRow value setter is slow!

Update 19 February 2016:
I've done the test again, using another computer and .Net 4.6.1. The speed of filling the DataTableReplacement class given at the end of the article, plus copying the data into a DataTable object is 30% faster than using a DataTable directly with BeginLoadData/EndLoadData and 50% faster than using DataTable without the LoadData methods.

Now for the original post:

It was about time I wrote a smashing IT entry. Here is to the obnoxious DataTable object, something about I have written before of bugs and difficulty in handling. Until now I haven't really thought about what kind of performance issues I might face when using it. I mean, yeah, everybody says it is slow, but how slow can it be? Twice as slow? Computers are getting faster and faster, I might not need a personal research into this. I tried to make a DataTable replacement object once and it was not really compatible with anything that needed DataTables so I gave up. But in this article I will show you how a simple piece of code became 7 times faster when taking into account some DataTable issues.

But let's get to the smashing part :) I was using C# to transform the values in a column from a datatable into columns. Something like this:
NameColumnValue
GeorgeMoney100
GeorgeAge31
GeorgeChildren1
JackMoney150
JackAge26
JackChildren0
JaneMoney300
JaneAge33
JaneChildren2


and it must look like this:

NameMoneyAgeChildren
George100311
Jack150260
Jane300332


I have no idea how to do this in SQL, if you have any advice, please leave a comment.
Update: Here are some links about how to do it in SQL and SSIS:
Give the New PIVOT and UNPIVOT Commands in SQL Server 2005 a Whirl
Using PIVOT and UNPIVOT
Transposing rows and columns in SQL Server Integration Services

Using PIVOT, the SQL query would look like this:
SELECT * 
FROM #input
PIVOT (
MAX([Value])
FOR [Column]
IN ([Money],[Age],[Children])
) as pivotTable

Anyway, the solution I had was to create the necessary table in the code behind add a row for each Name and a column for each of the distinct value of Column, then cycle through the rows of the original table and just place the values in the new table. All the values are present and already ordered so I only need to do it using row and column indexes that are easily computed.

The whole operation lasted 36 seconds. There were many rows and columns, you see. Anyway, I profiled the code, using the great JetBrains dotTrace program, and I noticed that 30 seconds from 36 were used by DataRow.set_Item(int, object)! I remembered then that the DataTable object has two BeginLoadData and EndLoadData methods that disable/enable the checks and constraints in the table. I did that and the operation went from 36 to 27 seconds.

Quite an improvement, but the bottleneck was still in the set_Item setter. So, I thought, what will happen if I don't use a DataTable at all. After all, the end result was being bound to a GridView and it, luckily, knows about object collections. But I was too lazy for that, as there was quite a complicated binding code mess waiting for refactoring. So I just used a List of object arrays instead of the datatable, then I used DataTable.Rows.Add(object[]) from this intermediary list to the DataTable that I originally wanted to obtain. The time spent on the operation went from... no, wait

The time spent on the operation went from the 27 seconds I had obtained to 5! 5 seconds! Instead of 225.351 calls to DataRow.set_Item, I had 1533 calls to DataRowCollection.Add, from 21 seconds to 175 miliseconds!

Researching the reflected source of System.Data.dll I noticed that the DataRow indexer with an integer index was going through
DataColumn column=_columns[index]; return this[column];
How bad can it get?! I mean, really! There are sites that recommend you find the integer index of table columns and then use them as integer variables. Apparently this is NOT the best practice. Best is to use the DataColumn directly!

So avoid the DataRow setter.

Update July 18, 2013:

Someone requested code, so here is a console application with some inline classes to replace the DataTable in GridView situations:

class Program
{
    static void Main(string[] args)
    {
        fillDataTable(false);
        fillDataTable(true);
        fillDataTableWriter();
        Console.ReadKey();
    }

    private static void fillDataTable(bool loadData)
    {
        var dt = new DataTable();
        dt.Columns.Add("cInt", typeof(int));
        dt.Columns.Add("cString", typeof(string));
        dt.Columns.Add("cBool", typeof(bool));
        dt.Columns.Add("cDateTime", typeof(DateTime));
        if (loadData) dt.BeginLoadData();
        for (var i = 0; i < 100000; i++)
        {
            dt.Rows.Add(dt.NewRow());
        }
        var now = DateTime.Now;
        for (var i = 0; i < 100000; i++)
        {
            dt.Rows[i]["cInt"] = 1;
            dt.Rows[i]["cString"] = "Some string";
            dt.Rows[i]["cBool"] = true;
            dt.Rows[i]["cDateTime"] = now;
        }
        if (loadData) dt.EndLoadData();
        Console.WriteLine("Filling DataTable"+(loadData?" with BeginLoadData/EndLoadData":"")+": "+(DateTime.Now - now).TotalMilliseconds);
    }

    private static void fillDataTableWriter()
    {
        var dt = new DataTableReplacement();
        dt.Columns.Add("cInt", typeof(int));
        dt.Columns.Add("cString", typeof(string));
        dt.Columns.Add("cBool", typeof(bool));
        dt.Columns.Add("cDateTime", typeof(DateTime));
        for (var i = 0; i < 100000; i++)
        {
            dt.Rows.Add(dt.NewRow());
        }
        var now = DateTime.Now;
        for (var i = 0; i < 100000; i++)
        {
            dt.Rows[i]["cInt"] = 1;
            dt.Rows[i]["cString"] = "Some string";
            dt.Rows[i]["cBool"] = true;
            dt.Rows[i]["cDateTime"] = now;
        }
        var fillingTime = (DateTime.Now - now).TotalMilliseconds;
        Console.WriteLine("Filling DataTableReplacement: "+fillingTime);
        now = DateTime.Now;
        var newDataTable = dt.ToDataTable();
        var translatingTime = (DateTime.Now - now).TotalMilliseconds;
        Console.WriteLine("Transforming DataTableReplacement to DataTable: " + translatingTime);
        Console.WriteLine("Total filling and transforming: " + (fillingTime+translatingTime));
    }
}

public class DataTableReplacement : IEnumerable<IEnumerable<object>>
{
    public DataTableReplacement()
    {
        _columns = new DtrColumnCollection();
        _rows = new DtrRowCollection();
    }

    private readonly DtrColumnCollection _columns;
    private readonly DtrRowCollection _rows;

    public DtrColumnCollection Columns
    {
        get { return _columns; }
    }

    public DtrRowCollection Rows { get { return _rows; } }

    public DtrRow NewRow()
    {
        return new DtrRow(this);
    }

    public DataTable ToDataTable()
    {
        var dt = new DataTable();
        dt.BeginLoadData();
        _columns.CreateColumns(dt);
        _rows.CreateRows(dt);
        dt.EndLoadData();
        return dt;
    }

    #region Implementation of IEnumerable

    public IEnumerator<IEnumerable<object>> GetEnumerator()
    {
        foreach (var row in _rows)
        {
            yield return row.ToArray();
        }
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    #endregion
}

public class DtrRowCollection : IEnumerable<DtrRow>
{
    private readonly List<DtrRow> _rows;

    public DtrRowCollection()
    {
        _rows = new List<DtrRow>();
    }

    public void Add(DtrRow newRow)
    {
        _rows.Add(newRow);
    }

    public DtrRow this[int i]
    {
        get { return _rows[i]; }
    }

    public void CreateRows(DataTable dt)
    {
        foreach (var dtrRow in _rows)
        {
            dt.Rows.Add(dtrRow.ToArray());
        }
    }

    #region Implementation of IEnumerable

    public IEnumerator<DtrRow> GetEnumerator()
    {
        return _rows.GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    #endregion
}

public class DtrRow
{
    private readonly object[] _arr;
    private readonly DataTableReplacement _dtr;

    public DtrRow(DataTableReplacement dtr)
    {
        _dtr = dtr;
        var columnCount = _dtr.Columns.Count;
        _arr = new object[columnCount];
    }

    public object this[string columnName]
    {
        get
        {
            var index = _dtr.Columns.GetIndex(columnName);
            return _arr[index];
        }
        set
        {
            var index = _dtr.Columns.GetIndex(columnName);
            _arr[index] = value;
        }
    }

    public object this[int columnIndex]
    {
        get
        {
            return _arr[columnIndex];
        }
        set
        {
            _arr[columnIndex] = value;
        }
    }

    public object[] ToArray()
    {
        return _arr;
    }
}

public class DtrColumnCollection
{
    private readonly Dictionary<string, int> _columnIndexes;
    private readonly Dictionary<string, Type> _columnTypes;

    public DtrColumnCollection()
    {
        _columnIndexes = new Dictionary<string, int>();
        _columnTypes = new Dictionary<string, Type>();
    }

    public int Count { get { return _columnIndexes.Count; } }

    public void Add(string columnName, Type columnType)
    {
        var index = _columnIndexes.Count;
        _columnIndexes.Add(columnName, index);
        _columnTypes.Add(columnName, columnType);
    }

    public int GetIndex(string columnName)
    {
        return _columnIndexes[columnName];
    }

    public void CreateColumns(DataTable dt)
    {
        foreach (var pair in _columnTypes)
        {
            dt.Columns.Add(pair.Key, pair.Value);
        }
    }
}

As you can see, there is a DataTableReplacement class which uses three other classes instead of DataColumnCollection, DataRowCollection and DataRow. For this example alone, the DtrRowCollection could have been easily replaced with a List<DtrRow>, but I wanted to allow people to replace DataTable wherever they had written code without any change to the use code.

In the example above, on my computer, it takes 1300 milliseconds to populate the DataTable the old fashioned way, 1000 to populate it with BeginLoadData/EndLoadData, 110 seconds to populate the DataTableReplacement. It takes another 920 seconds to create a new DataTable with the same data (just in case you really need a DataTable), which brings the total time to 1030. So this is the overhead the DataTable brings for simple scenarios such as these.

Tuesday, April 08, 2008

The Winamp Playlist Generator

Winamp Plugins
A while ago there was this site called Pandora (similar to lastFM, but better) that tried to match songs based on their internal structure not user preference. By choosing which songs you liked or you didn't like it would guess your preferences and try to play only songs you would listen to.

Apparently Winamp has a little known (or blogged) addin that does this. It is called the Nullsoft Playlist Generator and comes bundled with WinAmp. This is how you use it:
  1. Open Winamp and go to Media Library
  2. Create a playlist (or more) and add all your songs there
  3. Right click on the playlist and select Send To: Add to Local Media
  4. Go to Options, Preferences, Plug-ins, Media Library and click on Nullsoft Playlist Generator
  5. Click on Configure selected plug-in, select your options and click Scan. I recommend the background scanning option.
  6. After the scan is complete (or during it) you can right click on any song and select "Play similar song to..." and you will listen to songs that this software thinks are similar

Playlist Generator

That's it. The analysis is pretty superficial, but still better than nothing. It is perfect when you have gigs of songs and you don't want to browse forever, selecting which one you want or you don't want.

Wake Up Call - Prodigy

I am obsessed by this song, I am just listening to it again and again, typing at hyperspeed while I am doing it (I wish I would type meaningful things, too :) ). The video itself is from Smack My Bitch Up, but YouTube again blocked a cool clip so I had to take another from some other place. I couldn't find the original video for the song. (One that wasn't a fan made anime clip :-|)



Usage: Pump up the volume and get in front of a keyboard on monday morning. :)

Tuesday, April 01, 2008

How Would You Move Mount Fuji - William Poundstone

Finally, another book finished. It is not that I am illiterate, really, but lately I have been listening to podcasts every single free minute. I even fall asleep like that sometimes.

How Would You Move Mount Fuji book cover Anyway, this book is rather well written and gives some interesting insights into the Microsoft "puzzle" hiring interviews, revealing as well the highly competitive culture behind the system, as well as the most obvious flaws. One of the most famous such puzzles is in the very title of the book. The interviewer would ask one how would they move mount Fuji. The expected answer is a detailed analysis of the process and the more situations and data the candidate thinks of, the better. Of course, there are really stupid puzzles as well, used only to assess how the prospective employee reacts under stress. Others are deceptively simple, but hard to guess.

I will give you one. Please think about it as long as it takes to be CERTAIN the response is right. In fact, I won't even give you the answer. Here it goes:

You have four cards on the table. Each card has a digit on a side and a capital letter on the other. As placed on the table you see the cards like this:
4 G E 3


The request of the puzzle is this: what cards must be turned in order to verify that the four cards on the table verify the rule "Every card with a vowel has an even number on the other side". You need to give the exact cards not the number of cards and (of course) turn only the ones that you need, so as little card turning as possible.