Sunday, November 30, 2008

New Blog Format!

Hi, I am working on a new blog format. As I am lazy and a complete html and CSS noob, it will take a while. Please, feel free to comment on the new look. Actually, feel obligated to do so! :)

Windows Azure

I went to this presentation of a new Microsoft concept called Windows Azure. Well, it is not so much as a new concept, more like them entering the distributed computing competition. Like Google, IBM and - most notably - Amazon before it, Microsoft is using large computer centers to provide storage and computing as services. So, instead of having to worry about buying a zillion computers for your web farm, manage the failed components, the backups, etc. you just rent the storage and computing and create an application using the Windows Azure SDK.

As explained to me, it makes sense to use a large quantity of computers, especially structured for this cloud task, having centralized cooling, automated update, backup and recovery management, etc, rather than buying your own components. More than that. since the computers run the tasks of all customers, there is a more efficient use of CPU time and storage/memory use.

You may pay some extra money for the service, but it will closely follow the curve of your needs, rather than the ragged staircase that is usually a self managed system. You see, you would have to buy all the hardware resources for the maximum amount of use you expect from your application. Instead, with Azure, you just rent what you need and, more importantly, you can unrent when the usage goes down a bit. What you need to understand is that Azure is not a hosting service, nor a web farm proxy. It is what they call cloud computing, the separation of software from hardware.

Ok, ok, what does one install and how does one code against Azure? There are some SDKs. Included is a mock host for one machine and all the tools needed to build an application that can use the Azure model.

What is the Azure model? You have your resources split into storage and computing workers. You cannot access the storage directly and you have no visual interface for the computing workers. All the interaction between them or with you is done via REST requests, http in other words. You don't connect to SQL Servers, you use SQL Services, and so on and so on.

The graphical representation of the Azure vision

Moving a normal application to Azure may prove difficult, but I guess they will work something out. As with any new technology, people will find novell problems and the solutions for them.

I am myself unsure of what is the minimum size of an application where it becomes more effective to use Azure rather than your own servers, but I have to say that I like at least the model of software development. One can think of the SDK model (again, using only REST requests to communicate with the Azure host) as applicable to any service that would implement the Azure protocols. I can imagine building an application that would take a number of computers in one's home, office, datacenter, etc and transforming them into an Azure host clone. It wouldn't be the same, but assuming that many of your applications are working on Azure or something similar, maybe even a bit of the operating system, why not, then one can finally use all those computers gathering dust while sitting with 5% CPU usage or not even turned on to do real work. I certainly look forward to that! Or, that would be really cool, a peer to peer cloud computing network, free to use by anyone entering the cloud.

Saturday, November 22, 2008

Dexter in the Dark - Jeff Lindsay

book cover Well, I just said I can't wait for the third book, haven't I? :) Anyway, Dexter in the Dark was a bit of a disapointment to me. Apparently, Dexter's inner demons are just that, demons, liked to some ancient deity from the times of Solomon called Moloch which is like an alien parasite thing. Really... What did Lindsay do? Read Snow Crash? Watch Fallen? Try to mix Stargate Goa'ulds with Wicker Man and Eyes Wide Shut? Geez!

When I was getting so comfortable with the character of Dexter, thinking that Jeff Lindsay was a genius for portraying a type of character I was always thinking of writing, he just takes all that inner maniacal urge that both empowered and limited the character and transforms it into an external, fantasy like thing. Bad writer, bad!

Anyway, that doesn't mean I didn't enjoy the book. I just think that when the third series of the TV show became too far fetched, they were still safe when compared to it. I mean, until now Dexter was a brilliant guy with a dark path and also with a sort of artificial morality, mix in some police stuff, some blood spatter, the weird police seargent sister. It was a perfect setting for introspection and solitary struggle. I loved that! And now demons? As Doakes would have put it "the hell for?".

The fourth Dexter book is supposedly due for february 5th 2009. I hope Lindsay abandons the weird supernatural crap and instead focuses on Dexter's training of his adoptive children into the art of killing. Otherwise I can only see it turn toward so many bad directions like Blade or Hellboy or other green "hybrid saves the planet" thing.

Dearly Devoted Dexter - Jeff Lindsay

book coverDearly Devoted Dexter is much darker than the first Dexter book. Maybe it is just because all the facts about Dexter are clear and it starts with a gruesome murder, insane special forces style. The title comes from the fact that he helps his sister, now partially in the loop about his Dark Passenger, to solve the newest serial killer case. Of course, Deb, now a seargent after Laguerta has died, has a personal stake in this, since one of the people the murderer abducted and intends to do bad things to, is her boyfriend, with whom she is very much in love.

It is interesting to develop the Dexter character in this way, especially since he is described as totally indifferent to the horrible fate of people he doesn't care about, yet he is still compelled to help his sister out.

I was a bit disapointed by the police work involved. If I were to believe Lindsay, the Miami police are a bunch of morons, following (badly) a set of procedures without any real talent other than badmouthing.

Elements from this second book in the Dexter series were clearly used in the series, but it is already a completely different story. The FBI agent that Deborah briefly dates in the series was inspired by the character of Kyle, shady government agent that she falls in love with in this book. The stalking of Dexter by the grumpy seargent Doakes is also mirrored from this book, although the motives and the outcome are completely different.

Again, the series evolves the Dexter character more and the story is more complex than the book, but by now it is obvious the TV show and the book are going in completely different directions.

All in all, a bit better than the first, darker, but also funnier. I have never laughed as much reading a book for a long time. Can't wait for the third book now.

Friday, November 21, 2008

StringBuilder substring?

Well, it may have been obvious for many, but I had no idea. Whenever I was writing a piece of code that uses a StringBuilder I was terribly upset by the lack of a substring method.

It was there all along, only it is the ToString method. You give it the startIndex and length and you're set.

Looking at the source, I see that in .Net 1.1 this method is only a normal Substring on the internal string used by the string builder. In Net 2.0, the method uses a InternalSubStringWithChecks internal method of the string class, which is using a InternalSubString unsafe method that seems to be more basic and thus faster.

Darkly Dreaming Dexter - Jeff Lindsay

Book cover, obviously printed after the TV series startedI have been watching this TV Series called Dexter and slowly but surely I fell in love with it. It features a psychopathic serial killer that has a hobby of killing other killers. The story is long and I suggest you watch it to get it fully. Anyway, the series has reached season 3 and stars Michael C. Hall, which you may recognize from the Six Feet Under TV series. I've also noticed that the series is based on a book! So, naturally, I got the book and started reading it. It's Michael C. Hall on the cover there.

Darkly Dreaming Dexter is the first in a series of Dexter books by Jeff Lindsay. While it starts pretty much the same as the series, the series quickly moves away from the script in the book. However, the spirit is there, even if, of course, they had to make the lead character a little more likable in the series and the whole thing less bloody.

Imagine an emotionless killer, raised by his cop father to kill according to a code and also to be thorough and attentive to the details so that the police wouldn't catch him. He is also working for the Miami police department as a blood spatter analyst. The inner dialogues are really delicious, the way he sees the world as a cynical dark Data is both funny and deep. Lindsay manages to portray an alien being, silently watching the world we take for granted, hunting on the edge of our own morality.

And while I do enjoy the book, I have to say that the series is more complex and the story a bit more realistic. So, there, finally a movie or series that surpasses the book!

Wednesday, November 19, 2008

Cascading Dropdowns and 'Invalid postback or callback argument' error

Actually, I think this applies to any dynamic modification of drop down list options. Read on!

I have used a CascadingDropDown extender from the AjaxControlToolkit to select regions and provinces based on a web service. It was supposed to be painless and quick. And it was. Until another dev showed me a page giving the horribly obtuse 'Invalid postback or callback argument. Event validation is enabled using <pages enableEventValidation="true"/> in configuration or <%@ Page EnableEventValidation="true" %> in a page. For security purposes, this feature verifies that arguments to postback or callback events originate from the server control that originally rendered them. If the data is valid and expected, use the ClientScriptManager.RegisterForEventValidation method in order to register the postback or callback data for validation.'. As you can see, this little error message basically says there is a problem with a control, but doesn't care to disclose which. There are no events or overridable methods to enable some sort of debug.

Luckily, Visual Studio 2008 has source debug inside the .Net framework itself. Thus I could see that the error is caused by the drop down lists I mentioned above. Google told me that somewhere in the documentation of the CascadingDropDown extender there is a mention on setting enableEventValidation to false. I couldn't find the reference, but of course, I didn't look too hard, because that is simply stupid. Why disable event validation for the entire page because of a control? It seems reasonable that Microsoft left it enabled for a reason. (Not that I accuse them of being reasonable, mind you).

Analysing further, I realised that the error kind of made sense. You see, the dropdownlists were not binded with data that came from a postback. How can one POST a value from a select html element if the select did not have it as an option? It must be a hack. Well, of course it was a hack, since the cascade extender filled the dropdown list with values.

I have tried to find a way to override something, make only those two dropdownlists not have event validation enabled. Couldn't find any way to do that. Instead, I've decided to register all possible values with Page.ClientScript.RegisterForEventValidation. And it worked. What I don't understand is why did this error occur only now, and not in the first two pages I have built and tested. That is still to be determined.

Here is the code

foreach (var region in regions)
new PostBackOptions(ddlRegions,region)

It should be used in a Render override, since the RegisterForEventValidation method only allows its use in the Render stage of the page cycle.

And that is it. Is it ugly to load all possible values in order to validate the input? Yes. But how else could you validate the input? A little more work and a hidden bug that appears when you least expect it, but now even the input from those drop downs is more secure.

My control was used in two pages with EnableEventValidation="false" and that's why it didn't throw any error. Anyway, I don't recommend setting it to false. Use the code above. BUT, if you don't know where the code goes or you don't understand what it does, better use this solution and save us both a lot of grief.

Tuesday, November 18, 2008

Thoughts on Prize Extraction Games

We are working on these projects in which people either receive an email with an invitation or they register on a web site or they simply play and expect an instant result. What is the best way to compute the chance of winning?

Case 1: People register then, at a certain date, an extraction of prizes takes place. Ah, this is the best and simple situation. You have the list of players, the list of prizes. Just do an index = Random(0,number_of_players) and if index is smaller than number_of_prizes, you know the person won a prize. A second random on the number of prizes will determine the prize itself. The win probability is number_of_prizes/number_of_players.

Case 2: People enter a site that allows them to play instantly and win a prize. Here the problem is trickier. While the algorithm is basically the same, prizes over people, you don't have the total number of players. There are some subcases here based on the following parameters:
  • The win campaign lasts for a certain amount of time or is indefinite
  • The prizes must all be given at the end of the campaign or not
  • The players play after they have received an invitation (email, sms, etc) or just randomly coming from ad clicks or for some information on the site

Let's assume that the campaign doesn't last for a finite time. The only solution is to pick a win probability and be done with it until you remain out of prizes. You always compute this by considering the number of people playing over a period of time, in other words the speed of people playing. However, in this case the only thing influencing the selected probability is a psychological one: how many people would need to win in order to have a marketing effect?

Now, if the campaign does have a finite time, you would use the speed of the people playing to determine the total number of people that would play. Let's assume you know people are visiting your site at an average rate of 1000 per hour, then you see how many are playing and you remember this percentage, so you can estimate the number of players per hour, then you just multiply that number to the total number of hours in the campaign. Again, we get to the prizes over people formula.

However, it is very important to know how people are deciding the participate in the extraction!

If it is just a game added to one's site, then the people are coming and going based on that site's popularity and hourly/daily distribution (since the number of visitors fluctuates). So just computing this from the first hour of people coming to the site and playing doesn't help, but it might when using the first day and maybe the first week. One week of statistical data is best to estimate the number of people over time. Then the formula is number_of_prizes_available_per_week/people_visiting_per_week. Where the number of prizes available per week is either the total number of prizes over the finite campaign time or an arbitrary number chosen by the campaign creator.

If, instead, people are being invited to play, as following an email promotion campaign, let's say, then they will come as soon as they read their email. That means they will flock to your site in the first hours, then just trickle in the next week, then nothing. That means that estimating the total number of players from the first hour or day is not really feasible unless you are certain of a statistical distribution of people playing games after email campaigns. It is difficult as different messages and designs and game types might attract more or less people.

A mixed hybrid can also exist, with a game on a site that also people are invited to play over email. Then all the parameters from above must be used. In any case, the best estimation I can think of comes from the total of players in similar campaigns. The more similar the better.

But what if ALL the prizes must be given to people, as required by law or simple common sense (so as not to be seen as keeping some for you or your friends)? Then one can adjust the probability rate to suit the extraction speed. The same prizes over people formula is used, but only on the remaning values. The probability of winning is given by number_of_remaining_prizes/number_of_remaining_people.

But that has some disadvantages. If the number of total participating people is badly estimated it will result into a roller coaster of probabilities. People playing in the first part of the campaign would be either advantaged or disadvantaged than the people in the last part as the total number of players is being adjusted over time to compensate for the first bad estimation.

Let's do a small example:

Day 1Day 2Day 3Day 4Day 5Day 6Day 7
People playing75001000400300200300100
Estimated total players15000250001200011000100001000010000
Estimated remaining players75001650031001800600200100
Remaining prizes (day start)100504742362713
Win probability0.66%0.30%1.52%2.33%6.00%13.50%13.00%

As you can see, the people playing first were screwed pretty much, because it was expected the total players to be 15000 and the distribution closer to linear. After half of them played in the first day, panic made them all increase the expected players to 25000, while thinking what to do. Then they realised that the distribution of players is affected by the fact that all play after reading their emails and then they will probably not come play anymore. They adjust the win probability every day and as you can see, it is good to play in the last days.

But what would have happened if 1) they knew the percentual distribution of players would be 75,10,4,3,2,3,1 after an email campaign and 2) the total number of players will be a percentage out of all emails sent and so they estimated 10000 people playing and the right distribution?

Day 1Day 2Day 3Day 4Day 5Day 6Day 7
People playing75001000400300200300100
Estimated total players10000100001000010000100001000010000
Estimated remaining players250015001100800600300100
Remaining prizes (day start)100251511863
Win probability1.00%1.00%1.00%1.00%1.00%1.00%1.00%

Even if computing every day the number of remaining prizes over the remaining players, the probability was constantly 1%. Of course, one could say "Why didn't they stick to their 0.66% probability and be done with it? Like this:

Day 1Day 2Day 3Day 4Day 5Day 6Day 7
People playing75001000400300200300100
Estimated total playersNot important
Estimated remaining playersNot important
Remaining prizes (day start)100434038373534
Win probability0.66%0.66%0.66%0.66%0.66%0.66%0.66%

Everything is perfectly honest, only that they remained with a third of prices on hand. Now they have to give them to charity and be suspected of doing this on purpose for whatever distant relative that works at that charity.

Well, think about it, and let me know what you think. Are there smarter solutions? Is there a web repository of statistical data for things like that?

Monday, November 17, 2008

Claymore - a pretty cool manga/anime

The story in Claymore was pretty standard: monsters attack people, people are powerless, therefore an organization of hybrids (female warriors carrying deadly claymore swords) emerges to protect people from said monsters. So it's like Blade, in theory. But in reality it has the feel of Berserk (the first cool part, not the crappy lingering mess that it is now). Or you can imagine Naruto, with the monster and everything, fighting against a species of demon foxes. Only without the sillyness and all the mentoring.

I really liked the manga, I can barely wait for it to continue, unfortunately it is distributed like one chapter per month. The 26 episode anime series follows closely the manga story, but unfortunately ends prematurely with a different idea in the last two episodes. Not that it is not a lot better than Berserk leaving us in the dark at the end of the anime or other series that just ended in mid air.

Bottom line, if you liked Berserk, you will like this. If you like Naruto/Bleach, you will like this. I can even throw a little Akira in, to convince you, but it would probably be a stretch :)

Tuesday, November 11, 2008

How to return anonymous types from .Net 3.5 methods or how to cast objects to anonymous types

I have found this great link in Tomáš Petříček's blog which gives a very simple and elegant solution to casting to an anonymous type.

Short story shorter:

// Cast method - thanks to type inference when calling methods it
// is possible to cast object to type without knowing the type name
T Cast<T>(object obj, T objOfTypeT)
return (T)obj;

So simple! You see that the objOfTypeT parameter is never used, but the type infered from it is!

Correct usage:Cast(obj,anonObj)
Incorrect usage:Cast(obj,anonObj.GetType())

The Temporal Void - Peter F. Hamilton

Wee! Another Peter F. Hamilton book has been published. This time it is the second part of the Void trilogy, an ongoing series set up in the Commonwealth saga universe, but much later. Many characters are rented from said saga, so it would be a good idea to read that one first. Besides, as is Hamilton's style, the second book starts abruptly from the end of the first one and ends abruptly awaiting the third part.

Book cover

And, again, like in the Night's Dawn trilogy, the plot is a combination of stories, one set in the technological future of mankind and one in a feudal, fantasy like, universe. Hamilton's talent is to combine these two in a believable common narative. They are not so linked as in Night's Dawn and, I have to admit, I like the fantasy arch better, even if it is the classic Messiah myth. Maybe because it is not contiguous, but rather made up of small stories that have a beginning and an end.

Well, either way, it was a great book and I am waiting for the third part, due to be released in far away late 2009 or even 2010 :(

Monday, November 10, 2008

Oil is not dead yet!

First of all I want to say that I know I haven't been writing many tech articles lately and I've disappointed quite a few of the people reading this blog. I intend to rectify that, even if I am suferring from one of those dry tech spells at the moment :)

Anyway, about the oil. What if one could replicate the process that creates oil naturally, speed it up, and use it to not only for creating oil, but also for getting rid of a lot of organic waste? The technology is called Thermal Depolymerization and is pattented by the Changing World Technologies company. So, if one is to believe the Wikipedia article, while the test factory and the company itself have had problems ranging from technological hickups, to having to pay for the waste they use as fuel up to neverending complains from neighbours about the smell of bio waste, the technique was shown to work!

So, while the process does allow the production of slightly cheaper oil than the one extracted, it will certainly gain a big boost from the increase of prices in underground oil.

Here is a link to a 2003 interview with the CEO of Changing World Technologies, but utterly demolished by Paul Palmer, a chemistry PhD here. Also, this process is nothing very new or unique! there are other methods that are said to transform organic waste to Ethanol, as described in this link. So, oil may not dead yet.