.NET (295) administrative (41) Ajax (42) AngularJS (2) ASP.NET (144) bicycle (2) books (180) browser (8) C# (133) cars (1) chess (28) CodePlex (10) Coma (8) database (47) deployment (3) Entity Framework (2) essay (110) flash/shockwave (2) flex (1) food (3) friend (2) game (20) idea (5) IIS (8) javascript (82) LInQ (2) Linux (6) management (4) manga (42) misc (669) mobile (1) movies (90) MsAccess (1) murder (2) music (64) mysql (1) news (99) permanent (1) personal (68) PHP (1) physics (2) picture (307) places (12) politics (13) programming (501) rant (120) religion (3) science (43) Sharepoint (3) software (58) space (1) T4 (2) technology (11) Test Driven Development (4) translation (2) VB (2) video (97) Visual Studio (44) web design (45) Windows API (8) Windows Forms (3) Windows Server (4) WPF/Silverlight (63) XML (11)

Monday, August 18, 2014

I was mentioned on TMRO (formerly known as Spacevidcast)

I have been watching this weekly space show made by a husband and wife couple working for SpaceX. Initially called Spacevidcast, now it is called TMRO (pronounced Tomorrow). It is a great show, great quality, nice humor and, more than anything, a comprehensive video report on weekly events in space exploration, commercial or otherwise. If you are even remotely interested in space, you should subscribe. And they have been doing it all from their own resources and crowdfunding for seven years! You gotta love that.

But the selfish reason I am blogging about them is that I got mentioned in the TMRO show! Click here to see how they are trying and even succeeding to pronounce my Internet nom de guerre. The effort is appreciated.


Thursday, August 14, 2014

Google Hangouts hides its icon in the traybar

Because of idiotic firewall rules at my workplace I am forced to use Hangouts rather than Yahoo Messenger as an instant messenger. I am not going to rant here about which one is best, enough to say that most of my friends are on YM and being on Hangouts doesn't help. Hangouts has many annoyances for me, like its propensity to freeze when you lose Internet connection often or the lack of features that YM had. In fact I was so annoyed that I planned to do my own professional messenger to rule them all. But that's another story.

I am writing this post because of a behaviour of the Google Hangouts instant messenger (which, to be fair, is only a Chrome extension), mainly that after a while, the green traybar icon of the messenger goes in the "hidden icons" group. I have to customize it every day, sometimes twice, as it seems to reset this behavior after a period of use, not just on restarts. There is a Google product forum that discusses this here: System tray icon resets every time Chrome is started where you also see a few comments from your truly.

I immediately wanted to create a script or a C# program to fix this, but at first I just searched for a solution on the web and I found TrayManager, a C# app that does what the "Customize..." tray link does and more. One of the best features is a command line! So here is what you do after downloading the software and installing it somewhere: TrayManager.exe -t "Hangouts" 2. Now, probably that doesn't solve the problem long term. It is just as you would go into the Customize... link, but it's faster. Also, it has no side effects if run multiple times, so you can script it to run periodically. I will post a script to do that, if there is interest. Yatta!

Tuesday, August 12, 2014

How to Catch a Comet - a BBC Four documentary about the Rosetta comet

Comet 67P picture and trajectory BBC's show The Sky at Night did a coverage of the Rosetta mission, called How to Catch a Comet. It is the standard popular science show, with a lot of fake enthusiasm from the reporters and simple language and explanations, but for people who read this blog entry and wonder what the hell Rosetta is, it does the job. The fat black reporter is really annoying, and not because she's black, but because she feels completely fake whenever she says anything. Other than that the show is decent.

You get to learn about comet 67P, the Rosetta probe features and mission, walk around ESA, talk to scientists and even see a how-to about photographing comets - it was funny to see a shooting star in the night sky while the guy was preparing his camera and talking in the video. Of course, for me the show stopped just when it was getting interesting. I know you can't do much in 29 minutes, but still. I hope they do follow-up shows on Rosetta and I can't wait for November when the lander module will try to grapple the comet and land.

Just in case I've stirred your interest, here are some links that can cover the subject in a lot more detail:
ESA Euronews: Comet Hunters: Rosetta's race to map 67P - 8 minutes and a half of Euronews report from 11 August.
ESAHangout: How do we journey to a comet? - Google Hangout from ESA explaining the mission. It's one hour long and it dates from the 26th of June. Many other videos about Rosetta can be found on the ESA channel.
A playlist about Rosetta from Mars Underground. The most interesting is this video, published on 11 Aug 2014. It lasts an hour and a half and shows the first mission images and science results.
Comets - A wonder to Behold, A continuing Stream of Surprises - The Beauty and the Danger, not about Rosetta, but one hour and a half about comets. The documentary is trying to justify a controversial theory about the electric nature of comets. It is well done with a lot of proof, but I know too little about the theory so I can't recommend it. Interesting, though.

Monday, August 11, 2014

Ammonia from water and air!

In this post I will try to bring to your attention something that will probably change the world significantly. In 1909, German chemist Fritz Haber successfully fixed atmospheric nitrogen as ammonia in a laboratory and five years later a research team from BASF, led by Carl Bosch, developed the first industrial-scale application of the Haber process, sometimes called the Haber-Bosch process. Ammonia is extremely useful for many applications, the least of each is gunpowder and explosives and one of the most important is fertilizers. Without the Haber-Bosch process we probably wouldn't have the Green Revolution.

So today I found this article in Ars Technica that says that Researchers have developed a method to produce ammonia starting only with air and water. Not only is it more energy efficient than the century-old Haber-Bosch process that’s currently in use, but it’s also greener. The article goes on to say that almost 2% of the entire world energy is used to create ammonia; making the process more efficient is great! But I have to say that this is probably just the tip of the iceberg. Lowering the production cost of such a basic article will ripple throughout many industries, lead to innovation or the possibility to use some old innovation that until now was unfeasible.

I am not a chemist, so my enthusiasm may be way off-base, but my gut feeling is that this improvement on a century old process will have a great and positive effect.

Friday, August 08, 2014

Sending complex objects to T-SQL using the XML type

I have been working on a REST API lately and, while using Entity Framework or some other similar framework to abstract the database is certainly possible, I wanted to control every aspect of the implementation. I know, reinventing wheels, but this is how one learns. One of the most annoying bits was trying to translate some complex object from JSON to the (two dimensional) database relational tables. This post will explore my attempts and the solutions I have found.

My first attempt was straightforward: just send all types as DataTables, with some extra property to define identity and parent entity. This relies on the Microsoft Server SQL mechanism that allows sending of table variables to stored procedures. But this approach has several downsides. One of them is that in order to send a datatable to SQL you need... DataTables. As I have pointed out in several blog posts, the DataTable object is slow, and sometimes downright buggy. Even if I didn't care about performance that much, in order for SQL to receive the content of the DataTable one must create corresponding User Defined Types on the database side. Working with UDTs is very difficult for several reasons: you cannot alter a UDT (unless employing some SQL voodoo that changes system tables), you can only drop it and recreate it. This does not work if you use the UDT anywhere, so a lot of renaming needs to be done. Even if you automate the process, it's still very annoying. Then the UDT definition has to be an exact duplicate of the DataTable definition. Move some columns around and it fails. Debugging is also made difficult by the fact that the SQL profiler does not see the content of table variables when sending them to the server.

Long story short, I was looking for alternatives and I found XML. Now, you might think that this leads to a simple, and maybe even obvious, solution. But it's not that easy. Imagine that you send a list of objects in an XML. Each object is represented by an XML element and each property by a child element. In order to get the value of a property you need to do iterate through all the nodes, for each node find the properties, for each property find the one element that defines it, then get the attribute value or content of the property, all while making sure you select everything in a table. It's not that easy.

The solution I found, which simplifies the SQL code (and hopefully brings some well needed performance to the table) is to serialize the objects in a way that makes the selection simple enough. Here is an example: I have a Configuration object with an Id and a Name that also has a property called Servers, containing Server objects having an Id and a Url. Here is an example of XML serialization from the DataContractSerializer:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="http://schemas.datacontract.org/2004/07/SerializationTest" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
   <Id>1</Id>
   <Name>Test Config</Name>
   <Servers>
      <Server>
         <Id>1</Id>
         <Url>http://some.url</Url>
      </Server>
      <Server>
         <Id>2</Id>
         <Url>http://some.other.url</Url>
      </Server>
   </Servers>
</Configuration>
The SQL code to get the information from an XML variable with this content would look like this:
DECLARE @Xml XML='<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="http://schemas.datacontract.org/2004/07/SerializationTest" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
   <Id>1</Id>
   <Name>Test Config</Name>
   <Servers>
      <Server>
         <Id>1</Id>
         <Url>http://some.url</Url>
      </Server>
      <Server>
         <Id>2</Id>
         <Url>http://some.other.url</Url>
      </Server>
   </Servers>
</Configuration>'

;WITH XMLNAMESPACES(DEFAULT 'http://schemas.datacontract.org/2004/07/SerializationTest')
SELECT T.Item.value('(Id/text())[1]','INT')  as Id,
       T.Item.value('(Name/text())[1]','NVARCHAR(100)')  as Name
FROM @Xml.nodes('//Configuration') as T(Item)

;WITH XMLNAMESPACES(DEFAULT 'http://schemas.datacontract.org/2004/07/SerializationTest')
SELECT T.Item.value('(Id/text())[1]','INT')  as Id,
       T.Item.value('(Url/text())[1]','NVARCHAR(100)')  as Url
FROM @Xml.nodes('//Configuration/Servers/Server') as T(Item)

This works, but look at that code. In my case, the situation was worse, the object I was using was a wrapper which implemented IDictionary<string,object> and, even if it did implement ISerializable, both XmlSerializer and DataContractSerializer use the dictionary as their data and in the end I get ugly key elements and value elements that are even harder to get to and, I suppose, more inefficient to parse. Therefore I found the solution in IXmlSerializable, (yet) another serialization interface used exclusively by XML serializer classes. If every simple value would be saved as an attribute and every complex object in an element, then this could be the SQL code:
DECLARE @Xml XML='<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns="http://schemas.datacontract.org/2004/07/SerializationTest" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
   <Config Id="1" Name="Test Config">
   <Servers>
      <List>
        <Server Id="1" Url="http://some.url" />
        <Server Id="2" Url="http://some.other.url" />
      </List>
   </Servers>
   </Config>
</Configuration>'

;WITH XMLNAMESPACES(DEFAULT 'http://schemas.datacontract.org/2004/07/SerializationTest')
SELECT T.Item.value('@Id','INT')  as Id,
       T.Item.value('@Name','NVARCHAR(100)')  as Name
FROM @Xml.nodes('//Configuration/Config') as T(Item)

;WITH XMLNAMESPACES(DEFAULT 'http://schemas.datacontract.org/2004/07/SerializationTest')
SELECT T.Item.value('@Id','INT')  as Id,
       T.Item.value('@Url','NVARCHAR(100)')  as Url
FROM @Xml.nodes('//Configuration/Config/Servers/List/Server') as T(Item)

Much easier to read and hopefully to parse.

I am not going to write here about the actual implementation of IXmlSerializable. There are plenty of tutorials on the Internet about that. It's not pretty, :) but not too difficult, either.

What was the purpose of this exercise? Now I can send a complex object to SQL in a single query, making inserts and updates simple and not requiring at a call for each instance of each type of complex object. Now, is it fast? I have no idea. Certainly if performance is needed, perhaps the UDT/DataTable approach is faster. However you will have to define a type for each type that you send as a DataTable to a stored procedure. An alternative can be a binary serializer and a CLR SQL function that translates it into tables. However, in my project I need to easily implement very custom API methods and to control every aspect, including tracing and profiling the various SQL calls. I believe the customized IXmlSerializable/XML in SQL approach is a reasonable one.

As always, I hope this helps someone.