Thursday, March 27, 2014

Incredible loss of performance when encapsulating a DOM element in jQuery (1.7.2)

I was working on a pretty nice task that involved translating the text in a page in real time. For this I created a one page function that would do magic on elements that were added or changed in the page. On specific pages it moved with abysmal speed and I had no idea why. So I went to profile the thing and I was shocked to see that the problem did not come from my long piece of code, but from a simple encapsulation of an element in a jQuery object. I was using it only to have a nicer interface for getting the name of the element and changing an attribute. Here is the code:
var j=jQuery(elem);
if (j.is('img[alt]')) {
   j.attr('alt',translate(j.attr('alt')));
}

Replaced it with:
if (/^img$/i.test(elem.tagName)) {
  var alt=elem.getAttribute('alt');
  if (alt) {
    elem.setAttribute('alt',translate(alt));
  }
}

And it worked very fast indeed. The element might have been body so maybe the encapsulation tries to also parse the children or something like that or perhaps the problem was fixed with later versions of the library. However, think about how many times we used this kind of code without thinking twice about it. Think twice about it! :)

Tuesday, March 25, 2014

Accessing AngularJS services from outside AngularJS

If you read an Angular book or read a howto, you will think that Angular is the greatest discovery since fire. Everything is neatly stacked in modules, controllers, templates, directives, factories, etc. The problem comes when you want to use some code of your own, using simple Javascript that does specific work, and then you want to link it nicely with AngularJS. It is not always easy. My example concerns the simple display of a dialog which edits an object. I want it to work on every page, so I added it to the general layout template. The layout does not have a controller. Even if I add it, the dialog engine I have been using was buggy and I've decided to just use jQuery.dialog.

So here is my conundrum: How to load the content of a dialog from an Angular template, display it with jQuery.dialog, load the information with jQuery.get, then bind its input elements to an Angular scope object. I've tried the obvious: just load the template in the dialog and expect Angular to notice a new DOM element was added and parse it and work its magic. It didn't work. Why can't I just call an angular.refresh(elem); function and get it over with, I thought. There are several other solutions. One is to not create the content dynamically at all, just add it to the layout, mark it with ng-controller="something" and then, in the controller, save the object you are interested in or the scope as some sort of globally accessible object that you instantiate from jQuery.get. The dialog would just move the element around, afterwards. That means you need to create a controller, maybe in another file, to be nice, then load it into your page. Another is to create some sort of directive or script tag that loads the Angular template dynamically and to hope it works.

Long story short, none of these solutions appealed to me. I wanted a simple refresh(elem) function. And there is one. It is called angular.injector. You call it with the names of the modules you need to load ('ng' one of them and usually the main application module the second). The result is a function that can use invoke to get the same results as a controller constructor. And that is saying something: if you can do the work that the controller does in your block of code, you don't need a zillion controllers making your life miserable, nor do you need to mark the HTML uselessly for very simple functionality.

Without further due, here is a function that takes as parameters an element and a data object. The function will force angular to compile said element like it was part of the angular main application, then bind to the main scope the properties of the data object:
function angularCompile(elem, data) {
    // create an injector
    var $injector = angular.injector(['ng','app']);
            
    // use the type inference to auto inject arguments, or use implicit injection
    $injector.invoke(function($rootScope, $compile, $document){
        var compiled = $compile(elem || $document);
        compiled($rootScope);
        if (data) {
            for (var k in data) {
                if (data.hasOwnProperty(k)) {
                    $rootScope[k]=data[k];
                }
            }
        }
           $rootScope.$digest();
    });
}

Example usage:
angularCompile(dialog[0],{editedObject: obj}); // will take the jQuery dialog element, compile it, and add to the scope the editedObject property with the value of obj.

Full code:
OpenTranslationDialog=function(Rule, onProceed, onCancel) {
  jQuery.ajax({
          type: 'GET',
          url: '/Content/ng-templates/dialogs/Translation.html',
          data: Rule,
          success: function(data) {
            var dialog=jQuery('<div></div>')
              .html(data)
              .dialog({
                resizable:true,
                width:700,
                modal:true,
                    buttons: {
                      "Save": function() {
                    var url='/some/api/url';
                    jQuery.ajax({
                        type:'PUT',
                        url:url,
                        data:Rule,
                        success:function() {
                          if (onProceed) onProceed();
                              $(this).dialog( "close" );
                        },
                        error:function() {
                          alert('There was an error saving the rule');
                        }
                      });
                      },
                      Cancel: function() {
                    if (onCancel) onCancel();
                          $(this).dialog( "close" );
                      }
                    }
              });

            angularCompile(dialog[0],{Rule:Rule});
          },
          error:function() {
              alert('There was an error getting the dialog template');
                  }
      });
}

Before you take my word on it, though, beware: I am an Angular noob and my desire here was to hack away at it in order to merge my own code with the nice structured code of my colleagues, who now hate me. Although they liked angular.injector when I showed it to them :)

Monday, March 17, 2014

Creating a truly unique value from several others in T-SQL

Update 2015 August 28: I've replaced the function master.sys.fn_varbintohexstr with CONVERT, with the extra parameter 2, which translates a binary field into a hexadecimal string with no leading 0x. In addition to being ugly to use, fn_varbintohexstr is very slow.

Sometimes you need to create a unique identifier for a bunch of values so that you use it as an ID in the database. The immediately obvious choice is the CHECKSUM and BINARYCHECKSUM functions. But beware, the purpose of these functions is to detect changes in a string, not to uniquely identify it. It might seem strange, but the two concepts are very different. The change modification functionality is only meant to generate very different values on small changes. The uniqueness is trying to create a value as distinctive as possible for any string. That is why when you use a checksum you will get a lot of similar values for (very) different strings.

Enter HASHBYTES, another function that has the purpose of creating a cryptographic hash for a string. It is mainly used for password hashing, but it will fit nicely for our purpose. There are some caveats, though. First, CHECKSUM gets a variable number of parameters, HASHBYTES only accepts one, so we must take care of the cumbersome concatenation of multiple values. Unfortunately SQL functions do not have the option of variable parameters, which is truly a shame, so we can't hack it. Also, the value that HASHBYTES returns is a varbinary. We could cast it to NVARCHAR, but it turns into a weird Chinese characters string. In order to turn it into a proper string, we need to use the same function used by SQL Server to display varbinary when selecting it: master.sys.fn_varbintohexstr the CONVERT function with a parameter of 2 (hex string without the leading 0x).

So let's compare the two usages. Suppose we have this nice table that contains company data: company name, contact first name, contact last name, phone, email, yearly value. We need to create a unique ID based on these values.
First CHECKSUM:
SELECT CHECKSUM(companyName, firstName, lastName, phone, email, yearlyValue) FROM OurTable
So easy! Just add the columns, no matter how many or what type they have, and get a value as a result. You can even use * to select all columns in a row. You also have the advantage of getting the same checksum for differently capitalized strings. If you don't want this behaviour, use BINARYCHECSUM, which works even better.

Second HASHBYTES:
SELECT CONVERT(VARCHAR(Max),HASHBYTES('SHA1',companyName+'|'+firstName+'|'+lastName+'|'+phone+'|'+email+'|'+CAST(yearlyValue as NVARCHAR(100))),2) as id,*
FROM OurTable
Ugly! You need to create a string from different types, using ugly casts. Also, this works more like BINARYCHECKSUM. If you want to get the same functionality as CHECKSUM you need to use LOWER(LTRIM(RTRIM(value))). Horrid!
However, it works.

WARNING: using CAST to NVARCHAR from a FLOAT loses precision. You should use STR instead!

A middle solution is to use XCHECKSUM. What is that, you ask? A placeholder that can be replaced with some regular expression search and replace, of course :)

Update: I've created a query that creates the script to update the value of a column called 'ValuesHash', for tables that have it, with the hash of all columns that are not in a list of names, they are not primary keys and they are not foreign keys, plus they are not computed, rowguidcol or filestream.
Imagine the scenario where you have something like this:
  • Table A:
    1. Id: primary identity key
    2. Data1: some data
    3. Data2: some data
    4. CreateTime: the creation time
    5. ValuesHash: a VARBINARY(50) column - only 20 are required normally, but let's make sure :)
  • Table B:
    1. Id: primary identity key
    2. AId: foreign key to A
    3. Data1: some data
    4. Data2: some data
    5. ModifyTime: the modification time
    6. ValuesHash: a VARBINARY(50) column - only 20 are required normally, but let's make sure :)
  • Table C:
    1. Id: primary identity key
    2. AId: foreign key to A
    3. Data1: some data
    4. Data2: some data
The query below will update ValuesHash for A and B (because C doesn't have the ValuesHash column) with a hash constructed from the Data columns. The Id columns will be ignored for being primary keys (and for being in the list of columns to ignore), the AId columns will be ignored for being foreign keys, ValuesHash and CreateTime and ModifyTime will be ignored for being in a list of custom columns)

WARNING: each column data is always truncated to 4000 characters, then the corresponding string is also truncated to 4000 bytes before running HASHBYTES (which only accepts a maximum of 8000 bytes). This hash will help in determining unique records, but it is not 100%.

SELECT * 
FROM   (
SELECT t.name, 
  'UPDATE [' + t.name+ '] SET ValuesHash = HASHBYTES(''SHA1'',SUBSTRING(' 
  + Stuff( 
    (SELECT '+ ''|''+ ISNULL('+CASE 
      WHEN tp.name IN ('float', 'real') THEN 'STR('+c.name+',30,30)' 
      WHEN tp.name IN ('binary', 'varbinary') THEN 'CONVERT(NVARCHAR(4000),'+c.name+',2)'
      ELSE 'CONVERT(NVARCHAR(4000),'+c.name+')' END+','''')'
     FROM sys.all_columns c 
         INNER JOIN sys.types tp 
      ON c.system_type_id=tp.system_type_id
      AND c.user_type_id=tp.user_type_id 
     LEFT JOIN sys.index_columns ic
      ON ic.object_id=t.object_id
      AND ic.column_id=c.column_id
     LEFT JOIN sys.indexes i
      ON ic.object_id=i.object_id
      AND ic.index_id=i.index_id 
     LEFT JOIN sys.foreign_key_columns fc
      ON fc.parent_object_id=t.object_id 
            AND c.column_id=fc.parent_column_id 
     WHERE t.object_id=c.object_id
      AND ISNULL(c.is_identity, 0)=0
      AND ISNULL(c.is_computed, 0)=0
      AND ISNULL(c.is_filestream, 0)=0
      AND ISNULL(c.is_rowguidcol, 0)=0
      AND ISNULL(i.is_primary_key, 0)=0 
      AND fc.parent_column_id IS NULL
      AND c.name NOT IN ('Id', 'CreateTime' , 'AcquireTime' , 'IntermediateCreateTime', 'IntermediateModifyTime', 'IntermediateDeleteTime', 'ValuesHash')
     ORDER BY Sign(c.max_length) DESC, c.max_length, Lower(c.name)
     FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)')
    , 1, 7, '') 
  + ',0,4000)) WHERE ValuesHash IS NULL' AS computed 
FROM   sys.tables t 
INNER JOIN sys.all_columns c 
  ON t.object_id = c.object_id 
WHERE  c.name = 'ValuesHash') x 
WHERE  computed IS NOT NULL 
ORDER  BY name 

Change it to suit your needs. It is by no means perfect, but it's a start for whatever you need.

Update:

A new FORMAT function was introduced in SQL Server 2012, working somewhat similar to the .NET ToString method. Using that function is slightly more precise:

SELECT * 
FROM   (
SELECT t.name, 
  'UPDATE [' + t.name+ '] SET ValuesHash = HASHBYTES(''SHA1'',SUBSTRING(' 
  + Stuff( 
    (SELECT '+ ''|''+ ISNULL('+CASE 
      WHEN tp.name IN ('float', 'real') THEN 'FORMAT('+c.name+',''R'')' 
      WHEN tp.name IN ('decimal') THEN 'FORMAT('+c.name+',''G'')' 
      WHEN tp.name IN ('datetime','datetime2') THEN 'FORMAT('+c.name+',''O'')' 
      WHEN tp.name IN ('binary', 'varbinary') THEN 'CONVERT(NVARCHAR(4000),'+c.name+',2)'
      ELSE 'CONVERT(NVARCHAR(4000),'+c.name+')' END+','''')'
     FROM sys.all_columns c 
         INNER JOIN sys.types tp 
      ON c.system_type_id=tp.system_type_id
      AND c.user_type_id=tp.user_type_id 
     LEFT JOIN sys.index_columns ic
      ON ic.object_id=t.object_id
      AND ic.column_id=c.column_id
     LEFT JOIN sys.indexes i
      ON ic.object_id=i.object_id
      AND ic.index_id=i.index_id 
     LEFT JOIN sys.foreign_key_columns fc
      ON fc.parent_object_id=t.object_id 
            AND c.column_id=fc.parent_column_id 
     WHERE t.object_id=c.object_id
      AND ISNULL(c.is_identity, 0)=0
      AND ISNULL(c.is_computed, 0)=0
      AND ISNULL(c.is_filestream, 0)=0
      AND ISNULL(c.is_rowguidcol, 0)=0
      AND ISNULL(i.is_primary_key, 0)=0 
      AND fc.parent_column_id IS NULL
      AND c.name NOT IN ('Id', 'CreateTime' , 'AcquireTime' , 'IntermediateCreateTime', 'IntermediateModifyTime', 'IntermediateDeleteTime', 'ValuesHash')
     ORDER BY Sign(c.max_length) DESC, c.max_length, Lower(c.name)
     FOR XML PATH(''), TYPE).value('.', 'nvarchar(max)')
    , 1, 7, '') 
  + ',0,4000)) WHERE ValuesHash IS NULL' AS computed 
FROM   sys.tables t 
INNER JOIN sys.all_columns c 
  ON t.object_id = c.object_id 
WHERE  c.name = 'ValuesHash'
) x 
WHERE  computed IS NOT NULL 
ORDER  BY name 

Friday, March 14, 2014

Dragonlance - Chronicles, by Margaret Weis and Tracy Hickman

book covers I met a few friends for a drink and they recommended to me (or rather seemed amazed that I had not heard of it) Dragonlance. I looked it up and, to my chagrin, found that it is a huge series with over 20 books and a lot of short stories - actually, in 2008 there where over 190 novels in the same universe. Resigned myself to read them all, I googled for the right order in which to read the saga and came up with Chronicles, which is a trilogy of books, as the correct starting point.

As in the story, there is balance between the good and the bad in my assessment of the books. For one, I will not read the rest of the books and waste a lot of my time, but for the other, I already start regretting reading the first three. You see, the entire plot seems to have the only purpose of supporting a canon of the classic fantasy genre that the writers have thought up.

Probably emerging from games of Dungeons and Dragons, like many fantasy universes, the world of Krynn has nothing remotely original. There are elves, humans, dwarves, goblins, dragons, pegasi, unicorns, centaurs, and other races like that. From the very first pages, you meet the heroes that form the quest party and they seem to have gathered all the possible cliches in the genre in their travels: the dwarf is old and grumpy and complains a lot, the half-elf is tortured by his double ancestry, the knight is rigid and honorable, the mage is tiny and frail and frustrated about it, his big (twin) brother is huge and completely non-magical, etc. In fact, the mage character is the only one which seems remotely interesting, all the other being busy posturing most of the time, like real size commercials for their D&D class and specialization.

But what I thought was the most offensive of all was the premise of the trilogy. Beware, here be dragons... and spoilers. Do not read further if you think you might want to read the books.

You see, the world has been reeling after a huge Cataclysm, a fiery mountain hitting the planet and causing havoc. At the end of the book we learn that the gods, in their infinite wisdom, did that because the world was too unbalanced towards good! And we learn this from the good god, who for the entire duration of the story just nudged our heroes in one direction or the other while the evil god was amassing armies and killing everybody. How is that for balance?

Even so, you can hardly complain about a book being cliche if you don't read more of the genre and, to be honest, except for a few books, I didn't really read much fantasy. So I had an opportunity to enjoy this, even if the writing was simplistic, the characterization almost non existent and the story bland. But there was something in the books that kept me at arms length from enjoying it. It finally dawned on me in the middle of the second book, when, after reading about the emotional turmoil of everybody, having the men pair with the women - unless they were there for comic relief, like the dwarf and the kender (which one could consider a pair, if I think about it) - and making chaste promises to one another (like not having sex until they can focus on the relationship and stuff like that)... after all that, I realised that Dragonlance was written by two women.

I don't want to sound misogynistic here, I really wanted to read something cool written by women, but for a series entitled after a weapon - albeit something long and thin, with a thick bulbous appendage at the tip - the story was surprisingly devoid of any detailed battles, tactics, strategy or even decent brawls. The heroes are always running around, talking about their feelings or thinking about them and, in case there is a huge battle between the forces of good and evil, quickly skips forward to the conflict between the two women that love the same man.

Also, as if it all wasn't formulaic enough, no one really dies from the group, unless it is something that fulfills their purpose in life, while the support cast keeps perishing without anyone actually giving a damn. Check out the bit where an entire ship crew - including the woman captain and the minotaur second that I had read a lot about in previous pages - just die without the characters even remembering it. Or the battle of the knights with the dragon armies, where one phrase describes how the knights held, but half of them died. Just like that. I may have written more about that bit than there was written in the book.

To end this terrible rant, if you thought Wheel of Time was childish, as I did, this is worse. T'is true, the fair maiden that hath captured my heart and recommended the books hath read said scrolls of wisdom when she was 16, so that might explain her fond memories and my tortured journey towards the end of the story. I also really really wanted to believe that by writing more, the authors would become more skilled at it. It didn't seem to be the case. I refuse to read another dozen books just to keep the faith.

In conclusion, I cannot in good conscience recommend this to anyone, including children or young adults - to which I think the story would be tantamount to poison, teaching all the wrong lessons in the worst possible way. These books sucked lance!

Tuesday, March 04, 2014

Memories of you

I remember the first time it happened. We were just beginning to watch your struggle, the rise of such wonderful life, then the strike. It had happened before, but we weren't really paying attention. This time it was incredible drama. Galaxies cried out in anguish and desperation, but it wasn't the end. No, you survived, rebuilt, went on.

It's funny, the first time apes started acting smart there was a lot of disappointment. Ugly, cumbersome, a bit mad, everybody thought. But we continued to watch, mesmerized, as you got through so much that almost destroyed you. You changed, you evolved.

Then you bloomed. Started talking, singing, thinking and painting. Further on you started writing. What a wonderful flower you were. Worlds would die and form on the sounds of your wails or yells of joy. We loved it all, the wars, the art, the science, the suffering. We gulped it up, knowing that it was not going to last. They never do, we thought.

We cheered for you when you were dying in the great pandemics, we cried for you when you killed each other, you had our full attention when you almost destroyed everything with nuclear devices. Some of us rooted for that ending. They love violent, brusque ends, some of us do. But it didn't happen.

Oh how you danced, how you sang, the beautiful things you thought and put in writing, in films, did with your computers. We knew you were close to the end and we knew that it was all good stuff, because we didn't want it to end, but they all end, don't they?

We marvelled at your ships, at your resilience in hoping to contact others... us... and we cried. The space battles were magnificent. You took inspiration from your history and created fiction, then you used fiction to create your future. You bypassed your present altogether. A whole universe laughed and cheered. Those were good times.

Your enormous space habitats, floating around your sun, always changing, always growing, they gave us hope. Hope that some of you might make it, leave your system. Sometimes it happens. We love splinters, that we do, but it didn't happen. You stayed put.

And then your flower wilted away. You had altered yourselves, you had become faster, smarter, you had already merged with the machines you had built and defeated most of your biological problems. You were invincible and beautiful, immortal. But then you found it; after all, they all do, the universal link, the thing that finally allowed you to fulfil your dreams. You found us.

And now you listen to our songs, watch our histories, run our software, use our technology. But we remember you, how beautiful you were when you were young. We all loved you, humans. Now that you are old, like us, you have control. You are us. Join in watching this wonderful new life that emerges from the chaos. It's a great show. And maybe some will make it, some will manage to be different, somehow, sometime.

Monday, March 03, 2014

The Listeners, by James E. Gunn

Book cover As always, this post will reflect my personal opinion. I know that The Listeners is a classic book, one that has been cited by SETI as a major factor in the project becoming known and supported by others. I know that at that time, doing a reasonable sci-fi book was a feat. I know that the writer was a believer in the contact with aliens and human nature and so on, and thus he must have been a nice guy, with similar desires to mine and other space-looking people. However the book annoyed me to no end.

The first and biggest of all problems is the insistence of the writer to add to the book all kinds of quotes from various works, many of them in a foreign language - that is, other than English. It was the reason why originally publishers refused his manuscript. Now, even if I understand the language, I don't know the quote. There is an annex at the end of the book that translates everything, but really, when a character randomly interrupts a perfectly good conversation to spout something unintelligible in another language, that guy is an asshole!

Then there was the construction of the book, the Project being presented like something that held sway over the human heart. All you had to do to convince anyone of anything was turn on the speakers so that they hear static, while the main character would do PR work, knowing exactly what to say to manipulate the other person. I would not have a problem with that, if the manipulation would not be completely obvious and most of the time completely ridiculous. It felt like a Naruto episode where the other ninja, filled with power, suddenly decides to switch sides because Naruto is such a nice guy. I know I don't inspire confidence when I compare a classic sci-fi book with a Japanese manga, but for me it was the same quality of work, which may be entertaining, but not great.

All the people and events changed in order to conveniently support the plot. It felt fake and it is a lousy writing technique, more suited to pulp. I did not enjoy that.

As for the plot itself, it is about this Project, which is pretty much SETI, that suddenly receives an alien signal piggybacked on 90 years old radio transmissions. What people do and say is so underwhelming that it felt like I was wasting my time while reading the book. That is why it took so long to finish it. My conclusion: while a classic for the science fiction genre, I did not enjoy the book or empathise with its characters. The plot is difficult to swallow and the story is very dated. I would not recommend it.