Wednesday, April 27, 2016

Learning to program - again?

Corporate guy with other people on his shoulders... that's a dev, not a manager. I have been a professional in the IT business for a lot of years, less if you consider just software development, more if you count that my favorite activity since I was a kid was to mess with a computer or another. I think I know how to develop software, especially since I've kind of built my career on trying new places and new methods for doing that. And now people come to me and ask me: "Can I learn too? Can you teach me?". And the immediate answer is yes and no (heh! Learnt from the best politicians that line) Well, yes because I believe anyone who actually wants to learn can and no because I am a lousy teacher. But wait a minute... can't I become one?

You may think that it is easy to remember how it was when I was a code virgin, when I was writing Basic programs in a notebook in the hope that some day my father will buy me a computer, but it's not. My brain has picked up so many things that now they are applied automatically. I may not know what I know, but I know a lot and I am using it at all times. A few weeks ago I started thinking about these things and one of the first ideas that came to me was FizzBuzz! A program that allegedly people who can't program simple can't... err... program. Well, I thought, how would I write this best? How about worst? I even asked my wife and she gave me an idea that had never occurred to me, like not using the modulo function to determine divisibility.

And it dawned on me. To know if your code is good you need to know exactly what that code has to do. In other words, you can't program without having an idea on how to use or test it afterwards. You have to think about all the other people that would be stumbling unto your masterwork: other developers, for example, hired after you left the company, need to understand what they are looking at. You need to provide up to date and clear documentation to your users, as well. You need to handle all kinds of weird situations that your software might be subjected to. To sum it up: as a good developer you need to be a bit of all the people on the chain - users, testers, documenters, managers, marketers, colleagues - and to see the future as well. After all, you're an expert.

Of course, sketches like the one above are nothing but caricatures of people from the viewpoint of other people who don't understand them. After all, good managers need to be a little of everything as well. If you think about it, to be good at anything means you have to understand a little of everybody you work with and do your part well - exactly the opposite of specialization, the solution touted as solving every problem in the modern world. Anyway, enough philosophy. We were talking programming here.

What I mean to say is that for every bit of our craft, we developers are doing good things for other people. We code so that the computer does the job well, but we are telling it to do things that users need, we write concisely yet clear so that other developers can work from where we leave off, we write unit tests to make sure what we do is what we mean and ease the work of people who need to manually check that, we comment the code so that anyone can understand at a glance what a method does and maybe even automate the creation of documents explaining what the software does. And we draw lines in a form of a kitten so that marketers and managers sell the software - and we hate it, but we do it anyway.

So I ask, do we need to learn to write programs all over again? Because, to be frank, coders today write in TDD style because they think it's cutting edge, not that they are doing it for someone; they work in agile teams not because they know everybody will get a better understanding of what they are doing and prevent catastrophic crashes caused by lack of vision, but because they feel it takes managers off their backs and they can do their jobs; they don't write comments for the documentation team, but because they fear their small attention span might make them forget what the hell they were doing; they don't write several smaller methods instead of a large one because they believe in helping others read their code, but because some new gimmick tells them they have too much cyclomatic complexity. And so on and so on.

What if we would learn (and teach) that writing software is nothing but an abstraction layer thrown over helping all kinds of people in need and that even the least rockstar ninja superhero developer is still a hero if they do their job right? What if being a good cog in the machine is not such a bad thing?

While writing this I went all over the place, I know, and I didn't even touch what started me thinking about it: politics and laws. I was thinking that if we define the purpose of a law when we write it and package them together, anyone who can demonstrate that the effect is not the desired one can remove the law. How grand would that be? To know that something is applied upon you because no one could demonstrate that it is bad or wrong or ineffective.

We do that in software all the time, open software, for example, but also the internal processes in a programming shop designed to catch flaws early and to ensure people wrote things how they should have. Sometimes I feel so far removed from "the real world" because what I am doing seems to make more sense and in fact be more real than the crap I see all around me or on the media. What if we could apply this everywhere? Where people would take responsibility individually, not in social crowds? Where things would be working well not because a lot of people agree, but because no one can demonstrate they are working bad? What if the world is a big machine and we need to code for it?

Maybe learning to code is learning to live! Who wouldn't want to teach that?

Thursday, April 21, 2016

Getting thumbnail images for the most popular embedded video providers out there

Various video provider logos During one revamp of the blog I realized that I didn't have images for some of my posts. I had counted pretty much on the Blogger system that provides a post.thumbnailUrl post metadata that I can use in the display of the post, but the url is not always there. Of course if you have a nice image in the post somewhere prominently displayed, the thumbnail URL will be populated, but what if you have a video? Surprisingly, Blogger has a pretty shitty video to thumbnail mechanism that prompted me to build my own.

So the requirements would be: get me the image representing a video embedded in my page, using Javascript only.

Well, first of all, videos can be actual video tags, but most of the time they are iframe elements coming from a reliable global provider like YouTube, Dailymotion, Vimeo, etc, and all the information available is the URL of the display frame. Here is the way to get the thumbnail for these scenarios:

YouTube


Given the iframe src value:

// find youtube.com/embed/[videohash] or youtube.com/embed/video/[videohash]
var m = /youtube\.com\/embed(?:\/video)?\/([^\/\?]+)/.exec(src);
if (m) {
    // the thumbnail url is https://img.youtube.com/vi/[videohash]/0.jpg
    imgSrc = 'https://img.youtube.com/vi/' + m[1] + '/0.jpg';
}

If you have embeds in the old object format, it is best to replace them with the iframe one. If you can't change the content, it remains your job to create the code to give you the thumbnail image.

Dailymotion


Given the iframe src value:

//find dailymotion.com/embed/video/[videohash]
var m=/dailymotion\.com\/embed\/video\/([^\/\?]+)/.exec(src);
if (m) {
    // the thumbnail url is at the same URL with `thumbnail` replacing `embed`
    imgSrc=src.replace('embed','thumbnail');
}

Vimeo


Vimeo doesn't have a one URL thumbnail format that I am aware of, but they have a Javascript accessible API.

// find vimeo.com/video/[videohash]
m=/vimeo\.com\/video\/([^\/\?]+)/.exec(src);
if (m) {
    // set the value to the videohash initially
    imgSrc=m[1];
    $.ajax({
        //call the API video/[videohash].json
        url:'https://vimeo.com/api/v2/video/'+m[1]+'.json',
        method:'GET',
        success: function(data) {
            if (data&&data.length) {
                // and with the data replace the initial value with the thumbnail_medium value
                replaceUrl(data[0].thumbnail_medium,m[1]);
            }
        }
    });
}

In this example, the replaceUrl function would look for img elements to which the videohash value is attached and replace the url with the correct one, asynchronously.

TED


I am proud to announce that I was the one pestering them to make their API available over Javascript.

// find ted.com/talks/[video title].html
m=/ted\.com\/talks\/(.*)\.html/.exec(src);
if (m) {
    // associate the video title with the image element
    imgSrc=m[1];
    $.ajax({
        // call the oembed.json?url=frame_url
        url:'https://www.ted.com/services/v1/oembed.json?url='+encodeURIComponent(src),
        method:'GET',
        success: function(data) {
            // set the value of the image element asynchronously
            replaceUrl(removeSchemeForHttpsEnabledSites(data.thumbnail_url),m[1]);
        }
    });
    return false;
}

video tags


Of course there is no API to get the image from an arbitrary video URL, but the standard for the video tag specifies a poster attribute that can describe the static image associated with a video.

// if the element is a video with a poster value
if ($(this).is('video[poster]')) {
    // use it
    imgSrc=$(this).attr('poster');
}

Monday, April 18, 2016

Codex, by Lev Grossman

book cover The book was nothing if not captivating, its pace much better than for the Magicians trilogy, but as in those books, it was really really difficult to empathize with any of the characters.

The story in Codex is about this investment banker who is having the first vacation in years, actually a small transition time between switching from his US job to a London based one. This makes him feel disconnected and somehow gets tangled in a project to arrange the library of a very wealthy family. The problem with this character is that he doesn't seem to chose anything. Things just happen to him, kind of like with Quentin in the Magicians, and he goes with them, only to get disappointed or surprised at the end. His friends, the people he randomly meets, even his adversaries appear at the exact time when the story requires them, making the book feel like a string of unlikely happenstances a rather aimless character just stumbles through.

There are some interesting parallels in the book, analogies between the beginning of romantic literature and the emergence of computer game fantasy, and probably the literary and historical details in the book hide some deeper meaning as well, but it's Lev Grossman's infuriatingly detached, almost dreamlike perspective that made me not care about it at all. Strangely, all the literary research going on in the book and the altered mood made me think of House of Leaves, only that was orders of magnitude weirder and better written.

As for the ending, I think I can safely say now that it's typical Grossman: the main character becomes aware of the delusions he was living under, both relating to his goals and the people around him. The veil gets lifted and the world goes back to the usual confusing pointless drag that it is for most of us. The author doesn't seem to care about the need of the reader for a happy ending, and I would normally applaud that, yet to use a depressingly realistic ending to a story that felt torn out of a night's dream seems a bit pointless to me.

Bottom line: I actually enjoyed it better than The Magicians, perhaps because it was shorter, better paced and I could relate to the main character a bit more. However, it is not something I would recommend to people. It felt too much like describing a man during a heatstroke, always dazed and confused, spinning wildly out of control of the things happening around him, a hapless victim of his immature feelings and a situation he is out of his depth in.

Friday, April 15, 2016

.NET parsing command line arguments differently from CMD.EXE

Cracked command line screen 3D I came upon a StackOverflow question today that sounded plain silly to me. In it someone was complaining that parsing a command line like "x\" -y returns a single argument, not two. And that is ridiculous, since the operating system doesn't do that.

Just to prove myself right (because I love being right) I created a batch file that displays all command line parameters:
@ECHO OFF 
SET counter=1
:loop
IF "%1"=="" EXIT
ECHO %counter% %1
SET /A counter=counter+1
SHIFT
GOTO loop

I ran the batch file with the command line supplied in the SO question: -p -z "E:\temp.zip" -v "f:\" -r -o -s –c and the result was
1 -p
2 -z
3 "E:\temp.zip"
4 -v
5 "f:\"
6 -r
7 -o
8 -s
9 –c
See? I was right! The command line is properly parsed. But just to be sure, I created a .NET console application:
static void Main(string[] args)
{
    for (var i=0; i<args.Length; i++)
    {
        Console.WriteLine(i + "  " + args[i]);
    }
}
and the result was... different!
0  -p
1  -z
2  E:\temp.zip
3  -v
4  f:" -r -o -s -c
Never would I have imagined that .NET console applications would do different things from system applications, especially since they are both made by Microsoft.

Investigating the problem, I found a page that explained the rules of parsing command line arguments. It was for C++, but it applied perfectly. Apparently the caret (^) is treated just as any other character (not like by the operating system that treats it as an escape character) and a quote preceded by a backslash is considered an ordinary character as well (not like the operating system that ignores it).

So, how do I get the command line the same way the operating system does? First of all I need the raw unprocessed command line. I get that using Environment.CommandLine. Then I need to split it. Of course, I can make my own code, but I want to use the same stuff as the operating system - in this case Windows, we are not discussing .NET Core or Mono here - so I will be "Pinvoking" the Windows CommandLineToArgvW function.

And, of course, it didn't work. What I did here was basically recreating the Environment.GetCommandLineArgs method, which returns the exact same result as the args array!

Now that I had already gone through the rabbit hole, I got to this very informative link about it: How Command Line Parameters Are Parsed. In it, you may find the very aptly named chapter Everyone Parses Differently which shows that there are even differences between how C and VB programs parse the command line.

Bottom line: while you may be able to get the command line from the Environment class, there is no "standard" for parsing command line arguments, instead each compiler and operating system version implements their own. Consequently, I suggest that the only way to be consistent in how you parse command line arguments is to parse them yourself.

Wednesday, April 13, 2016

WordsAlive.js - make the words on your page come alive! Javascript library

Screenshot of a page with WordsAlive I had this crazy idea that I could make each word on my page come alive. The word "robot" would walk around the page, "explosion" would explode, "rotate" would rotate, color words would appear in the color they represent, no matter how weird named like OliveDrab , Chocolate , Crimson , DeepPink , DodgerBlue and so on, "radioactive" would pulse green, "Siderite" would appear in all its rocking glory and so on. And so I did!

The library is on GitHub and I urge you to come and bring your own ideas. Every effect that you see there is an addon that can be included or not in your setup.



Also see directly on GitHub pages.








Saturday, April 09, 2016

What I've learned from one month of answering on StackOverflow

Jon Skeet in a box Almost a month ago I got started being active on StackOverflow, a web site dedicated to answering computer related questions. It quickly got addictive, but the things that I found out there are many and subtle and I am happy with the experience.

The first thing you learn when you get into it is that you need to be fast. And I mean fast! Not your average typing-and-reading-and-going-back-to-fix-typos speed, but full on radioactive zombie attack typing. And without typos! If you don't, by the time you post your contribution the question would have been answered already. And that, in itself, is not bad, but when you have worked for minutes trying to get code working, looking good, being properly commented, taking care of all test cases, being effective, being efficient and you go there and you find someone else did the same thing, you feel cheated. And I know that my work is valid, too, and maybe even better than the answers already provided (otherwise I feel dumb), but to post it means I just reiterate what has been said before. In the spirit of good sportsmanship, I can only upvote the answer I feel is the best and eventually comment on what I think is missing. Now I realize that whenever I do post the answer first there are a lot of people feeling the same way I just described. Sorry about that, guys and gals!

The second thing you learn immediately after is that you need to not make mistakes. If you do, there will be people pointing them out to you immediately, and you get to fix them, which is not bad in itself, however, when you write something carelessly and you get told off or, worse, downvoted, you feel stupid. I am not the smartest guy in the world, but feeling stupid I don't like. True, sometimes I kind of cheat and post the answer as fast as possible and I edit it in the time I know the question poster will come check it out but before poor schmucks like me wanted to give their own answers. Hey, those are the rules! I feel bad about it, but what can you do?

Sometimes you see things that are not quite right. While you were busy explaining to the guy what he was doing wrong, somebody comes and posts the solution in code and gets the points for the good answer. Technically, he answered the question; educationally, not so much. And there are lot of people out there that ask the most silly of questions and only want quick cut-and-pastable answers. I pity them, but it's their job, somewhere in a remote software development sweat shop where they don't really want to work, but where the money is in their country. Luckily, for each question there are enough answers to get one thinking in the right direction, if that is what they meant to do.

The things you get afterwards become more and more subtle, yet more powerful as well. For example it is short term rewarding to give the answer to the question well and fast and first and to get the points for being best. But then you think it over and you realize that a silly question like that has probably been posted before. And I get best answer, get my five minutes of feeling smart for giving someone the code to add two values together, then the question gets marked as a duplicate. I learned that it is more satisfying and helpful to look first for the question before providing an answer. And not only it is the right thing to do, but then I get out of my head and see how other people solved the problem and I learn things. All the time.

The overall software development learning is also small, but steady. Soon enough you get to remember similar questions and just quickly google and mark new ones as duplicates. You don't get points for that, and I think that is a problem with StackOverflow: they should encourage this behavior more. Yet my point was that remembering similar questions makes you an expert on that field, however simple and narrow. If you go to work and you see the same problem there, the answer just comes off naturally, enforced by the confidence it is not only a good answer, but the answer voted best and improved upon by an army of passionate people.

Sometimes you work a lot to solve a complex problem, one that has been marked with a bounty and would give you in one shot maybe 30 times more points than getting best answer on a regular question. The situation is also more demanding, you have to not only do the work, but research novel ways of doing it, see how others have done it, explaining why you do things all the way. And yet, you don't get the bounty. Either it was not the best answer, or the poster doesn't even bother to assign the bounty to someone - asshole move, BTW, or maybe it is not yet a complete answer or even the poster snubs you for giving the answer to his question, but not what he was actually looking for. This is where you get your adrenaline pumping, but also the biggest reward. And I am not talking points here anymore. You actually work because you chose to, in the direction that you chose, with no restrictions on method of research or implementation and, at the end, you get to show off your work in an arena of your true peers that not only fight you, but also help you, improve on your results, point out inconsistencies or mistakes. So you don't get the points. Who cares? Doing great work is working great for me!

There is more. You can actually contribute not by answering questions, but by reviewing other people's questions, answers, comments, editing their content (then getting that edit approved by other reviewers) and so on. The quality of my understanding increases not only technically, but I also learn to communicate better. I learn to say things in a more concise way, so that people understand it quicker and better. I edit the words of people with less understanding of English and not only improve my own skills there, but help them avoid getting labelled "people in a remote software development sweat shop" just because their spelling is awful and their name sounds like John Jack or some other made up name that tries to hide their true origins. Yes, there is a lot of racism to go around and you learn to detect it, too.

I've found some interesting things while doing reviews, mostly that when I can't give the best edit, I usually prefer to leave the content as is, then before I know the content is subpar I can't really say it's OK or not OK, so I skip a lot of things. I just hope that people more courageous than me are not messing things up more than I would have. I understood how important it is for many people to do incremental improvements on something in order for it to better reach a larger audience, how important is that biases of language, race, sex, education, religion or psychology be eroded to nothing in order for a question to get the deserved answer.

What else? You realize that being "top 0.58% this week" or "top 0.0008% of all time" doesn't mean a lot when most of the people on StackOverflow are questioners only, but you feel a little better. Funny thing, I've never asked a question there yet. Does it mean that I never did anything cutting edge or that given the choice between asking and working on it myself I always chose the latter?

Most importantly, I think, I've learned a few things about myself. I know myself pretty well (I mean, I've lived with the guy for 39 years!) but sometimes I need to find out how I react in certain situations. For example I am pretty sure that given the text of a question with a large bounty, I gave the most efficient, most to the point, most usable answer. I didn't get the points, instead they went to a guy that gave a one liner answer that only worked in a subset of the context of the original question, which happened to be the one the poster was looking for. I fumed, I roared, I raged against the dying of the light, but in the end I held on to the joy of having found the answer, the pleasure of learning a new way of solving the same situation and the rightness of working for a few hours in the company of like-minded people on an interesting and challenging question. I've learned that I hate when people downvote me with no explanation even more than downvoting me with a good reason, that even if I am not always paying attention to detail, I do care a lot when people point out I missed something. And I also learned that given the choice between working on writing a book and doing what I already do best, I prefer doing the comfortable thing. Yeah, I suck!

It all started with a Tweet that claimed the best method of learning anything is to help people on StackOverflow who ask questions in the field. So far I've stayed in my comfort zone: C#, ASP.NET, WPF, Javascript, some CSS, but maybe later on I will get into some stuff that I've always planned on trying or even go all in. Why learn something when you can learn everything?!

Thursday, April 07, 2016

Blogger goes all HTTPS

Message from Blogger announcing the change My blog is hosted by Blogger, a Google site, and usually they are quite good, however this month they announced a change that, frankly, I think will hurt them in the short run because it was so sudden and leaves not only blog owners, but Blogger themselves unprepared. The news is about forcefully enabling Secure Hyper Text Transfer Protocol support for all free Blogspot sites. BTW, the link above that explains why Blogger forces HTTPS... doesn't work on HTTPS, which shows me a nice red error while editing this post.

The underlying idea is good: move everything to HTTPS, no matter how relevant it is for people to be safe and anonymous while reading my blog :). In theory, everything should be working the same as with HTTP, with some small issues that are easily fixed. In practice, we are talking about templates that people have installed without modifying or scripts that one can only find on HTTP sites or images and other resources that can only be found on unsecured web sites. For example, Google's default blog bar itself is causing an error in the console because it is trying to search the blog with an HTTP URL, even if the bar frame is loaded from an HTTPS location.

I had several problems:
  • The PGN Viewer that I use for chess games is only found on an HTTP site, therefore Google Chrome blocks loading those scripts when in HTTPS. I had to copy stuff in Google Drive and change the PGN Viewer scripts to use alternate URLs when under HTTPS and host files from Google Drive. I hope it will not reach some hosting limit and randomly fail.
  • Many thumbnails loaded for the related posts list and the blog main page are also loaded via HTTP, causing mostly annoying errors in the console. I tried to fix it programatically, but it relies on knowing which sites support HTTPS and which don't.
  • Videos, pictures and resources inside the blog posts themselves! I can't possibly change all the posts on my blog. There are over 1300 separate posts. While I don't have any posts that load remote scripts through HTTPS, it is still damn scary because I can't manually check everything. It would take me forever!
  • Caching. It is a myth that HTTPS requests cannot be cached, but it does depend on the server headers. If the HTTPS servers that I am blindly connecting to are misconfigured, people are going to perceive it as slowing down. Also, there is an interesting post that explains why loading scripts from third parties is breaking HTTPS security.

With this in mind, please help me test the blog with HTTPS and let me know if you find any issues with it. I've done what I could, but I am a developer, not a tester, so let me know, OK? Thanks!

While it is simple for blog owners to use a small Javascript in order to force users to go to the HTTP URL, not allowing this from the get go is a pretty ballsy, but asshole move. Will it make the internet safer? We'll just have to see.