Thursday, January 12, 2017

Way of the Kings (Stormlight Archive #1), by Brandon Sanderson

Brandon Sanderson proves again he is a brilliant writer. His Stormlight universe is not only vast and imaginative, but the characters are both compelling and well written.

Way of the Kings has some slow parts, though, and even if I kind of liked that, it is uneven in regards to its characters: some get more focus, some just a few chapters. That means that if you identify with the lead characters you will enjoy the book, but if you empathize with the lesser ones you will probably get frustrated.

I particularly enjoyed the climax. It was as it should be: the tension was rising and Sanderson just wouldn't let it go, it just kept pushing it and pushing it, filling in the motivations of the character, adding burden upon burden, making choices as difficult and as important as possible before finally allowing the release of his characters making one. Alas, the wonderful ending is followed by epilogues, several of them, which just seem boring afterwards, in comparison.

Great series, though, I recommend it highly.

Tuesday, January 03, 2017

Need help solving a code design problem: API validation

I am working on this issue that I can't seem to properly be able to solve. No matter what I do I can't shake the feeling of code smell from it. It involves a chain of validations and business operations that I want to separate. The simple scenario is this: I have a web API that receives a data transfer object and needs to perform some action, but then it gets murkier and murkier.

Simply code


The first implementation is pretty obvious. I get the DTO, perform checks on it, access the database to get more info, perform more checks, do more work until everything is done. It's easy to do, clear imperative linear programming, it does the job. However I get a bad smell from the code because I have mashed together validation and business logic. I would also like to reuse the validation in other areas of the project.

Manager and Validator


So the natural improvement is to create two flows instead of one, each with their own concern: a validator will perform checks so I know an operation is valid, then the manager performs the operation. But this soon hits a snag. Let say I want to validate that a record exists from the id that I send as a property in the DTO. If I do this check in the validator, then it accesses the database. It stinks! I should probably get the record with the manager, but then I have to validate that the record was returned and that the data in it is valid in the context of my DTO. What do I do?

Manager and Validator classes


OK, so I can split the validation and operational flows in methods that will live in their respective classes. In this scenario a validator checks the record id is not malformed, a manager attempts to get the record, then the validator executes another method to check the record exists and conforms to the expectations, and so on. Problem solved, right? But no. That means I have to pass the record to the validating method, or any number of records that I need to perform the validation. For example I want to check a large chain of records that need to be validated based on conditions in the DTO. I could get the list of all records, pass them to the validator and then check if they exist, but what if the database is very large?

Manager methods that help validation and Validator


Wait, I can be smart about it. I know that essentially I want to get a list of records then validate something. Can't I just create a specific method in the manager that gets only the records involved in validation? Just do manager.GetRelatedRecords(dto.Id) to get a list of records from the database, then pass them as parameters to the validator methods. Fixed it, right? Nope. What if there is a breaking condition that is related to validation? The original linear method would just go through the steps and exit when finished. It wouldn't get a bunch of unnecessary records then fail at the first later on. And there is even a worse problem: what if while I was busy getting the records and preparing for validation the records have changed? I am basically introducing a short lived cache, with all the problems arising from it, like concurrency and invalidation.

Transactions


Hey, I can just new TransactionScope the shit out of it, right? Do steps of Validate, GetMoreData and repeat all inside a transaction. Anything fails, just rollback, what is the big deal? Well, concurrent access being slowed or deadlocks would be problematic, but more than that I am terrified of the result. I started with a linear flow in a single method and now I have two classes, each with a lot of methods which get more and more parameters as I move on: first I validate the DTO values, then a record in relation to the DTO, then two records in relation to each other and the DTO and so on. All these methods are being executed inside a transaction using a local ad-hoc memory cache to hold all the data I presume I will need. The manager methods are either using a big list of parameters themselves or repeatedly instantiating the same providers for the data they get. It's a lot more complex than the original, working design. It's also duplicated flow, as it needs to at least partially go through the data to populate what I need, then it goes through the same flow during the validation.

Context


Ha! I know! Instead of passing an increasing number of parameters to validation methods, create a single context object in which I just populate properties as I go. No more mega methods. Anything can be solved by adding another layer of indirection, correct? Well, now I have three classes, not two, with the context class practically constituting a replacement of the local scope in the original method. The validation methods are now completely obscure unless you look inside them: they all receive the DTO and the context class. Who knows which properties in the context are being used and how?

Interfaces and Flow


Ok, maybe I rushed into things. Sometimes several layers of indirection are required to solve a single problem. In this case I will create an interface with the properties and methods used by every validation method. So the name validation will look like ValidateName(MyDTO dto, INameContext nameContext), the graph validation will look like ValidateGraph(MyDTO dto, IGraphStructure graph) and my context class will implement INameContext and IGraphStructure, even when the two interfaces have common members. I will do the same with the manager class, which will implement the interfaces required to populate the specific context interfaces.

I am not kidding!


This isn't one of those "how the junior and the senior write the same code" jokes. I actually don't know what to do. The most "elegant" solution turned a working method from an API class into three classes, transactions, a dozen interfaces and, best of all, still keeping the entire logical flow in the original method. You know how when a code block becomes too complex it needs to be split into several short methods, right? Well I feel like in this case I somehow managed to split each line in the original block into several methods. I can see the Medium article title now: "If your method has one line of code, it's too long!".

Unit Tests


Any software developer worth his salt will write unit tests, especially for a code such as this. That means any large validation method will need to be split just in order to not increase unit test size exponentially. Can you imagine writing unit tests for the methods that use the huge blob-like context class?

Cry for help


So help me out here, is there an general, elegant but simple solution to API separation of concerns in relation to validation?

Sunday, January 01, 2017

What I've learned from a year on social media

Last year I wrote a blog post detailing my experience with social media after four months. This is the followup, after I've had a whole year to take advantage of these tools.

Social Media - what is it?


To me social media means the big two: Facebook and Twitter. I still have no idea what Instagram is and I don't really consider LinkedIn as social media. And Google+ is not worth mentioning. I know there are a lot of other social sites, but I ignored completely photo and video platforms - since I rarely express myself visually, and I've tried some technical platforms like StackOverflow, HackerRank or GitHub, but again didn't consider that "social media". Probably I should, since I love software development and having people to share this with would truly be a social experience for me, but I started this experiment with focus on general social interaction. Also... Slack... what the hell is that?

What I used social for


In the previous post I said that I am using Blogger to express myself, as I have done for more than a decade, and use some tool to automatically share this on Facebook and Twitter. Not surprisingly, very little people were engaged by this method of communication. It works better than RSS feeds, that's for sure, but most of the time people on social media (including myself) want to shut off their brain and read something light, not my crazy ramblings or technical posts. I've created a Facebook page for my blog, so people can use that as an entry point, if they want, but all the posts there are shares from Blogger.

I was saying that I was pleasantly surprised by Twitter and the quality of content there and less enthusiastic about Facebook. However, Twitter changed some things lately, mostly allowing videos, images and smileys (what you, young folks, call emoticons - or is it emoji?) to take less space. The effect is that there are a lot more visual opinions (let's call them that) on Twitter and thus becoming harder and harder to read. Also the amount of postings there is overwhelming. I tried to look for some tools to limit the number of tweets or organizing them somehow, but unfortunately I found none that did what I envisioned. The result is that occasionally - every week or so - I scroll through Twitter until I get tired and put links in my to read list, but most of the time I only cover a day or two of content.

I found that whenever I see something that I believe is worth sharing I put it on Facebook, rather than Twitter, mostly because I have few friends on Twitter and there is that 140 character limitation. However, most of the time I just post the link anyway and maybe say a few words. I wonder if that short circuits me thinking about the subject and then writing my opinion on it, as I am doing with this blog, but most likely I would rather not share than write so much every time, so I don't know if the occasional Facebook posts are taking away Blogger posts. I will actively try to not make it the case. I noticed, though, that people that like my Tweets are often not in my friends list, so I guess it's more general an audience. That being said, all my Facebook posts are fully public, anyway.

Speaking of Facebook, when I compile my weekly reading list I also scroll down through the Facebook wall, but even with my extension to filter posts based on own content and less images, videos and likes, I still get bored rather quickly. Sometimes the jokes are funny or the pictures interesting, but I am not really a Facebook reader. Lately I have been unfollowing people in order to keep a modicum of content curation on my wall. I was really disappointed by the events system, as well. A lot of people just mark all events they could possible go to with 'Interested' and then they never go. Also the events that appear on Facebook feel like complete bullshit most of the time anyway. The ones that I would have liked to attend either don't even appear or they are so niche that I never hear about them until it is too late.

One thing that I thought I would use Facebook for was the messenger app. And I do use it, but very rarely. In the Yahoo Messenger days I would chat a lot with people. Somehow all that became frivolous, not only for me, but other people as well. Now I see young people just getting a lot of notifications and ignoring them. So what's the point, anyway?

And speaking of... God, notifications are annoying. Everything wants to notify you of the very important thing that happened on it. It does so by blinking, beeping, animating or any other histrionic method of getting your attention. They do it incessantly until I stop caring. Notify away, I will ignore you.

What I will be using social for


I don't foresee any change in the future other than maybe using less social media altogether. I am half convinced that I should try to develop meaningful human relationships, at least as another experiment. Clearly social media does NOT connect people on a personal level at all. I've heard about these young kids that share everything they do on social media. Maybe they do, but I am not following them. To me that's another network altogether. The occasional curiosity to see what is "trending" or "popular" disgusts me every single time. The things my friends share are not truly representative of them. And if I make the first step and post some weird feeling or situation I am in, I mostly get no reaction. People avoid negative emotions unless they are manic: hate, anger, disgust take first stage while depressive thoughts, sadness or desperation are avoided. Same for positive emotions, by the way, when people are extra happy about having a child or something like that. Just mildly enjoying something and feeling good about oneself is generally ignored.

Conclusion


I am not going to commit social media suicide or anything, but I concluded that I want to know what people think, rather than what people feel, and social media is used more for the latter. Therefore my commitment to online electronic expression is not going to increase towards Facebook and Twitter. As always, I do hope I will blog more meaningful posts. Wish me luck!

End of 2016

This is how 2016 ends, not with a bang but a whimper. After a relatively calm Christmas followed an eventless New Year celebration. Part of me was lamenting the oldness of it all: two people alone on New Year's Eve, drinking gin tonic and campari cola and sitting at their respective electronic devices. The other part of me was happy that no one bothers me, that I don't have to pretend to enjoy loud noises and strong lights and fake emotions. Was it a nice end of year or just another win for my comfort zone?

It was a strangely quiet year for me, as well. Most of it I was in a sabbatical during which I did nothing of note and the last part was about getting hired and getting acquainted with my new place of work, which was fine. No real drama, no dead relatives I cared about, no fuss. I wrote my code, I watched my TV series, I read my tech news. It was more than quiet, it was boring.

Meanwhile, though, everybody else was going crazy: beloved celebrities died, elections went to hell just about everywhere, terror scares, immigration issues, civil wars, cyber wars, cold wars, global warming and so on and so on. Weird contrast, isn't it? Part of me laments I am growing apart from the world and people, the other part enjoys the hell out of it. Is that what growing old is? Just getting off the train and raising a middle finger?

So what about my New Year resolutions? I have none. I have some hopes, but resolutions? Nah! I am too comfortable ignoring everything that matters. If I go down that road, defining priorities, finding solutions to get what I want, getting rid of what I don't want, setting up goals, then I have to change my entire life. I have to start over. I have to make an effort, hurt people, push the drama button. Who needs that? I do have the heart of a child. I keep it in the freezer, never to be thawed.

Saturday, December 24, 2016

Deserializing base type property with JSON.Net and having it work in WCF as well

Flexible JSON input Sometimes we want to serialize and deserialize objects that are dynamic in nature, like having different types of content based on a property. However, the strong typed nature of C# and the limitations of serializers make this frustrating.


In this post I will be discussing several things:
  • How to deserialize a JSON property that can have a value of multiple types that are not declared
  • How to serialize and deserialize a property declared as a base type in WCF
  • How to serialize it back using JSON.Net

The code can all be downloaded from GitHub. The first phase of the project can be found here.

Well, the scenario was this: I had a string in the database that was in JSON format. I would deserialize it using JSON.Net and use the resulting object. The object had a property that could be one of several types, but no special notation to describe its concrete type (like the $type notation for JSON.Net or the __type notation for DataContractSerializer). The only way I knew what type it was supposed to be was a type integer value.

My solution was to declare the property as JToken, meaning anything goes there, then deserialize it manually when I have both the JToken value and the integer type value. However, passing the object through a WCF service, which uses DataContractSerializer and which would throw the exception System.Runtime.Serialization.InvalidDataContractException: Type 'Newtonsoft.Json.Linq.JToken' is a recursive collection data contract which is not supported. Consider modifying the definition of collection 'Newtonsoft.Json.Linq.JToken' to remove references to itself..

I thought I could use two properties that pointed at the same data: one for JSON.Net, which would use JToken, and one for WCF, which would use a base type. It should have been easy, but it was not. People have tried this on the Internet and declared it impossible. Well, it's not!

Let's work with some real data. Database JSON:
{
  "MyStuff": [
    {
      "Configuration": {
        "Wheels": 2
      },
      "ConfigurationType": 1
    },
    {
      "Configuration": {
        "Legs": 4
      },
      "ConfigurationType": 2
    }
  ]
}

Here is the code that works:
[DataContract]
[JsonObject(MemberSerialization.OptOut)]
public class Stuff
{
    [DataMember]
    public List<Thing> MyStuff;
}

[DataContract]
[KnownType(typeof(Vehicle))]
[KnownType(typeof(Animal))]
public class Configuration
{
}

[DataContract]
public class Vehicle : Configuration
{
    [DataMember]
    public int Wheels;
}

[DataContract]
public class Animal : Configuration
{
    [DataMember]
    public int Legs;
}

[DataContract]
public class Thing
{
    private Configuration _configuration;
    private JToken _jtoken;

    [DataMember(Name = "ConfigurationType")]
    public int ConfigurationType;

    [IgnoreDataMember]
    public JToken Configuration
    {
        get
        {
            return _jtoken;
        }
        set
        {
            _jtoken = value;
            _configuration = null;
        }
    }

    [JsonIgnore]
    [DataMember(Name = "Configuration")]
    public Configuration ConfigurationObject
    {
        get
        {
            if (_configuration == null)
            {
                switch (ConfigurationType)
                {
                    case 1: _configuration = Configuration.ToObject<Vehicle>(); break;
                    case 2: _configuration = Configuration.ToObject<Animal>(); break;
                }
            }
            return _configuration;
        }
        set
        {
            _configuration = value;
            _jtoken = JRaw.FromObject(value);
        }
    }
}

public class ThingContractResolver : DefaultContractResolver
{
    protected override JsonProperty CreateProperty(System.Reflection.MemberInfo member,
            Newtonsoft.Json.MemberSerialization memberSerialization)
    {
        var prop = base.CreateProperty(member, memberSerialization);
        if (member.Name == "Configuration") prop.Ignored = false;
        return prop;
    }
}

So now this is what happens:
  • DataContractSerializer ignores the JToken property, and doesn't throw an exception
  • Instead it takes ConfigurationObject and serializes/deserializes it as "Configuration", thanks to the KnownTypeAttributes decorating the Configuration class and the DataMemberAttribute that sets the property's serialization name (in WCF only)
  • JSON.Net ignores DataContract as much as possible (JsonObject(MemberSerialization.OptOut)) and both properties, but then un-ignores the Configuration property when we use the ThingContractResolver
  • DataContractSerializer will add a property called __type to the resulting JSON, so that it knows how to deserialize it.
  • In order for this to work, the JSON deserialization of the original text needs to be done with JSON.Net (no __type property) and the JSON coming to the WCF service to be correctly decorated with __type when it comes to the server to be saved in the database

If you need to remove the __type property from the JSON, try the solution here: JSON.NET how to remove nodes. I didn't actually need to do it.

Of course, it would have been simpler to just replace the default serializer of the WCF service to Json.Net, but I didn't want to introduce other problems to an already complex system.


A More General Solution


To summarize, it would have been grand if I could have used IgnoreDataMemberAttribute and JSON.Net would just ignore it. To do that, I could use another ContractResolver:
public class JsonWCFContractResolver : DefaultContractResolver
{
    protected override JsonProperty CreateProperty(System.Reflection.MemberInfo member,
        Newtonsoft.Json.MemberSerialization memberSerialization)
    {
        var prop = base.CreateProperty(member, memberSerialization);
        var hasIgnoreDataMember = member.IsDefined(typeof(IgnoreDataMemberAttribute), false);
        var hasJsonIgnore = member.IsDefined(typeof(JsonIgnoreAttribute), false);
        if (hasIgnoreDataMember && !hasJsonIgnore)
        {
            prop.Ignored = false;
        }
        return prop;
    }
}
This class now automatically un-ignores properties declared with IgnoreDataMember that are also not declared with JsonIgnore. You might want to create your own custom attribute so that you don't modify JSON.Net's default behavior.

Now, while I thought it would be easy to implement a JsonConverter that works just like the default one, only it uses this contract resolver by default, it was actually a pain in the ass, mainly because there is no default JsonConverter. Instead, I declared the default contract resolver in a static constructor like this:
JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
    ContractResolver=new JsonWCFContractResolver()
};

Now the JSON.Net operations don't need an explicitly declared contract resolver. Updated source code on GitHub


Links that helped

Configure JSON.NET to ignore DataContract/DataMember attributes
replace WCF built-in JavascriptSerializer with Newtonsoft Json.Net json serializer
Cannot return JToken from WCF Web Service
Serialize and deserialize part of JSON object as string with WCF

Wednesday, December 21, 2016

The truth about post truth

Look at that title! If that doesn't trend on social media, I don't know what will.

Nothing spreads memes faster than an American election. The term post-truth was named the word of 2016 by Oxford Dictionaries, which is funny, considering the elections went on at the end if the year and that Oxford is in England, but it only emphasizes the impact that it had. However, like the cake, it is a lie. The truth of the matter is that Americans, like all masses of people, can't handle the truth, so they invent a more comfortable reality in which to dwell.

When Trump became president, people needed somebody to blame, someone other than themselves, of course. After all, only good things happen because of you and God, because you're awesome! And God, too, I'm sure. *The people* would never ever vote a narcissistic entertainer as their supreme leader, clearly. So they blamed social media. Now, I am not a fan of social media, but considering I've been blogging for more than a decade, I am not against it either. I've only recently and reluctantly joined Facebook, mainly for the messenger app, but when I saw the entire Internet rally against poor Zuckerberg (well, he is anything but poor, but you get the gist) I smirked, all superior and shit, because the idea was ludicrous. Not the idea that Facebook had a major impact on people's behavior, that one is totally true, but that this is a recent event, brought on by technology run amok and without checks.

Mass media is one of the pillars of American democracy, it has always swayed people one way or the other. The balance doesn't come because media is true to facts but because, like any other form of power, it is wielded by both sides equally. Facebook and the whole of Internet is just a distillation of that and when you distill shit you get... 3-methylindole - perhaps a bad metaphor. What I mean is that media has always been crap, it has always had an agenda and it was always under the control of people with power. Guttenberg himself was after all a goldsmith with political connections trying to satisfy investors. The Internet just gives you more granularity, more people to contribute in drowning facts into a sea of personal opinion.

We are not in the age of post truth, we are getting closer and closer to the actual truth which, as always, is not pretty, is not nice, is not politically correct. Instead it is painful, humbling and devastating. The truth, dear *the people*, is that this form of democracy is the worse political system that you can stomach while still functioning economically, mass media is not a pillar of anything, just another form of deadly power, and that when this political system that you wallow in turns its wheels you get people like Trump and Hillary Clinton representing you. And it's fine, because no one that is actually like you will ever reach a position of true power in any system. Normal people do not crave power, but comfort and security. Not even happiness.

So get over it, because after post-truth comes truth: you will forget about all the outrage in the customary six months and then everything will go #BackToNormal.