Important Javascript Fundamentals for OOP Developers

In this guide I have tried to put together all important nuts and bolts of Javascript. I had a really hard time learning JS, since all these parts were scattered over the net. I gradually found them over the years. I hope I have covered all of them. Please let me know in the comments if you feel I missed something important or something is incorrect.

Irrespective of the title, this guide should be helpful to all developers. However, you should be familiar with basics of JS before this guide could be helpful. This is more of a collection of JS concepts.

Loosely typed

Yup! No scratching head to decide if this should be float or double, int or short or long or even a String. Just declare the variable using – var my_variable; and be done with it. This is easiest to grasp concept of the lot.

Always ‘use strict’;

You can use the magic phrase "use strict"; at the start of a function or a JS file to turn on strict mode.

Just like any other language, JS language’s syntax, semantics and APIs are first proposed in a language specification document; which when approved is ready to be implemented by all browsers. This helps ward off incompatible implementations by different browsers and makes our, JS developers’, life easier. What a plague of incompatibility looks like can be seen in section ‘DOM is not JS’.

For JS, the language specification document is named – ECMAScript. The current version of JS we see in our modern browsers is based on ECMAScript 5. This specification describes a stricter version of JS. Frankly the non-strict version of JS allows and encourages extremely sloppy and bad coding practices, which at the end of the day will result into one hell of a messy product. The “strict” mode is much more cleaner, which all self respecting JS developers should be aware of and must use.

A full list of restrictions is available on MDN, but the most important one I feel is that all variables must be declared before use in strict mode.

So,

function () {
    'use strict';
    return a;
}

will result into error in strict mode, provided a is not already defined globally. In non-strict mode it will merrily create a variable a in global scope and continue. More about scopes later. However, I can give you a demonstration of this.

function f1() {
    a = 12;
    return a;
}
function f2() {
    return a;
}
alert(f1());
alert(f2());

Try running this code on jsfiddle.net. In both alerts you will see 12. This is because in both the functions a is in global scope.

Another important point to remember is that once strict mode is enabled, it can’t be disabled. Strict mode can be specified for a particular JS file, where that is enabled for all codes in that file; or, it could be enabled selectively inside functions.

Semi-colon is not optional

You must have read somewhere before that in JS semi-colons at the end of a statement is not required. However, this does not mean that there is absolutely no requirement of semi-colons. The language interpreter actually tries to guess where a semi-colon should have been and you missed it, and continues like that. I loath this “feature” a lot. Sometimes this results into hard to find bugs and this is allowed in strict mode too. >:(

function avg(a, b) {
    return
        (a + b) / 2
}
console.log(avg(2, 3))

What will be printed on the console? It will be undefined! An unambiguous language like C or Java makes no error here but JS does because of its overzealous attempt to “guess” when you need a semi-colon. In this case, JS “sees” this code as:-

function avg(a, b) {
    return;
        (a + b) / 2;
}
console.log(avg(2, 3));

However, if you wrote it like:-

function avg(a, b) {
    return (a
          + b) / 2
}
console.log(avg(2, 3))

Then the result would had been correct. In this case if JS tries to insert semi-colon after (a then that will result into invalid syntax as the ( needs to be closed.

The JS syntax specification goes into details about how to play this guessing game so that the guesswork works consistently across all implementations of JS – https://262.ecma-international.org/6.0/#sec-automatic-semicolon-insertion.

Hoisting and Scoping in JS

Unlike C, C++ or Java JS has only two simple scope types – Global and Function. So, in JS for, if, while, etc do not define a scope block. So, a code like:-

function () {
    if (someCondition) {
        var a;
    }
}

is actually treated as:-

function () {
    var a;
    if (someCondition) {
        // ...
    }
}

This behavior of JS is also known as hoisting. Like hoisting a flag it hoists all variable declaration to the topmost line in that scope.

Take another example.

function test() {
    'use strict';
    console.log(a);
    console.log(b);
    //console.log(x);
    var a = 10, b = 10;
    console.log(a);
    console.log(b);
}
test();

This is a unique example. The output in this case will be:-

undefined
undefined
10
10

However, if you uncomment the line console.log(x), you will end up with error – ReferenceError: x is not defined. This is because a and b are hoisted to top of function so they are present when console.log statement is run, however they were yet to be assigned values.

function test() {
    'use strict';
    console.log(a);
    console.log(b);
    var a = b = 10;
    console.log(a);
    console.log(b);
}
test();

Notice the var line. In this case not only the console.log(b) lines will error out, but also the var line, provided the outer scope doesn’t already have variable b defined. This is because in this case b is part of an expression, so var does not define b, but only a.

All variables defined outside the function blocks (be it in JS files or <script> blocks) are in global scope. Since there is only one global scope, so they all are accessible everywhere.

About functions

Functions are also objects

I will re-state this point throughout this guide. This is important to understand. The functions are objects of type Function. Like any other objects they too have methods! And like any object they too can be defined anywhere, returned from other functions or passed as arguments etc.

Take this example.

function gen() {
    return function ans(factor) {
        return 2 * factor;
    };
}

Does it look confusing? If yes, then let’s substitute the returned function by a variable.

function gen() {
    // ...
    return f;
}

Looks better? Since functions are just mere objects we can do any of the following.

function gen() {
    var f = function ans(factor) {
        return 2 * factor;
    };
    return f;
}

Or

function gen() {
    function ans(factor) {
        return 2 * factor;
    };
    return ans;
}

The assignment analogy

When you name a function (taking function f as example), like:-

function f(factor) {
    return 2 * factor;
};

then that is almost equivalent to:-

var f = function (factor) {
    return 2 * factor;
};

I say ‘almost’, since even though…

f(2);
var f = function (factor) {
    return 2 * factor;
};

will error out saying – TypeError: f is not a function, since f is actually undefined. But,…

f(2);
function f(factor) {
    return 2 * factor;
};

will not error out. Since, similar to var, function definitions too are hoisted.

Functions are not polymorphic

You cannot name two functions the same and hope to invoke either of them based on the parameter types. The function defined later will overwrite the previous. After all if you take the assignment analogy then the next function definition is going to reassign the same variable with its own function object.

function ans(f1, f2) { ... }

function ans(f1) { ... } // This replaces the previous definition.

But the point to note is, all arguments in functions are always optional.

function ans(a, b) {
    //...
}
ans(2); //ans will be invoked with a = 2, and b = undefined

Function return

In a function you may choose to return any data or nothing at all.

function () {
    if (cond1) {
        // Returns an object
        return {
            a: 10
        };
    } else if (cond2) {
        // Returns undefined
        return;
    } else if (cond3) {
        // Returns a number.
        return 1;
    }
}

What if all the conditions fail? Similar to cond2, this will return undefined.

Closure

JS has the power of lambda. Simply put lambdas are anonymous functions. This has proved to be one of the core pillars of the language. Now this has been introduced even into Java 8.

All functions in JS also have access to the outer scope, be it another function or global. It is also able to retain the outer scope even after the outer function has finished execution. This concept of hanging onto the outer scope is closure.

Java developers will be familiar with the concept of final. Where anonymous inner classes have access to final variables in outer scope and they hang onto it. This is like closure, but not 100%, since closure requires that the whole of outer scope be captured. Although JS interpreters optimize their implementation and only hold-on to the variables actually being referred. Also in true closure you are allowed to update values of variables in outer scope.

With this knowledge can you guess the output of the following?

function adder(factor) {
    return function (a) {
        return a + factor;
    };
}
var adder2 = adder(2);
console.log( adder2(5) );

If you guessed 7 then that is correct. adder2 variables refers to a function generated by adder which always adds 2 to any number passed to it.

If you find it difficult to understand then, this is what adder2 actually is:-

adder2 = function (a) {
    return a + 2;
};

Now make a guess for the following.

function gen() {
    var i = 1, f = [];
    for (; i <= 2; i++) {
        f.push(function (a) { return a + i; });
    }
    return f; // Returns an array with two functions in it.
}
var fs = gen();
console.log( fs[0](5) );
console.log( fs[1](5) );

If your answer is not 8 and 8 then it is wrong! fs[0] and fs[1] return the functions generated inside gen‘s for-loop. Remember that in this case both these functions hold-on to the same outer scope, not the values of i. When the for-loop ends the value of i is 3. So, both the functions are adding 3 to 5, and not 1 and 2 to 5.

Truthy and Falsy

Much like C and C++, but unlike Java, JS has a wide range of what can be treated as truthy or falsy. All objects (except empty string) and non-zero numbers are treated as truthy. Whereas empty string, zero, null and undefined are treated as falsy.

undefined is a special value. All variables when not assigned any value has the value undefined. Clear? 🙂 Similarly all functions which do not return a value actually return undefined. In fact it is a keyword. So, the following code is valid.

var a = undefined;

This is actually equivalent to:-

var a;

Value coercing

In JS when you try do something impossible with values then JS tries it best to make them compatible and come up with some meaningful result.

For example: !0 is actually boolean true since ! can work only with boolean values. 0 when coerced into boolean is false. '2' * 1 is actually number 2, since * cannot work on strings. But, '2' + 1 is string 21, since because of the presence of one string the number is coerced into string.

Here is a tip. Since, !0 is true. You can use this for a neat trick – var hasChildren = !!children.length;. This will set hasChildren to appropriate pure boolean value.

Prototype based programing

Unlike C, C++ or Java functions in JS are actually objects and, as an OOP developer would say, instance of a class Function. However, there are no classes in JS, just constructors. The constructors create objects by cloning another object. So, all functions in JS are clones of Function. Only functions are allowed to be constructors, i.e. new operator can be applied only on them.

In words of Douglas Crockford: you make prototype objects, and then … make new instances. Objects are mutable in JavaScript, so we can augment the new instances, giving them new fields and methods. These can then act as prototypes for even newer objects. We don’t need classes to make lots of similar objects….Objects inherit from objects. What could be more object oriented than that?

JS support two kinds of object creation – by cloning existing object (using Object.create(otherObj)) or ex nihilo (“from nothing”, using Object.create(null)). By the way, {} is the short-hand for Object.create(Object.prototype), and [] is short-hand for new Array().

Actually Object.create(obj) creates a new object (think of it like an empty shell) where obj is its prototype (this gives content to that shell). So, it doesn’t actually clone obj; instead it sets obj as its prototype. As its literal meaning, a prototype is an object from which the main object derives its properties and methods. However, you are allowed to directly add any property or method to the main object too.

Object.prototype itself is an ex nihilo object which other objects inherit, including Function.prototype. The prototype property in objects in itself is an object and can have other prototype forming a chain. More on this later. The standard way to get the prototype of an object is using Object.getPrototypeOf(obj). However, IE8 and below do not implement this. The non-standard way (also not supported by IE) is using __proto__ property. For IE and others you can use obj.constructor.prototype.

new operator

You can guess it. Similar to Java, new Foo() will create a new object of type Foo. When I say it is type of Foo, it means the object has the prototype set to Foo.prototype. As you will recall, you can do the same using Object.create() too. So, new Foo() is almost equivalent to Object.create(Foo.prototype). I say almost, since in former case Foo function is executed before the created object is returned. In later case the Foo function is not executed.

What is this?

This is one of the primary point of confusion for new JS developers. In JS the function is always executed in some context, implicit or explicit. That context decides the value of this inside the function. The same function can be invoked with any explicit context. When the context is not specified then it is ‘window’ for non-strict mode and undefined for strict mode. You can use the following code to test this.

function A() { return this; }
A(); // returns window
function B() {'use strict'; return this; }
B(); // returns undefined

Have a look below:-

var car = new Car();
car.honk('loud');

When you uses the new operator, you created a new object of type Car. When you did car.honk('loud'), first JS interpreter looked into car object for honk method, if it did not find that there, then next it would look into Car.prototype object for this method. If the method was not even there then it would look into Car.prototype.prototype object and so on. Once the method is located that method would be triggered in the context of car object. That means, in that method then, this will be car. These behaviors are part of JS language.

Recall that functions are themselves objects of type Function, which means they too have methods and can in-turn be used as objects! Functions have a method call using which you can explicitly specify the context in which to execute the function.

Car.prototype.honk.call(someObj, 'loud');

This will call honk such that this inside that refers to someObj. Practically someObj could be any object, not necessarily objects of type Car.

There is also an apply() method in Function class. The only difference between this and call() is that, here the second argument is an array of the arguments we need to send to the called function.

In next section we will put these information to use.

Prototypal inheritance is way different from OOPs

In class based inheritance the compiler takes care of making the inheritance work for you automatically. However, in prototypal inheritance (JS) it is left to developers to fend for themselves. Actually prototypal inheritance is a concept/trick developed by developers and is not something the language defines.

The main feature we expect in inheritance is the ability to inherit methods and fields from parent classes, and we should be able to override them if required.

Let’s now try to mimic this behavior in JS.

function Vehicle(color) {
    this.color = color;
}
Vehicle.prototype.honk = function() {
    alert('Honking!');
};

function Car(color) {
    Vehicle.call(this, color);
}
Car.prototype = Object.create(Vehicle.prototype);
Car.prototype.getWheelsCount = function() {
    return 4;
};

function Autorickshaw(color) { // OR TukTuk, take your pick
    Vehicle.call(this, color);
}
Autorickshaw.prototype = Object.create(Vehicle.prototype);
Autorickshaw.prototype.getWheelsCount = function() {
    return 3;
};

Above the Vehicle.call(this, color) statement executes the Vehicle function in the context of current object and passes on the color param. This way we technically made a super() call. So, this.color is like a field variable and this.honk() and this.getWheelsCount() are methods.

The prototype chain formed in this case is:-

Car.prototype -> Vehicle.prototype

Now there lots of boilerplate code above. Let’s try to cut that.

function define(superClass, definition) {
    function Class() {
        if (superClass)
            superClass.apply(this, arguments);
        if (definition.initialize)
            definition.initialize.apply(this, arguments);
    }
    if (superClass)
        Class.prototype = Object.create(superClass.prototype);
    var proto = Class.prototype;
    for (var key in definition) {
        proto[key] = definition[key];
    }
    return Class;
}

var Vehicle = define(null, {
    initialize: function(color) {
        this.color = color;
    },

    honk: function() {
        alert('Honking!');
    }
});

var Car = define(Vehicle, {
    getWheelsCount: function() {
        return 4;
    }
});

var Autorickshaw = define(Vehicle, {
    getWheelsCount: function() {
        return 3;
    }
});

The define method is pretty straight forward. Although, before I continue, note the arguments keyword. This magic variable is available inside a function. This is an “array” of all the arguments supplied to that function, when it is invoked. I say array in quotes, since this is not really a standard JS array. It has only few features and methods of an array.

This function internally defines another function which is the new class we are defining. One quick thing to note is that the function’s name is Class. That means to define its instance we should write new Class(). However, the name we write against the new operator has no significance. The name is just a reference to the action function object. So, if A = B = Class then new A() or new B() or new Class() will all yield the same result.

The function then iterates over the provided singleton object and simply copies them with the same keys to Class‘s prototype. Finally it returns that function – Class. Astute readers will notice that define feels similar to PrototypeJs’ Object.extends().

Now let’s add some features to define.

function define(superClass, definition) {
    function Class() {
        if (superClass) {
            this.$super = superClass.prototype; //New addition
            superClass.apply(this, arguments);
        }
        if (definition.initialize)
            definition.initialize.apply(this, arguments);
    }
    if (superClass)
        Class.prototype = Object.create(superClass.prototype);
    var proto = Class.prototype;
    for (var key in definition) {
        proto[key] = definition[key];
    }
    return Class;
}

We just now added a this.$super, which we can use to access super methods like we do in other languages.

var Vehicle = define(null, {
    initialize: function(color) {
        this.color = color;
    },

    honk: function() {
        alert('Honking!');
    }
});

var Car = define(Vehicle, {
    honk: function() {
        this.$super.honk(); // This will display the Honking! alert.
        alert('Beep Beep');
    }
});

You may ask, how to mimic private methods? Well we simply don’t. We prefix method name with _, denote that it is private. Convention is simpler than enforcing a rule. However, if you really want to enforce that then there is a another way to define a class.

function Vehicle(color) {
    var that = this;
    this.color = color;

    function _showColor() { // This is a private method
        alert(that.color);
    }

    this.honk = function() {
        _showColor();
        alert('Honking!');
    };
}

This will give an effect similar to what we have used till now, with the added benefit of private methods. Notice that we defined a variable that. This was needed so that _showColor() could refer to it. It simply can’t use this, since that has a special meaning.

In this approach we use the power of closures. However, you will notice that this is not as performant as previous approach, since a new function instance will be created per object of type Vehicle. Let’s how can Car inherit this.

function Car(color) {
    Vehicle.call(this, color);
    this.getWheelsCount = function () {
        return 4;
    };
}
Car.prototype = new Vehicle();

Here is the key difference. This type Car‘s prototype is not Vehicle.prototype but object of Vehicle.

The prototype chain formed in this case is:-

Car.prototype -> new Vehicle() -> Vehicle.prototype

There still another way to define a class.

function Vehicle(color) {
    this.color = color;
}
Vehicle.prototype = {
    honk: function() {
        alert('Honking!');
    }
};

Here we replaced the default prototype object with another object. Nothing changed, but this is more convenient to read and type.

ECMAScript 6 has a proposal to support class and extends keywords. Finally, in future, we might get actual classes support in JS.

The instanceof operator

Java developers will instantly recognize this. Same as in Java, this operator evaluates to true if the object on left is of a given class type (given on right). It’s syntax is:-

object instanceof function

This will make JS follow the prototype chain of object to look for function.prototype. So,

console.log(car instanceof Car); //Is true
console.log(car instanceof Vehicle); //Is true
console.log(car instanceof Autorickshaw); //Is false

But,

var o = Object.create(null);
console.log(o instanceof Object); // Is false

This is false since o is ex nihilo object and is not an object of anything. So you can say, it is just a shell. This too can be used as map, similar to {}, which is an object of type Object.

JS in browser is single threaded

JS language does not dictate that the interpreter be single threaded, in fact many server-side interpreters like NodeJs are not. However, the browser interpreters are single threaded. (Modern browsers now support Web Workers API which can launch background threads.)

This is an important fact to keep in mind. So, no matter when the ajax call completes, the callback you provided for that is not going to fire unless you are not done with what you are doing right now.

Also note when JS is processing, the browser gets locked up. It can’t respond to user inputs at that instant. So, if you have long running JS job then possibly the browser will stop responding. If you have modern browser then use Web Workers for that or divide your task into smaller chunks and periodically invoke setTimeout() to give control back to browser before continuing with your next chunk of task. I have a project CInk, which does some heavy rendering task; where I use this trick.

DOM is not JS

The magic objects we usually see in browser JS, like – window, document, etc. are DOM objects defined by the browser. DOM stands for Document Object Model. This is a tree data model to represent the Html code. These objects are injected by browser into JS domain, and they are not defined by JS language specification. DOMs have their own specification – Document Object Model (DOM) Level 1, 2 and 3. This specification was formed after ECMAScript was standardised1.

Unfortunately DOMs have a wide list of APIs which varies from browser to browser. For a short list you can see DOM Core on QuirksMode.

However, I am not here to talk about what is DOM. I only want to hammer in the point that DOM and JS are separate things. So, when coding on server-side JS, do not expect these DOM objects. You might find other magic global objects there.

Concluding

Before I let you go; here is a fun little project I worked on and which needs your help. I tried to implement the JS engine from scratch! That was a fun endeavour until I ran out of free time and motivation. I only managed to get the parser work. It is able to parse ES6 complaint JS codes and build AST tree. The parser is coded by hand and is based on the ES6 Grammar Specs. It is called Just (JS on Rust).

I hope this helps, and happy coding. 🙂

Tracing $scope events in AngularJS

Scope events in AngularJs is very powerful and useful in connecting disparate modules and directives. However, in a complex application things could become complicated and it would become hard to trace which event is triggered when and who act on those events.

I wrote a simple event tracer which displays a floating div with logs of all the events. The log is color coded, based on if the event was emitted, broadcasted or received.

The log looks like this:-

Event Trace

Event Trace

You can turn on or off the event data using the dataf link. The stackf toggles if the call stack or function code too will be shown which has emitted or consumed this event.

The code:-

var app = angular.module('tracer', []).run(Injector);

function Injector($rootScope) {
    'use strict';

    function log(msg, color) {
        var el = document.getElementById('tracerConsoleUl');
        el.innerHTML += itemHtml.format(color, msg)
    }

    function stringify(o) {
        try {
            return JSON.stringify(o, null, 3);
        } catch (err) {
            console.error('Scope Hack:', err, ' Actual object:', o);
            return '<i>Err: See console</i>';
        }
    }

    function getStackTrace() {
        var r;
        if (Error.captureStackTrace) {
            var obj = {};
            Error.captureStackTrace(obj, getStackTrace);
            r = obj.stack;
        } else if (Error.stack) {
            r = Error().stack;
        } else {
            r = '';
        }

        r = r.replace(/</gm, '&lt;').replace(/>/gm, '&gt;');
        r = r.replace(/^[^\s]+.+$/gm, ''); // removing first line
        r = r.replace(/^\s+at (.+) \([^(]+\)+$/gm, '$1');
        r = r.replace(/^\s+at [^()]+$/gm, ''); // removing rows with only text of format at ... and nothing in braces on right.
        r = r.replace(/\n\n/gm, '\n');
        r = r.replace(/\n/gm, ' &lt; ');

        return r;

    }

    function filter(e, clazz) {
        e.stopPropagation();
        e.preventDefault();

        var flag = angular.element('#tracerConsoleUl').hasClass(clazz);
        if (flag)
            angular.element('#tracerConsoleUl').removeClass(clazz);
        else
            angular.element('#tracerConsoleUl').addClass(clazz);
    }

    function fclear(e) {
        e.stopPropagation();
        e.preventDefault();

        angular.element('#tracerConsoleUl').empty();
    }

    function fmoveStart(e) {
        e.stopPropagation();
        e.preventDefault();

        angular.element(document).on('mousemove.scopehack', fmove).on('mouseup.scopehack', fmoveEnd);
        var el = angular.element('#tracerConsole');
        var left = e.pageX - parseInt(el.css('left'));
        var top = e.pageY - parseInt(el.css('top'));
        el.data('left', left);
        el.data('top', top);
    }

    function fmoveEnd(e) {
        e.stopPropagation();
        e.preventDefault();

        angular.element(document).off('.scopehack');
    }

    function fmove(e) {
        e.stopPropagation();
        e.preventDefault();

        var el = angular.element('#tracerConsole');

        var p = {
            left: (e.pageX - el.data('left')) + 'px',
            top: (e.pageY - el.data('top')) + 'px'
        };

        angular.element('#tracerConsole').css(p);

    }

    try {
        if (!String.prototype.format) {
            String.prototype.format = function() {

                var args = arguments;

                return this.replace(/{(\d+)}/g, function(match, number) {
                    return typeof args[number] != 'undefined' ? args[number] : '<i>NA</i>';
                });
            };

        }


        var html = "<div id='tracerConsole' style='position:absolute;top:50px;left:50px;background-color:white;border:2px solid black;z-index:1000;'>" +
            "<div><span class='ffilter' style='cursor:pointer;'>stackf</span> | <span class='dfilter' style='cursor:pointer;'>dataf</span> | <span class='fclear' style='cursor:pointer;'>clear</span> | <span class='fmove' style='cursor:move;'>move</span>" +
            "</div><ul id='tracerConsoleUl' style='display:block;list-style-type:none;max-height:500px;width:300px;overflow:scroll;'></ul></div>";



        var itemHtml = "<li style='border-bottom:1px solid gray;white-space:pre-wrap;background-color:{0};'>{1}</li>";

        var proto = Object.getPrototypeOf($rootScope);
        var oldOn = proto.$on;
        proto.$on = function mangaledOn(e, f) {
            var fWrapper = function fWrapper(e, d) {
                log('EVENT RECEIVED: {0} <span class="d">\nWITH DATA: {1}</span> <span class="f">\nBY f: {2}</span>'.format(e.name, stringify(d), f.toString()), '#7FD7B6');
                f.call(this, e, d);
            };

            oldOn.call(this, e, fWrapper);
        };

        var oldBroadcast = proto.$broadcast;
        proto.$broadcast = function mangaledBroadcast(e, d) {
            log('EVENT BROADCASTED: {0} <span class="d">\nWITH DATA: {1}</span> <span class="f">\nBY s: {2}</span>'.format(e, stringify(d), getStackTrace()), '#FF9C01');

            oldBroadcast.call(this, e, d);

        };

        var oldEmit = proto.$emit;
        proto.$emit = function mangaledEmit(e, d) {
            log('EVENT EMITTED: {0} <span class="d">\nWITH DATA: {1}</span> <span class="f">\nBY s: {2}</span>'.format(e, stringify(d), getStackTrace()), '#FFE78C');

            oldEmit.call(this, e, d);
        };

        console.log('Scope Hack Injected');
        angular.element('body')[0].innerHTML += html;
        angular.element('#tracerConsole .dfilter').on('click', function dfilter(e) {
            filter(e, 'dhide');
        });

        angular.element('#tracerConsole .ffilter').on('click', function ffilter(e) {
            filter(e, 'fhide');
        });
        angular.element('#tracerConsole .fclear').on('click', fclear);
        angular.element('#tracerConsole .fmove').on('mousedown', fmoveStart);
        angular.element(document).find('head').prepend('<style type="text/css">#tracerConsoleUl.dhide .d {display:none;}#tracerConsoleUl.fhide .f {display:none;} #tracerConsoleUl li{-moz-user-select:text;-webkit-user-select:text;-ms-user-select:text;}</style>');

    } catch (err) {
        console.error('Scope Hack:', err);
    }
}

To use this, paste this into some script block or some js file and make sure your app’s module depends on this module – tracer.

Using pinned self-signed SSL certificate with Android Volley

Recently for one of my Android apps I wanted to use a self-signed certificate on the server-side. However, if you try to connect to such a server with default settings, the connection is going to be refused. This is because it has been signed by you (your server domain), and ‘you’ is not trusted by any system.

Read the excellent article – Android security – Implementation of Self-signed SSL certificate for your App. This describes all the concepts and the pros and cons of pinning. However, it does not describe how to do this in Volley. The internet is littered with many articles on this, and most of them are outdated and can’t be used now.

So, here is an updated guide. I have tested this on my app which uses Android API 22 and Volley code downloaded on Jan 2015.

First use the official guide to create a singleton class to get the request queue. Now below is the modified code which takes care of the SSL pinning.

public class MySingleton {
    private static char[] KEYSTORE_PASSWORD = "YourKeyStorePass".toCharArray();

    ...

    public RequestQueue getRequestQueue() {
        if (mRequestQueue == null) {
            // getApplicationContext() is key, it keeps you from leaking the
            // Activity or BroadcastReceiver if someone passes one in.
            mRequestQueue = Volley.newRequestQueue(mCtx.getApplicationContext(), new HurlStack(null, newSslSocketFactory()));
        }
        return mRequestQueue;
    }

    ...

    private SSLSocketFactory newSslSocketFactory() {
        try {
            // Get an instance of the Bouncy Castle KeyStore format
            KeyStore trusted = KeyStore.getInstance("BKS");
            // Get the raw resource, which contains the keystore with
            // your trusted certificates (root and any intermediate certs)
            InputStream in = mCtx.getApplicationContext().getResources().openRawResource(R.raw.codeprojectssl);
            try {
                // Initialize the keystore with the provided trusted certificates
                // Provide the password of the keystore
                trusted.load(in, KEYSTORE_PASSWORD);
            } finally {
                in.close();
            }

            String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
            TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
            tmf.init(trusted);

            SSLContext context = SSLContext.getInstance("TLS");
            context.init(null, tmf.getTrustManagers(), null);

            SSLSocketFactory sf = context.getSocketFactory();
            return sf;
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    }
}

Please note, that the above solution will work for Android API 9 and above. To support the below versions you need to pass an instance of HttpClientStack instead of HurlStack. So, replacing the line

mRequestQueue = Volley.newRequestQueue(mCtx.getApplicationContext(), new HurlStack(null, newSslSocketFactory()));

with

mRequestQueue = Volley.newRequestQueue(mCtx.getApplicationContext(), new HttpClientStack(new MyHttpClient()));

might work. Where MyHttpClient is the one defined in the CodeProject article. I haven’t tested this part though.

ADF: Entity-State vs Post-State

There are lots of blog posts on this but all copy and paste the content of EntityImpl javadoc, which is very obscure. I finally found a sensible explanation in Dive into Oracle ADF, but that is usually buried deep in Google results.

I will restate the Dive into Oracle ADF post here, and will add few finer details.

In DB you can run the DML queries as many times as you want without committing them. When we are saving data to DB the call chain is like – transaction.postChanges() -> entity.postChanges() -> entity.prepareForDML() -> entity.doDML(). The entity’s DML operations depend on the post-state. So, transaction.postChanges() effectively make all entities to write their data to DB. They do that by checking their internal post-states and creating appropriate DML queries based on those states. After the DML action the post-state is updated. So, if before postChanges() if the post-state was STATUS_NEW, then after that it would be STATUS_UNMODIFIED, so that on next postChanges() call it doesn’t try to insert the row again.

Entity-state on the other hand represents the entity’s state irrespective of the fact that the changes has been posted (written) to the DB. So, even if all changes in entity are posted, the entity-state remains STATUS_MODIFIED. Entity-state changes only after commit is invoked on the transaction. Unless and until a transaction is committed the changes done in the transaction is not visible in other transactions (this is a feature of relational DB), so, when entity-state changes we know that those changes are now visible to all transactions.

An example:

#!java
row = vo.first();  /* read a row in from DB.  Both EO states are UNMODIFIED */

row.setAttribute("SomeAttr", someValue); /* After this, both states are MODIFIED */

am.getTransaction().postChanges(); /* Changes are written to DB. Trans not committed yet */
                                   /* After this, post-state is UNMODIFIED. Entity-state is MODIFIED */

am.getTransaction().commit(); /* Transaction committed. After this, both states are UNMODIFIED */

If you see EntityImpl’s javadoc then you will notice that we have one extra post-state – STATUS_INITIALIZED. Entities have this post-state only when it is newly created. The moment one of its attribute is set, this changes to STATUS_NEW. When postChanges() is invoked then DML action is skipped if post-state is STATUS_INITIALIZED. This prevents blank rows from getting inserted. Entity-state never has this value since entity-state is meant to be used for writing business logic, and from that perspective STATUS_NEW and STATUS_INITIALIZED makes no difference.

ADF: EntityImpl.refresh(…) has no effect

Many times we use entity.refresh(REFRESH_FORGET_NEW_ROWS | REFRESH_UNDO_CHANGES) to prevent any changes we made to that entity from getting committed to the DB. What the refresh() method effectively does is change the post state of the entity. (See difference between entity-state and post-state.) The post-state is later used to decided which DML operation to use for the entity, i.e. DML_DELETE, DML_INSERT or DML_UPDATE (ref).

I recently came across a code which called refresh() from the entity’s prepareForDML() but that had no effect. ADF documentation says nothing about such a behaviour. I read the EntityImpl code and got my answer there. When we are saving data to DB the call chain is like – transaction.postChanges() -> entity.postChanges() -> entity.prepareForDML() -> entity.doDML().

prepareForDML() and doDML() have first argument operation; this is the DML operation to undertake. The code to decide which DML operation to perform based on entity’s post-state is in EntityImpl.postChanges(). That method invokes prepareForDML() method with the DML operation to undertake. Changing post-state from prepareForDML() or doDML() is useless. So, post-state needs to be set no later than postChanges(). However, if you do need to do this from prepareForDML() or doDML() then you could modify the operation param’s value appropriately.