mXHR Playground

Recently I extended supplyJS to support streaming images. (This actually was implemented all the time, I just added the listener for the MIME types)

I guess streaming javascript & stylesheet files is not expressive as streaming images (in terms of measurable performance). So now you can test it yourself:

http://www.typeofnan.com/lab/mxhr-stream/

I'd be more than happy to get some responses from you guys. If you're in the mood, please post your Browser / version / performance to the comments!

! Important !
To make this work, I'm using base64 encoded images. The Internet Explorer does not handle base64 encoded images that well, so it might not work (properly)

Thanks.

"typeof " is fast ?

Recently, I read an article about the such called "micro optimizations". The article was about when to use typeof and when just to check against the undefined value, if you want to know whether a variable/property is undefined or not.

The conclusion was, to use typeof for any "global" variable and a direct conditional check against undefined within the scope of a function where you know the variables are at least declared. That makes sense, because if( some_undefined_variable !== undefined ) throws a reference error.

I played around with that in mind and I thought, this might be interesting for fellow Javascripters. My first testcase looked like this:

var loops = 1e6,
    foo;

console.time('typeof');
while(loops--) {
    if( typeof foo !== 'undefined' ) {}
}
console.timeEnd('typeof');

loops = 1e6;

console.time('direct');
while(loops--) {
    if( foo !== undefined) {}
}
console.timeEnd('direct');

with the result of:

typeof: 421ms
direct: 603ms

Pretty interesting. So here it looks like, typeof is faster than just comparing a variable against undefined. I couldn't really believe what I've seen here. I thought that is probably because I'm in the global context and my Javascript engine (SpiderMonkey) does some kind of wierd scope chain lookups for "undefined". Anyway, it should improve the overall performance to put that code into a function (-context), which I did:

(function() {
    var loops = 1e6,
        foo;

    console.time('typeof');
    while(loops--) {
        if( typeof foo !== 'undefined' ) {}
    }
    console.timeEnd('typeof');

    loops = 1e6;

    console.time('direct');
    while(loops--) {
        if( foo !== undefined) {}
    }
    console.timeEnd('direct');
}());

Results:

typeof: 54ms
direct: 185ms

As expected, this has a better overall performance, but still typeof is way faster than a plain conditional check vs. undefined. Well, the important thing I forgot here is to keep a local reference (some people call it: cache) for the undefined value:

(function(undef) {
    var loops = 1e6,
        foo;

    console.time('typeof');
    while(loops--) {
        if( typeof foo !== 'undefined' ) {}
    }
    console.timeEnd('typeof');

    loops = 1e6;

    console.time('direct');
    while(loops--) {
        if( foo !== undef) {}
    }
    console.timeEnd('direct');
}());

Final results:

typeof: 51ms
direct: 38ms

Yay! That's the way to go and that's why you always should try to cache objects / object propertys before accessing (if its not a problem to have a static reference).
My default Javascript opener when using jQuery looks like:

(function(win, doc, $, undef) {
    // lots of beautiful code that can access win(window), 
    // doc(document), $(jQuery) and undef(undefined)
}(this, this.document, jQuery));

However, to be honest I'm still not 100% sure why this affects the undefined value that much. Cached references are faster to access (fact), totally makes sense for objects and propertys, but the undefined value is not a property as far as I know.

update:

I was wrong. undefined really is a property in the global object (window).

'undefined' in window; // = true

This explains why the cached version of the undefined value is way faster. Thanks to Stackoverflow member CMS who pointed me on this fact here.

supplyJS - Updated

I recently updated SupplyJS, hooray!

There are some minor fixes and changes along with a new feature. SupplyJS now caches transfered files into the localStorage (if available). The underlying principle is pretty easy, when the Supply dealerscript fetches the file contents, it checks the mtime (last modified) and compares it with the value the Supply Javascript passed over. This happens for each file.
So the frontend part looks into the localStorage and checks if there already is an entry for a specific file and if so, send it's mtime value to the dealer.

--Snippet--

if( settings & options.localStorage ) {
    for(i = 0; i < len; i++) {
        ret.push([n, '=', arr[i], '~', (arr[i] in localStorage) ? JSON.parse(localStorage[arr[i]]).modified : '0'].join('')); 
    }
}

The serverscript for that part looks like (example in perl)

($filename, $modified) = split(/~/, $file);   
   $mtime = (stat($javascript_dir . '/' . $filename))->[9];
  
   if( int($mtime) > int($modified) || int($modified) == 0 ) {    
    open (JSFILE, $javascript_dir . '/' . $filename) or next;   
    
    while(<jsfile>) {
     $jscontent .= $_; 
    }
    
    close JSFILE;
   }
   else {
    $jscontent = 'cached';
   }

..and finally, the listeners for javascript & stylesheet files were adapted:

self.listen('text/javascript', function(payload, filename, mtime){
   if( settings & options.localStorage ) {
    if( payload === 'cached' ) {
     payload = JSON.parse(localStorage[filename]).src;
    }
    else {
     setTimeout(function() { 
      localStorage[filename] = JSON.stringify({
       modified: mtime,
       src:  payload
      });
     }, cacheDelay += 300);
    }
   }
   
   try{
    if( settings & options.useeval ){
     eval(payload);
    }
    else {
     var head    = document.getElementsByTagName('head')[0] || document.documentElement,
      nscr    = document.createElement('script');
      
      nscr.type   = 'text/javascript';
      nscr[fillMethod] = payload;
      nscr.setAttribute('name', filename);
      
     head.insertBefore(nscr, head.firstChild);
     
     if( settings & options.removescripts ) head.removeChild(nscr);
    }
   } catch(err) {
    if( 'console' in window ) console.error(filename, ': Exception. Details -> ', err);             
   }
  });

As you can see, the localStorage access happens asynchronously with setTimeout. That timeout increases with each file to avoid too heavy disc operations after loading has completed.

I broke my promise to provide some more backend-scripts for Supply (so far!), but I didn't forget about it. Anyway I still would highly appreciate some support from you guys here. Any port of the backend part is very welcome.

Cheers,  Andy

https://github.com/jAndreas/Supply

About Javascript shortcuts & trickery

There are lots of ways to write ECMA-/Javascript in shorthand. Some of those are just for the lazy ones, some of them are both, useful and short. And if I'm saying useful in that context, I mean fast(er).

Ladys and Gentlemen, here are my Top 3 Javascript shortcuts:

• The double bitwise not (~~)

This is a pretty neat replacement / enhancement for Math.floor(). Example:

var rand = ~~(Math.random() * 100);

What happens here?
~ is a bitwise operator. Everytime you're using a bitwise operator in ECMA-/Javascript, the number of invocation is converted into a 32 bit integer internally (normally all numbers are hold as 64 bit integers).
Finally it negates each '0' to '1' and each '1' to '0'. Long story short, this only remains true for integers. It also has  a tremendous better performance than Math.floor(). So for instance 24.7157 is translated into 24.

• The double negate (!!)

If you need to have a boolean expression from a given value, regardless which type it may be, you can call:

var someObject   = {foo: 'bar'},
    isFoo        = !!someObject.foo;

isFoo now is either true or false, regardless what foo contains, could be an Array, an integer, a string or even undefined. This can come in handy, if you want to check if a specific variable was passed into a function:

var myFunc = function(foo, bar) {
    var bar = !!bar;
    if( bar ) { /* ... */ }
};

You could say, well, I can do this without that !! thing. That is true, but you would have to explicitly make a typeof check against 'undefined' in this case.

• The plus cast (+)

Sometimes, you want to have an integer value, even if you know that you are dealing with a string for instance. Most people would come up with the idea to call .parseInt() to convert that string into an integer. Among other things why .parseInt() is evil (lot's of people forgot to pass the radix value which decides which numeral system to use) it has a pretty "wrong" behavior. For instance:

var foo      = "55abc",
    mynumber = parseInt(foo, 10);

mynumber would contain 55. This is, for my understanding, pretty wrong. It should be NaN!
What can we do about it ? We use this:

var foo      = "55abc",
    mynumber = +foo;

Now, mynumber really is NaN. If foo would contain "55", the plus operator would cast it correctly into an integer. This can also be used on booleans. +true evaluates to 1, +false evaluates to 0.

How to sort an Array of Strings ?

Recently I had the requirement to sort an Array, alphabetically. My first thoughts were, "pretty trivial with Array.prototype.sort(), ehh!".

As it turned out, it's not that trivial at all. First thing to mention here is, that the particular Array consisted out of Objects, which among other things, contained a string property.
The requirement was, to sort the Array based on this property.

Because of that, I couldn't ordinary call Array.sort(), I needed to pass in a custom sorting logic. So the first idea was just to call this:

// --
var arr = [{value: 'Foo'}, {value: 'Javascript'}, {value: 'MooTols'}, {value: 'Zebra'}, {value: 'Ape'}];

arr.sort(function(a,b) {
    return a.value > b.value;
});
// --

At first sight, this works pretty well. But what happens to this "operator based" comparison, if you have special characters, umlauts or numbers like '05' and '11' at the beginning of the strings? It will fail. That means the order will not be as you might expect it to be. For that reason, Javascript 1.2 introduced a String method called 'localeCompare(compareString)'.

From https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/String/localeCompare

Description

Returns a number indicating whether a reference string comes before or after or is the same as the given string in sort order. Returns -1 if the string occurs earlier in a sort thancompareString, returns 1 if the string occurs afterwards in such a sort, and returns 0 if they occur at the same level.

So our sorting function should look like:

// --
arr.sort(function(a,b) {
    return a.value.localeCompare(b.value);
});
// --

On the performance side, a quick measuring showed that the operator version is faster on Firefox & Safari. Chrome has a better performance with localeCompare. Don't forget that the former version brings wrong results, so you actually should always go with localeCompare.

http://jsperf.com/operator-vs-localecompage

See you next time!

- Andy

A MXHR loader - Part 3 - The Codes !

<script src="/js/supply.min.js?v=2"></script>
  <script type="text/javascript"> 
      supply.setDealer('/cgi-bin/supply.pl').files({
         debug: false,
         javascript: [
            '/js/jquery-1.4.2.min.js',
            '/js/jquery-ui-1.8.4.custom.min.js',
            '/js/init.min.js',
            '/js/box.min.js',
            '/js/app.min.js'
         ]   
     });    
  </script> 

That's how your script loading could look like in the future.
What happens here is simple. "supply.min.js" is loaded via the conventional method. After that, "supply" is available in the global namespace. The setDealer() method can optionally be called to tell the script where it can find it's backend counter part. Finally we call files() and pass in an object literal with the key "javascript" which value contains an array of javascript filenames.

Now supply creates a request to "supply.pl". Our Dealerscript does it's work, fetching and concatenating all our desired files on the server and sends us one big chunk of data back. Voila.

Supply.js codereview:





I have to apologize for the quality, it's my first record. Hope you like it anyway.

Source:

https://github.com/jAndreas/Supply

Download:

https://github.com/jAndreas/Supply/zipball/master

A MXHR loader - Part 2 - How does it work

In the last post I described the overhead problem when transfering lots of files in your browser. The basic idea of Multipart XHR is to have a little dealer script (that is my personal denotation) which sits and waits for requests. A request could look like:

Javascript: "Hey dealer, deliver me foobar.jpg, crockford.jpg, example.html and test.js"

That dealer script is doing nothing else but opening, reading and concatenating the files content into one big chunk of data. For images it needs to encode the data as base64 since we are only transfering a "string".
One more exercise for that script is to "mark" each data block with a MIME-Type, so our Javascript knows how to interpret that incoming data. Last thing to do here is to sepperate each data chunk. Sounds easier than it is because we're reading and concatening all kind of data (maybe even binary data encoded as base64), so we can't just declare a slash (or whatever) as our boundary character. We need a dead sure delimiter. As it turns out, the prime vathers of ASCII will help us here.

"ASCII reserves the first 32 codes (numbers 0–31 decimal) for control characters: codes originally intended not to represent printable information, but rather to control devices (such as printers) that make use of ASCII, or to provide meta-information about data streams such as those stored on magnetic tape."

Sounds pretty sweet. And it's indeed a great way to separate a stream. in Perl and PHP you can just call

chr(1) . "some text";

That would concat "some text" to the first ASCII control character (SOH). In Javascript we can encode it as unicode character like

var fieldDelimiter = '\u0001';

That dealer script can be written in any language, since it is really doing trivial work. I've written versions for Perl and PHP only so far. Since I'm releasing the complete code on Github the time I release this post's, feel free to contribute. I'd love to see a Rails or NodeJS version.

That whole concept is not actually new. I believe one of the very first releases and experiments were done from the Digg User Interface Library guys. I picked up the topic from the book "High Performance Javascript" by Nicholas C. Zakas
So a ton of credits goes to these guys.

Back to topic. The reason for using MXHR instead of normal XHR is performance. It's just faster transfering n-files in one request instead of n-requests (see Part 1).
But if we're after performance, we should look for more! So now we're receiving one big chunk of data instead of several small ones. That data stream is delimited with ASCII control characters so we should try to do something useful with the data as soon as we got one complete chunk, instead of waiting until we fetched the whole data string. That is where the XMLHttpRequest "Interactive" mode comes in handy. So when receiving our data we do ask the browser to inform us when data has arrived. We can do this by checking the readyState of the XHR object. If it's on "3", the responseText should contain any data that was received so far, see The XMLHttpRequest Object from W3C.

All major browsers do support readyState3 nowadays. Older versions (especially Internet Explorers) might have some trouble on this. On that browsers we have to workaround this issue. But if the browser supports it, we gain even more speed on loading data on the client.

Part 3 will finally show some code snippets and the complete resource of "Supply", my MXHR loader script.