supplyJS - Updated

I recently updated SupplyJS, hooray!

There are some minor fixes and changes along with a new feature. SupplyJS now caches transfered files into the localStorage (if available). The underlying principle is pretty easy, when the Supply dealerscript fetches the file contents, it checks the mtime (last modified) and compares it with the value the Supply Javascript passed over. This happens for each file.
So the frontend part looks into the localStorage and checks if there already is an entry for a specific file and if so, send it's mtime value to the dealer.

--Snippet--

if( settings & options.localStorage ) {
    for(i = 0; i < len; i++) {
        ret.push([n, '=', arr[i], '~', (arr[i] in localStorage) ? JSON.parse(localStorage[arr[i]]).modified : '0'].join('')); 
    }
}

The serverscript for that part looks like (example in perl)

($filename, $modified) = split(/~/, $file);   
   $mtime = (stat($javascript_dir . '/' . $filename))->[9];
  
   if( int($mtime) > int($modified) || int($modified) == 0 ) {    
    open (JSFILE, $javascript_dir . '/' . $filename) or next;   
    
    while(<jsfile>) {
     $jscontent .= $_; 
    }
    
    close JSFILE;
   }
   else {
    $jscontent = 'cached';
   }

..and finally, the listeners for javascript & stylesheet files were adapted:

self.listen('text/javascript', function(payload, filename, mtime){
   if( settings & options.localStorage ) {
    if( payload === 'cached' ) {
     payload = JSON.parse(localStorage[filename]).src;
    }
    else {
     setTimeout(function() { 
      localStorage[filename] = JSON.stringify({
       modified: mtime,
       src:  payload
      });
     }, cacheDelay += 300);
    }
   }
   
   try{
    if( settings & options.useeval ){
     eval(payload);
    }
    else {
     var head    = document.getElementsByTagName('head')[0] || document.documentElement,
      nscr    = document.createElement('script');
      
      nscr.type   = 'text/javascript';
      nscr[fillMethod] = payload;
      nscr.setAttribute('name', filename);
      
     head.insertBefore(nscr, head.firstChild);
     
     if( settings & options.removescripts ) head.removeChild(nscr);
    }
   } catch(err) {
    if( 'console' in window ) console.error(filename, ': Exception. Details -> ', err);             
   }
  });

As you can see, the localStorage access happens asynchronously with setTimeout. That timeout increases with each file to avoid too heavy disc operations after loading has completed.

I broke my promise to provide some more backend-scripts for Supply (so far!), but I didn't forget about it. Anyway I still would highly appreciate some support from you guys here. Any port of the backend part is very welcome.

Cheers,  Andy

https://github.com/jAndreas/Supply

About Javascript shortcuts & trickery

There are lots of ways to write ECMA-/Javascript in shorthand. Some of those are just for the lazy ones, some of them are both, useful and short. And if I'm saying useful in that context, I mean fast(er).

Ladys and Gentlemen, here are my Top 3 Javascript shortcuts:

• The double bitwise not (~~)

This is a pretty neat replacement / enhancement for Math.floor(). Example:

var rand = ~~(Math.random() * 100);

What happens here?
~ is a bitwise operator. Everytime you're using a bitwise operator in ECMA-/Javascript, the number of invocation is converted into a 32 bit integer internally (normally all numbers are hold as 64 bit integers).
Finally it negates each '0' to '1' and each '1' to '0'. Long story short, this only remains true for integers. It also has  a tremendous better performance than Math.floor(). So for instance 24.7157 is translated into 24.

• The double negate (!!)

If you need to have a boolean expression from a given value, regardless which type it may be, you can call:

var someObject   = {foo: 'bar'},
    isFoo        = !!someObject.foo;

isFoo now is either true or false, regardless what foo contains, could be an Array, an integer, a string or even undefined. This can come in handy, if you want to check if a specific variable was passed into a function:

var myFunc = function(foo, bar) {
    var bar = !!bar;
    if( bar ) { /* ... */ }
};

You could say, well, I can do this without that !! thing. That is true, but you would have to explicitly make a typeof check against 'undefined' in this case.

• The plus cast (+)

Sometimes, you want to have an integer value, even if you know that you are dealing with a string for instance. Most people would come up with the idea to call .parseInt() to convert that string into an integer. Among other things why .parseInt() is evil (lot's of people forgot to pass the radix value which decides which numeral system to use) it has a pretty "wrong" behavior. For instance:

var foo      = "55abc",
    mynumber = parseInt(foo, 10);

mynumber would contain 55. This is, for my understanding, pretty wrong. It should be NaN!
What can we do about it ? We use this:

var foo      = "55abc",
    mynumber = +foo;

Now, mynumber really is NaN. If foo would contain "55", the plus operator would cast it correctly into an integer. This can also be used on booleans. +true evaluates to 1, +false evaluates to 0.

How to sort an Array of Strings ?

Recently I had the requirement to sort an Array, alphabetically. My first thoughts were, "pretty trivial with Array.prototype.sort(), ehh!".

As it turned out, it's not that trivial at all. First thing to mention here is, that the particular Array consisted out of Objects, which among other things, contained a string property.
The requirement was, to sort the Array based on this property.

Because of that, I couldn't ordinary call Array.sort(), I needed to pass in a custom sorting logic. So the first idea was just to call this:

// --
var arr = [{value: 'Foo'}, {value: 'Javascript'}, {value: 'MooTols'}, {value: 'Zebra'}, {value: 'Ape'}];

arr.sort(function(a,b) {
    return a.value > b.value;
});
// --

At first sight, this works pretty well. But what happens to this "operator based" comparison, if you have special characters, umlauts or numbers like '05' and '11' at the beginning of the strings? It will fail. That means the order will not be as you might expect it to be. For that reason, Javascript 1.2 introduced a String method called 'localeCompare(compareString)'.

From https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/String/localeCompare

Description

Returns a number indicating whether a reference string comes before or after or is the same as the given string in sort order. Returns -1 if the string occurs earlier in a sort thancompareString, returns 1 if the string occurs afterwards in such a sort, and returns 0 if they occur at the same level.

So our sorting function should look like:

// --
arr.sort(function(a,b) {
    return a.value.localeCompare(b.value);
});
// --

On the performance side, a quick measuring showed that the operator version is faster on Firefox & Safari. Chrome has a better performance with localeCompare. Don't forget that the former version brings wrong results, so you actually should always go with localeCompare.

http://jsperf.com/operator-vs-localecompage

See you next time!

- Andy

A MXHR loader - Part 3 - The Codes !

<script src="/js/supply.min.js?v=2"></script>
  <script type="text/javascript"> 
      supply.setDealer('/cgi-bin/supply.pl').files({
         debug: false,
         javascript: [
            '/js/jquery-1.4.2.min.js',
            '/js/jquery-ui-1.8.4.custom.min.js',
            '/js/init.min.js',
            '/js/box.min.js',
            '/js/app.min.js'
         ]   
     });    
  </script> 

That's how your script loading could look like in the future.
What happens here is simple. "supply.min.js" is loaded via the conventional method. After that, "supply" is available in the global namespace. The setDealer() method can optionally be called to tell the script where it can find it's backend counter part. Finally we call files() and pass in an object literal with the key "javascript" which value contains an array of javascript filenames.

Now supply creates a request to "supply.pl". Our Dealerscript does it's work, fetching and concatenating all our desired files on the server and sends us one big chunk of data back. Voila.

Supply.js codereview:





I have to apologize for the quality, it's my first record. Hope you like it anyway.

Source:

https://github.com/jAndreas/Supply

Download:

https://github.com/jAndreas/Supply/zipball/master

A MXHR loader - Part 2 - How does it work

In the last post I described the overhead problem when transfering lots of files in your browser. The basic idea of Multipart XHR is to have a little dealer script (that is my personal denotation) which sits and waits for requests. A request could look like:

Javascript: "Hey dealer, deliver me foobar.jpg, crockford.jpg, example.html and test.js"

That dealer script is doing nothing else but opening, reading and concatenating the files content into one big chunk of data. For images it needs to encode the data as base64 since we are only transfering a "string".
One more exercise for that script is to "mark" each data block with a MIME-Type, so our Javascript knows how to interpret that incoming data. Last thing to do here is to sepperate each data chunk. Sounds easier than it is because we're reading and concatening all kind of data (maybe even binary data encoded as base64), so we can't just declare a slash (or whatever) as our boundary character. We need a dead sure delimiter. As it turns out, the prime vathers of ASCII will help us here.

"ASCII reserves the first 32 codes (numbers 0–31 decimal) for control characters: codes originally intended not to represent printable information, but rather to control devices (such as printers) that make use of ASCII, or to provide meta-information about data streams such as those stored on magnetic tape."

Sounds pretty sweet. And it's indeed a great way to separate a stream. in Perl and PHP you can just call

chr(1) . "some text";

That would concat "some text" to the first ASCII control character (SOH). In Javascript we can encode it as unicode character like

var fieldDelimiter = '\u0001';

That dealer script can be written in any language, since it is really doing trivial work. I've written versions for Perl and PHP only so far. Since I'm releasing the complete code on Github the time I release this post's, feel free to contribute. I'd love to see a Rails or NodeJS version.

That whole concept is not actually new. I believe one of the very first releases and experiments were done from the Digg User Interface Library guys. I picked up the topic from the book "High Performance Javascript" by Nicholas C. Zakas
So a ton of credits goes to these guys.

Back to topic. The reason for using MXHR instead of normal XHR is performance. It's just faster transfering n-files in one request instead of n-requests (see Part 1).
But if we're after performance, we should look for more! So now we're receiving one big chunk of data instead of several small ones. That data stream is delimited with ASCII control characters so we should try to do something useful with the data as soon as we got one complete chunk, instead of waiting until we fetched the whole data string. That is where the XMLHttpRequest "Interactive" mode comes in handy. So when receiving our data we do ask the browser to inform us when data has arrived. We can do this by checking the readyState of the XHR object. If it's on "3", the responseText should contain any data that was received so far, see The XMLHttpRequest Object from W3C.

All major browsers do support readyState3 nowadays. Older versions (especially Internet Explorers) might have some trouble on this. On that browsers we have to workaround this issue. But if the browser supports it, we gain even more speed on loading data on the client.

Part 3 will finally show some code snippets and the complete resource of "Supply", my MXHR loader script.

A MXHR loader - Part 1 - The downside of XHR

This is going to be somekind of a tutorial how to create a Multipart XMLHttpRequest Javascript loader script. I don't think I will go trough all of it in only one post, so it's probably going to be a series.

I guess a pretty good start for this is talking about MXHR in general. So,

what is MXHR ?

Ajax had a major impact on modern web development. It revolutionized the fashion of websites in the way that we no longer needed to reload a whole website to load new data into it. So that was a big chunk of the such called "Web 2.0".

Anyway, what is this all about. To create a such called "AJAX request" we need a "XMLHttpRequest" object in Javascript. That object provides us all necessary methods and propertys. A classical example would look like:

var _msxml_progid  = [
                    'Microsoft.XMLHTTP',     // no readystate === 3 support
                    'MSXML2.XMLHTTP.3.0',    // no readystate === 3 support
                    'MSXML3.XMLHTTP',                
                    'MSXML2.XMLHTTP.6.0'
                     ],

xhr = (function() {
         var req;
         try {      
              req = new XMLHttpRequest();                
         } catch(e){                
             var len = _msxml_progid.length;
             while(len--){
                 try{                        
                     req = new ActiveXObject(_msxml_progid[len]);
                     break;
                 } catch(e2){ }
             }
         } finally { return req; }
}());

Ok, to be honest that is a little bit more than a classical example, but it's probably the way a standard XHR setup should look like. If you don't understand what is going on here, a brief description:

_msxml_progid is an array with possible ActiveX XHR strings (used by Internet Explorer's of this world). I then call a self-invoking anonymous function to initialize the XHR object. I'm trying to create a "standard" XMLHttpRequest object, if that fails we are most likely in an Internet Explorer environment.

In that case, I'm trying to create an ActiveXObject with the strings of my array in reversed order.

The finally condition returns the newly created XHR object or undefined if we had no success.

So far so good. Where is the Multipart part you might wonder. Well it's not here yet. 

A big downside of an Ajax request is the overhead. The overhead consists of header information, cookies and other data beside the data you actually want to send or receive. That is a bad thing. Imagine we want to transfer five small HTML pages with Ajax requests, that would be a huge waste of bandwidth & performance due to the overhead each HTTP request creates.

But not only Ajax requests show this behavior. Let's assume you're displaying five images on your site. To do that, you place <img> tags in your markup. Guess what, each <img> tag will create a request to your server before loading the file you specify in the src attribute. In other words that method will also create overhead information. Of course that information is not completely unnecesarry, you're browser is telling the server what encodings, charsets, mime-types, etc. it accepts. The response from the server on the other hand, is telling your browser what data the server is actually going to send. So we need that information, but we don't need / want it for every single file!

Multipart XHR to the rescue


If there was a way, we could transfer multiple files in one request, that would be really great. That is the point were MXHR comes on the stage. It is actually doing exactly that, transfering multiple files over single XHR requests.
In a few days part 2 will follow with the topic - "How does it work?"

jQuery: Go for Gold!

I was pretty busy over the last weeks, which was the reason for not posting any content here.
Whatsoever, for this incidence I'm taking my time. You may know that I am a pretty active member of the Stackoverflow community. Yesterday I had quite an accomplishment, I earned the "jQuery Gold Badge". That makes me the 13th guy on the community holding that badge.

Not too shabby... :-)

Enough self-adulation for now. I will come up with a series of MXHR (Multipart XMLHttpRequest) posts soon. What that is? You will know in a few days.

localStorage === inter window communication ?

You maybe already heard about the HTML5 localStorage object. That is cool stuff indeed, since it allows us to store data locally (you think..?), in a very easy and robust manner.

But that is not the point of this post, another cool thing about the localStorage is, that the browser will fire a storage event, each time you add or remove data. This event is fired in all open windows / tabs which share the same domain.

That in turn means, you can think about it like IPC?..no, IWC! (Inter-Window-Communication). Using jQuery, we can listen like this:

$(window).bind('storage', function(e){
 alert(localStorage.newmsg);
});

We now can open a new window or tab and call:

localStorage.newmsg = 'I am a shared message!';

Suddenly, as if by magic, in our first window/tab an alert message appears. This concept
already works in all browser that support HTML5 localStorage. It would be even more nice
to know which key was written, what the old value was, what the new value is, etc.
So let's have a look at the W3C draft for the storage event object:

interface StorageEvent : Event {
  readonly attribute DOMString key;
  readonly attribute any oldValue;
  readonly attribute any newValue;
  readonly attribute DOMString url;
  readonly attribute Storage storageArea;
  void initStorageEvent(in DOMString typeArg, in boolean canBubbleArg, in boolean cancelableArg, in DOMString keyArg, in any oldValueArg, in any newValueArg, in DOMString urlArg, in Storage storageAreaArg);
};

Looks like we got everything we need? Well, at this point I have to disappoint you. This part from the draft is implemented pretty poorly. Infact, Firefox 3.6.8 does not implement even one from the above event propertys. Chrome at least, supports the "newValue" property, but that doesn't really help alot. I didn't check Safari and IE8+9 so far. Anyways, as I already mentioned, the concept already works. But you have to put any kind of logic within the stored object to process messages properly.

I can think of some usecases here. You could synchronously update messages (read/unread) in a web email client for instance or do synchronous UI updates without the need to invoke a server!
On the other hand, I can also think of a downside. Since the localStorage will store data on your harddisk, this mechanism could cause (on heavy operations) some rumble on your disk. I didn't test that hefty so far.

Anyway, the localStorage "storage" event might open a nice pure clientside way to communicate between windows and tabs, what do you think?