Alan Chandler

Alan Chandler

Having first written software at age 11 and still writing it today at age 70 Alan Chandler qualifies as an ancient geek. He enjoys playing the piano having started again in 2017 after not touching one for 55 years

The making of MBChat v3 - part 1 introducing Nested Closure Programming

MB chat is an Ajax chat application that I built for Melinda’s Backups. It was originally so that the leadership team could hold meetings without resorting to e-mail, but it became more of a general social thing for the members of the web site. In order for the software to work out if someone had said anything new, it polled a database every two seconds. Not something that would scale particularly but OK for v1. That was until Melinda herself came to chat one time, and a large number of people tried to come in at the same time. We found out that the scale limit was about 20 people. Beyond that and chat fell over.

In v2 I explored techniques for avoiding the need to poll a database and developed a solution based around the same technique as I mention in providing a high speed link between web clients using named pipes. Unfortunately the slight variation on this technique where essentially a message generated by one user has to be passed to all other users rather than just the one other, meant that you couldn’t wait for the two sides to synchronise (as is important in that solution). This means in some circumstances a message could get missed – so even with this technique a level of database interaction was necessary. I therefore also explored switching to the SQLite database as it seemed to offer a potential speed advantage. However I soon discovered that whilst it was almost faster the lock out mechanisms when multiple users attempted to access the database at the same time were crude and actually slowed things down. I got to 50 people, but that was the best I could achieve.

One thing I did discover in this test was that although SQLite was fairly fast with a single transaction per user interaction, it was lighteningly fast if multiple accesses could be done within the same transaction. I started thinking about a solution that could make that happen.

During this time, my daughter told me how they were considering a secure version of a chat program at her work. This got me thinking about how I could make MBChat secure. I set myself the following challenge:-

  • The chat server and the clients had to communicate with each other over the open internet
  • Only authorised users should be able to access the functionality or see any of the conversations
  • Clients can be sure they are talking to the server they think they are (to prevent an imposter capturing the messages)

A moments thought will immediately dictate that we should be encrypting at least the conversational messages with a key, but that since the software is in essence being downloaded from the server, it cannot contain that key within it as anyone watching the traffic would immediately be able to sniff out the key and use it to decrypt the conversation that was supposed to be secure .

In fact I decided that the only way to do this properly is for the client to generate a public/private key pair dynamically once it has started and transmit the public key of that pair to the server. The server will have to encrypt the key used to encrypt messages with that public key and pass it back to the client. The client (and only that client) who will be able to decrypt it with his private key and start using it in the subsequent client server communication.

Generating a public/private key pair turns out to be a non trivial task. If we are performing the task at the client end, inside the browser, then Javascript will most likely have to be the programming language which will be used to run the software which generates the public/private key pair. Even with a reasonably fast processor, it takes several seconds to calculate an appropriate 64bit key (which we use). If this was a normal script the browser would lock up solid, and would soon report a script failure.

I found Atsushi Oka had already solved this problem and provided a set of javascript routines that could generate the RSA public/private key pair. His solution he called Nested Closure Oriented Progarmming. If you take a look at the Nested Closure Oriented Programing module he has produced, you will see that part of his approach is to modify the base Object prototype to add the additional routines in. It was only after I built my first attempt to use his software to produce the key did I discover that there was major conflict with the mootools framework library that was an integral base for the rest of MBchat.

However after a detailed analysis of how the RSA key generation routines where built, I believed it was possible to make a mootools class that did almost the same job, but avoiding the need to alter the object prototype. I developed my own Nested Closure class which turned out to be considerably simpler that the library it replaced.

The key approach of nested closure programming is to have STEPS which are encapsulated within a function. A function can have part of its process broken into sub-steps, the key being that at the lowest level of steps the code can complete within the time allowed by the browser. Then all that is needed is an engine that runs one step, waits for a tick (a millisecond) and then executes the next step. Finally we need a mechanism to provide loops, for steps to indicate when they complete whether to exit this loop or to repeat the step or to continue to the next step (or loop if it was the last step of the loop), and a mechanism for passing results between steps.

Lets look at some small fragments of the RSA generation which show how it is used:-

Firstly – declaring the generator class, dealing with the result and starting the processes

RSA.prototype.generateAsync = function(keylen,exp,result) {
var self=this;
var generator = new NCOP( stepping_generate,true);
var _result = function(keyPair) {
result( self );
};
generator.EXEC([keylen,exp],_result);
…

We declare an instance of class NCOP (generator in this case) with the name of the highest level function used in the process, together with a flag which indicates that this instance of the NS class is unique. This allows the class to bind its key functions to the window object – making them easier to call (so you call can DO() rather than this.DO() or generator.DO()).

We start the process going and pass the parameters (as an array – which is an easy text transformation from the original code). We also pass the function that is called when the process terminates (_result).

Looking at the head of the stepping_generate function we can see how the parameters are picked up, and with a simple text transformation returned to two variables that were originally the functional parameters of that function

// Generate a new random private key B bits long, using public expt E
function stepping_generate (myArgs) {
var B,E;
B = myArgs[0];
E = myArgs[1];
…

Once this high level routine has done some simple initialisation of variables, it starts the stepping activity (the main process) by returning but calling the DO method (it could have alternatively called the STEP method) of the main NCOP class. We see here how the first few steps are declared by declaring an array (indicated by the “[” at the start of the function parameters) of functions. Notice also a nested arrays starting at step 1.3. Arrays represent loops with a list of steps that are repeated until an action is taken to exit the array. Finally notice in step 1.3.1 that in certain instances the code executes return AGAIN(). This causes that whole loop to be started from the beginning rather than the (default) action of proceeding to the next step.

return DO([
// Step1.ver2
function () {
RSA.log("RSAEngine:1.1");
self.p = new BigInteger();
rng = new SecureRandom();
},
function () {
RSA.log("RSAEngine:1.2");
// return self.p.stepping_fromNumber1( B-qs, 1, rng ).BREAK();
return self.p.stepping_fromNumber1( qs[0], 1, rng );
},
[
// Step1.3 ver3
function () {
RSA.log("RSAEngine:1.3.1");
if ( self.p.subtract(BigInteger.ONE).gcd(ee).compareTo(BigInteger.ONE) != 0 ) return AGAIN();
},
function () {
RSA.log("RSAEngine:1.3.2 : calling stepping_isProbablePrime");
return self.p.stepping_isProbablePrime(10);
},
…

It is also possible to pass the result from one function into the next step. This is shown in step 2.3.2 where a function “stepping_isProbablePrime” is called (it itself is using the NS methods to split its action into steps). The result of this step is then passed into the next step as the (only) parameter to the function which uses it to determine the flow of the following step (either exit the loop by calling DONE() or repeating the loop by calling AGAIN())

function() {
RSA.log("RSAEngine:2.3.2");
return self.q.stepping_isProbablePrime(10);
},
function(result) {
RSA.log( "RSAEngine:2.3.3:result="+result );
if ( result ) {
RSA.log("RSAEngine:2.3.3=>EXIT");
return DONE();
} else {
RSA.log("RSAEngine:2.3.3=>AGAIN");
return AGAIN();
}
},
…

This works very well. Dependant on the speed of the computer, it can generate a 64bit RSA public/private key pair in 1 to 2 seconds. This happens in the background – in some configurations of chat whilst the user is entering his username and password. In this instance that time delay is not even noticed. More importantly the browser never pops up a message to say that the script is taking too long to run.

Obviously when several things are happening at once (generating the key and awaiting a user to login) there is need to coordinate the completion. A simple mootools class to do that will be the subject of my next blog

UPDATE 21st April 2020 I am reading this post with the perspective of the state of javascript in 2020. The main thing that comes to mind, although I haven’t followed this thought through, is that async and await are the perfect tools to replicate this today. If I didn’t have so many projects queued up to work on it would have been a perfect experiment to develop a RSA key generator using the sofware I use here but using promises and async await structures.