Providing A High Speed Connection Between Two Web Clients

I have been considering the architecture for a web based real time game between two players. It doesn’t matter what the game is, any game in which there are two players. I’m thinking of an air hockey simulation, but it could be anything (tennis maybe). The basic idea is that each web browser will run a javascript/ajax application to control the player on their side, and will want to pass messages very rapidly to the other player.

My last application, a chat program, passed messages via a database, with the client polling the server every two seconds to see if anything new had arrived. For a game this would not be fast enough. So I needed an alternative. The obvious choice for passing data between two programs in unix is a pipe, and that is what I decided I would try and use. But there are limitations. I can pretty well only use php as a module under Apache web server for this activity. That means that each pipe has to be opened, read or written to, and then closed in the space of one request. Granted I can use ajax for that request to happen in parallel (ie not block the game), but how do you sync with the other client potentially on the other side of the world, who could go and walk away from the game at any time.

To make matters slightly more difficult, documentation on how precisely pipes work when they are opened, data is put in them and closed, or what happens when a pipe is opened for reading and there is no data yet in the pipe. Here is what I think I found out by trial and error:-

If you open a pipe for writing, write data to it, and close it again, then that data is lost, if no reader is connected whilst the pipe is open.¹
The above sequence does not block
If you attempt to open a pipe for reading, the call blocks until someone else opens it for writing.
If you attempt to read from an pipe opened for reading the call blocks until the other end closes it (even if no data is transmitted).

There are two other issues that need to be considered

Internet explorer queues requests after there are two outstanding requests Even if the requests are cancelled, the web server threads will remain blocked if they are hanging on opening or reading a pipe. This needs to be prevented to play nice with the web server. After experimentation, the way that this works can be shown by the following diagram

READ	SEND	COMMENTS
Open Ack pipe for writing		Indicate Read is there
Open Msg pipe for reading		Will hang until send side opens it
	Open Ack pipe for reading	Will hang until read is there
	Open Msg pipe for writing	At this point synchronised Read and Send sides
	Read Ack pipe	Waits for Ack pipe to close
Close Ack pipe		Indicates that Msg pipe is open, so write may proceed
	Close Ack pipe	Housekeeping
Read Msg pipe	Write Msg pipe	Data is transfered
Close Msg pipe	Close Msg pipe	Finish off

This is the send.php routine

<?php
/*
Copyright (c) 2009-2011 Alan Chandler
This file is part of AirHockey, an real time simulation of Air Hockey
for playing over the internet.

AirHockey is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

AirHockey is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with AirHockey (file supporting/COPYING.txt).  If not,
see <http://www.gnu.org/licenses/>.

*/
/* copied from index.php */
define(‘AIR_HOCKEY_PIPE_PATH’,    ‘/home/alan/dev/airhock/db/cf/’);

header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: -1"); // Date in the past
if(!(isset($_POST[‘uid’]) && isset($_POST[‘msg’]) && isset($_POST[‘ahv’]))) {
echo ‘<error>Invalid Parameters</error>’;
exit;
}
list($utime,$now1) = explode(" ",microtime());
$now1 .= substr($utime,2,3);

$readpipe=fopen(AIR_HOCKEY_DATABASE.$_POST[‘ahv’].’/ack’.$_POST[‘uid’],’rb’); //This waits until an read request is outstanding
$sendpipe=fopen(AIR_HOCKEY_DATABASE.$_POST[‘ahv’].’/msg’.$_POST[‘uid’],’r+b’);
$r=fread($readpipe,10); //not reading, but syncronising with other end (this will be satisfied as EOF as other side closes)
fclose($readpipe);
fwrite($sendpipe,"$".$_POST[‘msg’].’$’.$_POST[‘c’]);

list($utime,$time) = explode(" ",microtime());
$time .= substr($utime,2,3);

fclose($sendpipe);

echo ‘<status time="’.$time.’">OK</status>’;
?>

And the read.php side

/*
Copyright (c) 2009-2011 Alan Chandler
This file is part of AirHockey, an real time simulation of Air Hockey
for playing over the internet.

AirHockey is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

AirHockey is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with AirHockey (file supporting/COPYING.txt).  If not,
see <http://www.gnu.org/licenses/>.

*/
/* copied from db.inc */

define(‘AIR_HOCKEY_DATABASE’,’/home/alan/dev/airhock/db/’);

header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: -1"); // Date in the past
if(!(isset($_POST[‘oid’]) && isset($_POST[‘ahv’]))) {
?><error>Log – Hacking attempt – wrong parameters</error>
<?php
exit;
}
$sendpipe=fopen(AIR_HOCKEY_DATABASE.$_POST[‘ahv’].’/ack’.$_POST[‘oid’],’r+b’); //Say I am ready for a send from the other end
$readpipe=fopen(AIR_HOCKEY_DATABASE.$_POST[‘ahv’].’/msg’.$_POST[‘oid’],’rb’);
fclose($sendpipe);//this tells other end it may now write to the pipe
$response=fread($readpipe,400);

list($utime,$time) = explode(" ",microtime());
$time .= substr($utime,2,3);

fclose($readpipe);

if(strlen($response) > 0) {
list($n,$msg,$count) = explode(‘$’,$response);
echo ‘<message time="’.$time.’" count="’.$count.’">’.$msg.'</message>’;
} else {
echo ‘<error>zero length response = "’.$response.’"</error>’;
}

?>

The calls need to be cancelled from Javascript – because IE will not allow a third request outstanding (and typically there will be one read and one write from each side hanging on the same time. But the cancelled requests will not release the web server processes, so in addition and abort routine is called to clear the hanging pipes

And an abort routine to unlock a stuck call. This achieves it by opening a channel for writing that its own side has open for reading

Finally a set of javascript routines in a nice package (I use the mootools framework for the classes and requests)

/*
Copyright (c) 2009-2011 Alan Chandler
This file is part of AirHockey, an real time simulation of Air Hockey
for playing over the internet.

AirHockey is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

AirHockey is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with AirHockey (file supporting/COPYING.txt). If not,
see <http://www.gnu.org/licenses/>.

*/

Comms = function () {
  var sender = new Request({link:’chain’});
  var messageCallback;
  var failCallback;
  var readTimerID;
  var readTimeoutValue;
  var oid = 0;
  var me;
  var counter = 0;
  var reader = 0;
  var readerStarted = false;
  function readTimeout () {
  messageBoard.appendText(‘ [R:TO]’);
  reader.cancel();
  failCallback();
};
var messageBoard;

return {
  initialize: function (myself,opId,errDiv,fail) {
    me = myself;
    oid = opId;
    messageBoard = errDiv;
    if (oid != 0) {
      failCallback = fail;
      //Set up the read request
      reader = new Request({
        url:’read.php’,
        link:’chain’,
        onSuccess: function(html) {
          window.clearTimeout(readTimerID);
          readTimerID = readTimeout.delay(readTimeoutValue);
          var holder = new Element(‘div’).set(‘html’,html);
          if(holder.getElement(‘error’)) {
            messageBoard.appendText(holder.getElement(‘error’).get(‘text’));
          } else {
            var m = holder.getElement(‘message’);
            if(m) {
              var c = m.get(‘count’);
              if(++counter != c) messageBoard.appendText(‘ [C:’+counter+’:’+c+’]’); //Should catch out of sequence counts
              messageCallback(m.get(‘time’),m.get(‘text’));
            }
          }
          reader.post({oid:oid}); //Queue up next request
        }
      });
    }
  },
  Stream : new Class({
    initialize: function(myURL) {
      this.url = myURL;
      this.counter = 0;
    },
    send: function(myParams) {
      if (sender != 0) sender.send({
        url:this.url,
        data:Object.merge({c:++this.counter},me,myParams),
        method:’post’,
        onSuccess:function(html) {
          var holder = new Element(‘div’).set(‘html’,html);
          if(holder.getElement(‘error’)) messageBoard.appendText(‘ [S:’+holder.getElement(‘error’).get(‘text’)+’]’);
        }
      });
    }
  }),
  set: function(callback,timeout) {
    messageCallback = callback;
    readTimeoutValue = timeout;
    if(oid != 0) {
      //reset timeout
      window.clearTimeout(readTimerID);
      readTimerID = readTimeout.delay(readTimeoutValue);
      if(!readerStarted) {
        reader.post({oid:oid}); //Startup read request sequence if not already going
        readerStarted = true;
      }
    }
  },
  die: function() {
    if(sender != 0) {
      if (oid !=0 ) {
        window.clearTimeout(readTimerID);
        reader.cancel(); //Kill off any read requests as we are going to reset them
      }
      sender.cancel();
      sender = 0; //ensure nothing else goes
    }
  }
}
}();

updated 9th March after restructuring of code.

I have discovered that the “get” comms functions in the javascript are being cached by the browser. These need to be changed to “post”. I also didn’t say that I am using the mootools core framework

UPDATE 13th June 2010: You can get at the complete air hockey application in my git repositories at Github.

#u UPDATE 28th February 2011: I am not convinced that data put into a pipe is lost. I think it could still be there and is retrieved as part of the next successful read. I spent some time tracking down a bug where it appears messages were lost, and in the end discovered this was an error at the Javascript end.

As a result, the complete javascript end was rewritten to provide what I called the “Comms” package. I separated out the reads and writes. A single read request sits there with timeout waiting for information from the other end. It is initialised in Comms.initialize and parameters, such as timeout and callback routine can be changed in Comms.set. When the read request gets a response, it resets the timout and calls a callback function – if one fails to arrive then it will eventually timeout. For sending, there is a single request which is queued using the chain:’link’ parameter. A Comms.Stream class instantiates a stream to a particular url, but shares the single request. When someone writes to the stream, it is sent to this common request. If that request happens to be busy the new write is queued. This seems to prevent the browser from dropping any requests on the floor, and so far is proving robust.

The two PHP functions read and send where also changed – I moved away from using JSON in my messages as I think the mootools implementation is not as robust against errors as passing some xml like messages. As for the abort function, I found it not at all useful and although the code is still in the application repository it is not used. I found that it is better and easier to clean out the pipes as the application starts. As a result, this snippet of php code is used by the application as it initialises

// We delete any old fifo(s) for this user (to clear out stale messages) and create new ones
$old_umask = umask(0007);
if(file_exists(AIR_HOCKEY_PIPE_PATH."msg".$uid)) unlink(AIR_HOCKEY_PIPE_PATH."msg".$uid);
posix_mkfifo(AIR_HOCKEY_PIPE_PATH."msg".$uid,0660);
if(file_exists(AIR_HOCKEY_PIPE_PATH."ack".$uid)) unlink(AIR_HOCKEY_PIPE_PATH."ack".$uid);
posix_mkfifo(AIR_HOCKEY_PIPE_PATH."ack".$uid,0660);
umask($old_umask);

The new versions of all these routines have replaced the code in early versions of this post. If you want to see the more complete history look in the Air Hockey repository at Github

Twitter Facebook LinkedIn