Real time notifications with PHP


Real time communications are cool, isn’t it? Something impossible to do five years ago now (or almost impossible) is already available. Nowadays we have two possible solutions. WebSockets and Comet. WebSockets are probably the best solution but they’ve got two mayor problems:

  • Not all browsers support them.
  • Not all proxy servers allows the communications with websokets.

Because of that I prefer to use comet (at least now). It’s not as good as websockets but pretty straightforward ant it works (even on IE). Now I’m going to explain a little script that I’ve got to perform a comet communications, made with PHP. Probably it’s not a good idea to use it in a high traffic site, but it works like a charm in a small intranet. If you want to use comet in a high traffic site maybe you need have a look to Tornado, twisted, node.js or other comet dedicated servers.

Normally when we are speaking about real-time communications, all the people are thinking about a chat application. I want to build a simpler application. A want to detect when someone clicks on a link. Because of that I will need a combination of HTML, PHP and JavaScript. Let’s start:

For the example I’ll use jquery library, so we need to include the library in our HTML file. It will be a blend of JavaScrip and PHP:

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <title>Comet Test</title>
    </head>
    <body>
        <p><a class='customAlert' href="#">publish customAlert</a></p>
        <p><a class='customAlert2' href="#">publish customAlert2</a></p>
        <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js" type="text/javascript"></script>
        <script src="NovComet.js" type="text/javascript"></script>
        <script type="text/javascript">
NovComet.subscribe('customAlert', function(data){
    console.log('customAlert');
    //console.log(data);
}).subscribe('customAlert2', function(data){
    console.log('customAlert2');
    //console.log(data);
});

$(document).ready(function() {
    $("a.customAlert").click(function(event) {
        NovComet.publish('customAlert');
    });
    
    $("a.customAlert2").click(function(event) {
        NovComet.publish('customAlert2');
    });
    NovComet.run();
});
        </script>
    </body>
</html>

The client code:

//NovComet.js
NovComet = {
    sleepTime: 1000,
    _subscribed: {},
    _timeout: undefined,
    _baseurl: "comet.php",
    _args: '',
    _urlParam: 'subscribed',

    subscribe: function(id, callback) {
        NovComet._subscribed[id] = {
            cbk: callback,
            timestamp: NovComet._getCurrentTimestamp()
        };
        return NovComet;
    },

    _refresh: function() {
        NovComet._timeout = setTimeout(function() {
            NovComet.run()
        }, NovComet.sleepTime);
    },

    init: function(baseurl) {
        if (baseurl!=undefined) {
            NovComet._baseurl = baseurl;
        }
    },

    _getCurrentTimestamp: function() {
        return Math.round(new Date().getTime() / 1000);
    },

    run: function() {
        var cometCheckUrl = NovComet._baseurl + '?' + NovComet._args;
        for (var id in NovComet._subscribed) {
            var currentTimestamp = NovComet._subscribed[id]['timestamp'];

            cometCheckUrl += '&' + NovComet._urlParam+ '[' + id + ']=' +
               currentTimestamp;
        }
        cometCheckUrl += '&' + NovComet._getCurrentTimestamp();
        $.getJSON(cometCheckUrl, function(data){
            switch(data.s) {
                case 0: // sin cambios
                    NovComet._refresh();
                    break;
                case 1: // trigger
                    for (var id in data['k']) {
                        NovComet._subscribed[id]['timestamp'] = data['k'][id];
                        NovComet._subscribed[id].cbk(data.k);
                    }
                    NovComet._refresh();
                    break;
            }
        });

    },

    publish: function(id) {
        var cometPublishUrl = NovComet._baseurl + '?' + NovComet._args;
        cometPublishUrl += '&publish=' + id;
        $.getJSON(cometPublishUrl);
    }
};

The server-side PHP

// comet.php
include('NovComet.php');

$comet = new NovComet();
$publish = filter_input(INPUT_GET, 'publish', FILTER_SANITIZE_STRING);
if ($publish != '') {
    echo $comet->publish($publish);
} else {
    foreach (filter_var_array($_GET['subscribed'], FILTER_SANITIZE_NUMBER_INT) as $key => $value) {
        $comet->setVar($key, $value);
    }
    echo $comet->run();
}

and my comet library implementation:

// NovComet.php
class NovComet {
    const COMET_OK = 0;
    const COMET_CHANGED = 1;

    private $_tries;
    private $_var;
    private $_sleep;
    private $_ids = array();
    private $_callback = null;

    public function  __construct($tries = 20, $sleep = 2)
    {
        $this->_tries = $tries;
        $this->_sleep = $sleep;
    }

    public function setVar($key, $value)
    {
        $this->_vars[$key] = $value;
    }

    public function setTries($tries)
    {
        $this->_tries = $tries;
    }

    public function setSleepTime($sleep)
    {
        $this->_sleep = $sleep;
    }

    public function setCallbackCheck($callback)
    {
        $this->_callback = $callback;
    }

    const DEFAULT_COMET_PATH = "/dev/shm/%s.comet";

    public function run() {
        if (is_null($this->_callback)) {
            $defaultCometPAth = self::DEFAULT_COMET_PATH;
            $callback = function($id) use ($defaultCometPAth) {
                $cometFile = sprintf($defaultCometPAth, $id);
                return (is_file($cometFile)) ? filemtime($cometFile) : 0;
            };
        } else {
            $callback = $this->_callback;
        }

        for ($i = 0; $i < $this->_tries; $i++) {
            foreach ($this->_vars as $id => $timestamp) {
                if ((integer) $timestamp == 0) {
                    $timestamp = time();
                }
                $fileTimestamp = $callback($id);
                if ($fileTimestamp > $timestamp) {
                    $out[$id] = $fileTimestamp;
                }
                clearstatcache();
            }
            if (count($out) > 0) {
                return json_encode(array('s' => self::COMET_CHANGED, 'k' => $out));
            }
            sleep($this->_sleep);
        }
        return json_encode(array('s' => self::COMET_OK));
    }

    public function publish($id)
    {
        return json_encode(touch(sprintf(self::DEFAULT_COMET_PATH, $id)));
    }
}

As you can see in my example I’ve created a personal protocol for the communications between the client (js at browser), and the server (PHP). It’s a simple one. If you’re looking for a “standard” protocol maybe you need have a look to bayeux protocol from Dojo people.

Let me explain a little bit the usage of the script:

  • In the HTML page we start the listener (NovComet.subscribe).
  • We can subscribe to as many events we want (OK it depends on our resources)
  • When we subscribe to one event we pass a callback function to be triggered.
  • When we subscribe to the event, we pass the current timestamp to the server.
  • Client side script (js with jquery) will call to server-side script (PHP) with the timestamp and will wait until server finish.
  • Server side script will answer when even timestamp changes (someone has published the event)
  • Server side will no keep waiting forever. If nobody publish the event, server will answer after a pre-selected timeout
  • client side script will repeat the process again and again.

There’s something really important with this technique. Our server-side event check need to be as simpler as we can. We cannot execute a SQL query for example (our sysadmin will kill us if we do it). We need to bear in mind that this check will be performed again and again per user, because of that it must be as light as we can. In this example we are checking the last modification date of a file (filemtime). Another good solution is to use a memcached database and check a value.

For the test I’ve also created a publishing script (NovComet.publish). This is the simple part. We only call a server-side script that touch the event file (changing the last modification date), triggering the event.

Now I’m going to explain what we can see on the firebug console:

  1. The first iteration nothing happens. 200 OK Http code after the time-out set in the PHP script
  2. As we can see here the script returns a JSON with s=0 (nothing happens)
  3. Now we publish an event. Script returns a 200 OK but now the JSON is different. s=1 and the time-stamp of the event
  4. Our callback has been triggered
  5. And next iteration waiting

And that’s all. Simple and useful. But remember, you must take care if you are using this solution within a high traffic site. What do you think? Do you use lazy comet with PHP in production servers or would you rather another solution?

You can get the code at github here.

About these ads

About Gonzalo Ayuso

Web Architect specialized in Open Source technologies. PHP, Python, JQuery, Dojo, PostgreSQL, CouchDB and node.js but always learning.

Posted on March 14, 2011, in jQuery, js, php and tagged , , , . Bookmark the permalink. 32 Comments.

  1. Nice write up. You’re absolutely correct: we deployed this technique on a high traffic website and found the server frequently bottlenecking. We ended up using writing our own custom little server to handle the requests, as well as some back end caching to prevent unnecessary database calls.

    • I must admit I didn’t use it in a hight traffic site but it works like a charm in a medium intranet. I avoid any database connection in the comet script and it works without significant server load. BTW I’ve got in my mind a similar script using node.js instead of PHP.

  2. Very nice indeed…

    But line 41 in NowComet.php should be
    if (is_null($this->_callback)) {

  3. With this example you are still just doing short polling. The difference is that you have moved the polling from the client to the server. If you use a more event driven model, you can greatly improve performance. You won’t have blackout periods between polling memcached or while sending the data out to the client. I’ve written a small PHP HTTP/WebSocket server that allows you to take incoming messages from your main server and distribute them to many client connections at once. The smaller server (which is based off of Tornado and called WaterSpout) listens for events from the main server and then distributes them immediately to the proper client connections. It uses WebSockets to ensure a constant connection but will fail over to long polling if WebSockets aren’t available.

    If you are going to be at Tek11 this year, stop by my talk on Realtime Communications. I’d love to chat about the challenges you’ve faced and the way you went about solving them.

    • I know. I need to use websockets to improve the performance. The main problem I’ve got with websockets is the browser (only a few of them) and proxy servers (they normally kill websockes). But this solution works without any problem within a small/medium intranet (the number of users is limited). My sysadmin did not kill me (yet), and the server load isn’t altered. I’ve taken care with databases (no db connection at serverside), and I check the modification date of files located in RAM (/dev/shm). The apache connections are limited by a timeout (they’re not alive forever). OK, you’re right. That’s still short polling and it’s not a “real” real-time, but it looks like real-time. Good part: works with all browsers, and I don’t need to take care about internet proxies. Bad part: if I scale it, my sysadmin will definitely kill me. Now I’m playing with node.js + socket.io library (in order to use websockets)

      I’ve seen your talk about it at phpconference2010 (BCN). Definitely your talk inspired me a lot. In fact I’m the one who spoke with you (at the end of your great talk) at about the problems with proxy servers and websokets ;).

  4. How can I implement it on my site?

  5. Great work, I found it very helpful.

  6. Really interesting. Do you think Facebook/Gmail using the same way like this? I know they are using long-polling/comet. Is this the same? Why these two website still works like a charm? They have billions of users… Because of their amazing servers?

    • No. This technique is not suitable for big traffic. Your server load will raise to hell, and that’s too expensive. I’ve wrote a second part of the post (http://gonzalo123.wordpress.com/2011/05/23/real-time-notifications-part-ii-now-with-node-js-and-socket-io/) doing the same with node.js and websockets/socket.io. That’s a better solution if you need to scale. FB/Gmail/Twitter’s servers aren’t amazing. The only amazing thing is the number of servers :). The solution of this post “works” but the solution of the second post works too and scales too. The “problem” with the second solution is that you need one extra server for Real Time communications.

      The problem with the PHP’s solution (The first post) is the IDLE mechanism used (the sleep function). Node uses functional programming and its way to send process to IDLE is brilliant (with callbacks). So different ways to solve the same problem, wiht its pro and cons.

      • Thanks for your explanation. I’m still curious tho, is Facebook using node.js to get this work?

      • I don’t think so. They published couple of years ago one similar open source project called Tornado (python based). It’s similar than Twisted (python too). As far as I know FB uses python in the back-end and probably they have one customized version of Tornado. The idea is the same than node.js.

      • You are right. They are using tornado. How do I implement it on tornado? or I can simply use node.js? since i tried facebook on firebugs and found that they were using long-polling php as well. Should I set up another server to get node.js/tornado work?

        Thanks!

  7. hi i am using your script but not getting the correct manner to use it.While i used it i have got some error like / comet.php

    Warning: touch() [function.touch]: Unable to create file /dev/customAlert2.comet because No such file or directory in C:\xampp\htdocs\realtime\NovComet.php on line 73
    false

    in firebug response box.
    Actually i want to make real time notification in my php website but not geeting any idea.as go through the google i got your script but not working it too for me please help me.
    thanks

    • In the example I’m using /dev/shm to store files.That’s because it’s virtual file system mounted in memory (Linux systems). As I see within your comment you are using Windows, so you need to change the path of the files (hardcoded in DEFAULT_COMET_PATH constant). Anyway I recommend to use another alternatives with node.js or react (with PHP). The approach of this post works, but it will throw problems if you need to scale. (you can see examples in newer posts here for example)

  8. I’ve been programming notifications and all kinds of real-time updates (eg. live-chats) for the last 10 years using PHP and plain Javascript! “impossible 5 years ago”. What are you talking about???

    • I sad that it was “almost” impossible. In fact this implementation is also PHP and plain js and it was available 5 years ago, but nowadays websockets plays in another league. Without websockets we could do it using exotic implementations (like the example of this post) or “emulate” real time with timers. The problems. With a limited number of users those implementations was viable. I have something similar than this post in production for for years in a intranet and it works like a charm indeed, but I now that that if I need to scale it I will have serious problems. Nothing compared to the simplicity of a node.js server with socket.io or even PHP with rachet.

  9. Can you reliably compare timestamps issued by different machines?

    • If they have the same time (synchronized with a NTP server), it should be. Maybe it’s not perfect for ultra critical situations, but good enough for the rest of them

  10. I am getting syntax error

    Parse error: syntax error, unexpected T_FUNCTION in /home/user/web/test/notification/NovComet.php on line 44

    This is line no. 44
    $callback = function($id) use ($defaultCometPAth) {

  11. hi, your example is working great “out of the box”,
    I was wondering if it can support a publish action publish( ” my message”,”subscribed channel”) and subscribe (“subscribed channel” alert (“message” )?
    thanks
    jean-louis

  12. I was wondering if it was possible to modify it to support a publish function publish(“message”,”channel”) and a function subscribe ( “channel” ) ?
    thanks,
    jean-louis

  13. Great script, thank you

  14. Hie i need a help in php. when i insert a data into sql at that time after trigger is fired i need to send the data to client atomatically

    after trigger fired i need to run the php script automatically plz help for this problem

    Thank you

  1. Pingback: Anonymous

  2. Pingback: Real time notifications (part II). Now with node.js and socket.io « Gonzalo Ayuso | Web Architect

  3. Pingback: mymoe » 基于push技术的即时聊天室

  4. Pingback: Notifications in Node.js | t1u

  5. Pingback: [html] Real time notifications | PipisCrew Official Homepage

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,003 other followers

%d bloggers like this: