off the stack

Keep in touch with your servers using XMPP

2012-10-10 | xmpp, monitoring, python

Why

My first contact with XMPP was some years ago. Back then I had to glue together the facebook.com and hyves.nl instant messaging features in one application providing an improved social experience while shopping over the internet.

Time has gone by and I still have Psi installed on my machines to stay in touch with a few people. The services I am using don't give me any trouble, it tends to just work.

Lately my interest in server monitoring grew and while browsing through some old bookmarks I came across Jimbo and some posts from schrankmonster.de again. This got me thinking and I went on to create a little prototype of my own.

The plan

Sometimes a prototype can help to explore an idea. A prototype can give you an idea what works and can point you at flaws in your reasoning/design.

Giving the idea of (ad-hoc) server monitoring using a XMPP-client installed on the server a try, I started with the following requirements my client should meet:

Furthermore I wanted to keep it simple and small.

Shopping around for libraries - and looking for a target language too - I noticed that most major programming languages do have some client libraries for dealing with XMPP. I chose Python and after trying SleekXMPP first (which resulted in a "hanging" client somehow) I finally settled with xmpppy.

For package installation, I will use virtualenv along with virtualenvwrapper in order to get things working without screwing up my system's python installation.

Before I can start coding, I will need a XMPP server to connect to though.

Setup a server

Of course I could use a public server for this, but it seems more appropriate for testing purposes to be in charge of the server myself. There are several good XMPP servers out there - ejabberd, openfire and prosody just to name a few. Just head over to the XMPP standards foundation to get a more comprehensive list of servers.

I will stick with prosody for now as it is advertised as "easy to set up and configure, and light on resources" which seems just right.

For this experiment I will need at least one "monitor" account - which I will connect to and which is allowed to communicate with the client) and one "test" account which is used by the client to log in to the server.

So here we go (as root on my debian squeeze installation):

apt-get install prosody
prosodyctl register monitor localhost password
prosodyctl register test localhost password

Connect your client of choice

For this experiment I will use Psi, but it should not matter which of the available XMPP clients is used. For a list of clients head over to the xmpp.org website.

For Psi I had to go to "Account Setup", click on "Add", give a sensible name such as "monitor@localhost" leaving the "Register new account" checkbox unchecked. The Jabber ID for the next screen would be "monitor@localhost" and the password (as provided while setting up the server) should be entered here too. I then just hit "Save" and connected to the account using the main Psi window.

Setting up the client

First of all, I will have to install the xmpppy library which provides the xmpp module. For safety I will do this using virtualenv(wrapper):

mkvirtualenv test
pip install xmpppy

And finally, the client program:

#!/usr/bin/env python
"""
Prototype client for simple monitoring purposes.
"""
import os
import time
import traceback
import xmpp


def handle_messages(jids, commands):
    """
    Returns a stanza handler function which executes given commands
    from the provided jids.
    """
    def handler(client, stanza):
        """
        Handler which executes given commands from authorized jids.
        """
        sender = stanza.getFrom()
        message_type = stanza.getType()
        if any([sender.bareMatch(x) for x in jids]):
            command = stanza.getBody()
            if commands and command in commands:
                message = commands[command]()
            else:
                message = "Unknown command."
            client.send(xmpp.Message(sender, message, typ=message_type))
    return handler


def handle_presences(jids):
    """
    Returns a stanza handler function which automatically authorizes
    incoming presence requests from the provided jids.
    """
    def handler(client, stanza):
        """
        Handler which automatically authorizes subscription requests
        from authorized jids.
        """
        sender = stanza.getFrom()
        presence_type = stanza.getType()
        if presence_type == "subscribe":
            if any([sender.bareMatch(x) for x in jids]):
                client.send(xmpp.Presence(to=sender, typ="subscribed"))
    return handler


def run_client(client_jid, client_password, authorized_jids):
    """
    Initializes and runs a fresh xmpp client (blocking).
    """
    high_load = 5
    commands = {
        "stats": lambda *args: os.popen("uptime").read().strip(),
    }

    # Initialize and connect the client.
    jid = xmpp.JID(client_jid)
    client = xmpp.Client(jid.domain)
    if not client.connect():
        return

    # Authenticate.
    if not client.auth(jid.node, client_password, jid.resource):
        return

    # Register the message handler which is used to respond
    # to messages from the authorized contacts.
    message_callback = handle_messages(authorized_jids, commands)
    client.RegisterHandler("message", message_callback)

    # Register the presence handler which is used to automatically
    # authorize presence-subscription requests from authorized contacts.
    presence_callback = handle_presences(authorized_jids)
    client.RegisterHandler("presence", presence_callback)

    # Go "online".
    client.sendInitPresence()
    client.Process()

    # Finally, loop, update the presence according to the load and
    # "sleep" for a while.
    previous_load_maximum = 0
    while client.isConnected():
        # Determine the load averages and the maximum of the 1, 10 and 15 minute
        # averages.
        # In case the maximum is above the "high_load" value, the status will be
        # set to "do not disturb" with the load averages as status message.
        # The status will be updated continuously once we reached high load.
        # When the load decreases and the maximum is below the "high_load" value
        # again, the status is reset to "available".
        load = os.getloadavg()
        load_string = "load: %s %s %s" % load
        load_maximum = max(list(load))
        if load_maximum >= high_load:
            client.send(xmpp.Presence(show="dnd", status=load_string))
        elif load_maximum < high_load and previous_load_maximum >= high_load:
            client.send(xmpp.Presence())
        previous_load_maximum = load_maximum

        # Handle stanzas and sleep.
        client.Process()
        time.sleep(1)


if __name__ == "__main__":
    CLIENT_JID = "test@localhost"
    CLIENT_PASSWORD = "password"
    AUTHORIZED_JIDS = ["monitor@localhost"]

    # Loop until the program is terminated.
    # In case of connection problems and/or exceptions, the client is
    # run again after a delay.
    while True:
        print "Trying to connect ..."
        try:
            run_client(CLIENT_JID, CLIENT_PASSWORD, AUTHORIZED_JIDS)
        except Exception, e:
            print "Caught exception!"
            traceback.print_exc()
        print "Not connected - attempting reconnect in a moment."
        time.sleep(10)

It includes three functions, run_client for actually running the client, handle_messages which returns a handler function responding to received messages and handle_presences which returns a handler for presence information/requests.

The program starts at the end of the file. Here the login credentials for our XMPP server are defined as well as the authorized "jids" which is a list of accounts for which the client may execute commands and which may receive presence updates.

Then we enter a simple while loop which basically just starts and restarts the client over and over again in case of disconnection or other errors.

Within the run_client function some example command is defined together with the load average which the client should perceive as "high". Then client is initialized and started providing the presence and message handlers.

Well, starting the client (which I saved in client.py) with:

python client.py

Results in a working prototype which meets the criteria mentioned above. Sending "stats" to "test@localhost" is answered with statistics and I can see the client online (after adding the contact to my roster) with a status reflecting the load.

Further thoughts

The "responsiveness" of the client is sub-optimal; messages are processed with a noticeable delay because all processing happens synchronously/blocking and the interpreter is sleeping too within the main-loop. This could be helped with restructuring and using another (asynchronous) approach (multiprocessing/threading, twisted, node-xmpp a.s.o).

Furthermore the capabilities are limited (due to the scope of this test) and the communication could be improved too. Iq stanzas would be more appropriate for querying the client I suppose. Retrieving information from the client is in fact a request and each request should have a distinct response. Otherwise there would be no proper way to determine if we lost messages for example.

We also could use a custom XMPP component to further improve the communication between the monitor user and the client.

There is a lot left to do ...

Books that could be of interest

XMPP: The Definitive Guide: Building Real-Time Applications with Jabber Technologies

Professional XMPP Programming with JavaScript and jQuery

Jabber Developer's Handbook

Building the Realtime User Experience: Creating Immersive and Interactive Websites

Programming Jabber: Online book from o'reilly