I was recently working on a small hobbyist project - an IRC bot that notifies you by SMS when your name is mentioned - when I came across a problem I couldn't quickly solve. This is always the most exciting part of a personal project: an opportunity to learn a new technique, tool, or strategy for solving a problem.
In this case, the issue was interprocess communication. I wanted to allow users to reply to the users who mentioned them in IRC via SMS. To do this, the small Flask application that handles SMS replies (which are POSTed to it from Twilio) needed to be able to send a message to the IRC bot (a separate process), saying "user A is replying to user B with this message." The bot should receive this information and then send a private message to user B.
Interprocess communication is a problem that seems like it should be much more simple than it actually is. Python lists a variety of modules that can be used for interprocess communication, many of which can also be used for networking. However, none of these seemed to be what I wanted: a simple, asynchronous way to send and receive messages.
I was attracted to Redis mainly because of how incredibly simple it is to set up and use. As a key/value store, there are of course no schemas, tables, or anything else to set up. Literally all that needs to be done is install - as easy as $ brew install redis on OSX - and connect. Although I already was using one SQL database to keep track of users (their usernames and phone numbers), adding Redis required almost no extra overhead or time spent setting it up.
Redis's power in interprocess communication lies in its pub/sub capabilities. Redis's pub/sub system is just as simple as the rest of Redis - clients subscribe to a "channel," or a stream of messages, and other clients can publish messages (which are merely strings) to these channels.
In my SMS project, the IRC bot will subscribe to a channel called "sms_replies," while the Flask application will publish a message to it every time it receives a reply.
To communicate with Redis, the IRC bot and the Flask app use the redis-py module. Redis-py is a very simple API interface, and simply translates the various Redis commands to database methods (i.e. GET some_key' becomes redis_db.get('some key')).
The code to publish couldn't be simpler. Here, I've sent the message - containing the user it's from, the user it's to, and the reply itself - as a simple space-separated string, but you could send any sort of serialized data structure you'd like, if you need something more advanced.
The most notable aspect of this code is that it uses a separate thread for listening to Redis, since the Redis listening loop blocks execution of the rest of the code.
There's not much else in the way of details I have to offer. Redis is a rare piece of software that just works out of the box with zero configuration, and it's kind of amazing. Redis pub/sub is definitely not the only way to do interprocess communication, but it has a lot of advantages:
publish commands and a listening thread (or similar - in node.js, it's as easy as you'd expect)