Python UNIX Sockets

June 14, 2010

I’ve been tinkering with using UNIX sockets for IPC from Python and I thought I would share my most basic experiment.

This is a super simple example of client/server usage of a socket. Essentially the server is a blocking command socket that echo’s whatever is passed through it.

Listing: server.py

Listing: client.py

Here is the transcript of me running the client.

And here is the server transcript from that session.

Now all you need is a protocol and you’ll be set for basic IPC.

Categories: Geek
Tags: , , , ,

Comments

  1. I wrote a really large IRC bot in Python (with autoloading command modules and all sorts of neat stuff).

    I used raw sockets to do the network connectivity and came across an error condition that I could never quite solve. Any time the network connection dropped (IRC server shutdown, etc), the socket would still think it was connected, but the main while loop would just spin out of control and gobble up all the CPU.

    I tried various hacks like putting in a timed counter and stuff, but nothing seemed to work. The problem seemed to be that the Python socket object really seemed to think that it was still a valid, active connection. It didn’t know the other side had closed.

    Do you know of anything that can solve this problem?

  2. john says:

    I haven’t done any network socket work, just the local ones and I haven’t come across that error yet.

    I do have a similar problem consuming a Twitter stream using urllib2 though, which may use the socket library, I don’t really know. I haven’t debugged that one yet.

    I’ll let you know if I find anything useful.

    As far as IRC bots go, I wrote an extensible one based on Twisted a while back. That’s a nice framework, really enjoyed it. The bot lives at http://github.com/jmhobbs/pircie if you want to take a look.

  3. Ambuj says:

    Hi Jon,

    Your code works fine when communicating between a python client and a python server. I’ve been trying to use your client code to communicate with a server written in C. However, I got the following traceback when I try to connect:

    client.connect(“/tmp/python_unix_sockets_example”)
    File “”, line 1, in connect
    socket.error: (111, ‘Connection refused’)

    Is connecting with a C-based server different than connecting with a python based server?

  4. John Hobbs says:

    It should be able to connect fine. Keep in mind the Python script uses local sockets, AF_UNIX.

    Here is an example C++ server I put together which accepts connections from the Python client.

    Keep in mind I haven’t written C++ in years, so that’s kinda sloppy code.

  5. Fenhl says:

    I get the same error as Ambuj, but with a Python server. The relevant code is at http://pastie.org/8222330 — any ideas?

  6. badMan says:

    It`s very useful , Thanks a lot

  7. Tim Watkins says:

    New to Python and programming in general.

    I am using Python 3.4. After making the appropriate changes to the print statements, I am receiving a TypeError of ‘str’ does not support the buffer interface.

    Do you have the Unix Socket example using Python 3.4 in the basic format of the above?

    Tim

  8. John Hobbs says:

    Hi Tim!

    So, Python 3 changed some things (actually a good number of things).

    One of these things is that converting strings to bytes isn’t implicit anymore. But that’s okay! We can do that fairly easily.

    Essentially, you take a string and call .encode("encoding") to turn it into bytes, and then call .decode("encoding") on the bytes to turn it back into a string. Where “encoding” is a valid encoding for the string.

    Here’s a Python 3 version of the client and server. It’s not the cleanest code, because I tried to port it as directly from one version to the other without changing much.

    https://gist.github.com/jmhobbs/11276249

    Hope it helps!

  9. Tim Watkins says:

    John!

    Thanks so much! This is the start of exactly what I was looking for… A working concept!

    Huge appreciation on the quick response.

    If you were going to clean it up, what would you do?

    What is a good resource for wading thru the mud and finding what one really needs to know about Python 3.4 without knowing previous versions (basically, where would you start).

  10. John Hobbs says:

    No problem!

    I’m not exactly sure, I’d probably just tidy it up and make it more pythonic.

    For example, I wouldn’t check for existence of the socket file in the client;

    I would just try to connect and catch the exception. That, and there are a lot of superfluous print statements, but that’s just due to the explanatory nature of the code.

    I’m not sure where I would learn Python 3. There are some great resources for Python 2, you could learn that and then pick up the changes.

    Otherwise there is a tutorial in the docs, https://docs.python.org/3.3/tutorial/ and a list of resources here; http://docs.python-guide.org/en/latest/intro/learning/

    Sorry I can’t recommend anything specifically.

  11. Tim says:

    John!

    I have been getting more comfortable with your code.

    I need another nudge to finally make what I need it to do. I am going to be using Unix Sockets for Unit Testing.

    I have a Control Device and a Peripheral Device. At this point I will be doing unit testing on the Control Device.
    1 The Control Device will need a hook to receive messages from the Test Framework so that it can be controlled to do something from the Test Framework.
    2 The Control Device has a Transmit (TX) to the PD that I would like the Test Framework to Listen to.
    3 The Control Device also has a Receive (RX) from the PD that I would like to send Fake Peripheral messages to.
    4 The Control Device will need to have an Output to the Test Framework to be able to ensure that it responded correctly from input from the Peripheral.

    Not sure if you have played around with Unit Testing before. I will want the TFW to TX to the CP to do something (1). Have the TFW listen and RX on the CPs TX side to the PD (2). If the message is correct, I will then want the TFW to TX on the CPs RX side to the PD (3). Finally I will want to listen to the output of the CP side to ensure that the CP responded to the PD correctly (4)

    How simple/complex is it to build / teardown sockets between 3 devices (Test Framework (TFW), Control Device (CD), Peripheral Device (PD?)

    Can this be multithreaded and leave all four sockets up during the session?

    Do I need to have 4 sockets? Perhaps the CP could share a TX/RX with Test Framework?

    Looking at your thoughts and opinion on this one.

  12. John Hobbs says:

    Wow! That sounds… complicated :)

    Can you go end-around and just test the interface which generates the bytes for sending over the wire, instead of actually creating socket connections? That would probably be a simpler method, if your code has those separated out.

    My only other advice would be to look at the multiprocessing package for running these.

  13. Paul-Vincent Roll says:

    Is there a way to use the client to connect to a node.js server?

  14. John Hobbs says:

    It appears that node.js does not support UNIX datagram sockets, see https://groups.google.com/forum/#!topic/nodejs/iCzhcuxGP1I.

    However, we can tweak the client a bit to make it work.

    If you change SOCK_DGRAM to SOCK_STREAM then you can connect to a node.js net server, like this one:

    It looks like there is at least one unix-dgram userspace library out there, but I can’t imagine it’s super efficient. Unless you really want the datagram, protocol, I would just use the net module and change the client.

  15. Paul-Vincent Roll says:

    Amazing! Thank you!

  16. […] habe. Auf der Suche nach Beispielen für UNIX Sockets habe ich diesen Artikel gefunden, nach einem Kommentar, hat er ihn gleich noch um einen Node.js-Server ergänzt, was jetzt auch ohne Probleme […]

  17. […] modified the this nice example a bit (e.g., python server has to listen on TCP instead UDP socket to be compatible with nodejs […]

  18. shadow-c says:

    Very good explanation and code sample. Helped me a lot. Thanks John.

  19. John Hobbs says:

    Glad it helped you!

Leave A Comment

Your email will not be published.