Writing daemons in python

So I’m not a huge fan of Python for some particular reasons that are out of scope for this blog entry but it’s a language that exists and that is often used so sometimes on has to use it (by the way I like it for playing around with algorithms, fast hacks and as some advanced scripting language anyways). And then there are moments where one has to develop services around an extensive Python codebase - then it makes simply no sense to rewrite everything in a more sane and stable language than to use Python for the remaining parts of course.

Services on Unix like systems are usually (except for the new systemd area) realized as so called daemon processes. These are processes that are launched just like ordinary processes but then detach from their terminal and thus also from their process hierarchy and run in background. The Windows analogon are of course services - but they work somewhat different. On Unix like systems it’s the responsibility of the daemon to start up, detach itself from the terminal and optionally (especially important when launched as root) limit it’s own execution environment as far as possible to reduce attack surface.

Usually the daemonization process is the execution of the following steps:

• Cleaning up and sanitizing the environment variables
• Forking into a new process and exiting the parent process to signal to the launcher process that the process has successfully terminated
• Detaching from the controlling session:
  • Dissociating from the controlling tty by closing standard input and output as well as standard error output
    • Creating a new session
    • Becoming the own process group leader - i.e. detaching from the parent process to prevent termination in case the parent terminates
• Optionally the process might now fork again and terminate the old parent process to prevent acquiring the controlling tty by accident. This ensures the new child is not a session leader and thus has no way of gaining control of the tty again.
• Setting it’s own chroot directory to limit file system access as far as possible. In case one requires access to stuff like PID or logfiles these should be opened before
• Changing the umask
• Redirecting the stdin, stdout and stderr either to /dev/null or a logfile.

On some Unices the applications might even limit themselves more - for example on FreeBSD applications can drop their permissions to gain access to network ports, prevent access to filesystems and limit access to certain system calls. This is of course operating system specific.

Since these steps are security critical and tedious to do right there are libraries that allow applications to daemonize themselves. For Python there is the new PEP-3143 - the standard daemon process library that’s found in the python-daemon PyPi package. Unfortunately I never really got that working - the following recipe uses also an external library called daemonize. As usual I don’t really feel good when using an external dependency from PyPi but that’s the Python way as it seems. This package can be references in setup.cfg as daemonize and can be manually installed using

pip install daemonize

The mini recipe that I’m referencing here performs the following steps:

• Calling it’s own mainStartup routine in case the script is started as application
• Parsing command line arguments to determine if it should daemonize (and if then under which uid/gid, at which chroot and which PID file should be used) or if it should run in foreground.
• In case it should launch as daemon this is done using the mainDaemon trampoline function.
• The whole application is contained in the ExampleDaemon class. After daemonization - or without - the run function is invoked.
• The skeleton also:
  • Sets up logging into an optional logfile and selects the log level
  • Hooks the signal handlers for SIGHUP which I’m using to signal the process to re-read it’s configuration as well as SIGTERM and SIGINT which are hooked to signal a termination request.

The code

The whole code is available as a GitHub GIST

This article is tagged:

• Python
• Programming
• Basics