12.3 Deployment

When our web application is finally production ready, what are the steps necessary to get it deployed? In Go, an executable file encapsulating our application is created after we compile our programs. Programs written in C can run perfectly as background daemon processes, however Go does not yet have native support for daemons. The good news is that we can use third party tools to help us manage the deployment of our Go applications, examples of which are Supervisord, upstart and daemontools, among others. This section will introduce you to some basics of the Supervisord process control system.

Daemons

Currently, Go programs cannot cannot be run as daemon processes (for additional information, see the open issue on github here). It's difficult to fork existing threads in Go because there is no way of ensuring a consistent state in all threads that have been used.

We can, however, see many attempts at implementing daemons online, such as in the two following ways;

MarGo one implementation of the concept of using Command to deploy applications. If you really want to daemonize your applications, it is recommended to use code similar to the following:

    d := flag.Bool("d", false, "Whether or not to launch in the background(like a daemon)")
    if *d {
        cmd := exec.Command(os.Args[0],
            "-close-fds",
            "-addr", *addr,
            "-call", *call,
        )
        serr, err := cmd.StderrPipe()
        if err != nil {
            log.Fatalln(err)
        }
        err = cmd.Start()
        if err != nil {
            log.Fatalln(err)
        }
        s, err := ioutil.ReadAll(serr)
        s = bytes.TrimSpace(s)
        if bytes.HasPrefix(s, []byte("addr: ")) {
            fmt.Println(string(s))
            cmd.Process.Release()
        } else {
            log.Printf("unexpected response from MarGo: `%s` error: `%v`\n", s, err)
            cmd.Process.Kill()
        }
    }

Another solution is to use syscall, but this solution is not perfect:

    package main

    import (
        "log"
        "os"
        "syscall"
    )

    func daemon(nochdir, noclose int) int {
        var ret, ret2 uintptr
        var err uintptr

        darwin := syscall.OS == "darwin"

        // already a daemon
        if syscall.Getppid() == 1 {
            return 0
        }

        // fork off the parent process
        ret, ret2, err = syscall.RawSyscall(syscall.SYS_FORK, 0, 0, 0)
        if err != 0 {
            return -1
        }

        // failure
        if ret2 < 0 {
            os.Exit(-1)
        }

        // handle exception for darwin
        if darwin && ret2 == 1 {
            ret = 0
        }

        // if we got a good PID, then we call exit the parent process.
        if ret > 0 {
            os.Exit(0)
        }

        /* Change the file mode mask */
        _ = syscall.Umask(0)

        // create a new SID for the child process
        s_ret, s_errno := syscall.Setsid()
        if s_errno != 0 {
            log.Printf("Error: syscall.Setsid errno: %d", s_errno)
        }
        if s_ret < 0 {
            return -1
        }

        if nochdir == 0 {
            os.Chdir("/")
        }

        if noclose == 0 {
            f, e := os.OpenFile("/dev/null", os.O_RDWR, 0)
            if e == nil {
                fd := f.Fd()
                syscall.Dup2(fd, os.Stdin.Fd())
                syscall.Dup2(fd, os.Stdout.Fd())
                syscall.Dup2(fd, os.Stderr.Fd())
            }
        }

        return 0
    }

While the two solutions above implement daemonization in Go, I still cannot recommend that you use either methods since there is no official support for daemons in Go. Notwithstanding this fact, the first option is the more feasible one, and is currently being used by some well-known open source projects like skynet for implementing daemons.

Supervisord

Above, we've looked at two schemes that are commonly used to implement daemons in Go, however both methods lack official support. So, it's recommended that you use a third-party tool to manage application deployment. Here we take a look at the Supervisord project, implemented in Python, which provides extensive tools for process management. Supervisord will help you to daemonize your Go applications, also allowing you to do things like start, shut down and restart your applications with some simple commands, among many other actions. In addition, Supervisord managed processes can automatically restart processes which have crashed, ensuring that programs can recover from any interruptions.

As an aside, I recently fell into a common pitfall while trying to deploy an application using Supervisord. All applications deployed using Supervisord are born out of the Supervisord parent process. When you change an operating system file descriptor, don't forget to completely restart Supervisord -simply restarting the application it is managing will not suffice. When I first deployed an application with Supervisord, I modified the default file descriptor field, changing the default number from 1024 to 100,000 and then restarting my application. In reality, Supervisord continued using only 1024 file descriptors to manage all of my application's processes. Upon deploying my application, the logger began reporting a lack of file descriptors! It was a long process finding and fixing this mistake, so beware!

Installing Supervisord

Supervisord can easily be installed using sudo easy_install supervisor. Of course, there is also the option of directly downloading it from its official website, uncompressing it, going into the folder then running setup.py install to install it manually.

If you're going the easy_install route, then you need to first install setuptools

Go to http://pypi.python.org/pypi/setuptools#files and download the appropriate file, depending on your system's python version. Enter the directory and execute sh setuptoolsxxxx.egg. When then script is done, you'll be able to use the easy_install command to install Supervisord.

Configuring Supervisord

Supervisord's default configuration file path is /etc/supervisord.conf, and can be modified using a text editor. The following is what a typical configuration file may look like:

;/etc/supervisord.conf
[unix_http_server]
file = /var/run/supervisord.sock
chmod = 0777
chown= root:root

[inet_http_server]
# Web management interface settings
port=9001
username = admin
password = yourpassword

[supervisorctl]
; Must 'unix_http_server' match the settings inside
serverurl = unix:///var/run/supervisord.sock

[supervisord]
logfile=/var/log/supervisord/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=true              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
user=root                 ; (default is current user, required if root)
childlogdir=/var/log/supervisord/            ; ('AUTO' child log dir, default $TEMP)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
; Manage the configuration of a single process, you can add multiple program
[program: blogdemon]
command =/data/blog/blogdemon
autostart = true
startsecs = 5
user = root
redirect_stderr = true
stdout_logfile =/var/log/supervisord/blogdemon.log

Supervisord management

After installation is complete, two Supervisord commands become available to you on the command line: supervisor and supervisorctl. The commands are as follows:

supervisord: initial startup, launch, and process configuration management.
supervisorctl stop programxxx: stop the programxxx process, where programxxx is a value configured in your supervisord.conf file. For instance, if you have something like [program: blogdemon] configured, you would use the supervisorctl stop blogdemon command to kill the process.
supervisorctl start programxxx: start the programxxx process
supervisorctl restart programxxx: restart the programxxx process
supervisorctl stop all: stop all processes; note: start, restart, stop will not load the latest configuration files.
supervisorctl reload: load the latest configuration file, launch them, and manage all processes with the new configuration.

Summary

In this section, we described how to implement daemons in Go. We learned that Go does not natively support daemons, and that we need to use third-party tools to help us manage them. One such tool is the Supervisord process control system which we can use to easily deploy and manage our Go programs.

Deployment