Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 28 Mar 2011 17:41:10 +0000 (UTC)
From:      Mikolaj Golub <trociny@FreeBSD.org>
To:        src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org
Subject:   svn commit: r220104 - in stable/8: etc etc/mail etc/rc.d sbin/hastctl sbin/hastd
Message-ID:  <201103281741.p2SHfAFu001547@svn.freebsd.org>

next in thread | raw e-mail | index | archive | help
Author: trociny
Date: Mon Mar 28 17:41:10 2011
New Revision: 220104
URL: http://svn.freebsd.org/changeset/base/220104

Log:
  MFC r217729, r217730, r217731, r217732, r217737, r217784, r217958,
    r217961, r217962, r217964, r217965, r217966, r217967, r217969,
    r218040, r218041, r218042, r218043, r218044, r218045, r218048,
    r218049, r218119, r218132, r218138, r218139, r218147, r218148,
    r218158, r218185, r218191, r218192, r218193, r218194, r218201,
    r218214, r218215, r218217, r218218, r218370, r218373, r218374,
    r218375, r218376, r218464, r218465, r218474, r219082:
  
  r217729 (pjd):
  
  - On primary worker reload, update hr_exec field.
  - Update comment.
  
  r217730 (pjd):
  
  Use int16 for error.
  
  r217731 (pjd):
  
  Use more consistent function name with the others (pjdlogv_prefix_set()
  instead of pjdlog_prefix_setv()).
  
  r217732 (pjd):
  
  Add nv_assert() which allows to assert that the given name exists.
  
  r217737 (pjd):
  
  Add missing logs.
  
  r217784 (pjd):
  
  Don't open configuration file from worker process. Handle SIGHUP in the
  master process only and pass changes to the worker processes over control
  socket. This removes access to global namespace in preparation for capsicum
  sandboxing.
  
  r217958 (pjd):
  
  Remove __dead2 from pjdlog_verify() prototype, it does return sometimes.
  
  r217961 (pjd):
  
  - Remove obvious NOTREACHED comment after abort() call.
  - Remove redundant newline at the end of the file.
  
  r217962 (pjd):
  
  Add LOG_NDELAY flag to openlog(3) - we want descriptor to be immediately open
  so there are no surprises once we start chrooting or using capsicum.
  
  r217964 (pjd):
  
  Use pjd copyright for 2011 work.
  
  r217965 (pjd):
  
  Add functions to initialize/finalize pjdlog. This allows to open/close log
  file at will.
  
  r217966 (pjd):
  
  Extend pjdlog_verify() to support the following additional macros:
  PJDLOG_RVERIFY() - always check expression and on false log the given message
          and exit.
  PJDLOG_RASSERT() - check expression when NDEBUG is not defined and on false log
          given message and exit.
  PJDLOG_ABORT() - log the given message and exit.
  
  r217967 (pjd):
  
  Close the control socket before exiting, so it will be unlinked.
  
  r217969 (pjd):
  
  Remember created control connection so on fork(2) we can close it in child.
  
  r218040 (pjd):
  
  Initialize all global variables on pjdlog_init().
  
  r218041 (pjd):
  
  Add function to close all unneeded descriptors after fork(2).
  
  r218042 (pjd):
  
  Add comments to places where we treat errors as ciritical, but it is possible
  to handle them more gracefully.
  
  r218043 (pjd):
  
  Close all unneeded descriptors after fork(2).
  
  r218044 (pjd):
  
  Add function to assert that the only descriptors we have open are the ones
  we expect to be open. Also assert that they point at expected type.
  
  Because openlog(3) API is unable to tell us descriptor number it is using, we
  have to close syslog socket, remember assert message in local buffer and if we
  fail on assertion, reopen syslog socket and log the message.
  
  r218045 (pjd):
  
  Use newly added descriptors_assert() function to ensure only expected
  descriptors are open.
  
  r218046 (pjd), r218047 (pjd), r218119 (maxim):
  
  Add 'hast' user and 'hast' group that will be used by hastd (and maybe hastctl)
  to drop privileges.
  
  r218048 (pjd):
  
  Implement function that drops privileges by:
  - chrooting to /var/empty (user hast home directory),
  - setting groups to 'hast' (user hast primary group),
  - setting real group id, effective group id and saved group id to 'hast',
  - setting real user id, effective user id and saved user id to 'hast'.
  At the end verify that those operations where successfull.
  
  r218049 (pjd):
  
  Drop privileges in worker processes.
  
  Accepting connections and handshaking in secondary is still done before
  dropping privileges. It should be implemented by only accepting connections in
  privileged main process and passing connection descriptors to the worker, but
  is not implemented yet.
  
  r218132 (pjd):
  
  Rename pjdlog_verify() to pjdlog_abort() as it better describes what the
  the function does and mark it with __dead2.
  
  r218138 (pjd):
  
  - Use pjdlog for assertions and aborts as this will log assert/abort message
    to syslog if we run in background.
  - Asserts in proto.c that method we want to call is implemented and remove
    dummy methods from protocols implementation that are only there to abort
    the program with nice message.
  
  r218139 (pjd):
  
  Implement two new functions for sending descriptor and receving descriptor
  over UNIX domain sockets and socket pairs.
  This is in preparation for capsicum.
  
  r218147 (pjd), r218148 (pjd):
  
  Fix build on ia64.
  
  r218158 (pjd):
  
  Do not set socket send and receive buffer. It will be auto-tuned.
  
  Confirmed by:   rwatson
  
  r218185 (pjd):
  
  Be prepared that hp_client or hp_server might be NULL now.
  
  r218191 (pjd):
  
  Move protocol allocation and deallocation to separate functions.
  
  r218192 (pjd), r218201 (bz):
  
  Allow to specify connection timeout by the caller.
  
  r218193 (pjd):
  
  Add proto_connect_wait() to wait for connection to finish.
  If timeout argument to proto_connect() is -1, then the caller needs to use
  this new function to wait for connection.
  
  This change is in preparation for capsicum, where sandboxed worker wants
  to ask main process to connect in worker's behalf and pass descriptor
  to the worker. Because we don't want the main process to wait for the
  connection, it will start async connection and pass descriptor to the
  worker who will be responsible for waiting for the connection to finish.
  
  r218194 (pjd):
  
  - Rename proto_descriptor_{send,recv}() functions to
    proto_connection_{send,recv} and change them to return proto_conn
    structure. We don't operate directly on descriptors, but on
    proto_conns.
  - Add wrap method to wrap descriptor with proto_conn.
  - Remove methods to send and receive descriptors and implement this
    functionality as additional argument to send and receive methods.
  
  r218214 (pjd):
  
  Let the caller log info about successful privilege drop.
  We don't want to log this in hastctl.
  
  r218215 (pjd):
  
  Drop privileges after connecting to hastd, but before sending or receiving
  anything.
  
  r218217 (pjd):
  
  Add missing locking after moving keepalive_send() to remote send thread
  in r214692.
  
  r218218 (pjd):
  
  Setup another socketpair between parent and child, so that primary sandboxed
  worker can ask the main privileged process to connect in worker's behalf
  and then we can migrate descriptor using this socketpair to worker.
  This is not really needed now, but will be needed once we start to use
  capsicum for sandboxing.
  
  r218370 (pjd):
  
  Close more descriptors that can be open if the worker process for the given
  resource is already running.
  
  Submitted by:   Mikolaj Golub <to.my.trociny@gmail.com>
  
  r218373 (pjd):
  
  Open syslog when logging sysconf(3) failure.
  
  Reported by:    Mikolaj Golub <to.my.trociny@gmail.com>
  
  r218374 (pjd):
  
  Treat fstat(2) failure (different than EBADF) as fatal error.
  
  Reported by:    Mikolaj Golub <to.my.trociny@gmail.com>
  
  r218375 (pjd):
  
  Add (void) cast before snprintf(3)s for which we are not interested in return
  values.
  
  r218376 (pjd):
  
  Now that we break the loop on fstat(2) failure we no longer need to satisfy
  gcc's imperfections.
  
  r218464 (pjd):
  
  Unlink UNIX domain socket file only if:
  1. The descriptor is the one we are listening on (not the one when we connect
     as a client and not the one which is created on accept(2)).
  2. Descriptor was created by us (PID matches with the PID stored on bind(2)).
  
  Reported by:    Mikolaj Golub <to.my.trociny@gmail.com>
  
  r218465 (pjd):
  
  Explicitly include <sys/types.h> as suggested by getpid(2) and don't rely on
  <sys/un.h> including what's needed.
  
  r218474 (pjd):
  
  When we decide to unlink socket file, sun_path must be set. If it is set,
  but there is problem unlinking the file, log a warning.
  
  r219082 (pjd):
  
  Recognize 'reload' command, as hastd can be reloaded with the SIGHUP signal.
  
  Approved by:	pjd (mentor)

Modified:
  stable/8/etc/ftpusers
  stable/8/etc/group
  stable/8/etc/mail/aliases
  stable/8/etc/master.passwd
  stable/8/etc/rc.d/hastd
  stable/8/sbin/hastctl/hastctl.c
  stable/8/sbin/hastd/control.c
  stable/8/sbin/hastd/control.h
  stable/8/sbin/hastd/hast.h
  stable/8/sbin/hastd/hastd.c
  stable/8/sbin/hastd/hastd.h
  stable/8/sbin/hastd/nv.c
  stable/8/sbin/hastd/nv.h
  stable/8/sbin/hastd/pjdlog.c
  stable/8/sbin/hastd/pjdlog.h
  stable/8/sbin/hastd/primary.c
  stable/8/sbin/hastd/proto.c
  stable/8/sbin/hastd/proto.h
  stable/8/sbin/hastd/proto_common.c
  stable/8/sbin/hastd/proto_impl.h
  stable/8/sbin/hastd/proto_socketpair.c
  stable/8/sbin/hastd/proto_tcp4.c
  stable/8/sbin/hastd/proto_uds.c
  stable/8/sbin/hastd/secondary.c
  stable/8/sbin/hastd/subr.c
  stable/8/sbin/hastd/subr.h
Directory Properties:
  stable/8/etc/   (props changed)
  stable/8/sbin/hastctl/   (props changed)
  stable/8/sbin/hastd/   (props changed)

Modified: stable/8/etc/ftpusers
==============================================================================
--- stable/8/etc/ftpusers	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/etc/ftpusers	Mon Mar 28 17:41:10 2011	(r220104)
@@ -20,6 +20,7 @@ _dhcp
 uucp
 pop
 www
+hast
 nobody
 mailnull
 smmsp

Modified: stable/8/etc/group
==============================================================================
--- stable/8/etc/group	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/etc/group	Mon Mar 28 17:41:10 2011	(r220104)
@@ -27,5 +27,6 @@ dialer:*:68:
 network:*:69:
 audit:*:77:
 www:*:80:
+hast:*:845:
 nogroup:*:65533:
 nobody:*:65534:

Modified: stable/8/etc/mail/aliases
==============================================================================
--- stable/8/etc/mail/aliases	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/etc/mail/aliases	Mon Mar 28 17:41:10 2011	(r220104)
@@ -30,6 +30,7 @@ bin:	root
 bind:	root
 daemon:	root
 games:	root
+hast:	root
 kmem:	root
 mailnull: postmaster
 man:	root

Modified: stable/8/etc/master.passwd
==============================================================================
--- stable/8/etc/master.passwd	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/etc/master.passwd	Mon Mar 28 17:41:10 2011	(r220104)
@@ -20,4 +20,5 @@ _dhcp:*:65:65::0:0:dhcp programs:/var/em
 uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/local/libexec/uucp/uucico
 pop:*:68:6::0:0:Post Office Owner:/nonexistent:/usr/sbin/nologin
 www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin
+hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin
 nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin

Modified: stable/8/etc/rc.d/hastd
==============================================================================
--- stable/8/etc/rc.d/hastd	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/etc/rc.d/hastd	Mon Mar 28 17:41:10 2011	(r220104)
@@ -18,6 +18,7 @@ hastctl="/sbin/hastctl"
 required_files="/etc/hast.conf"
 stop_precmd="hastd_stop_precmd"
 required_modules="geom_gate:g_gate"
+extra_commands="reload"
 
 hastd_stop_precmd()
 {

Modified: stable/8/sbin/hastctl/hastctl.c
==============================================================================
--- stable/8/sbin/hastctl/hastctl.c	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastctl/hastctl.c	Mon Mar 28 17:41:10 2011	(r220104)
@@ -430,6 +430,7 @@ main(int argc, char *argv[])
 		break;
 	}
 
+	pjdlog_init(PJDLOG_MODE_STD);
 	pjdlog_debug_set(debug);
 
 	cfg = yy_config_parse(cfgpath, true);
@@ -486,10 +487,15 @@ main(int argc, char *argv[])
 		    cfg->hc_controladdr);
 	}
 	/* ...and connect to hastd. */
-	if (proto_connect(controlconn) < 0) {
+	if (proto_connect(controlconn, HAST_TIMEOUT) < 0) {
 		pjdlog_exit(EX_OSERR, "Unable to connect to hastd via %s",
 		    cfg->hc_controladdr);
 	}
+
+	if (drop_privs() != 0)
+		exit(EX_CONFIG);
+	pjdlog_debug(1, "Privileges successfully dropped.");
+
 	/* Send the command to the server... */
 	if (hast_proto_send(NULL, controlconn, nv, NULL, 0) < 0) {
 		pjdlog_exit(EX_UNAVAILABLE,

Modified: stable/8/sbin/hastd/control.c
==============================================================================
--- stable/8/sbin/hastd/control.c	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/control.c	Mon Mar 28 17:41:10 2011	(r220104)
@@ -62,6 +62,10 @@ child_cleanup(struct hast_resource *res)
 		proto_close(res->hr_event);
 		res->hr_event = NULL;
 	}
+	if (res->hr_conn != NULL) {
+		proto_close(res->hr_conn);
+		res->hr_conn = NULL;
+	}
 	res->hr_workerpid = 0;
 }
 
@@ -159,12 +163,13 @@ control_status_worker(struct hast_resour
 	nv_add_uint8(cnvout, HASTCTL_STATUS, "cmd");
 	error = nv_error(cnvout);
 	if (error != 0) {
-		/* LOG */
+		pjdlog_common(LOG_ERR, 0, error,
+		    "Unable to prepare control header");
 		goto end;
 	}
 	if (hast_proto_send(res, res->hr_ctrl, cnvout, NULL, 0) < 0) {
 		error = errno;
-		/* LOG */
+		pjdlog_errno(LOG_ERR, "Unable to send control header");
 		goto end;
 	}
 
@@ -173,17 +178,17 @@ control_status_worker(struct hast_resour
 	 */
 	if (hast_proto_recv_hdr(res->hr_ctrl, &cnvin) < 0) {
 		error = errno;
-		/* LOG */
+		pjdlog_errno(LOG_ERR, "Unable to receive control header");
 		goto end;
 	}
 
-	error = nv_get_int64(cnvin, "error");
+	error = nv_get_int16(cnvin, "error");
 	if (error != 0)
 		goto end;
 
 	if ((str = nv_get_string(cnvin, "status")) == NULL) {
 		error = ENOENT;
-		/* LOG */
+		pjdlog_errno(LOG_ERR, "Field 'status' is missing.");
 		goto end;
 	}
 	nv_add_string(nvout, str, "status%u", no);
@@ -277,6 +282,7 @@ control_handle(struct hastd_config *cfg)
 		return;
 	}
 
+	cfg->hc_controlin = conn;
 	nvin = nvout = NULL;
 	role = HAST_ROLE_UNDEF;
 
@@ -383,6 +389,7 @@ close:
 	if (nvout != NULL)
 		nv_free(nvout);
 	proto_close(conn);
+	cfg->hc_controlin = NULL;
 }
 
 /*
@@ -410,7 +417,6 @@ ctrl_thread(void *arg)
 			nv_free(nvin);
 			continue;
 		}
-		nv_free(nvin);
 		nvout = nv_alloc();
 		switch (cmd) {
 		case HASTCTL_STATUS:
@@ -432,11 +438,23 @@ ctrl_thread(void *arg)
 				nv_add_uint32(nvout, (uint32_t)0, "keepdirty");
 				nv_add_uint64(nvout, (uint64_t)0, "dirty");
 			}
+			nv_add_int16(nvout, 0, "error");
+			break;
+		case HASTCTL_RELOAD:
+			/*
+			 * When parent receives SIGHUP and discovers that
+			 * something related to us has changes, it sends reload
+			 * message to us.
+			 */
+			assert(res->hr_role == HAST_ROLE_PRIMARY);
+			primary_config_reload(res, nvin);
+			nv_add_int16(nvout, 0, "error");
 			break;
 		default:
 			nv_add_int16(nvout, EINVAL, "error");
 			break;
 		}
+		nv_free(nvin);
 		if (nv_error(nvout) != 0) {
 			pjdlog_error("Unable to create answer on control message.");
 			nv_free(nvout);

Modified: stable/8/sbin/hastd/control.h
==============================================================================
--- stable/8/sbin/hastd/control.h	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/control.h	Mon Mar 28 17:41:10 2011	(r220104)
@@ -34,6 +34,7 @@
 
 #define	HASTCTL_SET_ROLE	1
 #define	HASTCTL_STATUS		2
+#define	HASTCTL_RELOAD		3
 
 struct hastd_config;
 struct hast_resource;

Modified: stable/8/sbin/hastd/hast.h
==============================================================================
--- stable/8/sbin/hastd/hast.h	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/hast.h	Mon Mar 28 17:41:10 2011	(r220104)
@@ -81,6 +81,7 @@
 #define	HIO_FLUSH		4
 #define	HIO_KEEPALIVE		5
 
+#define	HAST_USER	"hast"
 #define	HAST_TIMEOUT	5
 #define	HAST_CONFIG	"/etc/hast.conf"
 #define	HAST_CONTROL	"/var/run/hastctl"
@@ -101,6 +102,8 @@ struct hastd_config {
 	char	 hc_controladdr[HAST_ADDRSIZE];
 	/* Protocol-specific data. */
 	struct proto_conn *hc_controlconn;
+	/* Incoming control connection. */
+	struct proto_conn *hc_controlin;
 	/* Address to listen on. */
 	char	 hc_listenaddr[HAST_ADDRSIZE];
 	/* Protocol-specific data. */
@@ -179,10 +182,12 @@ struct hast_resource {
 	int	hr_previous_role;
 	/* PID of child worker process. 0 - no child. */
 	pid_t	hr_workerpid;
-	/* Control connection between parent and child. */
+	/* Control commands from parent to child. */
 	struct proto_conn *hr_ctrl;
 	/* Events from child to parent. */
 	struct proto_conn *hr_event;
+	/* Connection requests from child to parent. */
+	struct proto_conn *hr_conn;
 
 	/* Activemap structure. */
 	struct activemap *hr_amp;

Modified: stable/8/sbin/hastd/hastd.c
==============================================================================
--- stable/8/sbin/hastd/hastd.c	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/hastd.c	Mon Mar 28 17:41:10 2011	(r220104)
@@ -1,6 +1,6 @@
 /*-
  * Copyright (c) 2009-2010 The FreeBSD Foundation
- * Copyright (c) 2010 Pawel Jakub Dawidek <pjd@FreeBSD.org>
+ * Copyright (c) 2010-2011 Pawel Jakub Dawidek <pjd@FreeBSD.org>
  * All rights reserved.
  *
  * This software was developed by Pawel Jakub Dawidek under sponsorship from
@@ -34,9 +34,9 @@ __FBSDID("$FreeBSD$");
 #include <sys/param.h>
 #include <sys/linker.h>
 #include <sys/module.h>
+#include <sys/stat.h>
 #include <sys/wait.h>
 
-#include <assert.h>
 #include <err.h>
 #include <errno.h>
 #include <libutil.h>
@@ -93,6 +93,194 @@ g_gate_load(void)
 	}
 }
 
+void
+descriptors_cleanup(struct hast_resource *res)
+{
+	struct hast_resource *tres;
+
+	TAILQ_FOREACH(tres, &cfg->hc_resources, hr_next) {
+		if (tres == res) {
+			PJDLOG_VERIFY(res->hr_role == HAST_ROLE_SECONDARY ||
+			    (res->hr_remotein == NULL &&
+			     res->hr_remoteout == NULL));
+			continue;
+		}
+		if (tres->hr_remotein != NULL)
+			proto_close(tres->hr_remotein);
+		if (tres->hr_remoteout != NULL)
+			proto_close(tres->hr_remoteout);
+		if (tres->hr_ctrl != NULL)
+			proto_close(tres->hr_ctrl);
+		if (tres->hr_event != NULL)
+			proto_close(tres->hr_event);
+		if (tres->hr_conn != NULL)
+			proto_close(tres->hr_conn);
+	}
+	if (cfg->hc_controlin != NULL)
+		proto_close(cfg->hc_controlin);
+	proto_close(cfg->hc_controlconn);
+	proto_close(cfg->hc_listenconn);
+	(void)pidfile_close(pfh);
+	hook_fini();
+	pjdlog_fini();
+}
+
+static const char *
+dtype2str(mode_t mode)
+{
+
+	if (S_ISBLK(mode))
+		return ("block device");
+	else if (S_ISCHR(mode)) 
+		return ("character device");
+	else if (S_ISDIR(mode)) 
+		return ("directory");
+	else if (S_ISFIFO(mode))
+		return ("pipe or FIFO");
+	else if (S_ISLNK(mode)) 
+		return ("symbolic link");
+	else if (S_ISREG(mode)) 
+		return ("regular file");
+	else if (S_ISSOCK(mode))
+		return ("socket");
+	else if (S_ISWHT(mode)) 
+		return ("whiteout");
+	else
+		return ("unknown");
+}
+
+void
+descriptors_assert(const struct hast_resource *res, int pjdlogmode)
+{
+	char msg[256];
+	struct stat sb;
+	long maxfd;
+	bool isopen;
+	mode_t mode;
+	int fd;
+
+	/*
+	 * At this point descriptor to syslog socket is closed, so if we want
+	 * to log assertion message, we have to first store it in 'msg' local
+	 * buffer and then open syslog socket and log it.
+	 */
+	msg[0] = '\0';
+
+	maxfd = sysconf(_SC_OPEN_MAX);
+	if (maxfd < 0) {
+		pjdlog_init(pjdlogmode);
+		pjdlog_prefix_set("[%s] (%s) ", res->hr_name,
+		    role2str(res->hr_role));
+		pjdlog_errno(LOG_WARNING, "sysconf(_SC_OPEN_MAX) failed");
+		pjdlog_fini();
+		maxfd = 16384;
+	}
+	for (fd = 0; fd <= maxfd; fd++) {
+		if (fstat(fd, &sb) == 0) {
+			isopen = true;
+			mode = sb.st_mode;
+		} else if (errno == EBADF) {
+			isopen = false;
+			mode = 0;
+		} else {
+			(void)snprintf(msg, sizeof(msg),
+			    "Unable to fstat descriptor %d: %s", fd,
+			    strerror(errno));
+			break;
+		}
+		if (fd == STDIN_FILENO || fd == STDOUT_FILENO ||
+		    fd == STDERR_FILENO) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (%s) is closed, but should be open.",
+				    fd, (fd == STDIN_FILENO ? "stdin" :
+				    (fd == STDOUT_FILENO ? "stdout" : "stderr")));
+				break;
+			}
+		} else if (fd == proto_descriptor(res->hr_event)) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (event) is closed, but should be open.",
+				    fd);
+				break;
+			}
+			if (!S_ISSOCK(mode)) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (event) is %s, but should be %s.",
+				    fd, dtype2str(mode), dtype2str(S_IFSOCK));
+				break;
+			}
+		} else if (fd == proto_descriptor(res->hr_ctrl)) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (ctrl) is closed, but should be open.",
+				    fd);
+				break;
+			}
+			if (!S_ISSOCK(mode)) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (ctrl) is %s, but should be %s.",
+				    fd, dtype2str(mode), dtype2str(S_IFSOCK));
+				break;
+			}
+		} else if (fd == proto_descriptor(res->hr_conn)) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (conn) is closed, but should be open.",
+				    fd);
+				break;
+			}
+			if (!S_ISSOCK(mode)) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (conn) is %s, but should be %s.",
+				    fd, dtype2str(mode), dtype2str(S_IFSOCK));
+				break;
+			}
+		} else if (res->hr_role == HAST_ROLE_SECONDARY &&
+		    fd == proto_descriptor(res->hr_remotein)) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (remote in) is closed, but should be open.",
+				    fd);
+				break;
+			}
+			if (!S_ISSOCK(mode)) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (remote in) is %s, but should be %s.",
+				    fd, dtype2str(mode), dtype2str(S_IFSOCK));
+				break;
+			}
+		} else if (res->hr_role == HAST_ROLE_SECONDARY &&
+		    fd == proto_descriptor(res->hr_remoteout)) {
+			if (!isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (remote out) is closed, but should be open.",
+				    fd);
+				break;
+			}
+			if (!S_ISSOCK(mode)) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d (remote out) is %s, but should be %s.",
+				    fd, dtype2str(mode), dtype2str(S_IFSOCK));
+				break;
+			}
+		} else {
+			if (isopen) {
+				(void)snprintf(msg, sizeof(msg),
+				    "Descriptor %d is open (%s), but should be closed.",
+				    fd, dtype2str(mode));
+				break;
+			}
+		}
+	}
+	if (msg[0] != '\0') {
+		pjdlog_init(pjdlogmode);
+		pjdlog_prefix_set("[%s] (%s) ", res->hr_name,
+		    role2str(res->hr_role));
+		PJDLOG_ABORT("%s", msg);
+	}
+}
+
 static void
 child_exit_log(unsigned int pid, int status)
 {
@@ -161,7 +349,7 @@ resource_needs_restart(const struct hast
     const struct hast_resource *res1)
 {
 
-	assert(strcmp(res0->hr_name, res1->hr_name) == 0);
+	PJDLOG_ASSERT(strcmp(res0->hr_name, res1->hr_name) == 0);
 
 	if (strcmp(res0->hr_provname, res1->hr_provname) != 0)
 		return (true);
@@ -186,9 +374,9 @@ resource_needs_reload(const struct hast_
     const struct hast_resource *res1)
 {
 
-	assert(strcmp(res0->hr_name, res1->hr_name) == 0);
-	assert(strcmp(res0->hr_provname, res1->hr_provname) == 0);
-	assert(strcmp(res0->hr_localpath, res1->hr_localpath) == 0);
+	PJDLOG_ASSERT(strcmp(res0->hr_name, res1->hr_name) == 0);
+	PJDLOG_ASSERT(strcmp(res0->hr_provname, res1->hr_provname) == 0);
+	PJDLOG_ASSERT(strcmp(res0->hr_localpath, res1->hr_localpath) == 0);
 
 	if (res0->hr_role != HAST_ROLE_PRIMARY)
 		return (false);
@@ -205,6 +393,45 @@ resource_needs_reload(const struct hast_
 }
 
 static void
+resource_reload(const struct hast_resource *res)
+{
+	struct nv *nvin, *nvout;
+	int error;
+
+	PJDLOG_ASSERT(res->hr_role == HAST_ROLE_PRIMARY);
+
+	nvout = nv_alloc();
+	nv_add_uint8(nvout, HASTCTL_RELOAD, "cmd");
+	nv_add_string(nvout, res->hr_remoteaddr, "remoteaddr");
+	nv_add_int32(nvout, (int32_t)res->hr_replication, "replication");
+	nv_add_int32(nvout, (int32_t)res->hr_timeout, "timeout");
+	nv_add_string(nvout, res->hr_exec, "exec");
+	if (nv_error(nvout) != 0) {
+		nv_free(nvout);
+		pjdlog_error("Unable to allocate header for reload message.");
+		return;
+	}
+	if (hast_proto_send(res, res->hr_ctrl, nvout, NULL, 0) < 0) {
+		pjdlog_errno(LOG_ERR, "Unable to send reload message");
+		nv_free(nvout);
+		return;
+	}
+	nv_free(nvout);
+
+	/* Receive response. */
+	if (hast_proto_recv_hdr(res->hr_ctrl, &nvin) < 0) {
+		pjdlog_errno(LOG_ERR, "Unable to receive reload reply");
+		return;
+	}
+	error = nv_get_int16(nvin, "error");
+	nv_free(nvin);
+	if (error != 0) {
+		pjdlog_common(LOG_ERR, 0, error, "Reload failed");
+		return;
+	}
+}
+
+static void
 hastd_reload(void)
 {
 	struct hastd_config *newcfg;
@@ -306,8 +533,9 @@ hastd_reload(void)
 	 * recreating it.
 	 *
 	 * We do just reload (send SIGHUP to worker process) if we act as
-	 * PRIMARY, but only remote address, replication mode and timeout
-	 * has changed. For those, there is no need to restart worker process.
+	 * PRIMARY, but only if remote address, replication mode, timeout or
+	 * execution path has changed. For those, there is no need to restart
+	 * worker process.
 	 * If PRIMARY receives SIGHUP, it will reconnect if remote address or
 	 * replication mode has changed or simply set new timeout if only
 	 * timeout has changed.
@@ -317,7 +545,7 @@ hastd_reload(void)
 			if (strcmp(cres->hr_name, nres->hr_name) == 0)
 				break;
 		}
-		assert(cres != NULL);
+		PJDLOG_ASSERT(cres != NULL);
 		if (resource_needs_restart(cres, nres)) {
 			pjdlog_info("Resource %s configuration was modified, restarting it.",
 			    cres->hr_name);
@@ -335,13 +563,10 @@ hastd_reload(void)
 			    sizeof(cres->hr_remoteaddr));
 			cres->hr_replication = nres->hr_replication;
 			cres->hr_timeout = nres->hr_timeout;
-			if (cres->hr_workerpid != 0) {
-				if (kill(cres->hr_workerpid, SIGHUP) < 0) {
-					pjdlog_errno(LOG_WARNING,
-					    "Unable to send SIGHUP to worker process %u",
-					    (unsigned int)cres->hr_workerpid);
-				}
-			}
+			strlcpy(cres->hr_exec, nres->hr_exec,
+			    sizeof(cres->hr_exec));
+			if (cres->hr_workerpid != 0)
+				resource_reload(cres);
 		}
 	}
 
@@ -496,10 +721,10 @@ listen_accept(void)
 	 * we have to cancel those and accept the new connection.
 	 */
 	if (token == NULL) {
-		assert(res->hr_remoteout == NULL);
+		PJDLOG_ASSERT(res->hr_remoteout == NULL);
 		pjdlog_debug(1, "Initial connection from %s.", raddr);
 		if (res->hr_workerpid != 0) {
-			assert(res->hr_remotein == NULL);
+			PJDLOG_ASSERT(res->hr_remotein == NULL);
 			pjdlog_debug(1,
 			    "Worker process exists (pid=%u), stopping it.",
 			    (unsigned int)res->hr_workerpid);
@@ -599,6 +824,41 @@ close:
 }
 
 static void
+connection_migrate(struct hast_resource *res)
+{
+	struct proto_conn *conn;
+	int16_t val = 0;
+
+	if (proto_recv(res->hr_conn, &val, sizeof(val)) < 0) {
+		pjdlog_errno(LOG_WARNING,
+		    "Unable to receive connection command");
+		return;
+	}
+	if (proto_client(res->hr_remoteaddr, &conn) < 0) {
+		val = errno;
+		pjdlog_errno(LOG_WARNING,
+		    "Unable to create outgoing connection to %s",
+		    res->hr_remoteaddr);
+		goto out;
+	}
+	if (proto_connect(conn, -1) < 0) {
+		val = errno;
+		pjdlog_errno(LOG_WARNING, "Unable to connect to %s",
+		    res->hr_remoteaddr);
+		proto_close(conn);
+		goto out;
+	}
+	val = 0;
+out:
+	if (proto_send(res->hr_conn, &val, sizeof(val)) < 0) {
+		pjdlog_errno(LOG_WARNING,
+		    "Unable to send reply to connection request");
+	}
+	if (val == 0 && proto_connection_send(res->hr_conn, conn) < 0)
+		pjdlog_errno(LOG_WARNING, "Unable to send connection");
+}
+
+static void
 main_loop(void)
 {
 	struct hast_resource *res;
@@ -629,6 +889,7 @@ main_loop(void)
 			case SIGTERM:
 				sigexit_received = true;
 				terminate_workers();
+				proto_close(cfg->hc_controlconn);
 				exit(EX_OK);
 				break;
 			case SIGCHLD:
@@ -638,29 +899,37 @@ main_loop(void)
 				hastd_reload();
 				break;
 			default:
-				assert(!"invalid condition");
+				PJDLOG_ABORT("Unexpected signal (%d).", signo);
 			}
 		}
 
 		/* Setup descriptors for select(2). */
 		FD_ZERO(&rfds);
 		maxfd = fd = proto_descriptor(cfg->hc_controlconn);
-		assert(fd >= 0);
+		PJDLOG_ASSERT(fd >= 0);
 		FD_SET(fd, &rfds);
 		fd = proto_descriptor(cfg->hc_listenconn);
-		assert(fd >= 0);
+		PJDLOG_ASSERT(fd >= 0);
 		FD_SET(fd, &rfds);
 		maxfd = fd > maxfd ? fd : maxfd;
 		TAILQ_FOREACH(res, &cfg->hc_resources, hr_next) {
 			if (res->hr_event == NULL)
 				continue;
+			PJDLOG_ASSERT(res->hr_conn != NULL);
 			fd = proto_descriptor(res->hr_event);
-			assert(fd >= 0);
+			PJDLOG_ASSERT(fd >= 0);
 			FD_SET(fd, &rfds);
 			maxfd = fd > maxfd ? fd : maxfd;
+			if (res->hr_role == HAST_ROLE_PRIMARY) {
+				/* Only primary workers asks for connections. */
+				fd = proto_descriptor(res->hr_conn);
+				PJDLOG_ASSERT(fd >= 0);
+				FD_SET(fd, &rfds);
+				maxfd = fd > maxfd ? fd : maxfd;
+			}
 		}
 
-		assert(maxfd + 1 <= (int)FD_SETSIZE);
+		PJDLOG_ASSERT(maxfd + 1 <= (int)FD_SETSIZE);
 		ret = select(maxfd + 1, &rfds, NULL, NULL, &seltimeout);
 		if (ret == 0)
 			hook_check();
@@ -678,12 +947,20 @@ main_loop(void)
 		TAILQ_FOREACH(res, &cfg->hc_resources, hr_next) {
 			if (res->hr_event == NULL)
 				continue;
+			PJDLOG_ASSERT(res->hr_conn != NULL);
 			if (FD_ISSET(proto_descriptor(res->hr_event), &rfds)) {
 				if (event_recv(res) == 0)
 					continue;
 				/* The worker process exited? */
 				proto_close(res->hr_event);
 				res->hr_event = NULL;
+				proto_close(res->hr_conn);
+				res->hr_conn = NULL;
+				continue;
+			}
+			if (res->hr_role == HAST_ROLE_PRIMARY &&
+			    FD_ISSET(proto_descriptor(res->hr_conn), &rfds)) {
+				connection_migrate(res);
 			}
 		}
 	}
@@ -735,6 +1012,7 @@ main(int argc, char *argv[])
 	argc -= optind;
 	argv += optind;
 
+	pjdlog_init(PJDLOG_MODE_STD);
 	pjdlog_debug_set(debuglevel);
 
 	g_gate_load();
@@ -751,7 +1029,7 @@ main(int argc, char *argv[])
 	}
 
 	cfg = yy_config_parse(cfgpath, true);
-	assert(cfg != NULL);
+	PJDLOG_ASSERT(cfg != NULL);
 
 	/*
 	 * Restore default actions for interesting signals in case parent

Modified: stable/8/sbin/hastd/hastd.h
==============================================================================
--- stable/8/sbin/hastd/hastd.h	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/hastd.h	Mon Mar 28 17:41:10 2011	(r220104)
@@ -43,7 +43,12 @@ extern const char *cfgpath;
 extern bool sigexit_received;
 extern struct pidfh *pfh;
 
+void descriptors_cleanup(struct hast_resource *res);
+void descriptors_assert(const struct hast_resource *res, int pjdlogmode);
+
 void hastd_primary(struct hast_resource *res);
 void hastd_secondary(struct hast_resource *res, struct nv *nvin);
 
+void primary_config_reload(struct hast_resource *res, struct nv *nv);
+
 #endif	/* !_HASTD_H_ */

Modified: stable/8/sbin/hastd/nv.c
==============================================================================
--- stable/8/sbin/hastd/nv.c	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/nv.c	Mon Mar 28 17:41:10 2011	(r220104)
@@ -563,11 +563,10 @@ nv_get_string(struct nv *nv, const char 
 	return (str);
 }
 
-bool
-nv_exists(struct nv *nv, const char *namefmt, ...)
+static bool
+nv_vexists(struct nv *nv, const char *namefmt, va_list nameap)
 {
 	struct nvhdr *nvh;
-	va_list nameap;
 	int snverror, serrno;
 
 	if (nv == NULL)
@@ -576,9 +575,7 @@ nv_exists(struct nv *nv, const char *nam
 	serrno = errno;
 	snverror = nv->nv_error;
 
-	va_start(nameap, namefmt);
 	nvh = nv_find(nv, NV_TYPE_NONE, namefmt, nameap);
-	va_end(nameap);
 
 	errno = serrno;
 	nv->nv_error = snverror;
@@ -586,6 +583,29 @@ nv_exists(struct nv *nv, const char *nam
 	return (nvh != NULL);
 }
 
+bool
+nv_exists(struct nv *nv, const char *namefmt, ...)
+{
+	va_list nameap;
+	bool ret;
+
+	va_start(nameap, namefmt);
+	ret = nv_vexists(nv, namefmt, nameap);
+	va_end(nameap);
+
+	return (ret);
+}
+
+void
+nv_assert(struct nv *nv, const char *namefmt, ...)
+{
+	va_list nameap;
+
+	va_start(nameap, namefmt);
+	assert(nv_vexists(nv, namefmt, nameap));
+	va_end(nameap);
+}
+
 /*
  * Dump content of the nv structure.
  */

Modified: stable/8/sbin/hastd/nv.h
==============================================================================
--- stable/8/sbin/hastd/nv.h	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/nv.h	Mon Mar 28 17:41:10 2011	(r220104)
@@ -127,6 +127,7 @@ const char *nv_get_string(struct nv *nv,
     __printflike(2, 3);
 
 bool nv_exists(struct nv *nv, const char *namefmt, ...) __printflike(2, 3);
+void nv_assert(struct nv *nv, const char *namefmt, ...) __printflike(2, 3);
 void nv_dump(struct nv *nv);
 
 #endif	/* !_NV_H_ */

Modified: stable/8/sbin/hastd/pjdlog.c
==============================================================================
--- stable/8/sbin/hastd/pjdlog.c	Mon Mar 28 16:58:48 2011	(r220103)
+++ stable/8/sbin/hastd/pjdlog.c	Mon Mar 28 17:41:10 2011	(r220104)
@@ -1,5 +1,6 @@
 /*-
  * Copyright (c) 2009-2010 The FreeBSD Foundation
+ * Copyright (c) 2011 Pawel Jakub Dawidek <pjd@FreeBSD.org>
  * All rights reserved.
  *
  * This software was developed by Pawel Jakub Dawidek under sponsorship from
@@ -33,6 +34,7 @@ __FBSDID("$FreeBSD$");
 #include <assert.h>
 #include <errno.h>
 #include <stdarg.h>
+#include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -40,10 +42,38 @@ __FBSDID("$FreeBSD$");
 
 #include "pjdlog.h"
 
-static int pjdlog_mode = PJDLOG_MODE_STD;
-static int pjdlog_debug_level = 0;
+static bool pjdlog_initialized = false;
+static int pjdlog_mode, pjdlog_debug_level;
 static char pjdlog_prefix[128];
 
+void
+pjdlog_init(int mode)
+{
+
+	assert(!pjdlog_initialized);
+	assert(mode == PJDLOG_MODE_STD || mode == PJDLOG_MODE_SYSLOG);
+
+	if (mode == PJDLOG_MODE_SYSLOG)
+		openlog(NULL, LOG_PID | LOG_NDELAY, LOG_DAEMON);
+	pjdlog_mode = mode;
+	pjdlog_debug_level = 0;
+	bzero(pjdlog_prefix, sizeof(pjdlog_prefix));
+
+	pjdlog_initialized = true;
+}
+
+void
+pjdlog_fini(void)
+{
+
+	assert(pjdlog_initialized);
+
+	if (pjdlog_mode == PJDLOG_MODE_SYSLOG)
+		closelog();
+
+	pjdlog_initialized = false;
+}
+
 /*
  * Configure where the logs should go.
  * By default they are send to stdout/stderr, but after going into background
@@ -54,12 +84,18 @@ void
 pjdlog_mode_set(int mode)
 {
 
+	assert(pjdlog_initialized);
 	assert(mode == PJDLOG_MODE_STD || mode == PJDLOG_MODE_SYSLOG);
 
-	pjdlog_mode = mode;
+	if (pjdlog_mode == mode)
+		return;
 
 	if (mode == PJDLOG_MODE_SYSLOG)
-		openlog(NULL, LOG_PID, LOG_DAEMON);
+		openlog(NULL, LOG_PID | LOG_NDELAY, LOG_DAEMON);
+	else /* if (mode == PJDLOG_MODE_STD) */
+		closelog();
+
+	pjdlog_mode = mode;
 }
 
 /*
@@ -69,6 +105,8 @@ int
 pjdlog_mode_get(void)
 {
 
+	assert(pjdlog_initialized);
+
 	return (pjdlog_mode);
 }
 
@@ -80,6 +118,7 @@ void
 pjdlog_debug_set(int level)
 {
 
+	assert(pjdlog_initialized);
 	assert(level >= 0);
 
 	pjdlog_debug_level = level;
@@ -92,6 +131,8 @@ int
 pjdlog_debug_get(void)
 {
 
+	assert(pjdlog_initialized);
+
 	return (pjdlog_debug_level);
 }
 
@@ -104,8 +145,10 @@ pjdlog_prefix_set(const char *fmt, ...)
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
-	pjdlog_prefix_setv(fmt, ap);
+	pjdlogv_prefix_set(fmt, ap);
 	va_end(ap);
 }
 
@@ -114,9 +157,10 @@ pjdlog_prefix_set(const char *fmt, ...)
  * Setting prefix to NULL will remove it.
  */
 void
-pjdlog_prefix_setv(const char *fmt, va_list ap)
+pjdlogv_prefix_set(const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
 	assert(fmt != NULL);
 
 	vsnprintf(pjdlog_prefix, sizeof(pjdlog_prefix), fmt, ap);
@@ -159,6 +203,8 @@ pjdlog_common(int loglevel, int debuglev
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv_common(loglevel, debuglevel, error, fmt, ap);
 	va_end(ap);
@@ -173,6 +219,7 @@ pjdlogv_common(int loglevel, int debugle
     va_list ap)
 {
 
+	assert(pjdlog_initialized);
 	assert(loglevel == LOG_EMERG || loglevel == LOG_ALERT ||
 	    loglevel == LOG_CRIT || loglevel == LOG_ERR ||
 	    loglevel == LOG_WARNING || loglevel == LOG_NOTICE ||
@@ -249,6 +296,8 @@ void
 pjdlogv(int loglevel, const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
+
 	/* LOG_DEBUG is invalid here, pjdlogv?_debug() should be used. */
 	assert(loglevel == LOG_EMERG || loglevel == LOG_ALERT ||
 	    loglevel == LOG_CRIT || loglevel == LOG_ERR ||
@@ -266,6 +315,8 @@ pjdlog(int loglevel, const char *fmt, ..
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv(loglevel, fmt, ap);
 	va_end(ap);
@@ -278,6 +329,8 @@ void
 pjdlogv_debug(int debuglevel, const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
+
 	pjdlogv_common(LOG_DEBUG, debuglevel, -1, fmt, ap);
 }
 
@@ -289,6 +342,8 @@ pjdlog_debug(int debuglevel, const char 
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv_debug(debuglevel, fmt, ap);
 	va_end(ap);
@@ -301,6 +356,8 @@ void
 pjdlogv_errno(int loglevel, const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
+
 	pjdlogv_common(loglevel, 0, errno, fmt, ap);
 }
 
@@ -312,6 +369,8 @@ pjdlog_errno(int loglevel, const char *f
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv_errno(loglevel, fmt, ap);
 	va_end(ap);
@@ -324,6 +383,8 @@ void
 pjdlogv_exit(int exitcode, const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
+
 	pjdlogv_errno(LOG_ERR, fmt, ap);
 	exit(exitcode);
 	/* NOTREACHED */
@@ -337,6 +398,8 @@ pjdlog_exit(int exitcode, const char *fm
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv_exit(exitcode, fmt, ap);
 	/* NOTREACHED */
@@ -350,6 +413,8 @@ void
 pjdlogv_exitx(int exitcode, const char *fmt, va_list ap)
 {
 
+	assert(pjdlog_initialized);
+
 	pjdlogv(LOG_ERR, fmt, ap);
 	exit(exitcode);
 	/* NOTREACHED */
@@ -363,6 +428,8 @@ pjdlog_exitx(int exitcode, const char *f
 {
 	va_list ap;
 
+	assert(pjdlog_initialized);
+
 	va_start(ap, fmt);
 	pjdlogv_exitx(exitcode, fmt, ap);
 	/* NOTREACHED */
@@ -370,21 +437,42 @@ pjdlog_exitx(int exitcode, const char *f

*** DIFF OUTPUT TRUNCATED AT 1000 LINES ***



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201103281741.p2SHfAFu001547>