From nobody Mon Oct 11 03:25:12 2021 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CCB7417DFEE9 for ; Mon, 11 Oct 2021 03:25:14 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from h2.pinyon.org (h2.pinyon.org [65.101.20.170]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HSPLf4F1cz4T2n for ; Mon, 11 Oct 2021 03:25:14 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from [10.0.10.15] (unknown [10.0.10.15]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by h2.pinyon.org (Postfix) with ESMTPSA id 1ACA79E73; Sun, 10 Oct 2021 20:25:13 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pinyon.org; s=dkim; t=1633922713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v3JxZbQykfZARE1cwNb6Jc5BCVNhdgjgMkefR+qGiNE=; b=jLmzRVZtCzGV49gseg8qOPU056u1B1xJrAH7mDSCoLK+19mS6Np49Vflc0ndUZcS5xqczQ ATFG05YuDdXYTZbAHpIbqWRk+ElR5g4DFw+CHfU0o8nmKMNnO9MQQoVkWt2LoZIEpYYOo8 IKKvGkgLkEwx+Z2vVkXhWYSBWKbiCC0= Message-ID: Date: Sun, 10 Oct 2021 20:25:12 -0700 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.2.0 Subject: Re: best software for managing multiple freebsd & linux machines ? Content-Language: en-US To: David Christensen References: <31ee138b-243f-ad88-aa46-9cbf5204aa03@pinyon.org> <9fd7c5b4-cf27-9940-f058-0762c4fc4a97@holgerdanske.com> From: "Russell L. Carter" Cc: freebsd-questions@freebsd.org In-Reply-To: <9fd7c5b4-cf27-9940-f058-0762c4fc4a97@holgerdanske.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.43 X-Rspamd-Server: h2 X-Rspamd-Queue-Id: 4HSPLf4F1cz4T2n X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi David, On 10/10/21 17:37, David Christensen wrote: > On 10/10/21 15:26, Russell L. Carter wrote: >> I use a single machine to keep host specific files git vc'd, > > > One Git project per host, or one Git project for everything? I don't think it matters, does it? I just use one for now. I make a flat set of OS dirs, then underneath each of those the OS specific hostnames. That way I can share common OS assumptions, included in the lower sh(1) scripts. > >> and then I just write scripts > > > What scripting language(s)? sh(1) > >> for tasks and rsync them over and execute them from the config box via >> ssh. > > So, the configuration of each host is determined by installation (e.g. > full configuration) followed by a sequence of task scripts (e.g. > differential configuration)? Yes, this is basically what something like salt does under the hood. So things like postfix/dovecot or postgresql or yadifa/unbound are going to need a bunch of post install actions. You can write them in the CMS DSL or you can write them in whatever you like. I detest sh(1) for anything complex, but realized that simple sh(1) is just fine for almost everything in configuration space. > > What if a script fails? Fail stop. A failure needs a postmortem, and then a response. Best to halt on first failure. > > Rollback? I thought this might be important when I was starting out but for my non-mission-critical stance, I rely on fail stop. I ought to state explicitly that watching py-salt fail, made me realize that rollback only works if you have the resources to do 2-phase deployment. You've got to vet that the deployment will (should?) succeed in a test environment first. Could be live, but it's still a test environment. I should also add that rollback is part of an audited infrastructure, and no, the overhead of that is something that I am not going to worry about as the sole person responsible for my infrastructure. Add significant money and of course it is required. > > > Idempotentcy? Cattle, not pets? Yeah, I do care about this. Things like sysrc(8) are helpful here, but I still don't have a good password file management story. Basically I bring a big hammer to the job if it looks complex: if things fail, stop. Triage and then fix the script(s), and then restart from the beginning. > > Do you record what scripts have run for each host, and the outcome? I should; when I have been paid to make things succeed, I always did. But for my personal systems, nope, fail stop, then triage. Never had a reason to regret that. I did regret all the time on py-salt, before I rebuilt using the lessons it taught me. > > How do you determine the configuration of a host at some point in the past? That's what the git repo is for. > > What if you perform configuration tasks by hand? I think this is a great idea when you're just starting out, or you are deploying a new software configuration, etc. Automate what you already understand. Great questions! HTH, Russell > > David >