Date: Thu, 26 Feb 2015 15:00:00 -0500 From: "Ellis H. Wilson III" <ellisw@panasas.com> To: <freebsd-sysinstall@freebsd.org> Subject: RFC: Patches to achieve bsdinstall script without dialogs Message-ID: <54EF7B40.2030706@panasas.com>
next in thread | raw e-mail | index | archive | help
--------------040805020502070509020205
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
All,
We use FreeBSD extensively here at Panasas, and in our day-to-day
software development it behooves us to have an automatic way to spin up
our various clients with fresh installs. Occasionally, one of these
automated installs, which relies on the various components of
bsdinstall, bails, locks up, or otherwise malfunctions. At that
juncture, checking our console logs is our best/only recourse since
these are head-less machines, but the existing code presumes (even in
the case of scripted installs) dialog will render properly. This is
typically not the case in our live console or console logs for a variety
of reasons, and this results in our engineers wading through a lot of
dialog "curses-gunk" to find the relevant pieces of ascii that have made
their way through.
Therefore, after positive discussion with Nathan Whitehorn about the
general utility of having an ascii-only install option, we have
implemented such and been running the attached patches for a few months
with success here. These enable a somewhat friendlier ASCII output when
scripted installation is requested, without sacrificing (most of) the
niceties of progress indication and the like. These also shouldn't
impact non-scripted installation, which still utilize dialogs. With the
ultimate hope of these being accepted upstream and benefiting others, I
am first presenting them here for your comment. These impact the
following files, and are named accordingly:
usr.sbin/bsdconfig/share/common.subr
usr.sbin/bsdinstall/distextract/distextract.c
usr.sbin/bsdinstall/scripts/checksum
usr.sbin/bsdinstall/scripts/script
usr.sbin/bsdinstall/partedit/partedit.c
usr.sbin/bsdinstall/partedit/scripted.c
usr.sbin/bsdinstall/partedit/partedit.h
These are patched from 10.1 releng, as that is what we are running. If
it will smooth the process towards upstreaming things, I can patch
against a different set of sources (i.e., CURRENT), just let me know.
Best,
ellis
--------------040805020502070509020205
Content-Type: text/x-patch; name="checksum.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="checksum.patch"
--- 10.1/usr.sbin/bsdinstall/scripts/checksum 2015-02-26 14:04:59.272966902 -0500
+++ usr.sbin/bsdinstall/scripts/checksum 2014-12-18 08:29:54.677947006 -0500
@@ -28,6 +28,10 @@
test -f $BSDINSTALL_DISTDIR/MANIFEST || exit 0
+if [ "$BSDINSTALL_SCRIPTED" -eq 1 ]; then
+ printf "$0: Verifying checksums of selected distributions:\n"
+fi
+
percentage=0
for dist in $DISTRIBUTIONS; do
distname=$(basename $dist .txz)
@@ -37,9 +41,14 @@
for i in $DISTRIBUTIONS; do
items="$items $i `eval echo \\\${status_$(basename $i .txz):-Pending}`"
done
- dialog --backtitle "FreeBSD Installer" --title "Checksum Verification" \
- --mixedgauge "Verifying checksums of selected distributions." \
- 0 0 $percentage $items
+
+ if [ -z "$BSDINSTALL_SCRIPTED" || "$BSDINSTALL_SCRIPTED" -ne 1 ]; then
+ dialog --backtitle "FreeBSD Installer" --title "Checksum Verification" \
+ --mixedgauge "Verifying checksums of selected distributions." \
+ 0 0 $percentage $items
+ else
+ printf "$0:\tVerifying $distname...\n"
+ fi
CK=`sha256 -q $BSDINSTALL_DISTDIR/$dist`
awk -v checksum=$CK -v dist=$dist -v found=0 '{
@@ -62,8 +71,12 @@
percentage=$(echo $percentage + 100/`echo $DISTRIBUTIONS | wc -w` | bc)
else
eval "status_$distname=1"
- dialog --backtitle "FreeBSD Installer" --title "Error" \
- --msgbox "The checksum for $dist does not match. It may have become corrupted, and should be redownloaded." 0 0
+ if [ -z "$BSDINSTALL_SCRIPTED" || "$BSDINSTALL_SCRIPTED" -ne 1 ]; then
+ dialog --backtitle "FreeBSD Installer" --title "Error" \
+ --msgbox "The checksum for $dist does not match. It may have become corrupted, and should be redownloaded." 0 0
+ else
+ printf "$0: Error: The checksum for $dist does not match. It may have become corrupted, and should be redownloaded.\n"
+ fi
exit 1
fi
done
--------------040805020502070509020205
Content-Type: text/x-patch; name="common.subr.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="common.subr.patch"
--- 10.1/usr.sbin/bsdconfig/share/common.subr 2015-02-26 14:05:08.708986060 -0500
+++ usr.sbin/bsdconfig/share/common.subr 2014-12-02 11:36:33.507794626 -0500
@@ -129,11 +129,12 @@
{
[ "$debug" ] || return $SUCCESS
local fmt="$1"; shift
+ local iso8601=$(date -u +"%Y-%m-%dT%H:%M:%SZ");
case "$debugFile" in ""|+*)
- printf "DEBUG: $fmt${fmt:+\n}" "$@" >&${TERMINAL_STDOUT_PASSTHRU:-1}
+ printf "$iso8601: DEBUG: $fmt${fmt:+\n}" "$@" >&${TERMINAL_STDOUT_PASSTHRU:-1}
esac
[ "${debugFile#+}" ] &&
- printf "DEBUG: $fmt${fmt:+\n}" "$@" >> "${debugFile#+}"
+ printf "$iso8601: DEBUG: $fmt${fmt:+\n}" "$@" >> "${debugFile#+}"
return $SUCCESS
}
--------------040805020502070509020205
Content-Type: text/x-patch; name="distextract.c.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="distextract.c.patch"
--- 10.1/usr.sbin/bsdinstall/distextract/distextract.c 2015-02-26 14:04:59.973016930 -0500
+++ usr.sbin/bsdinstall/distextract/distextract.c 2014-12-09 15:13:24.437675397 -0500
@@ -32,8 +32,13 @@
#include <limits.h>
#include <archive.h>
#include <dialog.h>
+#include <stdarg.h>
+#include <time.h>
static int extract_files(int nfiles, const char **files);
+static void scripted_printf(const char *fmt, ...);
+
+static int do_scripted = 0;
int
main(void)
@@ -63,22 +68,31 @@
for (i = 0; i < ndists; i++)
dists[i] = strsep(&diststring, " \t");
- init_dialog(stdin, stdout);
- dialog_vars.backtitle = __DECONST(char *, "FreeBSD Installer");
- dlg_put_backtitle();
+ if (getenv("BSDINSTALL_SCRIPTED") && strcmp(getenv("BSDINSTALL_SCRIPTED"), "1") == 0)
+ do_scripted = 1;
+
+ if (!do_scripted) {
+ init_dialog(stdin, stdout);
+ dialog_vars.backtitle = __DECONST(char *, "FreeBSD Installer");
+ dlg_put_backtitle();
+ }
if (chdir(getenv("BSDINSTALL_CHROOT")) != 0) {
char error[512];
sprintf(error, "Could could change to directory %s: %s\n",
getenv("BSDINSTALL_DISTDIR"), strerror(errno));
- dialog_msgbox("Error", error, 0, 0, TRUE);
- end_dialog();
+ if (!do_scripted) {
+ dialog_msgbox("Error", error, 0, 0, TRUE);
+ end_dialog();
+ } else
+ scripted_printf("Error: %s\n", error);
return (1);
}
retval = extract_files(ndists, dists);
- end_dialog();
+ if (!do_scripted)
+ end_dialog();
free(diststring);
free(dists);
@@ -133,7 +147,10 @@
snprintf(errormsg, sizeof(errormsg),
"Error while extracting %s: %s\n", file,
archive_error_string(archive));
- dialog_msgbox("Extract Error", errormsg, 0, 0, TRUE);
+ if (!do_scripted)
+ dialog_msgbox("Extract Error", errormsg, 0, 0, TRUE);
+ else
+ scripted_printf("Extract Error: %s\n", errormsg);
return (-1);
}
@@ -156,7 +173,7 @@
struct archive_entry *entry;
char errormsg[512];
char status[8];
- int i, err, progress, last_progress;
+ int i, err, progress, last_progress, last_progress_step;
err = 0;
progress = 0;
@@ -171,8 +188,11 @@
items[i*2 + 1] = "Pending";
}
- dialog_msgbox("",
- "Checking distribution archives.\nPlease wait...", 0, 0, FALSE);
+ if (!do_scripted)
+ dialog_msgbox("",
+ "Checking distribution archives.\nPlease wait...", 0, 0, FALSE);
+ else
+ scripted_printf("Checking distribution archives. Please wait...\n");
/* Count all the files */
total_files = 0;
@@ -184,6 +204,10 @@
}
current_files = 0;
+ last_progress_step = 0;
+
+ if (do_scripted)
+ scripted_printf("Extracting distribution files (0%%)");
for (i = 0; i < nfiles; i++) {
archive = archive_read_new();
@@ -204,11 +228,22 @@
(archive_file*100)/archive_files[i]);
items[i*2 + 1] = status;
- if (progress > last_progress)
- dialog_mixedgauge("Archive Extraction",
- "Extracting distribution files...", 0, 0,
- progress, nfiles,
- __DECONST(char **, items));
+ if (progress > last_progress) {
+ if (!do_scripted)
+ dialog_mixedgauge("Archive Extraction",
+ "Extracting distribution files...", 0, 0,
+ progress, nfiles,
+ __DECONST(char **, items));
+ else
+ if (progress >= last_progress_step + 10) {
+ printf("\n");
+ scripted_printf("Extracting distribution files (%d%%)", progress);
+ last_progress_step = progress;
+ } else {
+ printf(".");
+ fflush(stdout);
+ }
+ }
err = archive_read_extract(archive, entry,
ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_OWNER |
@@ -229,13 +264,41 @@
"Error while extracting %s: %s\n", items[i*2],
archive_error_string(archive));
items[i*2 + 1] = "Failed";
- dialog_msgbox("Extract Error", errormsg, 0, 0,
- TRUE);
+ if (!do_scripted)
+ dialog_msgbox("Extract Error", errormsg, 0, 0,
+ TRUE);
+ else
+ scripted_printf("Extract Error: %s\n", errormsg);
return (err);
}
archive_read_free(archive);
}
+ if (do_scripted) {
+ printf("\n");
+ scripted_printf("Extracting distribution files (100%%)\n");
+ }
+
+
return (0);
}
+
+static void
+scripted_printf(const char *fmt, ...)
+{
+ static const char fmtstr[] = "%Y-%m-%dT%H:%M:%SZ";
+ char timestr[32];
+ time_t tmval;
+ struct tm tm;
+ va_list ap;
+
+ time(&tmval);
+ gmtime_r(&tmval, &tm);
+ strftime(timestr, sizeof(timestr), fmtstr, &tm);
+ printf("%s: ", timestr);
+
+ va_start(ap, fmt);
+ vprintf(fmt, ap);
+ va_end(ap);
+}
--------------040805020502070509020205
Content-Type: text/x-patch; name="partedit.c.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="partedit.c.patch"
--- 10.1/usr.sbin/bsdinstall/partedit/partedit.c 2015-02-26 14:04:59.707606394 -0500
+++ usr.sbin/bsdinstall/partedit/partedit.c 2014-12-09 15:13:24.483607576 -0500
@@ -42,6 +42,7 @@
struct pmetadata_head part_metadata;
static int sade_mode = 0;
+static int scriptedpart_mode = 0;
static int apply_changes(struct gmesh *mesh);
static struct partedit_item *read_geom_mesh(struct gmesh *mesh, int *nitems);
@@ -61,7 +62,8 @@
gpart_revert_all(&mesh);
geom_deletetree(&mesh);
- end_dialog();
+ if (!scriptedpart_mode)
+ end_dialog();
exit(1);
}
@@ -75,40 +77,49 @@
struct gmesh mesh;
int i, op, nitems, nscroll;
int error;
+ int autopart_mode = 0;
if (strcmp(basename(argv[0]), "sade") == 0)
sade_mode = 1;
+ if (strcmp(basename(argv[0]), "autopart") == 0)
+ autopart_mode = 1;
+
+ if (strcmp(basename(argv[0]), "scriptedpart") == 0)
+ scriptedpart_mode = 1;
+
TAILQ_INIT(&part_metadata);
init_fstab_metadata();
- init_dialog(stdin, stdout);
- if (!sade_mode)
- dialog_vars.backtitle = __DECONST(char *, "FreeBSD Installer");
- dialog_vars.item_help = TRUE;
- nscroll = i = 0;
-
/* Revert changes on SIGINT */
signal(SIGINT, sigint_handler);
- if (strcmp(basename(argv[0]), "autopart") == 0) { /* Guided */
+ if (autopart_mode) /* Guided */
prompt = "Please review the disk setup. When complete, press "
"the Finish button.";
- part_wizard();
- } else if (strcmp(basename(argv[0]), "scriptedpart") == 0) {
- error = scripted_editor(argc, argv);
+ else if (scriptedpart_mode)
prompt = NULL;
- if (error != 0) {
- end_dialog();
- return (error);
- }
- } else {
+ else
prompt = "Create partitions for FreeBSD. No changes will be "
"made until you select Finish.";
+
+ if (scriptedpart_mode) {
+ error = scripted_editor(argc, argv);
+ if (error != 0)
+ return (error);
+ } else {
+ init_dialog(stdin, stdout);
+ if (!sade_mode)
+ dialog_vars.backtitle = __DECONST(char *, "FreeBSD Installer");
+ dialog_vars.item_help = TRUE;
}
+ if (autopart_mode)
+ part_wizard();
+
/* Show the part editor either immediately, or to confirm wizard */
+ nscroll = i = 0;
while (prompt != NULL) {
dlg_clear();
dlg_put_backtitle();
@@ -208,7 +219,8 @@
geom_deletetree(&mesh);
free(items);
- end_dialog();
+ if (!scriptedpart_mode)
+ end_dialog();
return (error);
}
@@ -271,9 +283,13 @@
}
if (root == NULL) {
- dialog_msgbox("Error", "No root partition was found. "
- "The root FreeBSD partition must have a mountpoint of '/'.",
- 0, 0, TRUE);
+ if (!scriptedpart_mode)
+ dialog_msgbox("Error", "No root partition was found. "
+ "The root FreeBSD partition must have a mountpoint of '/'.",
+ 0, 0, TRUE);
+ else
+ scripted_fprintf(stderr, "Error: No root partition was found. "
+ "The root FreeBSD partition must have a mountpoint of '/'.\n");
return (FALSE);
}
@@ -281,7 +297,7 @@
* Check for root partitions that we aren't formatting, which is
* usually a mistake
*/
- if (root->newfs == NULL && !sade_mode) {
+ if (root->newfs == NULL && !sade_mode && !scriptedpart_mode) {
dialog_vars.defaultno = TRUE;
cancel = dialog_yesno("Warning", "The chosen root partition "
"has a preexisting filesystem. If it contains an existing "
@@ -329,9 +345,12 @@
}
i = 0;
- dialog_mixedgauge("Initializing",
- "Initializing file systems. Please wait.", 0, 0, i*100/nitems,
- nitems, __DECONST(char **, items));
+ if (!scriptedpart_mode)
+ dialog_mixedgauge("Initializing",
+ "Initializing file systems. Please wait.", 0, 0, i*100/nitems,
+ nitems, __DECONST(char **, items));
+ else
+ scripted_fprintf(stdout, "Initializing file systems:\n");
gpart_commit(mesh);
items[i*2 + 1] = "3";
i++;
@@ -342,9 +361,12 @@
TAILQ_FOREACH(md, &part_metadata, metadata) {
if (md->newfs != NULL) {
items[i*2 + 1] = "7"; /* In progress */
- dialog_mixedgauge("Initializing",
- "Initializing file systems. Please wait.", 0, 0,
- i*100/nitems, nitems, __DECONST(char **, items));
+ if (!scriptedpart_mode)
+ dialog_mixedgauge("Initializing",
+ "Initializing file systems. Please wait.", 0, 0,
+ i*100/nitems, nitems, __DECONST(char **, items));
+ else
+ scripted_fprintf(stdout, "\t%s...\n", items[i*2]);
sprintf(message, "(echo %s; %s) >>%s 2>>%s",
md->newfs, md->newfs, getenv("BSDINSTALL_LOG"),
getenv("BSDINSTALL_LOG"));
@@ -353,9 +375,12 @@
i++;
}
}
- dialog_mixedgauge("Initializing",
- "Initializing file systems. Please wait.", 0, 0,
- i*100/nitems, nitems, __DECONST(char **, items));
+ if (!scriptedpart_mode)
+ dialog_mixedgauge("Initializing",
+ "Initializing file systems. Please wait.", 0, 0,
+ i*100/nitems, nitems, __DECONST(char **, items));
+ else
+ scripted_fprintf(stdout, "Completed initializing file systems!\n");
for (i = 1; i < nitems; i++)
free(__DECONST(char *, items[i*2]));
@@ -369,7 +394,10 @@
if (fstab == NULL) {
sprintf(message, "Cannot open fstab file %s for writing (%s)\n",
getenv("PATH_FSTAB"), strerror(errno));
- dialog_msgbox("Error", message, 0, 0, TRUE);
+ if (!scriptedpart_mode)
+ dialog_msgbox("Error", message, 0, 0, TRUE);
+ else
+ scripted_fprintf(stderr, "Error: %s\n", message);
return (-1);
}
fprintf(fstab, "# Device\tMountpoint\tFStype\tOptions\tDump\tPass#\n");
--------------040805020502070509020205
Content-Type: text/x-patch; name="partedit.h.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="partedit.h.patch"
--- 10.1/usr.sbin/bsdinstall/partedit/partedit.h 2015-02-26 14:04:59.712862309 -0500
+++ usr.sbin/bsdinstall/partedit/partedit.h 2014-12-09 15:13:24.515796476 -0500
@@ -32,6 +32,7 @@
#include <sys/queue.h>
#include <inttypes.h>
#include <fstab.h>
+#include <stdio.h>
struct gprovider;
struct gmesh;
@@ -56,6 +57,7 @@
int part_wizard(void);
int scripted_editor(int argc, const char **argv);
+void scripted_fprintf(FILE *fp, const char *fmt, ...);
int wizard_makeparts(struct gmesh *mesh, const char *disk, int interactive);
/* gpart operations */
--------------040805020502070509020205
Content-Type: text/x-patch; name="script.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="script.patch"
--- 10.1/usr.sbin/bsdinstall/scripts/script 2015-02-26 14:04:59.705037107 -0500
+++ usr.sbin/bsdinstall/scripts/script 2014-12-09 15:13:24.608203927 -0500
@@ -38,6 +38,7 @@
############################################################ CONFIGURATION
# VARIABLES:
+export BSDINSTALL_SCRIPTED=1
# PARTITIONS
# DISTRIBUTIONS
# BSDINSTALL_DISTDIR
@@ -47,7 +48,7 @@
#
# Strings that should be moved to an i18n file and loaded with f_include_lang()
#
-msg_installation_error="Installation Error!"
+msg_installation_error="Installation Error! See"
############################################################ FUNCTIONS
@@ -58,9 +59,7 @@
local file
f_getvar "$VAR_DEBUG_FILE#+" file
if [ "$file" ]; then
- f_dialog_title "$msg_installation_error"
- f_dialog_textbox "$file"
- # No need to restore title, pining for the fjords
+ printf "$0: $msg_installation_error $file"
fi
exit 1
@@ -74,7 +73,7 @@
SCRIPT="$1"
shift
-f_dprintf "Began Installation at %s" "$( date )"
+f_dprintf "$0: Began Installation at %s" "$( date )"
rm -rf $BSDINSTALL_TMPETC
mkdir $BSDINSTALL_TMPETC
@@ -89,7 +88,7 @@
export debugFile="$BSDINSTALL_LOG"
f_quietly f_debug_init
# NB: Being scripted, let debug go to terminal for invalid debugFile
- f_dprintf "Began Instalation at %s" "$( date )"
+ f_dprintf "$0: Began Installation at %s" "$( date )"
fi
# Make partitions
@@ -127,7 +126,7 @@
bsdinstall entropy
bsdinstall umount
-f_dprintf "Installation Completed at %s" "$( date )"
+f_dprintf "$0: Installation Completed at %s" "$( date )"
trap true EXIT
--------------040805020502070509020205
Content-Type: text/x-patch; name="scripted.c.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="scripted.c.patch"
--- 10.1/usr.sbin/bsdinstall/partedit/scripted.c 2015-02-26 14:04:59.277321560 -0500
+++ usr.sbin/bsdinstall/partedit/scripted.c 2014-12-09 15:13:24.541703483 -0500
@@ -34,6 +34,8 @@
#include <libgeom.h>
#include <dialog.h>
#include <dlg_keys.h>
+#include <stdarg.h>
+#include <time.h>
#include "partedit.h"
@@ -76,7 +78,7 @@
error = geom_gettree(&mesh);
if (provider_for_name(&mesh, disk) == NULL) {
- fprintf(stderr, "GEOM provider %s not found\n", disk);
+ scripted_fprintf(stderr, "GEOM provider %s not found\n", disk);
geom_deletetree(&mesh);
return (-1);
}
@@ -159,7 +161,7 @@
input++;
partconfig = strchr(input, '}');
if (partconfig == NULL) {
- fprintf(stderr, "Malformed partition setup "
+ scripted_fprintf(stderr, "Malformed partition setup "
"string: %s\n", input);
return (1);
}
@@ -174,7 +176,7 @@
else if (scheme == NULL)
scheme = strsep(&input, " \t\n");
else {
- fprintf(stderr, "Unknown directive: %s\n",
+ scripted_fprintf(stderr, "Unknown directive: %s\n",
strsep(&input, " \t\n"));
return (1);
}
@@ -211,3 +213,21 @@
return (0);
}
+void
+scripted_fprintf(FILE *fp, const char *fmt, ...)
+{
+ static const char fmtstr[] = "%Y-%m-%dT%H:%M:%SZ";
+ char timestr[32];
+ time_t tmval;
+ struct tm tm;
+ va_list ap;
+
+ time(&tmval);
+ gmtime_r(&tmval, &tm);
+ strftime(timestr, sizeof(timestr), fmtstr, &tm);
+ printf("%s: ", timestr);
+
+ va_start(ap, fmt);
+ vfprintf(fp, fmt, ap);
+ va_end(ap);
+}
--------------040805020502070509020205--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54EF7B40.2030706>