Date: Sun, 10 Oct 2010 15:49:39 -0700 From: Devin Teske <dteske@vicor.com> To: Garrett Cooper <gcooper@FreeBSD.org> Cc: Brandon Gooch <jamesbrandongooch@gmail.com>, freebsd-hackers@freebsd.org, Devin Teske <dteske@vicor.com> Subject: Re: sysrc -- a sysctl(8)-like utility for managing /etc/rc.conf et. al. Message-ID: <9BBDE5CB-F755-44DB-915A-1C90B9DE3482@vicor.com> In-Reply-To: <AANLkTinCxd9aDbr7GifEdeOpdcW4d%2B4ZcQWF0TK_mC=8@mail.gmail.com> References: <1286397912.27308.40.camel@localhost.localdomain> <AANLkTikoohMo5ng-RM3tctTH__P6cqhQpm=FPhSE9mMg@mail.gmail.com> <51B4504F-5AA4-47C5-BF23-FA51DE5BC8C8@vicor.com> <AANLkTim=BLkd229vdEst8U0ugpq3UsHPxjZZp2qaJxH-@mail.gmail.com> <238E0B24-AA12-4684-9651-84DA665BE893@vicor.com> <AANLkTinCxd9aDbr7GifEdeOpdcW4d%2B4ZcQWF0TK_mC=8@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Trimming further context...
On Oct 9, 2010, at 7:30 PM, Garrett Cooper wrote:
> Trimming out some context...
>=20
> On Sat, Oct 9, 2010 at 3:39 PM, Devin Teske <dteske@vicor.com> wrote:
>>=20
>=20
> ...
>=20
> Perhaps you meant env FAILURE=3D0 sysrc foo && reboot ?
>=20
> $ cat failure.sh
> #!/bin/sh
> echo "FAILURE: $FAILURE"
> $ FAILURE=3D0 && sh failure.sh
> FAILURE:
> $ env FAILURE=3D0 sh failure.sh
> FAILURE: 0
Ah, beautiful. I'd been searching for a way to set an environment =
variable prior to running a command in one-swift blow. I see env(1) is =
handy for that.
Though honestly, the reason it's not working for you is because FAILURE =
has not been exported...
$ while read LINE; do echo "$LINE" >> failure.sh; done =20
#!/bin/sh
echo "FAILURE: $FAILURE"
^D
### nifty way to create a file ^_^
$ cat failure.sh
#!/bin/sh
echo "FAILURE: $FAILURE"
### Yup, that's what we wrote to it.
$ unset FAILURE
### Let's start clean
$ FAILURE=3D0 && sh failure.sh
FAILURE:=20
### No effect... (cause it's not exported yet)
$ export FAILURE
### Should have an effect now, let's try
$ FAILURE=3D0 && sh failure.sh
FAILURE: 0
### Success ... once it has been exported by name (with a value or not) =
it is always exported
$ FAILURE=3D1 && sh failure.sh
FAILURE: 1
### no need to re-export, once exported by-name, new assignments are =
exported
$ unset FAILURE
### Only way to get it to be unexported once exported
$ FAILURE=3D0 && sh failure.sh
FAILURE:=20
### Assignment no longer exported to sub-shells
So, I guess an alternative to the usage of env(1) would be:
export FAILURE=3D0 && sh failure.sh
Works as expected...
$ unset FAILURE
$ export FAILURE=3D0 && sh failure.sh
FAILURE: 0
Hence why when adding environment-variable based tunables in ~/.bashrc, =
it's best to use export (either after initial assignment or with =
assignment specified on export line in-one-go) else the assignment won't =
survive the script...
Safe for four exceptions...
1. When the script itself executes the set(1) built-in with either `-a' =
flag or `-o allexport' arguments
2. The parent shell does the above and does not execute the child as a =
sub-shell but rather sources the child using the `.' built-in
3. The script itself has an invocation line resembling any of the =
following:
#!/bin/sh -a
#!/bin/sh -o allexport
#!/usr/bin/env sh -a
#!/usr/bin/env sh -o allexport
4. The parent shell does the above and does not execute the child as a =
sub-shell but rather sources the child using the `.' built-in.
Which, in any of the above cases, simple assignment to a variable name =
will cause it to be exported to all children/sub-shell environments.
Works for example in a live-shell too...
$ unset FAILURE
# start from a clean slate
$ FAILURE=3D0 && sh failure.sh=20
FAILURE:=20
# Success, we're not exporting on bare assignment
$ set -a
# Turn on allexport in the interactive shell
$ FAILURE=3D0 && sh failure.sh=20
FAILURE: 0
# Success, we're exporting on bare-assignment due to allexport
$ set +a
# Turn off allexport in the interactive shell
$ FAILURE=3D0 && sh failure.sh=20
FAILURE: 0
# It was still exported
$ unset FAILURE
# Let's unexport it
$ FAILURE=3D0 && sh failure.sh=20
FAILURE:=20
# success, no longer exported automatically via allexport feature
> Understood. There really isn't any degree of shell style in FreeBSD,
> but it would be nice if there was..
I find that to be the case quite often when dealing with shell =
scripting. I've been trying to bring a little style to the shell =
scripting world these days ^_^
>> Ah, coolness. command(1) is new to me just now ^_^
>=20
> Yeah.. I was looking for something 100% portable after I ran into
> issues with writing scripts for Solaris :).
I went back to our legacy systems just now (FreeBSD-4.11) and tried =
this...
$ uname -spr
FreeBSD 4.11-STABLE i386
$ /bin/sh -c "command -v '['"
command: unknown option: -v
Meanwhile:
$ uname -spr
FreeBSD 8.1-RELEASE-p1 amd64
$ /bin/sh -c "command -v '['"
[
So it would appear that on FreeBSD at least, type(1) built-in is more =
portable (this perhaps traces back to it's 4.4BSDLite roots).
I was thinking ... perhaps another flag. But alas, -p was the only valid =
option back then, which only causes a default PATH to be used rather =
than an inherited one.
> ...
>=20
>> If the variable expands to nothing, go ahead and let it. I've traced =
every
>> possible expansion of variables when used in the following manner:
>> [ "$VAR" ] ...
>> and it never fails. If $VAR is anything but null, the entire =
expression will
>> evaluate to true.
>> Again... coming from 15+ years of perl has made my eyes read the =
following
>> block of code:
>> if [ "$the_network_is_enabled" ]; then
>> aloud in my head as "if the network is enabled, then ..." (not too =
far of a
>> stretch)... which has a sort of quintessential humanized logic to it, =
don't
>> you think?
>> Now, contrast that with this block:
>> if [ "x$the_network_is_enabled" =3D x ]; then
>> (one might verbalize that in their head as "if x plus `the network is
>> enabled' is equal to x, then" ... which is more clear?)
>=20
> Yet, it's more complicated than that. I use the x because some
> versions are test(1) are more braindead than others and interpret the
> string as an option, not as an argument.
On legacy system:
$ uname -spr
FreeBSD 4.11-STABLE i386
$ /bin/sh -c '[ "-n" ] && echo true'
true
$ /bin/sh -c '[ "-e" ] && echo true'
true
$ /bin/sh -c '[ "-z" ] && echo true'
true
$ /bin/sh -c '[ "-r" ] && echo true'
true
$ /bin/sh -c '[ "-f" ] && echo true'
true
$ /bin/sh -c '[ "-s" ] && echo true'
true
Up-to-date is the same:
$ uname -spr
FreeBSD 8.1-RELEASE-p1 amd64
$ /bin/sh -c '[ "-n" ] && echo true'
true
$ /bin/sh -c '[ "-e" ] && echo true'
true
$ /bin/sh -c '[ "-z" ] && echo true'
true
$ /bin/sh -c '[ "-r" ] && echo true'
true
$ /bin/sh -c '[ "-f" ] && echo true'
true
$ /bin/sh -c '[ "-s" ] && echo true'
true
>> Meanwhile, in Perl, it's quite a difference to scope it to the loop =
rather
>> than the block. So, it all depends on whichever _looks_ nicer to you =
^_^
>=20
> Sure, and perl has the my keyword too :).
and the `our' keyword too ^_^
> Well, right... but if someone's taking the value out of context and
> you acted on the value in a different way, then really shouldn't be
> copy-pasting your code without understanding your intent :).
Indeed. Is it weird to have "code that is itself considerate and/or =
kind" in that respect? lol
>> Being pedantic, I would capitalize the P in permission to match
>> EACCES's output string.
>>=20
>> But, I actually copied the error verbatim from what the shell =
produces if
>> you actually try the command.
>> So... if you remove the check (if [ ! -w $file ] ... ... ...) and try =
the
>> script as non-root, you'll get exactly that error message (with =
lower-case
>> 'p' on 'permission denied').
>> It wouldn't make sense for my script to use upper-case 'P' unless the
>> bourne-shell is patched to do the same.
>> I'm simply fundamentally producing the same error message as the =
shell safe
>> for one difference... I try to detect the error before running into =
it
>> simply so I can throw a spurious newline before the error... causing =
the
>> output to more accurately mimick what sysctl(8) produces in the same =
exact
>> case (the case where a non-root user with insufficient privileges =
tries to
>> modify an MIB). Give it a shot...
>> $ sysctl security.jail.set_hostname_allowed=3D1
>> security.jail.set_hostname_allowed: 1
>> sysctl: security.jail.set_hostname_allowed: Operation not permitted
>> If I don't test for lack of write permissions first, and throw the =
error out
>> with a preceding new-line, the result would be:
>> $ sysrc foo=3Dbar
>> foo: barsysrc: cannot create /etc/rc.conf: permission denied
>> Rather than:
>> $sysrc foo=3Dbar
>> foo: bar
>> sysrc: cannot create /etc/rc.conf: permission denied
>=20
> I'm not sure which version you're using, but it looks like mine uses
> strerror(3):
>=20
> $ touch /etc/rc.conf
> touch: /etc/rc.conf: Permission denied
> $ > /etc/rc.conf
> cannot create /etc/rc.conf: Permission denied
> $ echo $SHELL
> /bin/sh
> $ uname -a
> FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #9 r211309M:
> Thu Aug 19 22:50:36 PDT 2010
> root@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA amd64
>=20
> *shrugs*
>=20
> ...
Oops! I intended to copy the message verbatim, but typo'd in the =
translation from one terminal to the next -- one of the cases where =
copy/paste could have been my friend. (realized this as I tried the same =
commands over again and got capital 'P' -- thanks)
>> I'll investigate lockf, however I think it's one of those things that =
you
>> just live with (for example... what happens if two people issue a =
sysctl(8)
>> call at the exact same time ... whoever gets there last sets the =
effective
>> value).
>=20
> There's a difference though. Most of sysctl(9) is locked with mutexes
> of various flavors; this method however is lock-free.
>=20
>> You'll notice that I do all my work in memory...
>> If the buffer is empty, I don't write out the buffer.
>> Much in the way that if an in-line sed (with -i for example) will =
also check
>> the memory contents before writing out the changes.
>> Since error-checking is performed, there's no difference between =
doing this
>> on a temporary file (essentially the memory buffer is the temporary =
file --
>> safe for wierd scenarios where memory fails you -- but then you have =
bigger
>> problems than possibly wiping out your rc.conf file -- like perhaps
>> scribbling on the disk in new and wonderful ways during memory =
corruption).
>> Also, since the calculations are done in memory and the read-in is =
decidedly
>> different than the write-out (read: not performed as a single =
command), if
>> two scripts operated simultaneously, here's what would happen:
>> script A reads rc.conf(5)
>> script B does the same
>> script A operates on in-memory buffer
>> script B does the same
>> script A writes out new rc.conf from modified memory buffer
>> script B does the same
>> whomever does the last write will have their contents preserved. The =
unlucky
>> first-writer will have his contents overwritten.
>> I do not believe the kernel will allow the two writes to intertwine =
even if
>> firing at the exact same precise moment. I do believe that one will =
block
>> until the other finishes (we could verify this by looking at perhaps =
the
>> bourne-shell's '>' redirect operator to see if it flock's the file =
during
>> the redirect, which it may, or perhaps such things are at lower =
levels).
>=20
> Even then, my concern was more about the atomicity of the operation
> than anything else. If person A modifies the file, then person B
> modifies it simultaneously, and for whatever reason person B finishes
> before person A, and person A's changes are written out to disk,
> there's not much that can be done (otherwise we'd need a database, but
> then that's smelling a lot like Windows registries, and those are a
> bi^%& to recover, if at all possible).
>=20
> I care more about the corruption case because that's a problem if the
> contents written out to disk get partially written (script killed,
> process interrupted, out of disk space, etc), or worse, the results
> get interleaved from process A and process B :/.
>=20
> There are some tricks that can be employed with test(1) (-nt, -ot),
> but it's probably just easier to use lockf when writing out the file
> because you're in a critical section of the script.
>=20
Hmmm, sysctl(9) is lock-free, which might imply that both sysctl(8) and =
sysctl(3) are also lock-free, and proposed sysrc(8) is lock-free, so =
might that imply that the atomicity tests would fare the same for all of =
the above?
Here's what I'm thinking we should do to solve this...
Since the atomicity of the write operation is anything-but singular =
(meaning, it takes incrementally larger blocks of time to write larger =
amounts of data, increasing the risk-curve for failure to occur by two =
operations coinciding at the same time -- I'm truly sorry, my wife has =
me helping her with her business statistics II course, forgive me, I'll =
clarify).
We make all our writes to a single temporary file (say, /etc/rc.conf.$$) =
and then rely on the much more atomic action of replacing the existing =
file with a move operation. For example:
Instead of this:
[ "$new_contents" ] && echo "$new_contents" > $file
Do this:
if [ "$new_contents" ]; then
echo "$new_contents" > $file.$$
mv -f $file.$$ $file
fi
The effective difference is that if two different instances of the same =
script, firing at the same precise moment, will avert writing to the =
same file at the same time, and the only point of common interest would =
be the final mv(1) command -- which I theorize the kernel would protect =
the filesystem inode structure when two programs try to move two =
different files into the same space).
Testing this in realtime would be an educational challenge/exercise. =
Though just at a glance (doing the steps slowly)... watch the inodes...
# First, make our sandbox to play in
$ sudo mkdir -p /sandbox/etc
$ sudo touch /sandbox/etc/rc.conf
# Next, show a reliable way to simulate the script's usage of $$
$ sudo /bin/sh -c 'echo "$$"'
87221
$ !!
sudo /bin/sh -c 'echo "$$"'
87222
$ !!
sudo /bin/sh -c 'echo "$$"'
87223
# The above works, "!!" re-runs the entire last-command entered and each
# instance gets a different value for the pid-expansion of "$$" as we =
expect
# Next, simulate the script writing out to the temporary file chosen
$ sudo /bin/sh -c 'echo 123 > /sandbox/etc/rc.conf.$$ && ls -li =
/sandbox/etc/rc.conf.$$'
23602 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 =
/sandbox/etc/rc.conf.87236
# Do it again, same exact code
$ !!
sudo /bin/sh -c 'echo 123 > /sandbox/etc/rc.conf.$$ && ls -li =
/sandbox/etc/rc.conf.$$'
23603 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 =
/sandbox/etc/rc.conf.87237
# What do we have at this snapshot-in-time?
$ ls -li /sandbox/etc/*
23601 -rw-r--r-- 1 root wheel 0 Oct 10 08:20 /sandbox/etc/rc.conf
23602 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 =
/sandbox/etc/rc.conf.87236
23603 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 =
/sandbox/etc/rc.conf.87237
# An untouched rc.conf, and two temporary files
# This snapshot is right before the mv command of each script
# Let's see what a mv(1) does...
$ sudo /bin/sh -c 'mv -f /sandbox/etc/rc.conf.87237 =
/sandbox/etc/rc.conf'
$ ls -li /sandbox/etc/*
23603 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 /sandbox/etc/rc.conf
23602 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 =
/sandbox/etc/rc.conf.87236
# the 23601 inode ceases to exist and in its place
# is the same filename pointing to the data at inode 23603.
# Let's now do the same with the other hypothetical instance
$ sudo /bin/sh -c 'mv -f /sandbox/etc/rc.conf.87236 =
/sandbox/etc/rc.conf'
$ ls -li /sandbox/etc/*
23602 -rw-r--r-- 1 root wheel 4 Oct 10 08:24 /sandbox/etc/rc.conf
# again the original inode ceases and the directory entry is
# updated with the newly desired inode
So, the question really comes to be...
What happens when mv(1) is called concurrently in the same exact moment?
According to `/usr/src/bin/mv/mv.c', the do_move(const char *from, const =
char *to) function ultimately relies on rename(2) (line 211, version =
1.51.2.2.2.1 RELENG_8).
And according to rename(2):
The rename() system call causes the link named `from' to be renamed as =
`to'. [snip] The rename() system call guarantees that if `to' already =
exists, an instance to `to' will always exist, even if the system should =
crash in the middle of the operation.
I think that sounds like our final solution...
Recap:
if [ "$new_contents" ]; then
echo "$new_contents" > $file.$$
mv -f $file.$$ $file
fi
> ...
>=20
>> The `-n' is already covered (see usage).
>> I do agree `-a' is both warranted and highly useful (provides system
>> administrator a snapshot of what /etc/rc sees at boot after =
performing a
>> source_rc_confs -- great for either trouble-shooting boot problems or
>> taint-checking everything before a reboot).
>=20
> Oooh -- cool (I'll have to look closer next time for `-n' :)..)!
>=20
> -a is helpful, but could become a bit tricky, esp.
Shouldn't be that hard.
Here's how we can systematically clear the environment:
for var in $(
set | awk -F=3D '/^[[:alpha:]_][[:alnum:]_]*=3D/ {print $1}'
); do
[ "$var" =3D "OPTIND" ] && continue
unset $var
done
unset var
For example:
$ uname -spr
FreeBSD 8.1-RELEASE i386
$ : start copy after semi-colon ;/bin/sh -c "# we're trailing; it's =
ok... till we hit closing quote
set | wc -l # approx. how many vars?
for var in \$(
set | awk -F=3D '/[[:alpha:]_][[:alnum:]_]*=3D/ {print \$1}'
); do
[ \"\$var\" =3D "OPTIND" ] && continue
echo unset \$var
unset \$var
done
unset var
set | /usr/bin/wc -l # how many things set now? notice PATH was unset
set # show what's currently set"(end copy at quote to the left; paste =
into shell)
24
unset BLOCKSIZE
unset FOOBAR
unset FTP_PASSIVE_MODE
unset HOME
unset IFS
unset LOGNAME
unset MAIL
unset PATH
unset PPID
unset PS1
unset PS2
unset PS4
unset PWD
unset SHELL
unset SHLVL
unset SSH_CLIENT
unset SSH_CONNECTION
unset SSH_TTY
unset TERM
unset USER
unset _
1
OPTIND=3D1
Note: The reason the first-result of "set | wc -l" produces 24 and we =
only unset 22 variables is because IFS and FOOBAR evaluate to strings =
with newlines in them.
We prune-out variables that contain newlines because the awk is =
instructed to only execute the "{print $1}" block when the line matches =
the following:
a. The line must be at least 2 characters long
b. The first character must be an alphabetic character ([a-zA-Z]) or =
underscore (_)
c. After an indeterminate number of characters, which must be =
alphanumeric ([a-zA-Z0-9]) or underscore (_), there must be an =
equals-sign (=3D).
That is all expressed in the awk regular expression =
"/^[[:alpha:]_][[:alnum:]_]*=3D/".
And, since awk is passed in a value of "=3D" to the `-F' operator, awk =
splits the fields on "=3D", so when the "{print $1}" block is executed =
(given only matching lines to the above regex), we get back a list of =
variable names that can be passed to the `unset' built-in.
Further, even if a variable had string contents that contained a newline =
and then a line that matched the awk pattern, it wouldn't matter, =
because:
a. the intent is to clear the entire environment (within a subshell) =
prior to calling source_rc_confs
b. the awk pattern ensures that we're only going to pass valid =
identifiers to `unset'
c. it doesn't matter if you try to unset something that is already unset =
(e.g., executing "unset nosuchvar" has no stderr output and returns with =
success)
I've crafted the regex pattern for awk based on what I think is the =
complete valid structure of a variable in POSIX bourne shell ... and =
tested it:
$ /bin/sh -c 'export x=3D'
# success (one-character, begins with letter)
$ /bin/sh -c 'export _x=3D'
# success (can begin with underscore)
$ /bin/sh -c 'export 3x=3D'
export: 3x: bad variable name
# can't begin with number
$ /bin/sh -c 'export _3x=3D'
# success (but can contain number)
$ /bin/sh -c 'export _3x-=3D'
export: _3x-: bad variable name
$ /bin/sh -c 'export -_3x=3D'
export: unknown option: -_
$ /bin/sh -c 'export -- -_3x=3D'
export: -_3x: bad variable name
# the dash should not be valid anywhere/anyway
$ uname -spr
FreeBSD 8.1-RELEASE i386
NOTE: "FreeBSD 4.11-STABLE i386" gave exact same results.
Last, but not least...
Not sure why, but...
$ uname -spr
FreeBSD 4.11-STABLE i386
$ /bin/sh -c 'echo OPTIND: $OPTIND; unset OPTIND; echo failed'
OPTIND: 1
unset: Illegal number:=20
$ uname -spr
FreeBSD 8.1-RELEASE-p1 amd64
$ /bin/sh -c 'echo OPTIND: $OPTIND; unset OPTIND; echo failed' =20
OPTIND: 1
unset: Illegal number:=20
> when some rc.d
> scripts live in /usr/local/etc/rc.d (can they live elsewhere? I don't
> remember OTOH..) and don't necessarily have the same constraints as
> rc.conf does... maybe some markup would need to be added to the
> scripts or external metadata, to deal with configuration information.
Nah... the purpose of sysrc(8) would be to print, munge, check, verify, =
etc. what source_rc_confs sees at boot time -- so in-turn being a tool =
to administer how those very rc.d scripts you speak of act at =
boot-time...=20
Because most all of the rc.d scripts I've seen (in `/etc/rc.d', =
`/usr/local/etc/rc.d', and any other directories that are configured in =
the `local_startup' option within rc.conf(5)).
Generally speaking though, things in these rc.d directories have two =
formats:
They either end in `.sh' or they don't.
The ones that end in `.sh' are old-style and tend to ignore =
`/etc/rc.subr'. The old-style scripts are passed in `start' at boot-time =
and they often use case/esac to switch on the first positional argument =
($1).
The ones that don't end in `.sh' are new-style and tend to source =
`/etc/rc.subr'. New-style scripts are expected to adhere to rcorder(8) =
markups placed in the script as strategic comments (preceded with `#') =
(see rc(8) for even more info).
The new-style functions are not passed a parameter at startup, rather =
the ${start_cmd}() function is called by /etc/rc at boot-time when [ =
"${name}_enable" =3D YES ] in rc.conf(5).
These scripts are more sophisticated and more modular in that they rely =
on shared-code in /etc/rc.subr
But, in the end-run, the new-style scripts tend to all run =
load_rc_config from /etc/rc.subr which ends up calling source_rc_confs =
anyway (among other things -- such as sourcing =
/etc/rc.conf.d/service_name, but we talked about sysrc(8) not having =
context of the service_name that load_rc_conf() does, though we could =
add some flag to cause sysrc(8) to emulate the startup of an rc.d script =
by-name which would follow the same actions performed by load_rc_conf, =
allowing the user of sysrc(8) to see additional values configured in =
/etc/rc.conf.d/service_name given the additional context).
>=20
> One thing that would be nice is mapping variables to humanized
> descriptions for the less understood values, but at that point it
> might be wise to point someone to a manpage for the service they're
> tweaking.
I think in a previous e-mail we talked about re-mapping `-d' from =
dependency to be more like sysctl(8)'s `-d' which is akin to "describe".
All we'd have to do is find the line in /etc/defaults/rc.conf and return =
"cat /etc/defaults/rc.conf | grep "[[:space:]]*$var=3D" | tail -1 | sed =
-e 's/.*#[[:space:]]\{1,\}//'" ... (finds last line that matches a =
bare-assignment of the variable by-name, trimming up-to first '#' =
including all additional white-space following first '#', retaining =
what's left up to the end of the line).
Admittedly, that's a poor-excuse of a parser, since it will miss =
multi-line comments. A full-on sub-shell would be much better (but I'll =
leave that for the final script which will hit the e-mail in another =
thread).
>=20
>> Well now....
>> If you really want to support ALL those possibilities... I _did_ have =
a more
>> complex routine which caught them all (each and every one), but it =
wasn't
>> quite as clean ^_^
>> If you really want me to break out the nuclear reactor, I'll work it =
back in
>> from one of the predecessors of this script which was 1,000+ lines of =
code.
>> However, I found that the need to catch such esoteric conditions was
>> far-out-weighed by the need to simplify the script and make a cleaner
>> approach.
>> Yes, the rc.conf(5) scripts (whether we're talking about =
/etc/rc.conf,
>> /etc/rc.conf.local, or ones that are appended by the end-user) can be =
quite
>> complex beasts...
>> And we could see things like this...
>> foo=3Dbar; bar=3Dbaz; baz=3D123
>> And the script would not be able to find the correct instance that =
needs to
>> be replaced to get "bar" to be some new value.
>> My nuclear-physics-type script could handle those instances (using =
sed to
>> reach into the line and replace only the baz portion and retain the =
existing
>> foo and baz declarations.
>> What would you prefer though? Something that is cleaner, more =
readable,
>> easier to digest, more efficient, and has fewer dependencies, or one =
that is
>> more robust but may require a degree to digest?
>=20
> Fair enough :P; I would clearly advertise the limitations of the
> tool with so it doesn't turn into a kitchen sink utility like
> pkg_install and sysinstall have become :/.. otherwise people love to
> add features into pieces of code that shouldn't really have those
> features.
Right-on. In the man-page would definitely be the place to advertise the =
limitations, such as:
Warning! If you have variables in rc.conf(5) which rely on the shell's =
interpretation of BLOCKs, then the modification of these variables will =
fail (currently with version 1.0 of sysrc(8) posted to this list). This =
includes BLOCKs of the form: "...", '...', {...}, (...), etc. where =
newlines are traversed innately without the need for a preceding =
back-slash before the EOL character. This is because the function that =
finds the line to replace when performing modification uses readline(3) =
(with the `-r' option), only the first line of the multi-line assignment =
will be replaced. Ultimately because readline(3) has no understanding of =
BLOCKs in the shell context and will stop reading at the EOL (note: =
back-slash not an option because `-r' is used).
For example,
$ echo 'abc=3D123
> 456
> xyz' | ( read LINE; echo "$LINE" )
abc=3D123
The above shows that `read LINE' invoked only once, read only up to the =
EOL (NOTE: we were able to embed the newline EOL character into the echo =
argument by enabling the shell's usage of BLOCKs, in this case a '...' =
block, which when we hit ENTER, the shell throws up a '> ' prompt asking =
for more data, until the block is ended with an ending apostrophe (').
This is supposed to simulate something like this in rc.conf(5):
multiline_var=3D"line 1
line 2
line 3"
Where, currently, sysrc(8) will turn the above into the following (when, =
for example, invoked as `sysrc multiline_var=3Dfoo'):
multiline_var=3D"foo"
line 2
line 3"
The remnants remain (which is not what we want, we wanted to replace the =
whole value, and the whole value includes lines 2 and 3, not to mention =
the bad-syntax arising from the prematurely terminated quote).
I could write a version that accounts for this, but it may be tricky (so =
the warning in a man-page could suffice until then).
Essentially, to solve this, we'll have to run tests on the line that was =
read-in. Unfortunately, it gets hairy even with egrep! What you really =
need is to fire-off a perl using the expert regex options like advanced =
look-ahead "(?=3D>>regex)" craziness (if that's even a valid "advanced =
look-ahead" regex, can't recall), to accurately detect a broken BLOCK, =
because you get into REAL craziness with compound strings...
=
compound_str_var=3D"strexp"_'lit-strexp'_"'strexp-surrby-sgl-quot'"_'"lit-=
strexp-w-dbl-quot"'
That's four valid blocks, "...", '...', "...", '...' (note the last-two =
are not special, they are the same as the first two, they just contain =
characters to make you think they are special... the shell treats the =
last two just as normal "..." and '...' blocks respectively).
The expansion of that compound string turns out to be a single string =
when expanded:
strexp_lit-strexp_'strexp-surrby-sql-quot'_"lit-strexp-w-dbl-qot"
If you did an `echo "$compound_str_var"' you'd actually get the above =
output (single quotes in the third block and double-quotes in the fourth =
block).
And... for our script to know whether one of the blocks was broken =
(which would cause the shell to keep reading to the next line without a =
back-slash preceding the EOL) we'd need the awesome power of perl's =
advanced look-ahead and look-behind regex in combination with variable =
quantifiers.
It's possible, but the solution, ... which indeed would be all-inclusive =
and catch all possible logic (by-way of understanding the POSIX shell =
line-interpreter at an awe-inspiring fundamental level)...
would then be dependent upon something that is no-longer installed as =
part of the base distribution ... Perl!
So... alas... the answer is...
We live with this fact and instead of trying to cow-tow to multi-line =
variable-assignment syntax in rc.conf(5), we simplify the script to =
handle all-but those cases which would require perl regex to handle ... =
have that warning in the man-page about such beasts, ... and move on ^_^ =
am I wrong?
:D
> Thanks!
> -Garrett
--
Cheers,
Devin Teske
-> CONTACT INFORMATION <-
Business Solutions Consultant II
FIS - fisglobal.com
510-735-5650 Mobile
510-621-2038 Office
510-621-2020 Office Fax
909-477-4578 Home/Fax
devin.teske@fisglobal.com
-> LEGAL DISCLAIMER <-
This message contains confidential and proprietary information
of the sender, and is intended only for the person(s) to whom it
is addressed. Any use, distribution, copying or disclosure by any
other person is strictly prohibited. If you have received this
message in error, please notify the e-mail sender immediately,
and delete the original message without making a copy.
-> END TRANSMISSION <-
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9BBDE5CB-F755-44DB-915A-1C90B9DE3482>