From owner-freebsd-questions@FreeBSD.ORG Thu Jul 2 18:19:15 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 069831065785 for ; Thu, 2 Jul 2009 18:19:15 +0000 (UTC) (envelope-from nullpt@gmail.com) Received: from mail-fx0-f218.google.com (mail-fx0-f218.google.com [209.85.220.218]) by mx1.freebsd.org (Postfix) with ESMTP id 582E28FC1C for ; Thu, 2 Jul 2009 18:19:13 +0000 (UTC) (envelope-from nullpt@gmail.com) Received: by fxm18 with SMTP id 18so1542740fxm.43 for ; Thu, 02 Jul 2009 11:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=2w7FD1pqoq+gPKre6Lo/wiD8QFAgunNZS79oLMD48fw=; b=JQQnFEG0hlHswpIRP/mu6YHxFTYVVj13Qik4uu4k6EJweH1X6GJjnu6heAYooAFXIW NVdEvP/GfZetp65ycjHt3yA+VdAY9n3GQqbUehiz8ozxKGXgF5bWua+gsh5X5Zfd2l9Y VZczwA/tEQ7p8lECKt5zUcxPjlTC/NPytDpQM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=OfDFxbiAOFEUA3k1ZU+u6/yOGXwBOPjjtpxTOPS+uetGh97GQkuCi00p1Uh0ALnaFS oRofFDp0zs24AtD2+ph6l+rLs8dc0gCLC4JWX1kwTGpXwMw1kbSPrnfgTJPkkn60s6Zy bXJ0WSVD2j4kPdX/4mnVPItCghCrq+pwVvsKI= MIME-Version: 1.0 Received: by 10.223.107.19 with SMTP id z19mr352462fao.27.1246558753063; Thu, 02 Jul 2009 11:19:13 -0700 (PDT) In-Reply-To: <87k52saz86.fsf@kobe.laptop> References: <755cb9fc0907011040o28b82cdbjd5760b139f797050@mail.gmail.com> <87tz1wqkmu.fsf@kobe.laptop> <87k52saz86.fsf@kobe.laptop> Date: Thu, 2 Jul 2009 19:19:13 +0100 Message-ID: <755cb9fc0907021119n26dee870t83ca3d1c9c5c3c90@mail.gmail.com> From: Alexandre Vieira To: Giorgos Keramidas Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Wojciech Puchar , freebsd-questions@freebsd.org Subject: Re: scripting tip needed X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jul 2009 18:19:15 -0000 On Wed, Jul 1, 2009 at 10:07 PM, Giorgos Keramidas wrote: > On Wed, 1 Jul 2009 22:02:48 +0200 (CEST), Wojciech Puchar < > wojtek@wojtek.tensor.gdynia.pl> wrote: > >> Using an interactive language like Python you can actually *test* the > >> code as you are writing it. This is a major win most of the time. > > > > could you explain what you mean? You can and you have to test a code on > > any language be it bash, ksh python or C > > Yes. I mean that one can directly interact with the interpret in a REPL > prompt, doing stuff like: > > >>> import re > >>> devre = re.compile(r'(/dev/\S+)\s+(\S+)\s.*$') > >>> devre > <_sre.SRE_Pattern object at 0x28462780> > >>> devre.match('/dev/ad0s1d 1012974 390512 541426 42% /var') > <_sre.SRE_Match object at 0x28432e78> > >>> devre.match('/dev/ad0s1d 1012974 390512 541426 42% /var').groups() > ('/dev/ad0s1d', '1012974') > >>> devre = > >>> re.compile(r'(/dev/\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+).*$') > >>> devre.match('/dev/ad0s1d 1012974 390512 541426 42% /var').groups() > ('/dev/ad0s1d', '1012974', '390512', '541426', '42%', '/var') > > See how I am 'refining' the initial regular expression without ever > leaving the Python prompt? That sort of interactivity is entirely lost > when you have to edit a file, save it, switch screen(1) windows or type > ^Z to background the editor, run a script, watch it fail and repeat. > > Then I can keep testing bits and pieces of code: > > >>> from subprocess import Popen, PIPE > >>> pipe = Popen(['df', '-k'], shell=False, stdout=PIPE).stdout > >>> for l in pipe: > ... m = devre.match(l) > ... if m: > ... print "device %s, size %ld KB" % (m.group(1), > long(m.group(2))) > ... > device /dev/ad0s1a, size 1012974 KB > device /dev/ad0s1d, size 1012974 KB > device /dev/ad0s1e, size 2026030 KB > device /dev/ad0s1f, size 10154158 KB > device /dev/ad0s1g, size 284455590 KB > device /dev/md0, size 19566 KB > >>> > > So piping df output to a Python bit of code works! That's nice. Then > once I have a 'rough idea' of how I want the script to work, I can > refactor a bit the repetitive bits: > > >>> def devsize(line): > ... m = devre.match(line) > ... if m: > ... return (m.group(1), m.group(2)) > ... > >>> devsize('/dev/ad0s1d 1012974 390512 541426 42% /var') > ('/dev/ad0s1d', '1012974') > > So here's a short function to return a nice 2-item tuple with two values > (device name, number of 1 KB blocks). Can we pipe df output through it? > > >>> pipe = Popen(['df', '-k'], shell=False, stdout=PIPE).stdout > >>> pipe = Popen(['df', '-k'], shell=False, stdout=PIPE).stdout > >>> map(devsize, pipe.readlines()) > [ None, ('/dev/ad0s1a', '1012974'), None, ('/dev/ad0s1d', '1012974'), > ('/dev/ad0s1e', '2026030'), ('/dev/ad0s1f', '10154158'), > ('/dev/ad0s1g', '284455590'), None, None, None, None, None, None, > None, None, None, None, None, None, None, None, > ('/dev/md0', '19566'), None] > >>> > > It looks we can do that too, but the tuple list may be more useful if we > trim the null items in the process: > > >>> pipe = Popen(['df', '-k'], shell=False, stdout=PIPE).stdout > >>> [t for t in map(devsize, pipe.readlines()) if t] > [ ('/dev/ad0s1a', '1012974'), ('/dev/ad0s1d', '1012974'), > ('/dev/ad0s1e', '2026030'), ('/dev/ad0s1f', '10154158'), > ('/dev/ad0s1g', '284455590'), ('/dev/md0', '19566') ] > > So there it is. A nice structure, supported by the core of the > language, using a readable, easy syntax, and listing all the /dev nodes > of my laptop along with their sizes in KBytes. > > The entire thing was built 'piece by piece', in the same Python session, > and I now have not only a 'rough idea' of how the code should work, but > also a working copy of the code in my history. > > Note the complete *lack* of care about how to append to a list, how to > create dynamic pairs of devicename-size tuples, how to map all elements > of a list through a function, and more importantly the complete and > utter lack of any sort of '"${[]}"' quoting for variable names, values, > nested expansions, and so on. > > That's what I am talking about. Shell scripts are nice, but if we are > not constrained for some reason to use only /bin/sh or ksh, there's no > excuse for wasting hours upon hours to decipher cryptic quoting rules > and exceptional edge-cases of "black quoting magic", just to get a short > job done. Being able to _easily_ use higher level structures than a > plain 'stream of bytes' is nice :) > > > Guys, I eventually found it with lots of rtfm on variable substitution and such.. [ne@dada~]$ z=0 [ne@dada~]$ y=1 [ne@dada~]$ x=aaa [ne@dada~]$ eval `echo MACHINE_DISK$z[$y]`=$x [ne@dada~]$ echo $(eval echo \${MACHINE_DISK$z[$y]}) aaa [ne@dada~]$ Thanks anyway! -- Alexandre Vieira - nullpt@gmail.com