From owner-freebsd-questions@FreeBSD.ORG Sat Mar 14 14:58:27 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E659E106566C for ; Sat, 14 Mar 2009 14:58:27 +0000 (UTC) (envelope-from cjk32@hermes.cam.ac.uk) Received: from ppsw-1.csi.cam.ac.uk (ppsw-1.csi.cam.ac.uk [131.111.8.131]) by mx1.freebsd.org (Postfix) with ESMTP id 7E14D8FC08 for ; Sat, 14 Mar 2009 14:58:27 +0000 (UTC) (envelope-from cjk32@hermes.cam.ac.uk) X-Cam-AntiVirus: no malware found X-Cam-SpamDetails: not scanned X-Cam-ScannerInfo: http://www.cam.ac.uk/cs/email/scanner/ Received: from hermes-2.csi.cam.ac.uk ([131.111.8.54]:42708) by ppsw-1.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.151]:25) with esmtpa (EXTERNAL:cjk32) id 1LiV2p-0006cb-5q (Exim 4.70) (return-path ); Sat, 14 Mar 2009 14:40:59 +0000 Received: from prayer by hermes-2.csi.cam.ac.uk (hermes.cam.ac.uk) with local (PRAYER:cjk32) id 1LiV2p-0004Pn-Pl (Exim 4.67) (return-path ); Sat, 14 Mar 2009 14:40:59 +0000 Received: from [88.97.163.221] by webmail.hermes.cam.ac.uk with HTTP (Prayer-1.3.1); 14 Mar 2009 14:40:59 +0000 Date: 14 Mar 2009 14:40:59 +0000 From: Christopher Key To: nightrecon@verizon.net Message-ID: In-Reply-To: References: X-Mailer: Prayer v1.3.1 Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Sender: Christopher Key Cc: freebsd-questions@freebsd.org Subject: Re: Bizarre behaviour of Linux binary under 7.1 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Mar 2009 14:58:28 -0000 On Mar 14 2009, Michael Powell wrote: >Christopher Key wrote: > >> Hello, >> >> I recently upgraded from 6.3 (i386) to 7.1p3 (amd64) with a view to >> experimenting with zfs. Mostly, everything went smoothly, but I am >> getting some very odd behaviour from a linux utility. >> >> The program is very simple, it has two executables, A and B. A is >> invoked by the user, and based upon the options given builds a list of >> files to process. B is then repeatedly invoked by A with the name of a >> file to process, and the name of a non existent file to write the >> results to. When B returns, A reads the results from the output file, >> deletes it and moves on. >> >> This all worked fine on 6.3, but cannot be made to work as intended on >> 7.1. >> >> After appropriate use of truss invoking B directly, I found that the >> source of problems was B being unable to create its output file /tmp/...: >> >> linux_open("/tmp/1234.tmp",0x42,0600) ERR#13 'Permission >> denied' >> >> which is odd. /tmp has suitable permissions: >> >> #ls -al /tmp >> drwxrwxrwt 12 root wheel 720 14 Mar 11:25 . >> drwxr-xr-x 21 root wheel 512 13 Mar 10:32 .. >> ... >> >> and I can quite happily create a identically named file in /tmp myself: >> #echo test >/tmp/1234.tmp >> #cat /tmp/1234.tmp >> test >> #rm /tmp/1234.tmp >> >> >> >> Bizarrely, however, if I instead invoke B and request its output go to >> /var/tmp/... instead of /tmp/..., it completes successfully. >> >> As a temporary workaround, I therefore tried to create a wrapper around >> B: >> >> #mv /usr/local/bin/B /usr/local/bin/B2 >> >> #cat /usr/local/bin/B #!/bin/sh B2 "$1" "$2" "/var$3" mv "/var$3" "$3" >> >> >> the idea being that the file would be written to /var/tmp/... by (as now) >> B2, then moved across by my script to where it was expected. >> >> When invoked directly, this works quite happily. However, even more >> bizarrely, when I now call A, allowing it to invoke (my) B, I get exactly >> the same behaviour from my wrapper script as (the original) B was showing >> previously, specifically, it is unable to create the file /tmp/.... >> >> As a final workaround, I inserted instead added a sleep to my script in >> place of mv ..., and instead had an external process detect the presence >> of /var/tmp/... and move it across to /tmp. This, unsurprisingly, worked. >> >> Interestingly, if I rewrote my wrapper script to, >> >> B2 "$1" "$2" "/var$3" >> sleep 3 >> cat "/var$3" > "$3" && rm "/var$3" >> >> and had the external process simply touch /tmp/..., my wrapper script >> worked, suggesting that the permissions problem is to do with creating a >> new file, not writing to an existing one. >> >> >> A few final points: >> I've tried both an md based /tmp and tmpfs with the same result. >> Everything worked perfectly on 6.3 i386. >> If I run A as root, everything works without error. >> >> My guess is that there's something a bit strange in linux_compat, either >> as a result of going to amd64 or to 7.1, and that affects both linux >> executables, and any processes that they create, but I'm not really sure >> beyond that. Can anyone shed any light on what might be going on? >> >> >> Kind Regards, >> >> Christopher Key >> > >Whenever you do an upgrade between major versions there is ABI breakage. >This necessitates either the rebuild of all installed ports or >deleting/reinstalling. > > Since my needs are relatively simple from a server perspective when I > change from one major version to another I start over from scratch and > then pull in configs from backups. When upgrading within a major version > this is not required, e.g., from 7.0 to 7.1 or 6.3 to 6.4. It's only a > consideration when it is a jump like 6.x to 7.x. > > The other approach is to use portupgrade to force all ports to be rebuilt > linked against 7.x libs. One thing to watch out for is if you're not > careful it is possible for some ports in a dependency chain to not be > rebuilt and still linked against 6.x and some do get rebuilt linked > against the new 7.x libs. This can give you flaky behavior. Byt forcing a > massive upgrade/rebuild of everything causes all ports will get linked > against 7.x libs during the rebuild. > >Don't know if this is the source of your problem, but it may be something >you can easily rule out. > Thanks Mike, This was a fresh install to an effectively clean disk. I then csupped and rebuilt with a custom kernel, before installing ports. The application in question isn't in the ports tree, and distributed in binary form, containing only the two executables. If relevant, I'm using linux_base-fc4. Having written the above, it occurs to me that I should probably try a generic kernel too. I'll give that a go and report back. Chris