Date: 14 Mar 2009 14:40:59 +0000 From: Christopher Key <cjk32@cam.ac.uk> To: nightrecon@verizon.net Cc: freebsd-questions@freebsd.org Subject: Re: Bizarre behaviour of Linux binary under 7.1 Message-ID: <Prayer.1.3.1.0903141440590.5872@hermes-2.csi.cam.ac.uk> In-Reply-To: <gpgali$83j$1@ger.gmane.org> References: <Prayer.1.3.1.0903141229390.8732@hermes-2.csi.cam.ac.uk> <gpgali$83j$1@ger.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mar 14 2009, Michael Powell wrote:
>Christopher Key wrote:
>
>> Hello,
>>
>> I recently upgraded from 6.3 (i386) to 7.1p3 (amd64) with a view to
>> experimenting with zfs. Mostly, everything went smoothly, but I am
>> getting some very odd behaviour from a linux utility.
>>
>> The program is very simple, it has two executables, A and B. A is
>> invoked by the user, and based upon the options given builds a list of
>> files to process. B is then repeatedly invoked by A with the name of a
>> file to process, and the name of a non existent file to write the
>> results to. When B returns, A reads the results from the output file,
>> deletes it and moves on.
>>
>> This all worked fine on 6.3, but cannot be made to work as intended on
>> 7.1.
>>
>> After appropriate use of truss invoking B directly, I found that the
>> source of problems was B being unable to create its output file /tmp/...:
>>
>> linux_open("/tmp/1234.tmp",0x42,0600) ERR#13 'Permission
>> denied'
>>
>> which is odd. /tmp has suitable permissions:
>>
>> #ls -al /tmp
>> drwxrwxrwt 12 root wheel 720 14 Mar 11:25 .
>> drwxr-xr-x 21 root wheel 512 13 Mar 10:32 ..
>> ...
>>
>> and I can quite happily create a identically named file in /tmp myself:
>> #echo test >/tmp/1234.tmp
>> #cat /tmp/1234.tmp
>> test
>> #rm /tmp/1234.tmp
>>
>>
>>
>> Bizarrely, however, if I instead invoke B and request its output go to
>> /var/tmp/... instead of /tmp/..., it completes successfully.
>>
>> As a temporary workaround, I therefore tried to create a wrapper around
>> B:
>>
>> #mv /usr/local/bin/B /usr/local/bin/B2
>>
>> #cat /usr/local/bin/B #!/bin/sh B2 "$1" "$2" "/var$3" mv "/var$3" "$3"
>>
>>
>> the idea being that the file would be written to /var/tmp/... by (as now)
>> B2, then moved across by my script to where it was expected.
>>
>> When invoked directly, this works quite happily. However, even more
>> bizarrely, when I now call A, allowing it to invoke (my) B, I get exactly
>> the same behaviour from my wrapper script as (the original) B was showing
>> previously, specifically, it is unable to create the file /tmp/....
>>
>> As a final workaround, I inserted instead added a sleep to my script in
>> place of mv ..., and instead had an external process detect the presence
>> of /var/tmp/... and move it across to /tmp. This, unsurprisingly, worked.
>>
>> Interestingly, if I rewrote my wrapper script to,
>>
>> B2 "$1" "$2" "/var$3"
>> sleep 3
>> cat "/var$3" > "$3" && rm "/var$3"
>>
>> and had the external process simply touch /tmp/..., my wrapper script
>> worked, suggesting that the permissions problem is to do with creating a
>> new file, not writing to an existing one.
>>
>>
>> A few final points:
>> I've tried both an md based /tmp and tmpfs with the same result.
>> Everything worked perfectly on 6.3 i386.
>> If I run A as root, everything works without error.
>>
>> My guess is that there's something a bit strange in linux_compat, either
>> as a result of going to amd64 or to 7.1, and that affects both linux
>> executables, and any processes that they create, but I'm not really sure
>> beyond that. Can anyone shed any light on what might be going on?
>>
>>
>> Kind Regards,
>>
>> Christopher Key
>>
>
>Whenever you do an upgrade between major versions there is ABI breakage.
>This necessitates either the rebuild of all installed ports or
>deleting/reinstalling.
>
> Since my needs are relatively simple from a server perspective when I
> change from one major version to another I start over from scratch and
> then pull in configs from backups. When upgrading within a major version
> this is not required, e.g., from 7.0 to 7.1 or 6.3 to 6.4. It's only a
> consideration when it is a jump like 6.x to 7.x.
>
> The other approach is to use portupgrade to force all ports to be rebuilt
> linked against 7.x libs. One thing to watch out for is if you're not
> careful it is possible for some ports in a dependency chain to not be
> rebuilt and still linked against 6.x and some do get rebuilt linked
> against the new 7.x libs. This can give you flaky behavior. Byt forcing a
> massive upgrade/rebuild of everything causes all ports will get linked
> against 7.x libs during the rebuild.
>
>Don't know if this is the source of your problem, but it may be something
>you can easily rule out.
>
Thanks Mike,
This was a fresh install to an effectively clean disk. I then csupped and
rebuilt with a custom kernel, before installing ports. The application in
question isn't in the ports tree, and distributed in binary form,
containing only the two executables. If relevant, I'm using linux_base-fc4.
Having written the above, it occurs to me that I should probably try a
generic kernel too. I'll give that a go and report back.
Chris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Prayer.1.3.1.0903141440590.5872>