From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 01:57:57 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 8BF21106566C; Sun, 20 Nov 2011 01:57:57 +0000 (UTC) Date: Sun, 20 Nov 2011 01:57:57 +0000 From: Alexander Best To: Gleb Kurtsou Message-ID: <20111120015757.GA71026@freebsd.org> References: <20111119100150.GA1560@reks> <20111119122538.GA47771@freebsd.org> <20111119133913.GA87101@reks> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111119133913.GA87101@reks> Cc: freebsd-hackers@FreeBSD.org Subject: Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 01:57:57 -0000 On Sat Nov 19 11, Gleb Kurtsou wrote: > On (19/11/2011 12:25), Alexander Best wrote: > > On Sat Nov 19 11, Gleb Kurtsou wrote: > > > Hi, > > > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > > correctly with -O2. Too keep long story short the code fails for gcc > > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > > -Os optimization levels are fine (I've tried with all -f* flags > > > mentioned in documentation) > > > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > > presume i386 should be fine. These options are also used for > > > compilation of kernel (with debugging enabled) and modules. > > > > > > I'm not able to share the code, but have a test case reproducing the > > > bug. I've encountered the issue over a week ago and tried narrowing it down > > > to a simple test I could share but without much success. > > > > > > The code itself is very common: initialize two structs on stack, call a > > > function with pointers to those stucts as arguments. A number of inlined > > > assertion functions. gcc fails to correctly optimize struct assignments > > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > > gcc decides not to use data coping but to assign fields directly. I've > > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > > to copy data. Replacing one particular assignment with memcpy produces > > > correct code, but that's not a solution. > > > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > > > does the issue also come up with '-fno-builtin' in addition to those flags? > > is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist > > when using -O1 and -O3? > > -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD > > -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! > > -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g > -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > -fno-strict-aliasing -- BAD > > -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > -fno-strict-aliasing -- OK btw: for any optimisation > -O1, -fno-strict-aliasing isn't needed, since it gets added automatically (see sys/conf/kern.pre.mk). cheers. alex > > So, it's -finline-functions that makes it work. But I'm afraid it just > masks the real problem. > > The rest of CFLAGS is warnings and includes: > -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W > -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type > -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter > -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 > -Wno-error=strict-aliasing > > > > > cheers. > > alex > > > > > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > > with gcc test case: > > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > > weekend as glk. > > > > > > > > > Thanks, > > > Gleb. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 04:48:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45256106566B; Sun, 20 Nov 2011 04:48:50 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 1E59D8FC0C; Sun, 20 Nov 2011 04:48:49 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id pAK4mmgV076400; Sun, 20 Nov 2011 04:48:48 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.119] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id 6w52fsxhty6xyzchypaimezhe6; Sun, 20 Nov 2011 04:48:48 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=windows-1252 From: Tim Kientzle In-Reply-To: <20111118221011.GA99985@triton8.kn-bremen.de> Date: Sat, 19 Nov 2011 20:48:47 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <649708B5-9CB4-4AEB-9409-FD27224D0365@kientzle.com> References: <20111115202450.GA73512@freebsd.org> <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118221011.GA99985@triton8.kn-bremen.de> To: Juergen Lock X-Mailer: Apple Mail (2.1251.1) Cc: Alexander Best , freebsd-hackers@freebsd.org Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 04:48:50 -0000 On Nov 18, 2011, at 2:10 PM, Juergen Lock wrote: > On Fri, Nov 18, 2011 at 12:00:07PM -0800, Tim Kientzle wrote: >>=20 >> On Nov 17, 2011, at 12:55 PM, Juergen Lock wrote: >>=20 >>>>=20 >>>> After a few experiments, bsdtar stopped using lseek() on >>>> FreeBSD for anything other than regular files and block >>>> devices. >>>=20 >>> Ah is that the reason why my patch never made it into FreeBSD 9? >>> =85. >>> Cheers, >>> Juergen (who would still like to see a faster "tar tfv = /dev/cd0"... :) >>=20 >> I would like to see that as well. >>=20 >> Take a look at=20 >>=20 >> = http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_fi= lename.c >>=20 >=20 > Ah so it's `just' a slow release cycle? >=20 > % grep DIOCGMEDIASIZE = /home/ncvs/src/lib/libarchive/archive_read_open_filename.c,v=20 > % >=20 > When will we see this code in FreeBSD? 10.0? 9.1? 8.3? :) Definitely in FreeBSD 10. Unfortunately libarchive 3.0 breaks the API and ABI, so it can't be imported into 9.1 or 8.3 wholesale. It should be possible to back port select pieces, though. Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 04:58:17 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C35F1106564A; Sun, 20 Nov 2011 04:58:17 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 9C92E8FC17; Sun, 20 Nov 2011 04:58:17 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id pAK4wGCL076489; Sun, 20 Nov 2011 04:58:16 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.119] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id 74gg7fpwxgq332x8q97n6z4qy2; Sun, 20 Nov 2011 04:58:16 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle In-Reply-To: <20111118203122.GA9508@freebsd.org> Date: Sat, 19 Nov 2011 20:58:16 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20111115202450.GA73512@freebsd.org> <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> To: Alexander Best X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 04:58:17 -0000 On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: > On Fri Nov 18 11, Tim Kientzle wrote: >>=20 >> Take a look at=20 >>=20 >> = http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_fi= lename.c >>=20 >> Especially the comments about detecting "disk-like" devices. >> I rewrote a bunch of this code to introduce an explicit >> notion of "strategy" so that we could optimize access >> to a variety of different devices. >>=20 >> This code has a notion of "disk-like" file descriptors and >> some optimizations for such. There are some comments >> in there outlining similar optimizations that could be made >> for "tape-like" or "socket-like" devices. >=20 > great you posted that file as reference. i believe most of the stuff = done there > should actually be done within lseek(). Libarchive runs on a lot of systems other than FreeBSD. FreeBSD is not the only Unix-like system with this issue, so that code isn't going to go out of libarchive regardless. If you think those same ideas can be used in dd or hd to speed them up, please send your patches. The key point: You cannot unconditionally call lseek() to skip over data. Instead, treat lseek() as an optimization that can be used under some circumstances. The question then becomes one of figuring out when that optimization can be enabled. Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 12:40:34 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 4017B1065689; Sun, 20 Nov 2011 12:40:34 +0000 (UTC) Date: Sun, 20 Nov 2011 12:40:34 +0000 From: Alexander Best To: Tim Kientzle Message-ID: <20111120124034.GA54811@freebsd.org> References: <20111115202450.GA73512@freebsd.org> <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 12:40:34 -0000 On Sat Nov 19 11, Tim Kientzle wrote: > > On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: > > > On Fri Nov 18 11, Tim Kientzle wrote: > >> > >> Take a look at > >> > >> http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > >> > >> Especially the comments about detecting "disk-like" devices. > >> I rewrote a bunch of this code to introduce an explicit > >> notion of "strategy" so that we could optimize access > >> to a variety of different devices. > >> > >> This code has a notion of "disk-like" file descriptors and > >> some optimizations for such. There are some comments > >> in there outlining similar optimizations that could be made > >> for "tape-like" or "socket-like" devices. > > > > great you posted that file as reference. i believe most of the stuff done there > > should actually be done within lseek(). > > Libarchive runs on a lot of systems other than FreeBSD. > FreeBSD is not the only Unix-like system with this issue, > so that code isn't going to go out of libarchive regardless. > > If you think those same ideas can be used in dd or hd > to speed them up, please send your patches. i'm on it. ;) i'll send in the patches in a few days. > > The key point: You cannot unconditionally call lseek() > to skip over data. Instead, treat lseek() as an optimization > that can be used under some circumstances. The > question then becomes one of figuring out when > that optimization can be enabled. ...however more importantly imo is that we fix the lseek(2) man page. i'm adding a CAVEATS section, which includes some wording from the POSIX definition, regarding the fact that lseek()'s behavior on devices that don't support seeking is undefined or better: one cannot expect lseek() to return an error, if the underlying device doesn't support seeking. cheers. alex > > Tim From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 14:03:10 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49F5F106564A for ; Sun, 20 Nov 2011 14:03:10 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id D9BBF8FC12 for ; Sun, 20 Nov 2011 14:03:09 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 3EC5F358C59; Sun, 20 Nov 2011 15:03:09 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 272E728468; Sun, 20 Nov 2011 15:03:09 +0100 (CET) Date: Sun, 20 Nov 2011 15:03:09 +0100 From: Jilles Tjoelker To: David Chisnall Message-ID: <20111120140308.GA97965@stack.nl> References: <20110919172214.GJ33993@hoeg.nl> <3A86EAEA-1861-43A6-95DC-FC700BE0E507@theravensnest.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org Subject: Re: xlocale patch X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 14:03:10 -0000 On Wed, Sep 21, 2011 at 08:28:54PM +0100, David Chisnall wrote: > And here's an updated version of the patch. I've fixed some other > bugs, including where wcstod() and wcstodl() in trunk return the wrong > value for any input string starting with spaces, wchar.h's violation > of POSIX by not declaring FILE, and a few C++ incompatibilities in > other headers (e.g. putchar being a macro, which breaks things like > std::putchar(foo)). All of my libcxxrt and libc++ changes have now > been pushed upstream, so this should now be repeatable. > > The libunwind port still has an irritating bug in the header, where > the extern "C" {} block ends with a semicolon, which causes it to be > rejected in any C++ program, but with that fixed you can compile > libcxxrt (I used cmake .. -DCMAKE_CXX_FLAGS="-I/usr/local/include > -nostdlib -g" - hopefully I'll work out how to make CMake add these > flags automatically soon...). Libc++ should build out of the box with > cmake. I think it is unfortunate that depends on #include order; we should be moving away from such ordering dependencies, not adding more. POSIX does not have such dependencies so if they add all these new _l functions, they will most likely do it differently. POSIX.1-2008 has some of the _l functions but adds them to the plain header; for example isalnum_l() is in like isalnum(), nl_langinfo_l() is in like nl_langinfo() and strcasecmp_l() is in like strcasecmp(). This appears more sensible to me. The header can then be an empty file. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 14:04:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 85B291065672; Sun, 20 Nov 2011 14:04:50 +0000 (UTC) Date: Sun, 20 Nov 2011 14:04:50 +0000 From: Alexander Best To: Tim Kientzle Message-ID: <20111120140450.GA62286@freebsd.org> References: <20111115202450.GA73512@freebsd.org> <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="XsQoSWH+UP9D9v3l" Content-Disposition: inline In-Reply-To: <20111120124034.GA54811@freebsd.org> Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 14:04:50 -0000 --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun Nov 20 11, Alexander Best wrote: > On Sat Nov 19 11, Tim Kientzle wrote: > > > > On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: > > > > > On Fri Nov 18 11, Tim Kientzle wrote: > > >> > > >> Take a look at > > >> > > >> http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > > >> > > >> Especially the comments about detecting "disk-like" devices. > > >> I rewrote a bunch of this code to introduce an explicit > > >> notion of "strategy" so that we could optimize access > > >> to a variety of different devices. > > >> > > >> This code has a notion of "disk-like" file descriptors and > > >> some optimizations for such. There are some comments > > >> in there outlining similar optimizations that could be made > > >> for "tape-like" or "socket-like" devices. > > > > > > great you posted that file as reference. i believe most of the stuff done there > > > should actually be done within lseek(). > > > > Libarchive runs on a lot of systems other than FreeBSD. > > FreeBSD is not the only Unix-like system with this issue, > > so that code isn't going to go out of libarchive regardless. > > > > If you think those same ideas can be used in dd or hd > > to speed them up, please send your patches. > > i'm on it. ;) i'll send in the patches in a few days. > > > > > The key point: You cannot unconditionally call lseek() > > to skip over data. Instead, treat lseek() as an optimization > > that can be used under some circumstances. The > > question then becomes one of figuring out when > > that optimization can be enabled. > > ...however more importantly imo is that we fix the lseek(2) man page. i'm > adding a CAVEATS section, which includes some wording from the POSIX > definition, regarding the fact that lseek()'s behavior on devices that don't > support seeking is undefined or better: one cannot expect lseek() to return > an error, if the underlying device doesn't support seeking. so...any thoughts regarding this man page patch? cheers. alex i was also wondering, wether the limits of lseek() - being, it cannot guarantee that s seek operation was actually successful - also apply to all of the libc functions in the fseek(3) man page, being: fseek(), ftell(), rewind(), fgetpos(), fsetpos(), fseeko() and ftello(). if that is the case, a similar not should also be added to the fseek(3) man page, imo. > > cheers. > alex > > > > > Tim --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="lseek2.diff" diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 index 874c523..ae183ea 100644 --- a/lib/libc/sys/lseek.2 +++ b/lib/libc/sys/lseek.2 @@ -28,7 +28,7 @@ .\" @(#)lseek.2 8.3 (Berkeley) 4/19/94 .\" $FreeBSD$ .\" -.Dd April 5, 2007 +.Dd November 20, 2011 .Dt LSEEK 2 .Os .Sh NAME @@ -114,10 +114,6 @@ If data is later written at this point, subsequent reads of the data in the gap return bytes of zeros (until data is actually written into the gap). .Pp -Some devices are incapable of seeking. -The value of the pointer -associated with such a device is undefined. -.Pp A .Qq hole is defined as a contiguous range of bytes in a file, all having the value of @@ -197,12 +193,28 @@ is associated with a pipe, socket, or FIFO. The .Fn lseek system call is expected to conform to -.St -p1003.1-90 . +.St -p1003.1-2008 . +.Pp +The +.Dv SEEK_HOLE +and +.Dv SEEK_DATA +directives, along with the +.Er ENXIO +error, are extensions to the specification. .Sh HISTORY The .Fn lseek function appeared in .At v7 . +.Sh CAVEATS +If +.Nm +is operating on a device, which is incapable of seeking, +it will request the seek operation and complete successfully. +The value of the pointer associated with such a device is undefined. +Device types which can be incapable of seeking include, +but are not limited to, tape drives. .Sh BUGS This document's use of .Fa whence --XsQoSWH+UP9D9v3l-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 14:21:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id EAB621065673; Sun, 20 Nov 2011 14:21:31 +0000 (UTC) Date: Sun, 20 Nov 2011 14:21:31 +0000 From: Alexander Best To: Tim Kientzle Message-ID: <20111120142131.GA64913@freebsd.org> References: <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="17pEHd4RhPHOinZp" Content-Disposition: inline In-Reply-To: <20111120140450.GA62286@freebsd.org> Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 14:21:32 -0000 --17pEHd4RhPHOinZp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun Nov 20 11, Alexander Best wrote: > On Sun Nov 20 11, Alexander Best wrote: > > On Sat Nov 19 11, Tim Kientzle wrote: > > > > > > On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: > > > > > > > On Fri Nov 18 11, Tim Kientzle wrote: > > > >> [snip] > > so...any thoughts regarding this man page patch? here's a revised patch. cheers. alex > > cheers. > alex > > i was also wondering, wether the limits of lseek() - being, it cannot guarantee > that s seek operation was actually successful - also apply to all of the > libc functions in the fseek(3) man page, being: > > fseek(), ftell(), rewind(), fgetpos(), fsetpos(), fseeko() and ftello(). > > if that is the case, a similar not should also be added to the fseek(3) man > page, imo. > > > [snip] --17pEHd4RhPHOinZp Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="lseek2.diff" diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 index 874c523..76102cd 100644 --- a/lib/libc/sys/lseek.2 +++ b/lib/libc/sys/lseek.2 @@ -28,7 +28,7 @@ .\" @(#)lseek.2 8.3 (Berkeley) 4/19/94 .\" $FreeBSD$ .\" -.Dd April 5, 2007 +.Dd November 20, 2011 .Dt LSEEK 2 .Os .Sh NAME @@ -113,10 +113,9 @@ of the existing end-of-file of the file. If data is later written at this point, subsequent reads of the data in the gap return bytes of zeros (until data is actually written into the gap). -.Pp -Some devices are incapable of seeking. -The value of the pointer -associated with such a device is undefined. +However, the +.Fn lseek +system call does not, by itself, extend the size of a file. .Pp A .Qq hole @@ -197,12 +196,28 @@ is associated with a pipe, socket, or FIFO. The .Fn lseek system call is expected to conform to -.St -p1003.1-90 . +.St -p1003.1-2008 . +.Pp +The +.Dv SEEK_HOLE +and +.Dv SEEK_DATA +directives, along with the +.Er ENXIO +error, are extensions to the specification. .Sh HISTORY The .Fn lseek function appeared in .At v7 . +.Sh CAVEATS +If the +.Fn lseek +system call is operating on a device, which is incapable of seeking, +it will request the seek operation and complete successfully. +The value of the pointer associated with such a device is undefined. +Device types which can be incapable of seeking include, +but are not limited to, tape drives. .Sh BUGS This document's use of .Fa whence --17pEHd4RhPHOinZp-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 15:36:55 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id BD1DC106566B; Sun, 20 Nov 2011 15:36:55 +0000 (UTC) Date: Sun, 20 Nov 2011 15:36:55 +0000 From: Alexander Best To: Tim Kientzle Message-ID: <20111120153655.GA72193@freebsd.org> References: <20111115202450.GA73512@freebsd.org> <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="zhXaljGHf11kAtnf" Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org, Juergen Lock Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 15:36:55 -0000 --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sat Nov 19 11, Tim Kientzle wrote: > > On Nov 18, 2011, at 12:31 PM, Alexander Best wrote: > > > On Fri Nov 18 11, Tim Kientzle wrote: > >> > >> Take a look at > >> > >> http://libarchive.googlecode.com/svn/trunk/libarchive/archive_read_open_filename.c > >> > >> Especially the comments about detecting "disk-like" devices. > >> I rewrote a bunch of this code to introduce an explicit > >> notion of "strategy" so that we could optimize access > >> to a variety of different devices. > >> > >> This code has a notion of "disk-like" file descriptors and > >> some optimizations for such. There are some comments > >> in there outlining similar optimizations that could be made > >> for "tape-like" or "socket-like" devices. > > > > great you posted that file as reference. i believe most of the stuff done there > > should actually be done within lseek(). > > Libarchive runs on a lot of systems other than FreeBSD. > FreeBSD is not the only Unix-like system with this issue, > so that code isn't going to go out of libarchive regardless. > > If you think those same ideas can be used in dd or hd > to speed them up, please send your patches. i'd like to propose the followup patch for hexdump(1). basically, the logic behind is is this: 1) if the file argument is a fifo, pipe or socket -- goto 4) 2) if the file argument is a tape drive -- goto 4) 3) for all other cases try fseeko(), if that fails -- goto 4) 4) use getchar() you should notice a dramtic increase in speed from something like the following: 'hexdump -s 500m -n32 /dev/random' cheers. alex > > The key point: You cannot unconditionally call lseek() > to skip over data. Instead, treat lseek() as an optimization > that can be used under some circumstances. The > question then becomes one of figuring out when > that optimization can be enabled. > > Tim > --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="hexdump.diff" diff --git a/usr.bin/hexdump/display.c b/usr.bin/hexdump/display.c index 991509d..8c8b065 100644 --- a/usr.bin/hexdump/display.c +++ b/usr.bin/hexdump/display.c @@ -35,8 +35,10 @@ static char sccsid[] = "@(#)display.c 8.1 (Berkeley) 6/6/93"; #include __FBSDID("$FreeBSD$"); +#include #include #include +#include #include #include @@ -368,7 +370,7 @@ next(char **argv) void doskip(const char *fname, int statok) { - int cnt; + int type; struct stat sb; if (statok) { @@ -380,16 +382,38 @@ doskip(const char *fname, int statok) return; } } - if (S_ISREG(sb.st_mode)) { - if (fseeko(stdin, skip, SEEK_SET)) + if (S_ISFIFO(sb.st_mode) || S_ISSOCK(sb.st_mode)) { + noseek(); + return; + } + if (S_ISCHR(sb.st_mode) || S_ISBLK(sb.st_mode)) { + if (ioctl(fileno(stdin), FIODTYPE, &type)) err(1, "%s", fname); - address += skip; - skip = 0; - } else { - for (cnt = 0; cnt < skip; ++cnt) - if (getchar() == EOF) - break; - address += cnt; - skip -= cnt; + /* + * Most tape drives don't support seeking, + * yet fseeko() would succeed. + */ + if (type & D_TAPE) { + noseek(); + return; + } + } + if (fseeko(stdin, skip, SEEK_SET)) { + noseek(); + return; } + address += skip; + skip = 0; +} + +void +noseek(void) +{ + int count; + + for (count = 0; count < skip; ++count) + if (getchar() == EOF) + break; + address += count; + skip -= count; } diff --git a/usr.bin/hexdump/hexdump.h b/usr.bin/hexdump/hexdump.h index be85bd9..1d4bb85 100644 --- a/usr.bin/hexdump/hexdump.h +++ b/usr.bin/hexdump/hexdump.h @@ -97,6 +97,7 @@ u_char *get(void); void newsyntax(int, char ***); int next(char **); void nomem(void); +void noseek(void); void oldsyntax(int, char ***); size_t peek(u_char *, size_t); void rewrite(FS *); --zhXaljGHf11kAtnf-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 17:40:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 035B51065670 for ; Sun, 20 Nov 2011 17:40:29 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id A43B68FC17 for ; Sun, 20 Nov 2011 17:40:28 +0000 (UTC) Received: (qmail 17760 invoked by uid 0); 20 Nov 2011 17:40:27 -0000 Received: from 67.206.161.167 by rms-us017 with HTTP Content-Type: text/plain; charset="utf-8" Date: Sun, 20 Nov 2011 12:40:16 -0500 From: "Dieter BSD" Message-ID: <20111120174025.233420@gmx.com> MIME-Version: 1.0 To: freebsd-hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: vQn+eUciynCoPwXIBmBnbW5vcmZ1Zpy7 Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 17:40:29 -0000 > something like the following inside lseek() would take care of tape drives: > >        if (S_ISCHR(sb.st_mode) || S_ISBLK(sb.st_mode)) { >                if (ioctl(io->fd, FIODTYPE, &type) == -1) >                        err(1, "%s", io->name); > >                if (type & D_TAPE) >                        return(EBADF) >        } I'd suggest ENODEV ("Operation not supported by device") rather than EBADF ("Bad file descriptor"). To do this correctly, we'd need some standard way to ask the device driver if the device can perform the seek or not. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 17:51:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id C937C1065676; Sun, 20 Nov 2011 17:51:00 +0000 (UTC) Date: Sun, 20 Nov 2011 17:51:00 +0000 From: Alexander Best To: Dieter BSD Message-ID: <20111120175100.GA10240@freebsd.org> References: <20111120174025.233420@gmx.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20111120174025.233420@gmx.com> Cc: freebsd-hackers@freebsd.org Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 17:51:00 -0000 On Sun Nov 20 11, Dieter BSD wrote: > > something like the following inside lseek() would take care of tape drives: > > > >        if (S_ISCHR(sb.st_mode) || S_ISBLK(sb.st_mode)) { > >                if (ioctl(io->fd, FIODTYPE, &type) == -1) > >                        err(1, "%s", io->name); > > > >                if (type & D_TAPE) > >                        return(EBADF) > >        } > > I'd suggest ENODEV ("Operation not supported by device") rather than > EBADF ("Bad file descriptor"). > > To do this correctly, we'd need some standard way to ask the > device driver if the device can perform the seek or not. this might be a bit of a problem, since those devices and drivers that don't support seeking are mostly legacy tape drives. i don't think these drivers are actively being maintained atm. another issue are removable media. running the patched version of hexdump (i posted a patch earlier today) shows that running fseeko() against /dev/cd0, where no media is present, succeeds. with a media inserted however, /dev/cd0 is very well capable of seeking. cheers. alex > From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 18:24:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67355106566B; Sun, 20 Nov 2011 18:24:33 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id B04A78FC08; Sun, 20 Nov 2011 18:24:32 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so7556103bkb.13 for ; Sun, 20 Nov 2011 10:24:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=mj0GghX4ejGsJoKQcZbSqGGb1gl/Sk3NXpbkghkT6No=; b=FkSIOy3ofvjOUX/K+19UUH2asgi9x5EXGT6dQkPLw43MiCdDtoQcQMTIdwKEqPBWX8 AcH0LP8nctHVndG9uo/Fanz2DrvLeVWdMVAhpI+1jFNC+sgqr3rwxNJKb7s7oi7Im/3Y qEEyDoyiK80KqTPa+tqtMB741tY4cmMSfnXb8= Received: by 10.204.10.81 with SMTP id o17mr11339442bko.65.1321813471367; Sun, 20 Nov 2011 10:24:31 -0800 (PST) Received: from localhost ([78.157.92.5]) by mx.google.com with ESMTPS id e18sm5379459bkr.15.2011.11.20.10.24.29 (version=SSLv3 cipher=OTHER); Sun, 20 Nov 2011 10:24:29 -0800 (PST) Date: Sun, 20 Nov 2011 20:24:30 +0200 From: Gleb Kurtsou To: mdf@FreeBSD.org Message-ID: <20111120182430.GA1672@reks> References: <20111119100150.GA1560@reks> <20111119161958.GA91681@reks> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org Subject: Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 18:24:33 -0000 On (19/11/2011 09:11), mdf@FreeBSD.org wrote: > On Sat, Nov 19, 2011 at 8:19 AM, Gleb Kurtsou wrote: > > On (19/11/2011 07:26), mdf@FreeBSD.org wrote: > >> On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou wrote: > >> > Hi, > >> > > >> > I was lucky to write a bit of code which gcc 4.2 fails to compile > >> > correctly with -O2. Too keep long story short the code fails for gcc > >> > from base system and last gcc 4.2 snapshot from ports. It works with gcc > >> > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > >> > -Os optimization levels are fine (I've tried with all -f* flags > >> > mentioned in documentation) > >> > > >> > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > >> > presume i386 should be fine. These options are also used for > >> > compilation of kernel (with debugging enabled) and modules. > >> > > >> > I'm not able to share the code, but have a test case reproducing the > >> > bug. I've encountered the issue over a week ago and tried narrowing it down > >> > to a simple test I could share but without much success. > >> > > >> > The code itself is very common: initialize two structs on stack, call a > >> > function with pointers to those stucts as arguments. A number of inlined > >> > assertion functions. gcc fails to correctly optimize struct assignments > >> > with -fno-omit-frame-pointer, I have a number of small structs assigned, > >> > gcc decides not to use data coping but to assign fields directly. I've > >> > tried disabling sra, tweaking sra parameters -- no luck in forcing it > >> > to copy data. Replacing one particular assignment with memcpy produces > >> > correct code, but that's not a solution. > >> > >> How small are the structs?  gcc has an optimization for structs that > >> are no larger than a register, but it's buggy in 4.2 and we disabled > >> it at $WORK.  I can dig up the patch if this is the problem. > > struct sockaddr_in in this particular test. 16 bytes. > > > > Register size structs are rather common, e.g. struct in_addr. > > > > I could test the patch. Adding -finline-functions seems to fix the issue > > for me. > > I can't find the thing I'm thinking of. The only potentially relevant > patch I see in our gcc sources is this: It could be related but doesn't fix bug I observe. I've installed fresh 9.0-RC2 virtual machine and reran tests in clean environment. Do you plan committing it? > > > Index: opts.c > =================================================================== > --- opts.c (.../vendor.branches/freebsd/stable/7/src/contrib/gcc/opts.c) (revision > 211574) > +++ opts.c (.../head/src/contrib/gcc/opts.c) (revision 211574) > @@ -457,11 +457,7 @@ > flag_tree_dse = 1; > flag_tree_ter = 1; > flag_tree_live_range_split = 1; > + /** > + * 7dot1MERGE: tree-sra in gcc 4.2.x is buggy and > + * breaks bitfield structs. > + */ > + flag_tree_sra = 0; > - flag_tree_sra = 1; > flag_tree_copyrename = 1; > flag_tree_fre = 1; > flag_tree_copy_prop = 1; > > Thanks, > matthew From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 18:40:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3DBE1065672; Sun, 20 Nov 2011 18:40:49 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 47FED8FC0C; Sun, 20 Nov 2011 18:40:48 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so7571273bkb.13 for ; Sun, 20 Nov 2011 10:40:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=U+BbxSyqS+zQtshUV2zJgcpQiesxbwnh0TjQS+FhlQU=; b=iiPsA86ptfciUHPv86heoeJQs1l1CNaSbDRNvMvmSWwzQTEQqflM/PN/+MTyQH3kAM TriJkzpzDsJwrypz0cWZIRKgOaEJQmWB+TNiFiuAXA6Z5jdHwLJZHWysoLKv7EopKIkM Lga/QoSKxPL5uWsj+7pt8P/JePdBGZPkPdwUs= Received: by 10.204.154.25 with SMTP id m25mr11114366bkw.140.1321814447353; Sun, 20 Nov 2011 10:40:47 -0800 (PST) Received: from localhost ([78.157.92.5]) by mx.google.com with ESMTPS id d19sm5436116bkq.5.2011.11.20.10.40.45 (version=SSLv3 cipher=OTHER); Sun, 20 Nov 2011 10:40:46 -0800 (PST) Date: Sun, 20 Nov 2011 20:40:47 +0200 From: Gleb Kurtsou To: Alexander Best Message-ID: <20111120182824.GB1672@reks> References: <20111119100150.GA1560@reks> <20111119122538.GA47771@freebsd.org> <20111119133913.GA87101@reks> <20111120015757.GA71026@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20111120015757.GA71026@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org Subject: Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 18:40:50 -0000 On (20/11/2011 01:57), Alexander Best wrote: > On Sat Nov 19 11, Gleb Kurtsou wrote: > > On (19/11/2011 12:25), Alexander Best wrote: > > > On Sat Nov 19 11, Gleb Kurtsou wrote: > > > > Hi, > > > > > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > > > correctly with -O2. Too keep long story short the code fails for gcc > > > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > > > -Os optimization levels are fine (I've tried with all -f* flags > > > > mentioned in documentation) > > > > > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > > > presume i386 should be fine. These options are also used for > > > > compilation of kernel (with debugging enabled) and modules. > > > > > > > > I'm not able to share the code, but have a test case reproducing the > > > > bug. I've encountered the issue over a week ago and tried narrowing it down > > > > to a simple test I could share but without much success. > > > > > > > > The code itself is very common: initialize two structs on stack, call a > > > > function with pointers to those stucts as arguments. A number of inlined > > > > assertion functions. gcc fails to correctly optimize struct assignments > > > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > > > gcc decides not to use data coping but to assign fields directly. I've > > > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > > > to copy data. Replacing one particular assignment with memcpy produces > > > > correct code, but that's not a solution. > > > > > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > > > > > does the issue also come up with '-fno-builtin' in addition to those flags? > > > is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist > > > when using -O1 and -O3? > > > > -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD > > > > -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! > > > > -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g > > -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > -fno-strict-aliasing -- BAD > > > > -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > -fno-strict-aliasing -- OK > > btw: for any optimisation > -O1, -fno-strict-aliasing isn't needed, since it > gets added automatically (see sys/conf/kern.pre.mk). For kernel builds. I'm building 3rd party software with base system compiler. On the other hand kernel and modules are built with -O2 -fno-omit-frame-pointer.. > > cheers. > alex > > > > > So, it's -finline-functions that makes it work. But I'm afraid it just > > masks the real problem. > > > > The rest of CFLAGS is warnings and includes: > > -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W > > -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type > > -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter > > -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 > > -Wno-error=strict-aliasing > > > > > > > > cheers. > > > alex > > > > > > > > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > > > with gcc test case: > > > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > > > weekend as glk. > > > > > > > > > > > > Thanks, > > > > Gleb. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 18:57:38 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 0A166106566C; Sun, 20 Nov 2011 18:57:38 +0000 (UTC) Date: Sun, 20 Nov 2011 18:57:38 +0000 From: Alexander Best To: Gleb Kurtsou Message-ID: <20111120185738.GA19861@freebsd.org> References: <20111119100150.GA1560@reks> <20111119122538.GA47771@freebsd.org> <20111119133913.GA87101@reks> <20111120015757.GA71026@freebsd.org> <20111120182824.GB1672@reks> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111120182824.GB1672@reks> Cc: freebsd-hackers@FreeBSD.org Subject: Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 18:57:38 -0000 On Sun Nov 20 11, Gleb Kurtsou wrote: > On (20/11/2011 01:57), Alexander Best wrote: > > On Sat Nov 19 11, Gleb Kurtsou wrote: > > > On (19/11/2011 12:25), Alexander Best wrote: > > > > On Sat Nov 19 11, Gleb Kurtsou wrote: > > > > > Hi, > > > > > > > > > > I was lucky to write a bit of code which gcc 4.2 fails to compile > > > > > correctly with -O2. Too keep long story short the code fails for gcc > > > > > from base system and last gcc 4.2 snapshot from ports. It works with gcc > > > > > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > > > > > -Os optimization levels are fine (I've tried with all -f* flags > > > > > mentioned in documentation) > > > > > > > > > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > > > > > presume i386 should be fine. These options are also used for > > > > > compilation of kernel (with debugging enabled) and modules. > > > > > > > > > > I'm not able to share the code, but have a test case reproducing the > > > > > bug. I've encountered the issue over a week ago and tried narrowing it down > > > > > to a simple test I could share but without much success. > > > > > > > > > > The code itself is very common: initialize two structs on stack, call a > > > > > function with pointers to those stucts as arguments. A number of inlined > > > > > assertion functions. gcc fails to correctly optimize struct assignments > > > > > with -fno-omit-frame-pointer, I have a number of small structs assigned, > > > > > gcc decides not to use data coping but to assign fields directly. I've > > > > > tried disabling sra, tweaking sra parameters -- no luck in forcing it > > > > > to copy data. Replacing one particular assignment with memcpy produces > > > > > correct code, but that's not a solution. > > > > > > > > > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > > > > > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > > > > > > > does the issue also come up with '-fno-builtin' in addition to those flags? > > > > is '-O2 -fno-omit-frame-pointer' alone also buggy? does the issue also exist > > > > when using -O1 and -O3? > > > > > > -O0 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > > > -Os -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK > > > > > > -O -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > > > -O1 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -- BAD > > > > > > -O2 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-builtin -fno-strict-aliasing -- BAD > > > > > > -O3 -g -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- OK!! > > > > > > -O3 -fno-inline-functions -fno-unswitch-loops -fno-gcse-after-reload -g > > > -std=gnu99 -pipe -fno-omit-frame-pointer -fno-strict-aliasing -- BAD > > > > > > -O3 -fno-inline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > > -fno-strict-aliasing -- BAD > > > > > > -O2 -finline-functions -g -std=gnu99 -pipe -fno-omit-frame-pointer > > > -fno-strict-aliasing -- OK > > > > btw: for any optimisation > -O1, -fno-strict-aliasing isn't needed, since it > > gets added automatically (see sys/conf/kern.pre.mk). > > For kernel builds. I'm building 3rd party software with base system > compiler. On the other hand kernel and modules are built with > -O2 -fno-omit-frame-pointer.. right. was was i thinking. ;) sorry for the noise. cheers. alex > > > > > > cheers. > > alex > > > > > > > > So, it's -finline-functions that makes it work. But I'm afraid it just > > > masks the real problem. > > > > > > The rest of CFLAGS is warnings and includes: > > > -D_GNU_SOURCE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W > > > -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type > > > -Wcast-qual -Wno-write-strings -Wswitch -Wshadow -Wunused-parameter > > > -Wcast-align -Wno-pointer-sign -Wstrict-aliasing=2 > > > -Wno-error=strict-aliasing > > > > > > > > > > > cheers. > > > > alex > > > > > > > > > > > > > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > > > > > with gcc test case: > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > > > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > > > > > > > > > > I'll be glad to help debugging it and will be hanging on #bsddev during > > > > > weekend as glk. > > > > > > > > > > > > > > > Thanks, > > > > > Gleb. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 19:16:09 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9FC1B1065672 for ; Sun, 20 Nov 2011 19:16:09 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f44.google.com (mail-pz0-f44.google.com [209.85.210.44]) by mx1.freebsd.org (Postfix) with ESMTP id 744918FC08 for ; Sun, 20 Nov 2011 19:16:09 +0000 (UTC) Received: by pzk33 with SMTP id 33so23637163pzk.3 for ; Sun, 20 Nov 2011 11:16:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=QXdgnlDNH628JVbMh+n8q1ZLGDMBwxnBDoJjk+nXvlc=; b=WmS1d3HJG7jw/I/OHyKl5knK3sajAeBYlx2XqjOyAikQUII6Cu9ni/xdewlAFVqGG5 wi444T2ZVEDfjuuQBH/Ek528Jv9E7B88NaUvsQ/nwt/jRS2YQOW5QxTjmzH6SnHP4YmQ X2X+pajidEDkrR3vp7oHU33D6FF5DfPEFXzcg= MIME-Version: 1.0 Received: by 10.68.33.134 with SMTP id r6mr29219905pbi.76.1321816568945; Sun, 20 Nov 2011 11:16:08 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.56.97 with HTTP; Sun, 20 Nov 2011 11:16:08 -0800 (PST) In-Reply-To: <20111120182430.GA1672@reks> References: <20111119100150.GA1560@reks> <20111119161958.GA91681@reks> <20111120182430.GA1672@reks> Date: Sun, 20 Nov 2011 11:16:08 -0800 X-Google-Sender-Auth: YZZvC6iz3cN6Z8l3naB743C5SNg Message-ID: From: mdf@FreeBSD.org To: Gleb Kurtsou Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 19:16:09 -0000 On Sun, Nov 20, 2011 at 10:24 AM, Gleb Kurtsou wro= te: > On (19/11/2011 09:11), mdf@FreeBSD.org wrote: >> On Sat, Nov 19, 2011 at 8:19 AM, Gleb Kurtsou w= rote: >> > On (19/11/2011 07:26), mdf@FreeBSD.org wrote: >> >> On Sat, Nov 19, 2011 at 2:01 AM, Gleb Kurtsou wrote: >> >> > Hi, >> >> > >> >> > I was lucky to write a bit of code which gcc 4.2 fails to compile >> >> > correctly with -O2. Too keep long story short the code fails for gc= c >> >> > from base system and last gcc 4.2 snapshot from ports. It works wit= h gcc >> >> > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O= and >> >> > -Os optimization levels are fine (I've tried with all -f* flags >> >> > mentioned in documentation) >> >> > >> >> > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I >> >> > presume i386 should be fine. These options are also used for >> >> > compilation of kernel (with debugging enabled) and modules. >> >> > >> >> > I'm not able to share the code, but have a test case reproducing th= e >> >> > bug. I've encountered the issue over a week ago and tried narrowing= it down >> >> > to a simple test I could share but without much success. >> >> > >> >> > The code itself is very common: initialize two structs on stack, ca= ll a >> >> > function with pointers to those stucts as arguments. A number of in= lined >> >> > assertion functions. gcc fails to correctly optimize struct assignm= ents >> >> > with -fno-omit-frame-pointer, I have a number of small structs assi= gned, >> >> > gcc decides not to use data coping but to assign fields directly. I= 've >> >> > tried disabling sra, tweaking sra parameters -- no luck in forcing = it >> >> > to copy data. Replacing one particular assignment with memcpy produ= ces >> >> > correct code, but that's not a solution. >> >> >> >> How small are the structs? =A0gcc has an optimization for structs tha= t >> >> are no larger than a register, but it's buggy in 4.2 and we disabled >> >> it at $WORK. =A0I can dig up the patch if this is the problem. >> > struct sockaddr_in in this particular test. 16 bytes. >> > >> > Register size structs are rather common, e.g. struct in_addr. >> > >> > I could test the patch. Adding -finline-functions seems to fix the iss= ue >> > for me. >> >> I can't find the thing I'm thinking of. =A0The only potentially relevant >> patch I see in our gcc sources is this: > > It could be related but doesn't fix bug I observe. I've installed fresh > 9.0-RC2 virtual machine and reran tests in clean environment. > > Do you plan committing it? We probably should, but I haven't heard that anyone else has had a problem with this. I'm sure when I do the next FreeBSD merge at $WORK I'll re-remember it and probably commit it then. Thanks, matthew >> Index: opts.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- opts.c =A0 =A0(.../vendor.branches/freebsd/stable/7/src/contrib/gcc/= opts.c) =A0 (revision >> 211574) >> +++ opts.c =A0 =A0(.../head/src/contrib/gcc/opts.c) =A0 =A0 =A0 (revisio= n 211574) >> @@ -457,11 +457,7 @@ >> =A0 =A0 =A0 =A0flag_tree_dse =3D 1; >> =A0 =A0 =A0 =A0flag_tree_ter =3D 1; >> =A0 =A0 =A0 =A0flag_tree_live_range_split =3D 1; >> + =A0 =A0 =A0/** >> + =A0 =A0 =A0 * 7dot1MERGE: tree-sra in gcc 4.2.x is buggy and >> + =A0 =A0 =A0 * breaks bitfield structs. >> + =A0 =A0 =A0 */ >> + =A0 =A0 =A0flag_tree_sra =3D 0; >> - =A0 =A0 =A0flag_tree_sra =3D 1; >> =A0 =A0 =A0 =A0flag_tree_copyrename =3D 1; >> =A0 =A0 =A0 =A0flag_tree_fre =3D 1; >> =A0 =A0 =A0 =A0flag_tree_copy_prop =3D 1; >> >> Thanks, >> matthew > From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 20 20:38:40 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77946106564A; Sun, 20 Nov 2011 20:38:40 +0000 (UTC) (envelope-from wkoszek@freebsd.czest.pl) Received: from freebsd.czest.pl (freebsd.czest.pl [212.87.224.105]) by mx1.freebsd.org (Postfix) with ESMTP id 014D78FC0C; Sun, 20 Nov 2011 20:38:39 +0000 (UTC) Received: from wkoszek-thinkpad-t410 (freebsd.czest.pl [212.87.224.105]) by freebsd.czest.pl (8.14.5/8.14.5) with ESMTP id pAKKHWXY049236; Sun, 20 Nov 2011 20:17:32 GMT (envelope-from wkoszek@freebsd.czest.pl) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-hackers@freebsd.org Date: Sun, 20 Nov 2011 21:18:38 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Wojciech A. Koszek" Organization: FreeBSD.czest.pl Message-ID: User-Agent: Opera Mail/11.60 (Linux) X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.2.7 (freebsd.czest.pl [127.0.0.2]); Sun, 20 Nov 2011 20:17:32 +0000 (UTC) X-Mailman-Approved-At: Sun, 20 Nov 2011 22:18:54 +0000 Cc: Subject: The FreeBSD Project in the Google Code-In 2011 contest X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Nov 2011 20:38:40 -0000 Hello, (cross-posted message; please keep eventual comments on freebsd-hackers@) The FreeBSD project has been accepted to the Google Code-In 2011 contest. http://google-opensource.blogspot.com/2011/11/google-code-in-2011-participating.html We have proposed 50 tasks so far, and more are still coming! This is an event similar to the Google Summer of Code, but is targetted to people in the 13-17 age range. Young people will be working on FreeBSD-related tasks for the next weeks. If you know any potential candidates, feel free to forward this message. Brief summary: T-shirts and possibility of earning some $$$ for participants. The best participants get a chance to see the Google complex in Mountain View, Silicon Valley, California, USA. FreeBSD tasks are here: http://wiki.freebsd.org/GoogleCodeIn/2011 Ideas page can be extended till December, 16th! Contest's home page: http://www.google-melange.com/gci/homepage/google/gci2011 After you create an account, you can acquire tasks which you're interested in (if any of them are left!) Start date: November, 21st (tomorrow) Official communication channels: IRC (EFNet): #freebsd-soc Q/A: wkoszek@FreeBSD.ORG, jceel@FreeBSD.ORG, eadler@FreeBSD.org wkoszek, jceel, eadler on IRC Mailing list: freebsd-hackers@ Please coordinate communication with your mentor. Include '[GCIN]' header when posting to freebsd-hackers@ In case of potential task candidates, ideas and suggestions, feel free to contact me. -- Wojciech A. Koszek wkoszek@freebsd.czest.pl http://FreeBSD.czest.pl/~wkoszek/ From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 05:25:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF5201065675; Mon, 21 Nov 2011 05:25:45 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (unknown [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 865B08FC0A; Mon, 21 Nov 2011 05:25:45 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id pAL5Phnk011515 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 20 Nov 2011 21:25:44 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id pAL5PhNn011514; Sun, 20 Nov 2011 21:25:43 -0800 (PST) Received: from fbsd81 ([192.168.200.81]) by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA05959; Sun, 20 Nov 11 21:16:09 PST Date: Mon, 21 Nov 2011 04:15:49 -0800 From: perryh@pluto.rain.com To: arundel@freebsd.org Message-Id: <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> References: <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> In-Reply-To: <20111120142131.GA64913@freebsd.org> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 05:25:45 -0000 Alexander Best wrote: > here's a revised patch. > ... > +.Sh CAVEATS > +If the > +.Fn lseek > +system call is operating on a device, which is incapable of seeking, > +it will request the seek operation and complete successfully. I think it would be better without the first comma (after "device"). IMO the whole section should be added to BUGS rather than creating a new CAVEATS section. Returning "success" when the requested operation actually _failed_ is a serious POLA violation. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 06:01:15 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9886E106564A; Mon, 21 Nov 2011 06:01:15 +0000 (UTC) (envelope-from Ramanjaneyulu.Talla@citrix.com) Received: from SMTP.CITRIX.COM.AU (smtp.citrix.com.au [203.166.19.134]) by mx1.freebsd.org (Postfix) with ESMTP id 07B588FC08; Mon, 21 Nov 2011 06:01:14 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.69,545,1315180800"; d="scan'208";a="9358120" Received: from banpmailmx02.citrite.net ([10.103.128.74]) by SYDPIPO01.CITRIX.COM.AU with ESMTP/TLS/RC4-MD5; 21 Nov 2011 05:31:28 +0000 Received: from BANPMAILBOX01.citrite.net ([10.103.128.72]) by BANPMAILMX02.citrite.net ([10.103.128.74]) with mapi; Mon, 21 Nov 2011 11:01:27 +0530 From: Ramanjaneyulu Talla To: "perryh@pluto.rain.com" , "arundel@freebsd.org" Date: Mon, 21 Nov 2011 11:01:20 +0530 Thread-Topic: easy way to determine if a stream or fd is seekable Thread-Index: AcyoDtCE15ZCEzLOQ8qbHZ++1YrSeQ== Message-ID: In-Reply-To: <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.10.0.110310 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" , "nox@jelal.kn-bremen.de" Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 06:01:15 -0000 On 21/11/11 5:45 PM, "perryh@pluto.rain.com" wrote: >Alexander Best wrote: > >> here's a revised patch. >> ... >> +.Sh CAVEATS >> +If the >> +.Fn lseek >> +system call is operating on a device, which is incapable of seeking, >> +it will request the seek operation and complete successfully. > >I think it would be better without the first comma (after "device"). > >IMO the whole section should be added to BUGS rather than creating >a new CAVEATS section. Returning "success" when the requested >operation actually _failed_ is a serious POLA violation. >_______________________________________________ >freebsd-hackers@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 06:20:56 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0B50106564A; Mon, 21 Nov 2011 06:20:56 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-5.mit.edu (DMZ-MAILSEC-SCANNER-5.MIT.EDU [18.7.68.34]) by mx1.freebsd.org (Postfix) with ESMTP id 47B2A8FC14; Mon, 21 Nov 2011 06:20:55 +0000 (UTC) X-AuditID: 12074422-b7ff56d00000092f-46-4ec9edc7aae4 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP id 3A.CC.02351.7CDE9CE4; Mon, 21 Nov 2011 01:20:55 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id pAL6Kspa023891; Mon, 21 Nov 2011 01:20:54 -0500 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id pAL6Kpms022001 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 Nov 2011 01:20:53 -0500 (EST) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id pAL6Kp78016534; Mon, 21 Nov 2011 01:20:51 -0500 (EST) Date: Mon, 21 Nov 2011 01:20:51 -0500 (EST) From: Benjamin Kaduk To: arundel@freebsd.org In-Reply-To: <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> Message-ID: References: <20111116102239.GA2687@britannica.bec.de> <20111116131428.GA40723@freebsd.org> <20111116232152.GC21793@britannica.bec.de> <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLIsWRmVeSWpSXmKPExsUixG6nonv87Uk/g0tvFCzaP89jsti++R+j ReO+qywW8zrPMTuweMz4NJ/F48HUDkaP6XuvMwUwR3HZpKTmZJalFunbJXBl9F+TL5jMWdG4 bA5LA+MC9i5GTg4JAROJl4dusUHYYhIX7q0Hsrk4hAT2MUr8+vuZFcLZwCix/sJhKOcAk0T/ gpOMEE4Do8TEF9NYQPpZBLQlDjyZxQhiswmoSMx8sxFsroiAuMTztevAapgF/CROLf8LFhcW cJCY0/CQFcTmFLCT2P/uPjOIzStgL3Hj/GyoO3pYJJp33AUrEhXQkVi9fwoLRJGgxMmZT6CG Wkqc+3OdbQKj4CwkqVlIUgsYmVYxyqbkVunmJmbmFKcm6xYnJ+blpRbpmurlZpbopaaUbmIE h7KL0g7GnweVDjEKcDAq8fDOXHnST4g1say4MvcQoyQHk5Iob9oLoBBfUn5KZUZicUZ8UWlO avEhRgkOZiUR3hxnoBxvSmJlVWpRPkxKmoNFSZyXa6eDn5BAemJJanZqakFqEUxWhoNDSYKX HRizQoJFqempFWmZOSUIaSYOTpDhPEDD2UBqeIsLEnOLM9Mh8qcYFaXEeYVBEgIgiYzSPLhe WKp5xSgO9IowLytIFQ8wTcF1vwIazAQ0eNraEyCDSxIRUlINjHbxHOd2L9iu0fRYs4b/8aK1 QmnrHIMav7SsCrqgeMtPN7vx23KeayfPztgh9M3M/HOMzcyjmlvnPNV78fc8b9u+q8tne01/ 7bN6h/hNb2vRd5/t465lZ/62eLOS/YL4yck9SmpfrTUfybgqHHGYeGVBxLW2i6eOrTwv9jPq 8ZOcRfO95whPPuqvxFKckWioxVxUnAgAbSVylRADAAA= Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 06:20:56 -0000 On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: > Alexander Best wrote: > >> here's a revised patch. >> ... >> +.Sh CAVEATS >> +If the >> +.Fn lseek >> +system call is operating on a device, which is incapable of seeking, >> +it will request the seek operation and complete successfully. > > I think it would be better without the first comma (after "device"). Definitely. Also, +.Sh CAVEATS +If the +.Fn lseek +system call is operating on a device, which is incapable of seeking, +it will request the seek operation and complete successfully. I would prefer something like "request the seek operation and return as if the seek was successful, even though no seek was performed." +The value of the pointer associated with such a device is undefined. "Which pointer?" That it is "the file offset" was clear from context where this line was moved from, but is no longer, here. +Device types which can be incapable of seeking include, +but are not limited to, tape drives. This is an awkward phrasing; perhaps just "Many tape drives are incapable of seeking and can trigger this bug."? -Ben Kaduk From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 06:23:29 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B76D1065670 for ; Mon, 21 Nov 2011 06:23:29 +0000 (UTC) (envelope-from eugen@grosbein.pp.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id E1C478FC14 for ; Mon, 21 Nov 2011 06:23:28 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id pAL6NRWQ076385 for ; Mon, 21 Nov 2011 13:23:27 +0700 (NOVT) (envelope-from eugen@grosbein.pp.ru) Message-ID: <4EC9EE5A.2070204@grosbein.pp.ru> Date: Mon, 21 Nov 2011 13:23:22 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: hackers@freebsd.org References: <4EC39367.4030109@rdtc.ru> In-Reply-To: <4EC39367.4030109@rdtc.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 06:23:29 -0000 Hi! I need to draw graph of dummynet's CPU usage. "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. Now I see this time has 9 seconds increase during 60 seconds of real time. This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. Where is my error? Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 06:36:19 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0564D1065670 for ; Mon, 21 Nov 2011 06:36:19 +0000 (UTC) (envelope-from aram_baghomian@hotmail.com) Received: from snt0-omc2-s33.snt0.hotmail.com (snt0-omc2-s33.snt0.hotmail.com [65.55.90.108]) by mx1.freebsd.org (Postfix) with ESMTP id CECA88FC08 for ; Mon, 21 Nov 2011 06:36:18 +0000 (UTC) Received: from SNT140-W38 ([65.55.90.73]) by snt0-omc2-s33.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Sun, 20 Nov 2011 22:24:17 -0800 Message-ID: X-Originating-IP: [2.145.221.134] From: aram baghomian To: Date: Mon, 21 Nov 2011 06:24:17 +0000 Importance: Normal In-Reply-To: References: , , , MIME-Version: 1.0 X-OriginalArrivalTime: 21 Nov 2011 06:24:17.0704 (UTC) FILETIME=[32822A80:01CCA816] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: How to compile opencrypto separately X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 06:36:19 -0000 Hi I'm sorry=2C i forgot to register my email address to your mailing list. n= ow i registered in it and resend my question. I have some problems with opencrypto project and i hope you want to help m= e for solve them. - I want to compile opencrypto code without to force compile it as a subsys= tem of kernel and with whole code of my kernel. How can i do that? - I trying to add my hash algorithm into opencrypto project and compile it = for use it later with cryptodev and IPSEC vpn. I use rmd160 as a sample and do this steps: 1- rename rmd160.c & rmd160.h to myhash.c & myhash.h . 2- change the name and the contents of the functions of rmd160 to myhash.(= used standard form in the title of functions) 3- add my function titles and variables in xform.c & xform.h . 4- add my fixed value of my algorithm in the cryptodev.h . 5- recompile my kernel with IPSEC option and cryptodev device . But in the build time i get this error : in function 'MYHASHUpdate_int' : xform.c:759: undefined refrence to 'MYHASHUpdate' what isn't enough in my steps? what should i do? i'm sorry for my english. thanks. = From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 08:32:48 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A0E9106566C for ; Mon, 21 Nov 2011 08:32:48 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2B8DD8FC08 for ; Mon, 21 Nov 2011 08:32:48 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 812837300A; Mon, 21 Nov 2011 09:30:03 +0100 (CET) Date: Mon, 21 Nov 2011 09:30:03 +0100 From: Luigi Rizzo To: Eugene Grosbein Message-ID: <20111121083003.GA54883@onelab2.iet.unipi.it> References: <4EC39367.4030109@rdtc.ru> <4EC9EE5A.2070204@grosbein.pp.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EC9EE5A.2070204@grosbein.pp.ru> User-Agent: Mutt/1.4.2.3i Cc: hackers@freebsd.org Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 08:32:48 -0000 On Mon, Nov 21, 2011 at 01:23:22PM +0700, Eugene Grosbein wrote: > Hi! > > I need to draw graph of dummynet's CPU usage. > "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. > "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. > > Now I see this time has 9 seconds increase during 60 seconds of real time. > This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. apart from the scaling on number of cores (e.g. if you have 8 cores the 15% becomes a bare 2%) remember that percentages are computed with some kind of filtering (EWMA ?) so if the load of dummynet threads is bursty, the filter might eat most of it. not completely sure this explains a steady 0.00%, if that's is what you are seeing cheers luigi > Where is my error? > > Eugene Grosbein > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 11:02:06 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id E6AB4106566C; Mon, 21 Nov 2011 11:02:06 +0000 (UTC) Date: Mon, 21 Nov 2011 11:02:06 +0000 From: Alexander Best To: Benjamin Kaduk Message-ID: <20111121110206.GA63744@freebsd.org> References: <20111117002438.GA55931@freebsd.org> <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 11:02:07 -0000 On Mon Nov 21 11, Benjamin Kaduk wrote: > On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: > > >Alexander Best wrote: > > > >>here's a revised patch. > >>... > >>+.Sh CAVEATS > >>+If the > >>+.Fn lseek > >>+system call is operating on a device, which is incapable of seeking, > >>+it will request the seek operation and complete successfully. > > > >I think it would be better without the first comma (after "device"). > > Definitely. > > Also, > > +.Sh CAVEATS > +If the > +.Fn lseek > +system call is operating on a device, which is incapable of seeking, > +it will request the seek operation and complete successfully. > > I would prefer something like "request the seek operation and return as if > the seek was successful, even though no seek was performed." > > +The value of the pointer associated with such a device is undefined. > > "Which pointer?" That it is "the file offset" was clear from context > where this line was moved from, but is no longer, here. > > +Device types which can be incapable of seeking include, > +but are not limited to, tape drives. > > This is an awkward phrasing; perhaps just "Many tape drives are incapable > of seeking and can trigger this bug."? this is too limited. this suggests that only certain tape drives won't seek after a successfull return of lseek(). as i mentioned beforehand, this is also the case with device with insertable media, such as dvd and blue-ray drives. here lseek() will sucessfully return, without a media inserted. i'll rephrase the whole patch and will submit a revised version. i think a reference to POLA, would also be a good idea, as suggested by perry@ thanks for all the suggestions. cheers. alex > > -Ben Kaduk From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 12:07:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id B14381065675; Mon, 21 Nov 2011 12:07:51 +0000 (UTC) Date: Mon, 21 Nov 2011 12:07:51 +0000 From: Alexander Best To: Benjamin Kaduk Message-ID: <20111121120751.GA85679@freebsd.org> References: <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> <20111121110206.GA63744@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline In-Reply-To: <20111121110206.GA63744@freebsd.org> Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 12:07:51 -0000 --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon Nov 21 11, Alexander Best wrote: > On Mon Nov 21 11, Benjamin Kaduk wrote: > > On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: > > > > >Alexander Best wrote: > > > > > >>here's a revised patch. > > >>... > > >>+.Sh CAVEATS > > >>+If the > > >>+.Fn lseek > > >>+system call is operating on a device, which is incapable of seeking, > > >>+it will request the seek operation and complete successfully. > > > > > >I think it would be better without the first comma (after "device"). > > > > Definitely. > > > > Also, > > > > +.Sh CAVEATS > > +If the > > +.Fn lseek > > +system call is operating on a device, which is incapable of seeking, > > +it will request the seek operation and complete successfully. > > > > I would prefer something like "request the seek operation and return as if > > the seek was successful, even though no seek was performed." > > > > +The value of the pointer associated with such a device is undefined. > > > > "Which pointer?" That it is "the file offset" was clear from context > > where this line was moved from, but is no longer, here. > > > > +Device types which can be incapable of seeking include, > > +but are not limited to, tape drives. > > > > This is an awkward phrasing; perhaps just "Many tape drives are incapable > > of seeking and can trigger this bug."? > > this is too limited. this suggests that only certain tape drives won't seek > after a successfull return of lseek(). as i mentioned beforehand, this is also > the case with device with insertable media, such as dvd and blue-ray drives. > here lseek() will sucessfully return, without a media inserted. > > i'll rephrase the whole patch and will submit a revised version. i think a > reference to POLA, would also be a good idea, as suggested by perry@ here's a revised patch. cheers. alex > > thanks for all the suggestions. > > cheers. > alex > > > > > -Ben Kaduk --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="lseek.2.diff" diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 index 874c523..bcd9d20 100644 --- a/lib/libc/sys/lseek.2 +++ b/lib/libc/sys/lseek.2 @@ -28,7 +28,7 @@ .\" @(#)lseek.2 8.3 (Berkeley) 4/19/94 .\" $FreeBSD$ .\" -.Dd April 5, 2007 +.Dd November 21, 2011 .Dt LSEEK 2 .Os .Sh NAME @@ -113,10 +113,9 @@ of the existing end-of-file of the file. If data is later written at this point, subsequent reads of the data in the gap return bytes of zeros (until data is actually written into the gap). -.Pp -Some devices are incapable of seeking. -The value of the pointer -associated with such a device is undefined. +However, the +.Fn lseek +system call does not, by itself, extend the size of a file. .Pp A .Qq hole @@ -197,13 +196,43 @@ is associated with a pipe, socket, or FIFO. The .Fn lseek system call is expected to conform to -.St -p1003.1-90 . +.St -p1003.1-2008 . +.Pp +The +.Dv SEEK_HOLE +and +.Dv SEEK_DATA +directives, along with the +.Er ENXIO +error, are extensions to that specification. .Sh HISTORY The .Fn lseek function appeared in .At v7 . .Sh BUGS +If the +.Fn lseek +system call is operating on a device which is incapable of seeking, +it will request the seek operation and return successfully, +even though no seek was performed. +Because the +.Ar offset +argument will be stored in the file descriptor of that device, +there is no way to verifying/falsify the seek operation afterwards +(e.g. using the +.Fn ftell +function). +Device types which are known to be incapable of seeking include +tape drives. +.Pp +The +.Fn lseek +system call will not detect, whether media are present in changeable +media devices, such as DVD or Blue-ray devices. +A requested seek operation will therefore return sucessfully in case +of an uninserted medium. +.Pp This document's use of .Fa whence is incorrect English, but is maintained for historical reasons. --ew6BAiZeqk4r7MaW-- From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 05:58:29 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66D82106564A for ; Mon, 21 Nov 2011 05:58:29 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id ADFEB8FC15 for ; Mon, 21 Nov 2011 05:58:28 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id pAL5wPUs075943 for ; Mon, 21 Nov 2011 12:58:25 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4EC9E87C.9040008@rdtc.ru> Date: Mon, 21 Nov 2011 12:58:20 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: hackers@freebsd.org References: <4EC39367.4030109@rdtc.ru> In-Reply-To: <4EC39367.4030109@rdtc.ru> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 21 Nov 2011 12:17:26 +0000 Cc: Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 05:58:29 -0000 Hi! I need to draw graph of dummynet's CPU usage. "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. Now I see this time has 9 seconds increase during 60 seconds of real time. This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. Where is my error? Please CC: me as I'm not in the list. Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 16:49:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72AFB106566C; Mon, 21 Nov 2011 16:49:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 4A4578FC12; Mon, 21 Nov 2011 16:49:41 +0000 (UTC) Received: from bigwig.baldwin.cx (96.47.65.170.static.nyinternet.net [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 0079746B42; Mon, 21 Nov 2011 11:49:41 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C2220B977; Mon, 21 Nov 2011 11:49:37 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 21 Nov 2011 11:26:15 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201111211126.15627.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 21 Nov 2011 11:49:37 -0500 (EST) Cc: "Paul B. Mahol" , FreeBSD-Current Subject: Re: Reprobing of devices after module load? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 16:49:41 -0000 On Friday, November 18, 2011 11:48:20 am Paul B. Mahol wrote: > Hi, > > Is there nice way in FreeBSD to force reprobe of devices for specific > driver like it is done when kernel module is loaded (via > DRIVER_MODULE(...) stuff)? Note that those probes happen for specific buses rather than for specific drivers. The routine that does this currently is static (devclass_driver_added() in sys/kern/subr_bus.c). What specific problem are you trying to solve? You might be able to use BUS_DRIVER_ADDED() or device_probe_and_attach() to achieve what you are trying to do. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 16:51:18 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C4F6106564A for ; Mon, 21 Nov 2011 16:51:18 +0000 (UTC) (envelope-from eugen@grosbein.pp.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id CB5518FC23 for ; Mon, 21 Nov 2011 16:51:17 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id pALGpERb080992; Mon, 21 Nov 2011 23:51:14 +0700 (NOVT) (envelope-from eugen@grosbein.pp.ru) Message-ID: <4ECA817D.5020205@grosbein.pp.ru> Date: Mon, 21 Nov 2011 23:51:09 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Ryan Stone References: <4EC39367.4030109@rdtc.ru> <4EC9EE5A.2070204@grosbein.pp.ru> In-Reply-To: Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: hackers@freebsd.org Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 16:51:18 -0000 21.11.2011 23:37, Ryan Stone ÐÉÛÅÔ: > On Mon, Nov 21, 2011 at 1:23 AM, Eugene Grosbein wrote: >> Hi! >> >> I need to draw graph of dummynet's CPU usage. >> "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. >> "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. >> >> Now I see this time has 9 seconds increase during 60 seconds of real time. >> This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. >> >> Where is my error? > Which version are you running? 8.1-RELEASE and older have the problem > that the scheduler and the CPU statistics gatherer are driven from the > same clock. For threads that tend to frequently wake up and run for > less than a full tick(dummynet would appear to fall in this category) > this means that CPU usage statistics are never captured at a point > where those threads are running, so top shows 0% for those threads. > > I am told that some relatively recent timer-related work(eventtimer?) > should have resolved the issue, which is definitely in 9.0 and might > be in stable/8 and may even have made it into 8.2-RELEASE. I run 8.2-STABLE of 16 October. During most busy hours top shows some nonzero values but TIME obtained from ps(1) is still several times larger. Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 17:00:02 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0D1C1065674 for ; Mon, 21 Nov 2011 17:00:02 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 545688FC1D for ; Mon, 21 Nov 2011 17:00:02 +0000 (UTC) Received: by eyd10 with SMTP id 10so7911788eyd.13 for ; Mon, 21 Nov 2011 09:00:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=syuaTA6QUvliN5mpZb+2QBak7o1SN6f+vwW4uZWKbco=; b=XoqFdYkkpPDO6y8tLHpoWBBSXDK0U3DhU5aZK8/tHO1vxbohNFAGxopkvVuWJlvTmg UQo2BzeNRMzJEYasNGe+EWrR/BPbAvT4+KlvekDdX48hZFJ1oraLHSqnAGyK62v3Fbdm GlMyZC8lCTdqU85JiCSfANR1z/LDWn17dVagY= MIME-Version: 1.0 Received: by 10.180.24.65 with SMTP id s1mr14652240wif.59.1321893441182; Mon, 21 Nov 2011 08:37:21 -0800 (PST) Received: by 10.180.95.104 with HTTP; Mon, 21 Nov 2011 08:37:21 -0800 (PST) In-Reply-To: <4EC9EE5A.2070204@grosbein.pp.ru> References: <4EC39367.4030109@rdtc.ru> <4EC9EE5A.2070204@grosbein.pp.ru> Date: Mon, 21 Nov 2011 11:37:21 -0500 Message-ID: From: Ryan Stone To: Eugene Grosbein Content-Type: text/plain; charset=ISO-8859-1 Cc: hackers@freebsd.org Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 17:00:02 -0000 On Mon, Nov 21, 2011 at 1:23 AM, Eugene Grosbein wrote: > Hi! > > I need to draw graph of dummynet's CPU usage. > "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. > "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. > > Now I see this time has 9 seconds increase during 60 seconds of real time. > This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. > > Where is my error? > > Eugene Grosbein Which version are you running? 8.1-RELEASE and older have the problem that the scheduler and the CPU statistics gatherer are driven from the same clock. For threads that tend to frequently wake up and run for less than a full tick(dummynet would appear to fall in this category) this means that CPU usage statistics are never captured at a point where those threads are running, so top shows 0% for those threads. I am told that some relatively recent timer-related work(eventtimer?) should have resolved the issue, which is definitely in 9.0 and might be in stable/8 and may even have made it into 8.2-RELEASE. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 17:15:23 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FF911065679; Mon, 21 Nov 2011 17:15:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id EBB778FC14; Mon, 21 Nov 2011 17:15:22 +0000 (UTC) Received: from [10.30.101.53] ([209.117.142.2]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pALH9i5D060437 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Mon, 21 Nov 2011 10:09:46 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <201111211126.15627.jhb@freebsd.org> Date: Mon, 21 Nov 2011 10:09:37 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8D7CE344-05B3-4A48-8E5B-91100A48F0B1@bsdimp.com> References: <201111211126.15627.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Mon, 21 Nov 2011 10:09:47 -0700 (MST) Cc: freebsd-hackers@FreeBSD.org, FreeBSD-Current Subject: Re: Reprobing of devices after module load? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 17:15:23 -0000 On Nov 21, 2011, at 9:26 AM, John Baldwin wrote: > On Friday, November 18, 2011 11:48:20 am Paul B. Mahol wrote: >> Hi, >>=20 >> Is there nice way in FreeBSD to force reprobe of devices for specific >> driver like it is done when kernel module is loaded (via >> DRIVER_MODULE(...) stuff)? >=20 > Note that those probes happen for specific buses rather than for = specific=20 > drivers. The routine that does this currently is static=20 > (devclass_driver_added() in sys/kern/subr_bus.c). What specific = problem are=20 > you trying to solve? You might be able to use BUS_DRIVER_ADDED() or=20= > device_probe_and_attach() to achieve what you are trying to do. You can load a dummy module that has an attach point to the bus that = you're wanting to force a rescan on. Warner From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 17:47:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F02DD106566B for ; Mon, 21 Nov 2011 17:47:29 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id BBE8F8FC0C for ; Mon, 21 Nov 2011 17:47:29 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id A8F026E85B for ; Mon, 21 Nov 2011 18:47:28 +0100 (CET) From: Stefan Bethke Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Mon, 21 Nov 2011 18:47:27 +0100 Message-Id: To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Apple Message framework v1251.1) X-Mailer: Apple Mail (2.1251.1) Subject: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 17:47:30 -0000 I have a process that tends to eat up all available disk bandwidth. I = have other processes that I would like to have preference over this one = process. Is there a facility that would allow me to assign priorities = based on jail ID or uid? This is on 8-stable (but will upgrade to 9 soon) on ZFS. The straightforward solution is to separate the datasets onto their own = disks, which I'm planning to do, but a software facility would be that = much more flexible. Thanks, Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 18:42:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B32C106566B; Mon, 21 Nov 2011 18:42:12 +0000 (UTC) (envelope-from onemda@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8B60D8FC08; Mon, 21 Nov 2011 18:42:11 +0000 (UTC) Received: by vcbfl10 with SMTP id fl10so3017413vcb.13 for ; Mon, 21 Nov 2011 10:42:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=C5AZ9F9jmwPKdRcnShIseP+FJpeEndss3YRKdRsCXkQ=; b=b2TfhYSmWhlslc9RmwVYH0GhSy5VNWAHuM43+JQ3eFxxPYnC/mciflcmnFOPVFAzL7 D4uagbsHJnCAx4opvW656r4GdYvk68L0lWQj5NyiZgGdgPDgFaj7PiNY2c/F1rWJCcsi ZlyA5TUMcBcw6HqQOUDnDxBUYob7hPfhJPCI0= MIME-Version: 1.0 Received: by 10.52.65.77 with SMTP id v13mr16439341vds.95.1321900930857; Mon, 21 Nov 2011 10:42:10 -0800 (PST) Received: by 10.52.101.132 with HTTP; Mon, 21 Nov 2011 10:42:10 -0800 (PST) In-Reply-To: <201111211126.15627.jhb@freebsd.org> References: <201111211126.15627.jhb@freebsd.org> Date: Mon, 21 Nov 2011 18:42:10 +0000 Message-ID: From: "Paul B. Mahol" To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, FreeBSD-Current Subject: Re: Reprobing of devices after module load? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 18:42:12 -0000 On 11/21/11, John Baldwin wrote: > On Friday, November 18, 2011 11:48:20 am Paul B. Mahol wrote: >> Hi, >> >> Is there nice way in FreeBSD to force reprobe of devices for specific >> driver like it is done when kernel module is loaded (via >> DRIVER_MODULE(...) stuff)? > > Note that those probes happen for specific buses rather than for specific > drivers. The routine that does this currently is static > (devclass_driver_added() in sys/kern/subr_bus.c). What specific problem are > you trying to solve? You might be able to use BUS_DRIVER_ADDED() or > device_probe_and_attach() to achieve what you are trying to do. I have changed NDISulator to load 3rd party *.SYS from userspace via ioctl on /dev/ndis. For now i can do reprobe by reloading if_ndis.ko module. Loading if_ndis.ko ideally should not cause reprobe but reprobe should be done after ioctl on /dev/ndis. Perhaps devclass_add_driver from sys/kern/subr_bus can do this? From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 18:59:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B680F106566B for ; Mon, 21 Nov 2011 18:59:45 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 498398FC2B for ; Mon, 21 Nov 2011 18:59:45 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so9262378bkb.13 for ; Mon, 21 Nov 2011 10:59:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CdU08Kw6H5cZM8QrIU2ITidTTFdpzfEKvwBVLZOZJnU=; b=eE931j+TV57SpNyUpmKeobgf09qBtGUyEQWaI6A9+BigDGCrlzrQeU8fxD3RFE3rE9 F1XUnhyY4Go6V8ArSLYDkpr5+U7HV7pUyjCChC377NWYiUYZ69VTSvSS3HkaeF8/ic1d pbMT6KXKXVu9au0vGBSuWm+vE3LXUmUW+R/XQ= MIME-Version: 1.0 Received: by 10.204.9.209 with SMTP id m17mr15569383bkm.101.1321900349366; Mon, 21 Nov 2011 10:32:29 -0800 (PST) Received: by 10.223.88.72 with HTTP; Mon, 21 Nov 2011 10:32:29 -0800 (PST) In-Reply-To: References: Date: Mon, 21 Nov 2011 12:32:29 -0600 Message-ID: From: Adam Vande More To: Stefan Bethke Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 18:59:45 -0000 On Mon, Nov 21, 2011 at 11:47 AM, Stefan Bethke wrote: > I have a process that tends to eat up all available disk bandwidth. I > have other processes that I would like to have preference over this one > process. Is there a facility that would allow me to assign priorities > based on jail ID or uid? > > This is on 8-stable (but will upgrade to 9 soon) on ZFS. > > The straightforward solution is to separate the datasets onto their own > disks, which I'm planning to do, but a software facility would be that much > more flexible. > http://wiki.freebsd.org/Hierarchical_Resource_Limits -- Adam Vande More From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 19:58:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 473DA1065670 for ; Mon, 21 Nov 2011 19:58:25 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 0F7188FC24 for ; Mon, 21 Nov 2011 19:58:25 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 4B0E274EFD; Mon, 21 Nov 2011 20:58:24 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Stefan Bethke In-Reply-To: Date: Mon, 21 Nov 2011 20:58:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Adam Vande More X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 19:58:25 -0000 Am 21.11.2011 um 19:32 schrieb Adam Vande More: > On Mon, Nov 21, 2011 at 11:47 AM, Stefan Bethke = wrote: > I have a process that tends to eat up all available disk bandwidth. I = have other processes that I would like to have preference over this one = process. Is there a facility that would allow me to assign priorities = based on jail ID or uid? >=20 > This is on 8-stable (but will upgrade to 9 soon) on ZFS. >=20 > The straightforward solution is to separate the datasets onto their = own disks, which I'm planning to do, but a software facility would be = that much more flexible. >=20 > =20 > http://wiki.freebsd.org/Hierarchical_Resource_Limits Interesting, but it doesn't seem to offer limiting the I/O bandwidth = induced by a process or jail, or assigning different priorities, which = would need to be implemented in the ZFS or GEOM schedulers, I suppose. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 20:41:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5019106566C for ; Mon, 21 Nov 2011 20:41:00 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 31E088FC18 for ; Mon, 21 Nov 2011 20:40:59 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so9394147bkb.13 for ; Mon, 21 Nov 2011 12:40:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Xv8xuVEr1RmfuQBYHHNYpXG0oED7Tsv3Q/S+pUUoIDo=; b=mT6Qrj8zlkMTuWnLV7ihyx2RvpN/pXWRm6POeyMspuJXPoiJ/DjXuJXLvuBapKftIp mx3bnlIPjaS9zyllGZR97U7K+oCVVuA+aHU57SZmpnzxkoXD37hUs1SyxJWJNIPGecTH Fe6BqdESI8COnfrAIhP/2OdZDqrD4tKG+fbWI= MIME-Version: 1.0 Received: by 10.204.10.81 with SMTP id o17mr16047390bko.65.1321908058788; Mon, 21 Nov 2011 12:40:58 -0800 (PST) Received: by 10.223.88.72 with HTTP; Mon, 21 Nov 2011 12:40:58 -0800 (PST) In-Reply-To: References: Date: Mon, 21 Nov 2011 14:40:58 -0600 Message-ID: From: Adam Vande More To: Stefan Bethke Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 20:41:00 -0000 On Mon, Nov 21, 2011 at 1:58 PM, Stefan Bethke wrote: > > Interesting, but it doesn't seem to offer limiting the I/O bandwidth > induced by a process or jail, or assigning different priorities, which > would need to be implemented in the ZFS or GEOM schedulers, I suppose. > Limiting CPU has long been the poor man's IO scheduler, and has usually worked pretty well for me but has required some trial and error. YMMV -- Adam Vande More From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 20:42:22 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A0BD1065675 for ; Mon, 21 Nov 2011 20:42:22 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 308208FC1C for ; Mon, 21 Nov 2011 20:42:22 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 7E5D67BCB1; Mon, 21 Nov 2011 21:42:21 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Stefan Bethke In-Reply-To: Date: Mon, 21 Nov 2011 21:42:21 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> References: To: Adam Vande More X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 20:42:22 -0000 Am 21.11.2011 um 21:40 schrieb Adam Vande More: > On Mon, Nov 21, 2011 at 1:58 PM, Stefan Bethke wrote: >=20 > Interesting, but it doesn't seem to offer limiting the I/O bandwidth = induced by a process or jail, or assigning different priorities, which = would need to be implemented in the ZFS or GEOM schedulers, I suppose. >=20 > Limiting CPU has long been the poor man's IO scheduler, and has = usually worked pretty well for me but has required some trial and error. = YMMV=20 Good point, I'll give that a try. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 21:25:38 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61136106566B for ; Mon, 21 Nov 2011 21:25:38 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 25EA58FC15 for ; Mon, 21 Nov 2011 21:25:38 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 0FB829D66A; Mon, 21 Nov 2011 22:25:37 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Stefan Bethke In-Reply-To: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> Date: Mon, 21 Nov 2011 22:25:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> References: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> To: Adam Vande More X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 21:25:38 -0000 Am 21.11.2011 um 21:42 schrieb Stefan Bethke: > Am 21.11.2011 um 21:40 schrieb Adam Vande More: >=20 >> On Mon, Nov 21, 2011 at 1:58 PM, Stefan Bethke = wrote: >>=20 >> Interesting, but it doesn't seem to offer limiting the I/O bandwidth = induced by a process or jail, or assigning different priorities, which = would need to be implemented in the ZFS or GEOM schedulers, I suppose. >>=20 >> Limiting CPU has long been the poor man's IO scheduler, and has = usually worked pretty well for me but has required some trial and error. = YMMV=20 >=20 > Good point, I'll give that a try. Unfortunately, the process I want to limit is not sufficiently CPU bound = to be limited that way vs. all the other processes. I guess I'll put in = a second disk. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 22:26:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AF1A106564A for ; Mon, 21 Nov 2011 22:26:44 +0000 (UTC) (envelope-from aram_baghomian@hotmail.com) Received: from snt0-omc2-s27.snt0.hotmail.com (snt0-omc2-s27.snt0.hotmail.com [65.55.90.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7144B8FC16 for ; Mon, 21 Nov 2011 22:26:44 +0000 (UTC) Received: from SNT140-W29 ([65.55.90.71]) by snt0-omc2-s27.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 21 Nov 2011 14:26:43 -0800 Message-ID: X-Originating-IP: [2.146.169.208] From: aram baghomian To: Date: Mon, 21 Nov 2011 22:26:43 +0000 Importance: Normal In-Reply-To: References: , , , , MIME-Version: 1.0 X-OriginalArrivalTime: 21 Nov 2011 22:26:43.0941 (UTC) FILETIME=[A5F2A950:01CCA89C] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: FW: How to compile opencrypto separately X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 22:26:44 -0000 Can any hacker help me? From: aram_baghomian@hotmail.com To: freebsd-hackers@freebsd.org Subject: How to compile opencrypto separately Date: Mon=2C 21 Nov 2011 06:24:17 +0000 Hi I have some problems with opencrypto project and i hope you want to help m= e for solve them. - I trying to add my hash algorithm into opencrypto project and compile it = for use it later with cryptodev and IPSEC vpn. I use rmd160 as a sample and do this steps: 1- rename rmd160.c & rmd160.h to myhash.c & myhash.h . 2- change the name and the contents of the functions of rmd160 to myhash.(= used standard form in the title of functions) 3- add my function titles and variables in xform.c & xform.h . 4- add my fixed value of my algorithm in the cryptodev.h . 5- recompile my kernel with IPSEC option and cryptodev device . But in the make time i get this error : in function 'MYHASHUpdate_int' : xform.c:759: undefined refrence to 'MYHASHUpdate' what isn't enough in my steps? what should i do? thanks. = From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 21 22:35:14 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E3271065670 for ; Mon, 21 Nov 2011 22:35:14 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id A08558FC19 for ; Mon, 21 Nov 2011 22:35:13 +0000 (UTC) Received: by faap15 with SMTP id p15so7180376faa.13 for ; Mon, 21 Nov 2011 14:35:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=j27qKiK4cmG8L7CZs5aGXVQ/DqxXmvCuFqPXpngkyjA=; b=jpM3ifGtXFSfnBnWB6mqJt80P9DRA2wyw2aCpnzPyG8HgBlm+PurFA806aiMsSWHM0 IFWy+OqMT3H0EATMgySZ/Oza/IK+BDXtKrPftjd+v9vc9iZ3YV0/zrN+gnQxmDGECZ30 PSQHbSPGJD0q58ytOIkfQYXELkacf7jBd7SdY= MIME-Version: 1.0 Received: by 10.180.24.65 with SMTP id s1mr15901735wif.59.1321914912493; Mon, 21 Nov 2011 14:35:12 -0800 (PST) Received: by 10.180.95.104 with HTTP; Mon, 21 Nov 2011 14:35:12 -0800 (PST) In-Reply-To: References: Date: Mon, 21 Nov 2011 17:35:12 -0500 Message-ID: From: Ryan Stone To: aram baghomian Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: How to compile opencrypto separately X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Nov 2011 22:35:14 -0000 2011/11/21 aram baghomian : > > Hi > > I'm sorry, i forgot to register my email =A0address to your mailing list.= now i registered in it and resend > my question. > > I have some problems with opencrypto project =A0and i hope you want to he= lp me for solve them. > > - I want to compile opencrypto code without to force compile it as a subs= ystem of kernel and > with whole code of my kernel. How can i do that? > > - I trying to add my hash algorithm into opencrypto project and compile i= t for use it later with cryptodev > and IPSEC vpn. > > I use rmd160 as a sample and do this steps: > =A01- rename rmd160.c & rmd160.h to myhash.c & myhash.h . > =A02- change the name and the contents of the functions of rmd160 to myha= sh.(used standard form in the title of functions) > =A03- add my function titles and variables in xform.c & xform.h . > =A04- add my fixed value of my algorithm in the cryptodev.h . > =A05- recompile my kernel with IPSEC option and cryptodev device . > > But in the build time i get this error : > > in function 'MYHASHUpdate_int' : > xform.c:759: undefined refrence to 'MYHASHUpdate' > > what isn't enough in my steps? > what should i do? > > i'm sorry for my english. > > thanks. You need to add myhash.c to sys/conf/files (use what is done for rmd160.c as a template) and then rebuild your kernel. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 00:41:09 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBA16106566B for ; Tue, 22 Nov 2011 00:41:09 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7323A8FC0A for ; Tue, 22 Nov 2011 00:41:09 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so9659181bkb.13 for ; Mon, 21 Nov 2011 16:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sEGwkh2nTib8g70r7/+SvBscOpxnAMoy7OJ1LMav3Pk=; b=XflsPSHRRa6XhJYcnKKb3nP72dVRsYt9kHhZcAjSIMxn2e17eb9D/tbgiAHDbX1wk6 mMMkr+yP9NxzmGeUzY5w0qwd011zpXSEztmCtadmogkVJT/QmA/6xfhjRjKLKyhrse9W 6v/ejA9VzHlHc2uAV1SL77VYfWIt8pvwxsr5Y= MIME-Version: 1.0 Received: by 10.204.154.77 with SMTP id n13mr16877832bkw.83.1321922468153; Mon, 21 Nov 2011 16:41:08 -0800 (PST) Received: by 10.223.88.72 with HTTP; Mon, 21 Nov 2011 16:41:08 -0800 (PST) In-Reply-To: <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> References: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> Date: Mon, 21 Nov 2011 18:41:08 -0600 Message-ID: From: Adam Vande More To: Stefan Bethke Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 00:41:10 -0000 On Mon, Nov 21, 2011 at 3:25 PM, Stefan Bethke wrote: > > Unfortunately, the process I want to limit is not sufficiently CPU bound > to be limited that way vs. all the other processes. I guess I'll put in a > second disk. > > Well, a couple other suggestions. Have you tried with gsched? It's pretty easy to turn on and might be good enough to keep the system responsive in your workload. Also AFAIK ZFS has a built in scheduler, not sure if it's adequate or tunable. Finally workaround in VirtualBox. The "VBoxManage bandwidthctl" allows you to set bandwidth per disk image. -- Adam Vande More From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 04:30:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84F44106564A; Tue, 22 Nov 2011 04:30:03 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU [18.9.25.15]) by mx1.freebsd.org (Postfix) with ESMTP id 029FD8FC18; Tue, 22 Nov 2011 04:30:02 +0000 (UTC) X-AuditID: 1209190f-b7f6e6d0000008df-0f-4ecb254a6de1 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id B0.8F.02271.A452BCE4; Mon, 21 Nov 2011 23:30:02 -0500 (EST) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id pAM4U09Y027854; Mon, 21 Nov 2011 23:30:01 -0500 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id pAM4TwPR007471 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 Nov 2011 23:29:59 -0500 (EST) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id pAM4Tv3o004774; Mon, 21 Nov 2011 23:29:57 -0500 (EST) Date: Mon, 21 Nov 2011 23:29:56 -0500 (EST) From: Benjamin Kaduk To: Alexander Best In-Reply-To: <20111121120751.GA85679@freebsd.org> Message-ID: References: <201111172055.pAHKtZso061118@triton8.kn-bremen.de> <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> <20111121110206.GA63744@freebsd.org> <20111121120751.GA85679@freebsd.org> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrDIsWRmVeSWpSXmKPExsUixG6nruuletrPoOEpu0X753lMFts3/2O0 aNx3lcViXuc5ZgcWjxmf5rN4PJjawegxfe91pgDmKC6blNSczLLUIn27BK6MP5M+sBccU6iY cSimgfGlZBcjJ4eEgInEz3mLGSFsMYkL99azdTFycQgJ7GOUmP+ggwXC2cAoMX3NCqjMASaJ pp197BBOA6PEvG+PmED6WQS0JU5MPcwKYrMJqEjMfLORDcQWEdCQ2NbwmBnEZhbwkzi1/C9Y XFjAQWJOw0Owek4BQ4mFS7rA7uAVsJfYfG0l2EwhgdfMEsuu1IHYogI6Eqv3T2GBqBGUODnz CQvETEuJf2t/sU5gFJyFJDULSWoBI9MqRtmU3Crd3MTMnOLUZN3i5MS8vNQiXRO93MwSvdSU 0k2MoEDmlOTfwfjtoNIhRgEORiUe3sUTT/kJsSaWFVfmHmKU5GBSEuVdonLaT4gvKT+lMiOx OCO+qDQntfgQowQHs5IIr+x5oHLelMTKqtSifJiUNAeLkjhv4w4HPyGB9MSS1OzU1ILUIpis DAeHkgTvHJChgkWp6akVaZk5JQhpJg5OkOE8QMNPgtTwFhck5hZnpkPkTzEqSonzngNJCIAk Mkrz4HphieYVozjQK8K8y0CqeIBJCq77FdBgJqDB09aeABlckoiQkmpgTOs7xbVQJDqASVf2 nEpYlH+qfmToxj7XcmaVFO1/l4PfbyzIncQed/LrVqs/kbNeHVhen7nrMvu+Yy/uT6qYoBFT FOklJKHfsmjr7pmzukpTtGPFnjg765bvOdUju03k0xyPecIXDu9ea3XxD5Nb+9qOwDvZ3y9r bFl/9pqxypSIHx2HdRdtVGIpzkg01GIuKk4EALv0LjcPAwAA Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 04:30:03 -0000 On Mon, 21 Nov 2011, Alexander Best wrote: > On Mon Nov 21 11, Alexander Best wrote: >> On Mon Nov 21 11, Benjamin Kaduk wrote: >>> On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: >>> >>>> Alexander Best wrote: >>>> >>>>> here's a revised patch. >>>>> ... >>>>> +.Sh CAVEATS >>>>> +If the >>>>> +.Fn lseek >>>>> +system call is operating on a device, which is incapable of seeking, >>>>> +it will request the seek operation and complete successfully. >>>> >>>> I think it would be better without the first comma (after "device"). >>> >>> Definitely. >>> >>> Also, >>> >>> +.Sh CAVEATS >>> +If the >>> +.Fn lseek >>> +system call is operating on a device, which is incapable of seeking, >>> +it will request the seek operation and complete successfully. >>> >>> I would prefer something like "request the seek operation and return as if >>> the seek was successful, even though no seek was performed." >>> >>> +The value of the pointer associated with such a device is undefined. >>> >>> "Which pointer?" That it is "the file offset" was clear from context >>> where this line was moved from, but is no longer, here. >>> >>> +Device types which can be incapable of seeking include, >>> +but are not limited to, tape drives. >>> >>> This is an awkward phrasing; perhaps just "Many tape drives are incapable >>> of seeking and can trigger this bug."? >> >> this is too limited. this suggests that only certain tape drives won't seek >> after a successfull return of lseek(). as i mentioned beforehand, this is also >> the case with device with insertable media, such as dvd and blue-ray drives. >> here lseek() will sucessfully return, without a media inserted. >> >> i'll rephrase the whole patch and will submit a revised version. i think a >> reference to POLA, would also be a good idea, as suggested by perry@ > > here's a revised patch. % diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 % index 874c523..bcd9d20 100644 % --- a/lib/libc/sys/lseek.2 % +++ b/lib/libc/sys/lseek.2 % @@ -197,13 +196,43 @@ is associated with a pipe, socket, or FIFO. % The % .Fn lseek % system call is expected to conform to % -.St -p1003.1-90 . % +.St -p1003.1-2008 . % +.Pp % +The % +.Dv SEEK_HOLE % +and % +.Dv SEEK_DATA % +directives, along with the % +.Er ENXIO % +error, are extensions to that specification. % .Sh HISTORY % The % .Fn lseek % function appeared in % .At v7 . % .Sh BUGS % +If the % +.Fn lseek % +system call is operating on a device which is incapable of seeking, % +it will request the seek operation and return successfully, % +even though no seek was performed. % +Because the % +.Ar offset % +argument will be stored in the file descriptor of that device, This sentence assumes more familiarity with file i/o implementation than seems prudent. Perhaps "stored in the file descriptor of that device and thus used for future queries" or something similar? % +there is no way to verifying/falsify the seek operation afterwards "verifying" is incorrect, here. Just "verify" would work, but the combo "verify/falsify" doesn't feel quite right. I guess I want "no way to confirm success of the seek operation" (no 'afterwards'). % +(e.g. using the % +.Fn ftell % +function). % +Device types which are known to be incapable of seeking include % +tape drives. "most"? I think someone said that certain (old) drives could actually seek under some circumstances. % +.Pp % +The % +.Fn lseek % +system call will not detect, whether media are present in changeable % +media devices, such as DVD or Blue-ray devices. The first comma is bogus; the second comma could be removed, and probably should be. Also, "Blu-ray" has no 'e' (but apparently is capitalized in that way). % +A requested seek operation will therefore return sucessfully in case % +of an uninserted medium. s/in case of an uninserted medium/when no medium is present/. Thanks for the fixups, Ben % +.Pp % This document's use of % .Fa whence % is incorrect English, but is maintained for historical reasons. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 06:35:59 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3404F106564A for ; Tue, 22 Nov 2011 06:35:59 +0000 (UTC) (envelope-from eugen@grosbein.pp.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13::5]) by mx1.freebsd.org (Postfix) with ESMTP id 92EAA8FC0C for ; Tue, 22 Nov 2011 06:35:57 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.5/8.14.5) with ESMTP id pAM6Zsdn086788; Tue, 22 Nov 2011 13:35:54 +0700 (NOVT) (envelope-from eugen@grosbein.pp.ru) Message-ID: <4ECB42C5.6050108@grosbein.pp.ru> Date: Tue, 22 Nov 2011 13:35:49 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: Luigi Rizzo References: <4EC39367.4030109@rdtc.ru> <4EC9EE5A.2070204@grosbein.pp.ru> <20111121083003.GA54883@onelab2.iet.unipi.it> In-Reply-To: <20111121083003.GA54883@onelab2.iet.unipi.it> Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 8bit Cc: hackers@freebsd.org Subject: Re: dummynet(4) kernel process CPU usage monitoring X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 06:35:59 -0000 21.11.2011 15:30, Luigi Rizzo ÐÉÛÅÔ: > On Mon, Nov 21, 2011 at 01:23:22PM +0700, Eugene Grosbein wrote: >> Hi! >> >> I need to draw graph of dummynet's CPU usage. >> "procstat -t 0" shows me TID (thread id) of dummynet kernel thread. >> "ps -Hxo time,lwp" shows me total CPU time consumed by this thread. >> >> Now I see this time has 9 seconds increase during 60 seconds of real time. >> This should be 9/60=15% CPU usage, but "top -SHP" shows me 0.00% meantime. > > apart from the scaling on number of cores (e.g. if you have 8 cores > the 15% becomes a bare 2%) I have 4 cores (no hyperthreading) and it seems that "top -SHP" does not scale anything in this mode. > remember that percentages are computed > with some kind of filtering (EWMA ?) so if the load of dummynet threads > is bursty, the filter might eat most of it. I do not understand what this filter is. Btw, dummynet load is steady here. > not completely sure this explains a steady 0.00%, if that's is > what you are seeing Yes, I see 0.00% except of hours of most load where top shows about 10% and ps shows about 30%. Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 15:29:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 517EA106564A for ; Tue, 22 Nov 2011 15:29:11 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id E04188FC20 for ; Tue, 22 Nov 2011 15:29:10 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so454384bkb.13 for ; Tue, 22 Nov 2011 07:29:09 -0800 (PST) MIME-Version: 1.0 Received: by 10.205.128.15 with SMTP id hc15mr19166586bkc.110.1321975749409; Tue, 22 Nov 2011 07:29:09 -0800 (PST) Received: by 10.223.70.205 with HTTP; Tue, 22 Nov 2011 07:29:09 -0800 (PST) X-Originating-IP: [209.66.78.50] Date: Tue, 22 Nov 2011 10:29:09 -0500 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Approaching the limit on PV entries X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 15:29:11 -0000 Hello All I want to get to the bottom of a warning in dmesg. On 7.2-RELEASE and 7.3-RELEASE I have seen the following warning in dmesg. Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. So looking around I see a few posts here and there about how to tune the sysctls to address the warning however I am not 100% sure what each value does. It appears changing vm.pmap.shpgperproc affects the value of vm.pmap.pv_entry_max . Can someone explain the relationship of the two sysctls. Also what pitfalls of changing them are. Also why would setting kern.ipc.shm_use_phys=1 effect the pv entries. Is this supposed to lower the pv entries ? Some details on the systems where I saw this. FreeBSD 7.3-RELEASE-p5 amd64 2G RAM + 2G SWAP 1 single core opteron Superpages are enabled The servers are high traffic web-servers serving static content. tunings kern.ipc.maxsockbuf=2097152 kern.ipc.nmbclusters=32768 kern.ipc.somaxconn=1024 kern.maxfiles=131072 kern.maxfilesperproc=32768 net.inet.tcp.inflight.enable=0 net.inet.tcp.path_mtu_discovery=0 net.inet.tcp.recvbuf_max=2097152 net.inet.tcp.sendbuf_max=2097152 net.isr.direct=1 vm.pmap.shpgperproc=600 vfs.read_max=64 http://lists.freebsd.org/pipermail/freebsd-hackers/2009-December/030193.html http://lists.freebsd.org/pipermail/freebsd-hackers/2009-July/029076.html http://forums.freebsd.org/showthread.php?t=17786 -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 20:30:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6FA5106564A for ; Tue, 22 Nov 2011 20:30:57 +0000 (UTC) (envelope-from aram_baghomian@hotmail.com) Received: from snt0-omc2-s50.snt0.hotmail.com (snt0-omc2-s50.snt0.hotmail.com [65.54.61.101]) by mx1.freebsd.org (Postfix) with ESMTP id BF1BC8FC0A for ; Tue, 22 Nov 2011 20:30:57 +0000 (UTC) Received: from SNT140-W64 ([65.55.90.72]) by snt0-omc2-s50.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Tue, 22 Nov 2011 12:30:57 -0800 Message-ID: X-Originating-IP: [2.146.142.238] From: aram baghomian To: Date: Tue, 22 Nov 2011 20:30:57 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 22 Nov 2011 20:30:57.0110 (UTC) FILETIME=[A3BA1F60:01CCA955] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: crypto.ko module problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 20:30:58 -0000 Hi If you can remember i wanted to add my custom hash algorithm to the opencry= pto project=2C after i added it=20 and compile my kernel source( by your advise)=2Ci load the crypto.ko module= using kldload and give this error. link_elf: symbol MYHASHUpdate undefined MYHASHUpdate is one of my hash functions that i add to the source. what should i do that i forget?=20 = From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 20:36:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3BC4106566B for ; Tue, 22 Nov 2011 20:36:45 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 9A9CD8FC15 for ; Tue, 22 Nov 2011 20:36:45 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 666323A19; Tue, 22 Nov 2011 12:36:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1321994205; bh=4E8nPDirIrL3v+CFt8kS6W99EGoxI2k95+/tITVFmj4=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=RzZOq3sw9xl/89ky3wR4n/miPZANiKfuBCWrfttirpNyznyifoTYcgK3XioQsbDOH wy6w1JQKZi4v790kgPflYMtxKs1y6zM4CmcCBR2ETOJOt9TJH+h2OmKUpVR5QKon4i EPdsInB0sYYKRpcvtI/8JQHVq0u7J5y/7c1a3BX0= Message-ID: <4ECC07DC.2070403@delphij.net> Date: Tue, 22 Nov 2011 12:36:44 -0800 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: aram baghomian References: In-Reply-To: OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: crypto.ko module problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 20:36:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 11/22/11 12:30, aram baghomian wrote: > > Hi > > If you can remember i wanted to add my custom hash algorithm to the > opencrypto project, after i added it and compile my kernel source( > by your advise),i load the crypto.ko module using kldload and give > this error. > > link_elf: symbol MYHASHUpdate undefined > > MYHASHUpdate is one of my hash functions that i add to the source. > > what should i do that i forget? Looks like it's expecting something that is not linked into your module? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOzAfcAAoJEATO+BI/yjfB4JcIAIB6dwTlsvj9djUdKGkRI8ix G/pJaZIONh3mUVV7TLdsl2OUtzDTmvGbV1hh1LCo/cmKfZMUOWJWoH6Sx+LnMQYR ftfHnpFk4bQP/38aP3yTHjGX1mGzhCFWCyHYm4d45Jl+ukjRwlZw7eeTzDTFYtF9 cJe/fRs8GGWIDh3MvLC5ZjMInPneOJk4jEKv0SARbDk+bG23q//xf5HQkml4Q35g 2sxaE7750IDtqER/9cKxtd4Akr1blRkp0jOxzH3vkgubjY63k2fCj9mXNKxR82wb rx9igsYzOjl6rLpv0agJINpyl1gXHrl28ielJAmbWT5uwfEawG1LJw/N57o5EY8= =Gkm6 -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 20:44:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 1D90B1065675; Tue, 22 Nov 2011 20:44:25 +0000 (UTC) Date: Tue, 22 Nov 2011 20:44:25 +0000 From: Alexander Best To: Benjamin Kaduk Message-ID: <20111122204425.GA35693@freebsd.org> References: <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> <20111121110206.GA63744@freebsd.org> <20111121120751.GA85679@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 20:44:25 -0000 On Mon Nov 21 11, Benjamin Kaduk wrote: > On Mon, 21 Nov 2011, Alexander Best wrote: > > >On Mon Nov 21 11, Alexander Best wrote: > >>On Mon Nov 21 11, Benjamin Kaduk wrote: > >>>On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: > >>> > >>>>Alexander Best wrote: > >>>> > >>>>>here's a revised patch. > >>>>>... > >>>>>+.Sh CAVEATS > >>>>>+If the > >>>>>+.Fn lseek > >>>>>+system call is operating on a device, which is incapable of seeking, > >>>>>+it will request the seek operation and complete successfully. > >>>> > >>>>I think it would be better without the first comma (after "device"). > >>> > >>>Definitely. > >>> > >>>Also, > >>> > >>>+.Sh CAVEATS > >>>+If the > >>>+.Fn lseek > >>>+system call is operating on a device, which is incapable of seeking, > >>>+it will request the seek operation and complete successfully. > >>> > >>>I would prefer something like "request the seek operation and return as > >>>if > >>>the seek was successful, even though no seek was performed." > >>> > >>>+The value of the pointer associated with such a device is undefined. > >>> > >>>"Which pointer?" That it is "the file offset" was clear from context > >>>where this line was moved from, but is no longer, here. > >>> > >>>+Device types which can be incapable of seeking include, > >>>+but are not limited to, tape drives. > >>> > >>>This is an awkward phrasing; perhaps just "Many tape drives are incapable > >>>of seeking and can trigger this bug."? > >> > >>this is too limited. this suggests that only certain tape drives won't > >>seek > >>after a successfull return of lseek(). as i mentioned beforehand, this is > >>also > >>the case with device with insertable media, such as dvd and blue-ray > >>drives. > >>here lseek() will sucessfully return, without a media inserted. > >> > >>i'll rephrase the whole patch and will submit a revised version. i think a > >>reference to POLA, would also be a good idea, as suggested by perry@ > > > >here's a revised patch. > > % diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 > % index 874c523..bcd9d20 100644 > % --- a/lib/libc/sys/lseek.2 > % +++ b/lib/libc/sys/lseek.2 > % @@ -197,13 +196,43 @@ is associated with a pipe, socket, or FIFO. > % The > % .Fn lseek > % system call is expected to conform to > % -.St -p1003.1-90 . > % +.St -p1003.1-2008 . > % +.Pp > % +The > % +.Dv SEEK_HOLE > % +and > % +.Dv SEEK_DATA > % +directives, along with the > % +.Er ENXIO > % +error, are extensions to that specification. > % .Sh HISTORY > % The > % .Fn lseek > % function appeared in > % .At v7 . > % .Sh BUGS > % +If the > % +.Fn lseek > % +system call is operating on a device which is incapable of seeking, > % +it will request the seek operation and return successfully, > % +even though no seek was performed. > % +Because the > % +.Ar offset > % +argument will be stored in the file descriptor of that device, > > This sentence assumes more familiarity with file i/o implementation than > seems prudent. Perhaps "stored in the file descriptor of that device and > thus used for future queries" or something similar? > > % +there is no way to verifying/falsify the seek operation afterwards > > "verifying" is incorrect, here. Just "verify" would work, but the combo > "verify/falsify" doesn't feel quite right. > I guess I want "no way to confirm success of the seek operation" (no > 'afterwards'). > > % +(e.g. using the > % +.Fn ftell > % +function). > % +Device types which are known to be incapable of seeking include > % +tape drives. > > "most"? I think someone said that certain (old) drives could actually > seek under some circumstances. > > % +.Pp > % +The > % +.Fn lseek > % +system call will not detect, whether media are present in changeable > % +media devices, such as DVD or Blue-ray devices. > > The first comma is bogus; the second comma could be removed, and probably > should be. > Also, "Blu-ray" has no 'e' (but apparently is capitalized in that way). > > % +A requested seek operation will therefore return sucessfully in case > % +of an uninserted medium. > > s/in case of an uninserted medium/when no medium is present/. > > Thanks for the fixups, thanks for your suggestions. i included some of them into the patch and also added a few stuff myself. maybe you could have a look at the problem report i submitted [1] and post a followup mail, what you think. this worked quite well the last time, in case of the du(1) man page patch, imo. ;) cheers. alex [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=162765 > > Ben > > > % +.Pp > % This document's use of > % .Fa whence > % is incorrect English, but is maintained for historical reasons. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 21:23:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F9881065677 for ; Tue, 22 Nov 2011 21:23:31 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 5430D8FC24 for ; Tue, 22 Nov 2011 21:23:31 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 04EA83CC6; Tue, 22 Nov 2011 13:23:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1321997011; bh=/zslYaIHF5hZL8k4Gbmwg5eLyOL+r6TnErhI3dlb6bE=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=N+Alh9v4Wef6MEWS1JfdIgS1gI1LwfigjTl56fOesv1H99Ji1RESNh7jpf1tJFOKl /PSTZik/6GvILB525xwvCXxrIimV0ugd6+OVabpkgBOqoxuEPcbY9a0n1lCgvtMgTM anx0s8LClHeY5fZWK6+RnX+zaYYmUa52kZ1ePdrE= Message-ID: <4ECC12D2.2090300@delphij.net> Date: Tue, 22 Nov 2011 13:23:30 -0800 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: aram baghomian , FreeBSD-Hackers References: , <4ECC07DC.2070403@delphij.net> , <4ECC0CDA.9060108@delphij.net> In-Reply-To: OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: d@delphij.net Subject: Re: crypto.ko module problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 21:23:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 [Added -hackers@ back to Cc list] On 11/22/11 13:13, aram baghomian wrote: > I add myhash.c address in '/sys/conf/files' and my make process > done completely whitout any error. I don't know is there any place > that i should add my hash program address.(i searched all the > "/sys/conf" files for same algorithms such as rmd160 and SHA256) If you are compiling a kernel module, it would be /sys/modules/*/Makefile (for your exact case would be /sys/modules/crypto/Makefile I think). >> Date: Tue, 22 Nov 2011 12:58:02 -0800 From: delphij@delphij.net >> To: aram_baghomian@hotmail.com CC: d@delphij.net Subject: Re: >> crypto.ko module problem >> > On 11/22/11 12:49, aram baghomian wrote: >> I can understand what do you want to refer. > > Well be more specific -- check if you have everything included in > your Makefile, my guess is there is some .c missing. > > >>> Date: Tue, 22 Nov 2011 12:36:44 -0800 From: >>> delphij@delphij.net To: aram_baghomian@hotmail.com CC: >>> freebsd-hackers@freebsd.org Subject: Re: crypto.ko module >>> problem > >> On 11/22/11 12:30, aram baghomian wrote: > >>> Hi > >>> If you can remember i wanted to add my custom hash algorithm >>> to the opencrypto project, after i added it and compile my >>> kernel source( by your advise),i load the crypto.ko module >>> using kldload and give this error. > >>> link_elf: symbol MYHASHUpdate undefined > >>> MYHASHUpdate is one of my hash functions that i add to the >>> source. > >>> what should i do that i forget? > >> Looks like it's expecting something that is not linked into your >> module? > >> Cheers, > - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOzBLRAAoJEATO+BI/yjfBgxQH/A36S+IZt/V0D/LcDZWcBubC CoHSoVCBkMcyFIa4SF7MJvuLT1zCec+lsPr9/E7GGVqLqFy/XyaMC0TpHi4J+hg2 xOvwvEkSPWmwe9SoEpo91BpL+hK5P/uTQ+MBoMBV4kz+eI7VbLZpRFjlEOh2ZhGK kWuRRsHZWXSiPDsZW4Pbcmt6HxhvvCjGHN/RPrQ4lmcU52SBgvvexXiJeTXdcZ4F 9hO3oPLBXat4aypX9BeGvqi6vydOiqXdIsFKBd3LG3V4l+QIjb9Rs2epciYNXgQi u6cspLayVcnyNxEoA9j09Mi/iuGxy7qcGc/YsnEo0K0tzi25ZW2w/TJX+2Nmg7Y= =vzBy -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 22 22:35:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A07071065677 for ; Tue, 22 Nov 2011 22:35:23 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 81C058FC1B for ; Tue, 22 Nov 2011 22:35:23 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 6DFBF6104; Tue, 22 Nov 2011 14:35:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1322001323; bh=U+qfGsMjYneUmBJWbtAA2yO/NrsxCh4u9VH5iEtERGQ=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=K9n2uU4D0pq1CygOnzppVNM8pqxoBQVoQvKWJkW5ENhmgKSwJUjyhqLbqxrXnRmp6 If9PYQW4VeWMuS3h4Bjhx5WnrnqAmQXMoRfuo5MBH+xSWrUIfykE24xpqs6/RbHxiC Dei8kqZMAlaouuDyPjVNutx4rYRSTmWExE1CuTiE= Message-ID: <4ECC23A8.4080807@delphij.net> Date: Tue, 22 Nov 2011 14:35:20 -0800 From: Xin LI Organization: The FreeBSD Project MIME-Version: 1.0 To: aram baghomian References: , <4ECC07DC.2070403@delphij.net>, , <4ECC0CDA.9060108@delphij.net>, , <4ECC12D2.2090300@delphij.net>, , In-Reply-To: OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Hackers , d@delphij.net Subject: Re: crypto.ko module problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Nov 2011 22:35:23 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 11/22/11 14:25, aram baghomian wrote: > > one more question. the crypto.ko module need zlib.ko module for > dependence, how can i config kernel to load zlib.ko before > crypto.ko in boot time? Add a dependency, something like: MODULE_DEPEND(crypto, zlib, 1, 1, 1); (Note I think crypto.ko already do that though). > > ------------------------------------------------------------------------ > > From: aram_baghomian@hotmail.com > To: d@delphij.net Subject: RE: crypto.ko module problem Date: Tue, > 22 Nov 2011 21:57:40 +0000 > > I add my hash program address to /sys/modules/crypto/Makefile and > recompile the module. it loaded successfully. Thanks alot. > >> Date: Tue, 22 Nov 2011 13:23:30 -0800 From: delphij@delphij.net >> To: aram_baghomian@hotmail.com; freebsd-hackers@freebsd.org CC: >> d@delphij.net Subject: Re: crypto.ko module problem >> > [Added -hackers@ back to Cc list] > > On 11/22/11 13:13, aram baghomian wrote: >> I add myhash.c address in '/sys/conf/files' and my make process >> done completely whitout any error. I don't know is there any >> place that i should add my hash program address.(i searched all >> the "/sys/conf" files for same algorithms such as rmd160 and >> SHA256) > > If you are compiling a kernel module, it would be > /sys/modules/*/Makefile (for your exact case would be > /sys/modules/crypto/Makefile I think). > >>> Date: Tue, 22 Nov 2011 12:58:02 -0800 From: >>> delphij@delphij.net To: aram_baghomian@hotmail.com CC: >>> d@delphij.net Subject: Re: crypto.ko module problem > >> On 11/22/11 12:49, aram baghomian wrote: >>> I can understand what do you want to refer. > >> Well be more specific -- check if you have everything included >> in your Makefile, my guess is there is some .c missing. > > >>>> Date: Tue, 22 Nov 2011 12:36:44 -0800 From: >>>> delphij@delphij.net To: aram_baghomian@hotmail.com CC: >>>> freebsd-hackers@freebsd.org Subject: Re: crypto.ko module >>>> problem > >>> On 11/22/11 12:30, aram baghomian wrote: > >>>> Hi > >>>> If you can remember i wanted to add my custom hash algorithm >>>> to the opencrypto project, after i added it and compile my >>>> kernel source( by your advise),i load the crypto.ko module >>>> using kldload and give this error. > >>>> link_elf: symbol MYHASHUpdate undefined > >>>> MYHASHUpdate is one of my hash functions that i add to the >>>> source. > >>>> what should i do that i forget? > >>> Looks like it's expecting something that is not linked into >>> your module? > >>> Cheers, > > - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJOzCOoAAoJEATO+BI/yjfBisEH/2ompnPAk5Doi5VACeJ8zZaz kfuYfq8HHoit5WzbDsXFt5A+uii5brafTKxFYRVb0zB6BmjnFbgIxqCebSw3jNll TBjzhnSI5g1O43n2/XAQT2RjIJdfWnZqfBI0JLpZqjedObMbmQiyAyxz2bpDPV7H gtb/ElUa17bRVCwjoAy2iKC9hvwIixjMhcWvlrg49kVJyFFd6gVjBxUplS2SMEfv ZTLDIY8ZeQGCIolVZv8gcjeT1ToSmKQHIGL5V25bb22TVCX/oONuhfM46A/cOaSN Jsj06Hp/BkDMlYA2mFYAXyQ+OotuW9lz+JVC12o8EBoFpILVNcbAuMVKLa4a6hs= =JXyV -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 24 08:44:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 544BE106564A for ; Thu, 24 Nov 2011 08:44:10 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id D0D918FC13 for ; Thu, 24 Nov 2011 08:44:09 +0000 (UTC) Received: by faap15 with SMTP id p15so3775013faa.13 for ; Thu, 24 Nov 2011 00:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=3ioXrVI/OnGOer7ObwC/BaaaIISPOB20NmrN7mDA3J4=; b=ateqFatJLfZ0g0jLtSNdYG7yPhx5IGf7/VoFjjHYUjbpZHMyzkuX0wTuGJOzT3GwHa ODBxPeVSpnoiTx7tNEnjBuXV8imgbeWlrYrX4esoZDjO0ctTgV7angMm4BLZMjD3TbvM plavYgdIYNyX/Jbmw86N7n48YhyMUswoeKSSw= Received: by 10.180.85.4 with SMTP id d4mr12553889wiz.19.1322122547850; Thu, 24 Nov 2011 00:15:47 -0800 (PST) Received: from DataIX.net (ppp-21.212.dialinfree.com. [209.172.21.212]) by mx.google.com with ESMTPS id j5sm9124581wix.20.2011.11.24.00.15.41 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Nov 2011 00:15:45 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAO8FZcx011077 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 24 Nov 2011 03:15:36 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAO8FWMf011076; Thu, 24 Nov 2011 03:15:32 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Thu, 24 Nov 2011 03:15:32 -0500 From: Jason Hellenthal To: Stefan Bethke Message-ID: <20111124081532.GA10540@DataIX.net> References: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> Cc: Adam Vande More , freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Nov 2011 08:44:10 -0000 Hi Stefan, This is probably what you are looking for. Give it a shot. https://jhell.googlecode.com/hg/base/vendor/vehosting/slowdown.c On Mon, Nov 21, 2011 at 10:25:36PM +0100, Stefan Bethke wrote: > > Am 21.11.2011 um 21:42 schrieb Stefan Bethke: > > > Am 21.11.2011 um 21:40 schrieb Adam Vande More: > > > >> On Mon, Nov 21, 2011 at 1:58 PM, Stefan Bethke wrote: > >> > >> Interesting, but it doesn't seem to offer limiting the I/O bandwidth induced by a process or jail, or assigning different priorities, which would need to be implemented in the ZFS or GEOM schedulers, I suppose. > >> > >> Limiting CPU has long been the poor man's IO scheduler, and has usually worked pretty well for me but has required some trial and error. YMMV > > > > Good point, I'll give that a try. > > > Unfortunately, the process I want to limit is not sufficiently CPU bound to be limited that way vs. all the other processes. I guess I'll put in a second disk. > > > Stefan > > -- > Stefan Bethke Fon +49 151 14070811 > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 24 10:15:06 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00503106566B for ; Thu, 24 Nov 2011 10:15:05 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id B1A598FC12 for ; Thu, 24 Nov 2011 10:15:05 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 778AA110EA5; Thu, 24 Nov 2011 11:15:02 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20111124081532.GA10540@DataIX.net> Date: Thu, 24 Nov 2011 11:14:58 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> <20111124081532.GA10540@DataIX.net> To: Jason Hellenthal X-Mailer: Apple Mail (2.1084) Cc: Adam Vande More , freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Nov 2011 10:15:06 -0000 Am 24.11.2011 um 09:15 schrieb Jason Hellenthal: >=20 > Hi Stefan, >=20 > This is probably what you are looking for. Give it a shot. >=20 > https://jhell.googlecode.com/hg/base/vendor/vehosting/slowdown.c Just tried this: Unfortunately, my app is not slowed down sufficiently. = I'm afraid I need something that actually re-prioritizes actual disk = I/O. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 24 14:51:27 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC09F106564A for ; Thu, 24 Nov 2011 14:51:26 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 742008FC08 for ; Thu, 24 Nov 2011 14:51:26 +0000 (UTC) Received: by wwe5 with SMTP id 5so133083wwe.31 for ; Thu, 24 Nov 2011 06:51:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=bAh9cibSKDYXj2HKzJrG8gHJkNe5CKaoE/v9smgw3p8=; b=pcanPHGtsA/bD1ecwJVUWd75sKZ2BYjtbtk14zWDDH972ra9mruy2ENFeoZuMJBCc0 4wWlk5E1QsqkuWzhe37HZ1CnuvVgXCoF2UmWht+DPixaQHJfMULNN8+CweIQtjQzh7WT 7wg/Ya+Rr+LfbhqSVpR4sFcAlX6se+eoM9u2Q= Received: by 10.227.209.9 with SMTP id ge9mr18351835wbb.1.1322146285337; Thu, 24 Nov 2011 06:51:25 -0800 (PST) Received: from DataIX.net (ppp-21.226.dialinfree.com. [209.172.21.226]) by mx.google.com with ESMTPS id a27sm24357408wbp.16.2011.11.24.06.51.20 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Nov 2011 06:51:24 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAOEpD4Y027833 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 24 Nov 2011 09:51:14 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAOEpDZx027832; Thu, 24 Nov 2011 09:51:13 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Thu, 24 Nov 2011 09:51:13 -0500 From: Jason Hellenthal To: Stefan Bethke Message-ID: <20111124145113.GB10540@DataIX.net> References: <33122B07-5473-4C84-A89D-B4C2F9677BC0@lassitu.de> <6354F6F2-959D-4451-A434-32C5C7335C25@lassitu.de> <20111124081532.GA10540@DataIX.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HlL+5n6rz5pIUxbD" Content-Disposition: inline In-Reply-To: Cc: Adam Vande More , freebsd-hackers@freebsd.org Subject: Re: Limiting disk I/O by jail or uid? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Nov 2011 14:51:27 -0000 --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 24, 2011 at 11:14:58AM +0100, Stefan Bethke wrote: > Am 24.11.2011 um 09:15 schrieb Jason Hellenthal: >=20 > >=20 > > Hi Stefan, > >=20 > > This is probably what you are looking for. Give it a shot. > >=20 > > https://jhell.googlecode.com/hg/base/vendor/vehosting/slowdown.c >=20 > Just tried this: Unfortunately, my app is not slowed down sufficiently. = I'm afraid I need something that actually re-prioritizes actual disk I/O. What app is this ? Considering the above code is a wrapper for stat() fstat() lstat() syscalls= and you only seen a small decrease I would believe its using something els= e quite intensively but without the application at hand or at least a ktrac= e(1) or a truss(1) of the running proccess, I don't think there is much tha= t can be done. It would be nice if something like this was natively available but we tend = to lack a few things here in FreeBSD that can be seen elsewhere. --HlL+5n6rz5pIUxbD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJOzlngAAoJEJBXh4mJ2FR+jXMH/3wznogcLe2YAv0VbSfbBccS 8f3/K/OkJWB8AyC49YSy8jDTJe4xxAt75YDY/bmgSFI95cUVCi+dvM382IMjcvxP 48xl6D6EDUXX04Y+I5xmywXw2l4lME3a38sY1T7XRb5DSTxqd5qytWPlLbdxso6m iIQaCv0KorJK+GMj/nXRVqMa8TvoNidt6Q02On93K0eKK3UJCtskTjh1GNOawB5g NTjhm7bkUeMJCk0MY3f/FCrQBVmgX45OJxtf7FdXrWMkIyywml+Fki4EsQK23EG7 +03NUN0gEbTgAt+CriUD1biOwSqbCwLxFCbzyfueakuG2Jz3nPh7tb9xbW1V3ig= =kdNH -----END PGP SIGNATURE----- --HlL+5n6rz5pIUxbD-- From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 07:31:15 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1652E106566B for ; Fri, 25 Nov 2011 07:31:15 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 9987E8FC0C for ; Fri, 25 Nov 2011 07:31:14 +0000 (UTC) Received: by wwe5 with SMTP id 5so1419171wwe.31 for ; Thu, 24 Nov 2011 23:31:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition; bh=8qmUZkIrZWWcWbPZwpcpgupQwCi4gleOJ5p9wTxWOLE=; b=UafKBt4uOwTAdUclpmQfbUYDCnpRFwVShXQ69C0LuPIxmm7kLwGTqe1gRGKN+tUuRI xkxmTCmTMaWjSBiuhMuV2nIbgdfqgxna+hgyXG+U71HiAOmj+q2OiDSRVRfsHBeCiQtb 7Kbh//1t0E7Lw98RFksCHaLve9tLLBogSBh50= Received: by 10.180.19.42 with SMTP id b10mr39038297wie.39.1322204575274; Thu, 24 Nov 2011 23:02:55 -0800 (PST) Received: from DataIX.net (ppp-21.79.dialinfree.com. [209.172.21.79]) by mx.google.com with ESMTPS id fw16sm26838638wbb.13.2011.11.24.23.02.51 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Nov 2011 23:02:54 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAP72hNo010117 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 25 Nov 2011 02:02:43 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAP72fvt010116 for hackers@FreeBSD.org; Fri, 25 Nov 2011 02:02:41 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Fri, 25 Nov 2011 02:02:41 -0500 From: Jason Hellenthal To: hackers@FreeBSD.org Message-ID: <20111125070241.GA7915@DataIX.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cWoXeonUoKmBZSoM" Content-Disposition: inline Cc: Subject: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 07:31:15 -0000 --cWoXeonUoKmBZSoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline List, When using @reboot with cron you expect your proccesses to always start when the system boots up and only when the system boots. But long after the system in question had been booted, my @reboot processes ran again! after a (/etc/rc.d/cron restart). This is normally fine and dandy until one of your @reboot jobs needs to contain a process that purges files "files that are already in use by a running daemon since the system has not rebooted" and becomes hazardous. So with that said... is there a way we could actually make this run @reboot only ? Compare the system boottime (kern.boottime) to the current time and if it is greater than ?5 minutes? do not run on any @reboot's ? or add yet another extension @boottime so it does not throw off current functionality ? Surely I could modify the scripts which do this but I find it unproductive and counter intuitive for the need to explain that @reboot means "When cron is restarted" even though the name means something completely opposite. Regards & Happy Thanks Giving. --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJOzz2QAAoJEJBXh4mJ2FR+jo4H/0pJ5OFg7JsUbu2hMw5tm5xE yLt2eMIiPiaM2PEOTBA1eo0iMF48U2BQhA5DIkAWEwO3zIMrj/HjVbOW7NT83Y7z 5MLjIWZGZmaWqlE7chlFpnQNAgLwtnzr9AljXggs9j7WH9uPYkNOoZc7Ybm0lwSe 6qEPaLc1F/IYw7FNyuOJlkjmAydDY0BBks4NyzlQB1lLAAmC2cijelpJWK3XwNmx NfOgHxCscW7C1ZDLGwCzKKHWhl6ibF+HJ/yeOCYwBs+N+p3yXQmTfzoXzxq7+Idb 3jHNsx4YZkkmyM/2/6vxPajpJxPwc+C5PcX7F2YsIx4ixxzTtzUWLrL2GLr7hTk= =P9LA -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM-- From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 07:36:47 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C035F106564A for ; Fri, 25 Nov 2011 07:36:47 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 516128FC12 for ; Fri, 25 Nov 2011 07:36:47 +0000 (UTC) Received: by wwe5 with SMTP id 5so1427600wwe.31 for ; Thu, 24 Nov 2011 23:36:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition; bh=aDFY83TEpJMZ5rGO4dcQBtIqjDE0vBCPn7K0J17VBUI=; b=wDawC0/gVXlMhR7WyY9OWwnSGxqN0/rQu3eZiZOEO+PdwhTq2mDYTp/JSFOZEnBEpX EaiAGycLMOgYQNs3JpLHjFFE/TLeRFjDNlgjSS3aQzYjZ64RiN0fEeqtXbwOmUKIJXoL gF5uTsteaHdjRd54kAVgWsrLgv+xNDneu6tco= Received: by 10.180.3.71 with SMTP id a7mr32148078wia.0.1322206606431; Thu, 24 Nov 2011 23:36:46 -0800 (PST) Received: from DataIX.net (ppp-21.79.dialinfree.com. [209.172.21.79]) by mx.google.com with ESMTPS id f18sm10349700wiv.14.2011.11.24.23.36.43 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Nov 2011 23:36:45 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAP7abRl013994 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 25 Nov 2011 02:36:37 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAP7aVo0013993 for hackers@FreeBSD.org; Fri, 25 Nov 2011 02:36:31 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Fri, 25 Nov 2011 02:36:30 -0500 From: Jason Hellenthal To: hackers@FreeBSD.org Message-ID: <20111125073630.GC7915@DataIX.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p2kqVDKq5asng8Dg" Content-Disposition: inline Cc: Subject: sysctl description spillover and also setting the sysctl ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 07:36:47 -0000 --p2kqVDKq5asng8Dg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Found a troubling result of the following and figured someone might want to take a look. Pay close attention to the output and behavior. sysctl net.inet.udp.blackhole=0 sysctl net.inet.udp.blackhole sysctl -d net.inet.udp.blackhole=1 sysctl net.inet.udp.blackhole Is this expected ? should it not just display the description instead of adjusting ? as well not display the description like it is adjusting the description too ? # sysctl net.inet.udp.blackhole=0 net.inet.udp.blackhole: 0 -> 0 # sysctl net.inet.udp.blackhole net.inet.udp.blackhole: 0 # sysctl -d net.inet.udp.blackhole=1 net.inet.udp.blackhole: Do not send port unreachables for refused connects -> Do not send port unreachables for refused connects # sysctl net.inet.udp.blackhole net.inet.udp.blackhole: 1 --p2kqVDKq5asng8Dg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJOz0V9AAoJEJBXh4mJ2FR+LDgH/1lahamQ4SFsl1OG7suciLyw GnRLhxFzMZXNytfksUC3P+zo14WlaoaZLhDJftDk3qs5ZbLN5Qw8Zcoev9Y5xJme VJx8zIvrKFOZuurC4ydN5hLquvxWSVJg7tgTDttTP07ieZ60MIxihESTq6ZP7FDN AlCv192v5XLZ88NQmmu3oUKju7vhfDGi5JLfS2rbuE7graktQ11Nf3jOyUghSOy9 Xy/r5hVhZU2om7oh/jzv93HxAeBSGB3X5ukbUnUtR09QGnkShl6eziUbIHntDLPF ddGorEGjzLgwOezN8vfk2PMRAQ20tLX+pqSBwHf/Zmg7BFjgagEpAIYP8T6OAZg= =/uI4 -----END PGP SIGNATURE----- --p2kqVDKq5asng8Dg-- From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 08:27:50 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D98781065672 for ; Fri, 25 Nov 2011 08:27:50 +0000 (UTC) (envelope-from Cy.Schubert@komquats.com) Received: from idcmail-mo1so.shaw.ca (idcmail-mo1so.shaw.ca [24.71.223.10]) by mx1.freebsd.org (Postfix) with ESMTP id 9D4928FC0C for ; Fri, 25 Nov 2011 08:27:50 +0000 (UTC) Received: from pd3ml3so-ssvc.prod.shaw.ca ([10.0.141.149]) by pd2mo1so-svcs.prod.shaw.ca with ESMTP; 25 Nov 2011 01:12:48 -0700 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=svA9syYi2VW8WjUw5Fmdp5V1toRmC2VQL6MasMQUDjw= c=1 sm=1 a=FNQ37yD0mI0A:10 a=QrugwKR0C_UA:10 a=wAGQQ9Az6v0A:10 a=BLceEmwcHowA:10 a=ICAaq7hcmGcA:10 a=kj9zAlcOel0A:10 a=2Er20JxOMs3KTlR2XTlUiQ==:17 a=M0VwM5ZyAAAA:8 a=BWvPGDcYAAAA:8 a=6I5d2MoRAAAA:8 a=OT2V39PqUMkCechenPIA:9 a=CjuIK1q_8ugA:10 a=Xclt7nbD0wcA:10 a=V7tsTZBp22UA:10 a=SV7veod9ZcQA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Received: from unknown (HELO spqr.komquats.com) ([24.68.73.211]) by pd3ml3so-dmz.prod.shaw.ca with ESMTP; 25 Nov 2011 01:12:48 -0700 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 4AC1B46B6E; Fri, 25 Nov 2011 00:12:48 -0800 (PST) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.14.5/8.14.5) with ESMTP id pAP8ClDw011348; Fri, 25 Nov 2011 00:12:47 -0800 (PST) (envelope-from Cy.Schubert@komquats.com) Message-Id: <201111250812.pAP8ClDw011348@slippy.cwsent.com> X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.komquats.com/ To: Jason Hellenthal In-Reply-To: Message from Jason Hellenthal of "Fri, 25 Nov 2011 02:02:41 EST." <20111125070241.GA7915@DataIX.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 25 Nov 2011 00:12:47 -0800 Cc: hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Cy Schubert List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 08:27:50 -0000 In message <20111125070241.GA7915@DataIX.net>, Jason Hellenthal writes: > List, > > When using @reboot with cron you expect your proccesses to always start when > the system boots up and only when the system boots. But long after the system > in question had been booted, my @reboot processes ran again! after a (/etc/r > c.d/cron restart). This is normally fine and dandy until one of your @reboot > jobs needs to contain a process that purges files "files that are already in > use by a running daemon since the system has not rebooted" and becomes hazard > ous. > > So with that said... is there a way we could actually make this run @reboot o > nly ? > > Compare the system boottime (kern.boottime) to the current time and if it is > greater than ?5 minutes? do not run on any @reboot's ? or add yet another ext > ension @boottime so it does not throw off current functionality ? > > Surely I could modify the scripts which do this but I find it unproductive an > d counter intuitive for the need to explain that @reboot means "When cron is > restarted" even though the name means something completely opposite. I don't see how cron could run reboot jobs again while running. It calls run_reboot_jobs only during startup. Could it be possible that cron died on your system and you restarted it? -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 08:29:52 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39BB9106564A for ; Fri, 25 Nov 2011 08:29:52 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 0357F8FC1B for ; Fri, 25 Nov 2011 08:29:52 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id DCB16102509; Fri, 25 Nov 2011 09:29:50 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20111125070241.GA7915@DataIX.net> Date: Fri, 25 Nov 2011 09:29:50 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <166BF54D-CEA6-49F5-B8A0-14A647F94766@lassitu.de> References: <20111125070241.GA7915@DataIX.net> To: Jason Hellenthal X-Mailer: Apple Mail (2.1251.1) Cc: hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 08:29:52 -0000 Am 25.11.2011 um 08:02 schrieb Jason Hellenthal: > When using @reboot with cron you expect your proccesses to always = start when the system boots up and only when the system boots. But long = after the system in question had been booted, my @reboot processes ran = again! after a (/etc/rc.d/cron restart). This is normally fine and dandy = until one of your @reboot jobs needs to contain a process that purges = files "files that are already in use by a running daemon since the = system has not rebooted" and becomes hazardous. >=20 > So with that said... is there a way we could actually make this run = @reboot only ? I didn't even know cron had this feature. Why wouldn't you add custom = rc.d scripts for these tasks, or add the commands to rc.local? Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 08:42:26 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 1C3791065675 for ; Fri, 25 Nov 2011 08:42:26 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id B98EC14FF8E; Fri, 25 Nov 2011 08:42:25 +0000 (UTC) Message-ID: <4ECF54F1.50203@FreeBSD.org> Date: Fri, 25 Nov 2011 00:42:25 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Cy Schubert References: <201111250812.pAP8ClDw011348@slippy.cwsent.com> In-Reply-To: <201111250812.pAP8ClDw011348@slippy.cwsent.com> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Jason Hellenthal , hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 08:42:26 -0000 On 11/25/2011 00:12, Cy Schubert wrote: > In message <20111125070241.GA7915@DataIX.net>, Jason Hellenthal writes: >> List, >> >> When using @reboot with cron you expect your proccesses to always start when >> the system boots up and only when the system boots. But long after the system >> in question had been booted, my @reboot processes ran again! after a (/etc/r >> c.d/cron restart). This is normally fine and dandy until one of your @reboot >> jobs needs to contain a process that purges files "files that are already in >> use by a running daemon since the system has not rebooted" and becomes hazard >> ous. >> >> So with that said... is there a way we could actually make this run @reboot o >> nly ? >> >> Compare the system boottime (kern.boottime) to the current time and if it is >> greater than ?5 minutes? do not run on any @reboot's ? or add yet another ext >> ension @boottime so it does not throw off current functionality ? >> >> Surely I could modify the scripts which do this but I find it unproductive an >> d counter intuitive for the need to explain that @reboot means "When cron is >> restarted" even though the name means something completely opposite. > > I don't see how cron could run reboot jobs again while running. It calls > run_reboot_jobs only during startup. Could it be possible that cron died on > your system and you restarted it? Please read the OP again carefully. -- "We could put the whole Internet into a book." "Too practical." Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 08:44:19 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 8913E106566C for ; Fri, 25 Nov 2011 08:44:19 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 45FB614F148; Fri, 25 Nov 2011 08:44:19 +0000 (UTC) Message-ID: <4ECF5562.7060909@FreeBSD.org> Date: Fri, 25 Nov 2011 00:44:18 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Stefan Bethke References: <20111125070241.GA7915@DataIX.net> <166BF54D-CEA6-49F5-B8A0-14A647F94766@lassitu.de> In-Reply-To: <166BF54D-CEA6-49F5-B8A0-14A647F94766@lassitu.de> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Jason Hellenthal , hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 08:44:19 -0000 On 11/25/2011 00:29, Stefan Bethke wrote: > I didn't even know cron had this feature. Why wouldn't you add custom rc.d scripts for these tasks, or add the commands to rc.local? Personally I find this feature very useful for unprivileged users to do their own stuff at startup. Doug -- "We could put the whole Internet into a book." "Too practical." Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 11:06:10 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 213C2106566B for ; Fri, 25 Nov 2011 11:06:10 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id CF17C8FC16 for ; Fri, 25 Nov 2011 11:06:09 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so4459561vbb.13 for ; Fri, 25 Nov 2011 03:06:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=bc/tZNJP3g0gDrhysnSfAu7jp5YeEiPwxda0IqneDzc=; b=pY6oKynltAQ19nbtyC/0X+2tuZCr1GVJc6FZJ2XXY9PxJnEy3/IoT4UaAsVa8uHVlj emFQDFe4iLBQ19oGPQGYEHj4OOsW7hau2o5tLh0Pu6G9XvdlouQe1tHrHAZO4WIcG3pw TCnBv9qWufcMKY9R9ezw3sOfNiYp24Mmzv52Y= MIME-Version: 1.0 Received: by 10.52.68.79 with SMTP id u15mr33350050vdt.5.1322217794576; Fri, 25 Nov 2011 02:43:14 -0800 (PST) Received: by 10.52.182.40 with HTTP; Fri, 25 Nov 2011 02:43:14 -0800 (PST) In-Reply-To: <20111125070241.GA7915@DataIX.net> References: <20111125070241.GA7915@DataIX.net> Date: Fri, 25 Nov 2011 10:43:14 +0000 Message-ID: From: Tom Evans To: Jason Hellenthal Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 11:06:10 -0000 On Fri, Nov 25, 2011 at 7:02 AM, Jason Hellenthal wrote: > > List, > > When using @reboot with cron you expect your proccesses to always start w= hen the system boots up and only when the system boots. But long after the = system in question had been booted, my @reboot processes ran again! after a= (/etc/rc.d/cron restart). This is normally fine and dandy until one of you= r @reboot jobs needs to contain a process that purges files "files that are= already in use by a running daemon since the system has not rebooted" and = becomes hazardous. > > So with that said... is there a way we could actually make this run @rebo= ot only ? > > Compare the system boottime (kern.boottime) to the current time and if it= is greater than ?5 minutes? do not run on any @reboot's ? or add yet anoth= er extension @boottime so it does not throw off current functionality ? > > Surely I could modify the scripts which do this but I find it unproductiv= e and counter intuitive for the need to explain that @reboot means "When cr= on is restarted" even though the name means something completely opposite. > > > Regards & Happy Thanks Giving. > Yep, @reboot is super dangerous if you don't realise it is 'crond reboot' not 'system reboot'. We went through a period where all our production servers crond kept dieing and getting restarted at odd times. The simplest solution is to not run stuff you want to run at boot time through cron, and instead use rc.d, which is designed to do this. Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 10:06:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25E43106566B for ; Fri, 25 Nov 2011 10:06:47 +0000 (UTC) (envelope-from tplou@lbl.gov) Received: from ironport3.lbl.gov (ironport3.lbl.gov [128.3.41.25]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD698FC1B for ; Fri, 25 Nov 2011 10:06:46 +0000 (UTC) X-Ironport-SBRS: 5.2 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvYAANdhz07RVdK0mGdsb2JhbABEFqpWCCIBAQEBAQYLDQcUJYILAiaBVAEFAVeHa5YsglwKnDaDX4NugjJjBIghmgE9hBc X-IronPort-AV: E=Sophos;i="4.69,570,1315206000"; d="scan'208";a="58355478" Received: from mail-iy0-f180.google.com ([209.85.210.180]) by ironport3.lbl.gov with ESMTP; 25 Nov 2011 01:38:09 -0800 Received: by iagz35 with SMTP id z35so6671206iag.39 for ; Fri, 25 Nov 2011 01:38:09 -0800 (PST) Received: by 10.42.137.69 with SMTP id x5mr11565488ict.19.1322213889627; Fri, 25 Nov 2011 01:38:09 -0800 (PST) Received: from kalliste.redcotton.org (kalliste.redcotton.org. [208.106.25.235]) by mx.google.com with ESMTPS id ai7sm55610632igc.0.2011.11.25.01.38.08 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 25 Nov 2011 01:38:08 -0800 (PST) From: Tak Pui Lou Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 25 Nov 2011 01:38:05 -0800 Message-Id: <08E5746B-621E-47D6-AE0E-8D359608284F@LBL.gov> To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Apple Message framework v1084) X-Mailer: Apple Mail (2.1084) X-Mailman-Approved-At: Fri, 25 Nov 2011 14:44:31 +0000 Subject: Porting PathScale's EKOPath Compiler Suite X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 10:06:47 -0000 Hello, I have tested the port from = http://people.freebsd.org/~jkim/path64-devel-20111117.tar.bz2 and = http://people.freebsd.org/~jkim/path64-20111115.tar.xz but the compiler = failed in the following tests: 3/6 Test #3: regression_tests .................***Failed 0.81 sec Start 4: hello_c 4/6 Test #4: hello_c .......................... Passed 0.14 sec Start 5: hello_cpp 5/6 Test #5: hello_cpp ........................ Passed 0.67 sec Start 6: path64_bootstrap_test 6/6 Test #6: path64_bootstrap_test ............***Failed 42.28 sec 67% tests passed, 2 tests failed out of 6 Total Test time (real) =3D 44.74 sec The following tests FAILED: 3 - regression_tests (Failed) 6 - path64_bootstrap_test (Failed) Errors while running CTest Are these known errors for that build? I also tested it on a fortran code. Here is the runtime result: 0.923u /usr/local/path64/bin/pathf95 -O3 -LANG:copyinout=3DON:recursive=3D= ON -OPT:goto=3DON 1.283u gfortran46 -O3 I actually compiled gfortran with CLooG-PPL but the optimization flags = from GRAPHITE does not change the run time of this code. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 16:09:10 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EF73106566B for ; Fri, 25 Nov 2011 16:09:10 +0000 (UTC) (envelope-from Cy.Schubert@komquats.com) Received: from idcmail-mo1so.shaw.ca (idcmail-mo1so.shaw.ca [24.71.223.10]) by mx1.freebsd.org (Postfix) with ESMTP id D5BFA8FC0C for ; Fri, 25 Nov 2011 16:09:09 +0000 (UTC) Received: from pd2ml1so-ssvc.prod.shaw.ca ([10.0.141.139]) by pd2mo1so-svcs.prod.shaw.ca with ESMTP; 25 Nov 2011 09:09:09 -0700 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=n2W5COKzyCYKBlY96L0A3my6r0xTlngINTUh0kppeyk= c=1 sm=1 a=FNQ37yD0mI0A:10 a=QrugwKR0C_UA:10 a=wAGQQ9Az6v0A:10 a=BLceEmwcHowA:10 a=ICAaq7hcmGcA:10 a=kj9zAlcOel0A:10 a=2Er20JxOMs3KTlR2XTlUiQ==:17 a=6I5d2MoRAAAA:8 a=M0VwM5ZyAAAA:8 a=BWvPGDcYAAAA:8 a=Lt116vQMuqQpeSF3j_UA:9 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 a=Xclt7nbD0wcA:10 a=V7tsTZBp22UA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Received: from unknown (HELO spqr.komquats.com) ([24.68.73.211]) by pd2ml1so-dmz.prod.shaw.ca with ESMTP; 25 Nov 2011 09:09:08 -0700 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id BBE4A46B74; Fri, 25 Nov 2011 08:09:07 -0800 (PST) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.14.5/8.14.5) with ESMTP id pAPG97dT008848; Fri, 25 Nov 2011 08:09:07 -0800 (PST) (envelope-from Cy.Schubert@komquats.com) Message-Id: <201111251609.pAPG97dT008848@slippy.cwsent.com> X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.komquats.com/ To: Doug Barton In-Reply-To: Message from Doug Barton of "Fri, 25 Nov 2011 00:42:25 PST." <4ECF54F1.50203@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 25 Nov 2011 08:09:07 -0800 Cc: Jason Hellenthal , hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Cy Schubert List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 16:09:10 -0000 In message <4ECF54F1.50203@FreeBSD.org>, Doug Barton writes: > On 11/25/2011 00:12, Cy Schubert wrote: > > In message <20111125070241.GA7915@DataIX.net>, Jason Hellenthal writes: > >> List, > >> > >> When using @reboot with cron you expect your proccesses to always start wh > en > >> the system boots up and only when the system boots. But long after the sys > tem > >> in question had been booted, my @reboot processes ran again! after a (/et > c/r > >> c.d/cron restart). This is normally fine and dandy until one of your @rebo > ot > >> jobs needs to contain a process that purges files "files that are already > in > >> use by a running daemon since the system has not rebooted" and becomes haz > ard > >> ous. > >> > >> So with that said... is there a way we could actually make this run @reboo > t o > >> nly ? > >> > >> Compare the system boottime (kern.boottime) to the current time and if it > is > >> greater than ?5 minutes? do not run on any @reboot's ? or add yet another > ext > >> ension @boottime so it does not throw off current functionality ? > >> > >> Surely I could modify the scripts which do this but I find it unproductive > an > >> d counter intuitive for the need to explain that @reboot means "When cron > is > >> restarted" even though the name means something completely opposite. > > > > I don't see how cron could run reboot jobs again while running. It calls > > run_reboot_jobs only during startup. Could it be possible that cron died on > > > your system and you restarted it? > > Please read the OP again carefully. You're right. Sorry. It was late, after a long night of O/T. Let's try this again.... Add an option to cron to check lastlog and if within 5 or 10 minutes of the last reboot, then call run_reboot_jobs(). Alternatively, any cron job that needed to run after reboot could do this itself. Changing the behaviour by default would change the semantics of @reboot, altering the behaviour of cron jobs which rely on the brokenness. What if both behaviours are wanted on the same system? Unlikely, as I can't see anyone relying on this broken behaviour. Having said that, I'm sure there are cron jobs that do rely on the broken behaviour, so it may be best to simply deprecate the broken behaviour and make one or the other a command line option. Apologies for not reading the original posting carefully. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 16:37:22 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5510A1065674 for ; Fri, 25 Nov 2011 16:37:22 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1278A8FC14 for ; Fri, 25 Nov 2011 16:37:21 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so2684029vcb.13 for ; Fri, 25 Nov 2011 08:37:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=bk2jkoDXM11RAtpIgLKMfafCAIkMUD/0KRnH3TSq1eE=; b=bq2jmCaFPn7fkFmxnwNTTt4K3oGMcc25avqEGf4ukmj7z5WFJr1U7DoLf5K0nBM3i6 3k1vYTmuqXN9ZZKSCkk5bQzlQJ/+CHA0vhJZ+gq7SJ/ymH545bFqebfkgjXih74ZIeQv iVkdl4a6hMQEELEdAk/Y3GQKdb2JYo2yEiKwU= MIME-Version: 1.0 Received: by 10.52.68.79 with SMTP id u15mr34363635vdt.5.1322239041334; Fri, 25 Nov 2011 08:37:21 -0800 (PST) Received: by 10.52.182.40 with HTTP; Fri, 25 Nov 2011 08:37:21 -0800 (PST) In-Reply-To: <201111251609.pAPG97dT008848@slippy.cwsent.com> References: <4ECF54F1.50203@FreeBSD.org> <201111251609.pAPG97dT008848@slippy.cwsent.com> Date: Fri, 25 Nov 2011 16:37:21 +0000 Message-ID: From: Tom Evans To: Cy Schubert Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 16:37:22 -0000 On Fri, Nov 25, 2011 at 4:09 PM, Cy Schubert wro= te: > Changing the behaviour by default would change the semantics of @reboot, > altering =C2=A0the behaviour of cron jobs which rely on the brokenness. W= hat if > both behaviours are wanted on the same system? Unlikely, as I can't see > anyone relying on this broken behaviour. Having said that, I'm sure there > are cron jobs that do rely on the broken behaviour, so it may be best to > simply deprecate the broken behaviour and make one or the other a command > line option. The problem is that the behaviour is not broken, it works exactly as described in crontab(5) - it is just confusing. It's also slightly nonsensical - the command isn't run at reboot, it is run at boot. How about leaving @reboot exactly as it is, improving the docs so that it is clear what @reboot does, and add a @boot to vixie-cron which would only run at boot time. I think it isn't wise to change how one strange to use format behaves under FreeBSD when vixie-cron is a quite ubiquitous choice amongst nix variants. With regards to anything that runs at boot, I think dumbly checking that the boot time was less than N minutes ago is also sub-optimal. If cron keeps dieing and respawning close to boot time, your cronjob could trigger multiple times. I don't know the exact semantics of kern.boottime - at what point is the boot time stored? If it is before file systems are mounted, an unclean file system triggering a foreground fsck could delay boot time by hours, thereby forcing cron to not think that it is running at boot time when it is finally started. Cheers Tom Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 17:02:36 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F00101065670 for ; Fri, 25 Nov 2011 17:02:36 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A53598FC0A for ; Fri, 25 Nov 2011 17:02:36 +0000 (UTC) Received: by vbbfa15 with SMTP id fa15so4791057vbb.13 for ; Fri, 25 Nov 2011 09:02:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sTonm8G8rL4CdHSItwnr7POAZO91n8s5Vb6i733tJxk=; b=o7gQuLL14HKTG9TDcPdqEW59ea4cPMP3+ICLc0xQXo6uk1Kb9Jc2T2Zf0q2nnIpB0M FswsvQ3U1hmpgQ/uI5an828o8yhLktOlFoVCplKZL9U3pmDuc6KGTrYLwRb9Tsz2c51/ k5Uatv1czeIYJiQCCLAEUhPUET8uvAl7og+aY= MIME-Version: 1.0 Received: by 10.52.35.177 with SMTP id i17mr810194vdj.21.1322239224641; Fri, 25 Nov 2011 08:40:24 -0800 (PST) Received: by 10.220.183.13 with HTTP; Fri, 25 Nov 2011 08:40:24 -0800 (PST) In-Reply-To: References: <4ECF54F1.50203@FreeBSD.org> <201111251609.pAPG97dT008848@slippy.cwsent.com> Date: Fri, 25 Nov 2011 08:40:24 -0800 Message-ID: From: Freddie Cash To: Tom Evans Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 17:02:37 -0000 On Fri, Nov 25, 2011 at 8:37 AM, Tom Evans wrote: > On Fri, Nov 25, 2011 at 4:09 PM, Cy Schubert > wrote: > > Changing the behaviour by default would change the semantics of @reboot, > > altering the behaviour of cron jobs which rely on the brokenness. What > if > > both behaviours are wanted on the same system? Unlikely, as I can't see > > anyone relying on this broken behaviour. Having said that, I'm sure there > > are cron jobs that do rely on the broken behaviour, so it may be best to > > simply deprecate the broken behaviour and make one or the other a command > > line option. > > The problem is that the behaviour is not broken, it works exactly as > described in crontab(5) - it is just confusing. > > It's also slightly nonsensical - the command isn't run at reboot, it > is run at boot. How about leaving @reboot exactly as it is, improving > the docs so that it is clear what @reboot does, and add a @boot to > vixie-cron which would only run at boot time. I think it isn't wise to > change how one strange to use format behaves under FreeBSD when > vixie-cron is a quite ubiquitous choice amongst nix variants. > > With regards to anything that runs at boot, I think dumbly checking > that the boot time was less than N minutes ago is also sub-optimal. If > cron keeps dieing and respawning close to boot time, your cronjob > could trigger multiple times. > > I don't know the exact semantics of kern.boottime - at what point is > the boot time stored? If it is before file systems are mounted, an > unclean file system triggering a foreground fsck could delay boot time > by hours, thereby forcing cron to not think that it is running at boot > time when it is finally started. > Perhaps a call to uptime(1) would be enough? -- Freddie Cash fjwcash@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 17:53:48 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F02DD106566C for ; Fri, 25 Nov 2011 17:53:48 +0000 (UTC) (envelope-from dieterbsd@engineer.com) Received: from mailout-us.gmx.com (mailout-us.gmx.com [74.208.5.67]) by mx1.freebsd.org (Postfix) with SMTP id 9972E8FC16 for ; Fri, 25 Nov 2011 17:53:48 +0000 (UTC) Received: (qmail 8347 invoked by uid 0); 25 Nov 2011 17:53:47 -0000 Received: from 67.206.186.8 by rms-us003.v300.gmx.net with HTTP Content-Type: text/plain; charset="utf-8" Date: Fri, 25 Nov 2011 12:53:43 -0500 From: "Dieter BSD" Message-ID: <20111125175344.233450@gmx.com> MIME-Version: 1.0 To: hackers@freebsd.org X-Authenticated: #74169980 X-Flags: 0001 X-Mailer: GMX.com Web Mailer x-registered: 0 Content-Transfer-Encoding: 8bit X-GMX-UID: Z6OuKlg6zVK0IR3RHjUzbss/Njh6dI4W Cc: Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 17:53:49 -0000 The system doesn't go multiuser until the rc jobs complete, even if you attempt to background them with '&'.  This can be a problem with long running jobs.  I started using cron @reboot for this reason. I haven't run into the problem since I've never needed to run /etc/rc.d/cron restart. > Add an option to cron to check lastlog and if within 5 or 10 minutes > of the last reboot, then call run_reboot_jobs(). Depending on timestamps might be okay as a temporary quick-and-dirty workaround, but there is likely to be a case where it will also do the wrong thing.  What if you need to restart cron within the 5-10 minutes? Maybe something like: rc script sets a flag, cron_reboot script checks and resets flag.  The flag could be a file ("> /tmp/rebooting_system"). Better yet, a run-the-reboot-script command line option could be added to cron. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 18:16:58 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 526021065675 for ; Fri, 25 Nov 2011 18:16:58 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id D98C28FC0C for ; Fri, 25 Nov 2011 18:16:57 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id pAPHrd99080755; Fri, 25 Nov 2011 10:53:39 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id pAPHrdrd080752; Fri, 25 Nov 2011 10:53:39 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Fri, 25 Nov 2011 10:53:39 -0700 (MST) From: Warren Block To: Tom Evans In-Reply-To: Message-ID: References: <4ECF54F1.50203@FreeBSD.org> <201111251609.pAPG97dT008848@slippy.cwsent.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-902635197-1839580335-1322243619=:80691" X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Fri, 25 Nov 2011 10:53:39 -0700 (MST) Cc: hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 18:16:58 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---902635197-1839580335-1322243619=:80691 Content-Type: TEXT/PLAIN; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT On Fri, 25 Nov 2011, Tom Evans wrote: > On Fri, Nov 25, 2011 at 4:09 PM, Cy Schubert wrote: >> Changing the behaviour by default would change the semantics of @reboot, >> altering  the behaviour of cron jobs which rely on the brokenness. What if >> both behaviours are wanted on the same system? Unlikely, as I can't see >> anyone relying on this broken behaviour. Having said that, I'm sure there >> are cron jobs that do rely on the broken behaviour, so it may be best to >> simply deprecate the broken behaviour and make one or the other a command >> line option. > > > The problem is that the behaviour is not broken, it works exactly as > described in crontab(5) - it is just confusing. But crontab(5) just says "startup", when really it means "cron startup", so: http://svnweb.freebsd.org/base?view=revision&revision=227981 > It's also slightly nonsensical - the command isn't run at reboot, it > is run at boot. It isn't just at boot, even. Really it should be called @cronstart. But that ship probably sailed a long time ago. A better alias could be added and @reboot marked as deprecated. (This does not address the technical problem of really only running something at system startup. IMHO, rc scripts are a better fit for that.) ---902635197-1839580335-1322243619=:80691-- From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 20:48:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25C17106564A for ; Fri, 25 Nov 2011 20:48:49 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id DD3468FC16 for ; Fri, 25 Nov 2011 20:48:48 +0000 (UTC) Received: by yenq9 with SMTP id q9so5435608yen.13 for ; Fri, 25 Nov 2011 12:48:48 -0800 (PST) Received: by 10.182.145.38 with SMTP id sr6mr9739690obb.65.1322252620260; Fri, 25 Nov 2011 12:23:40 -0800 (PST) Received: from [192.168.1.42] (ppp-115-87-225-143.revip4.asianet.co.th. [115.87.225.143]) by mx.google.com with ESMTPS id h4sm274016obt.9.2011.11.25.12.23.31 (version=SSLv3 cipher=OTHER); Fri, 25 Nov 2011 12:23:38 -0800 (PST) Message-ID: <4ECFF924.9010403@pathscale.com> Date: Sat, 26 Nov 2011 03:23:00 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Mozilla/5.0 (X11; SunOS i86pc; rv:6.0.2) Gecko/20111003 Thunderbird/6.0.2 MIME-Version: 1.0 To: Tak Pui Lou References: <08E5746B-621E-47D6-AE0E-8D359608284F@LBL.gov> In-Reply-To: <08E5746B-621E-47D6-AE0E-8D359608284F@LBL.gov> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Porting PathScale's EKOPath Compiler Suite X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 20:48:49 -0000 On 11/25/11 04:38 PM, Tak Pui Lou wrote: > Hello, > > I have tested the port from http://people.freebsd.org/~jkim/path64-devel-20111117.tar.bz2 and http://people.freebsd.org/~jkim/path64-20111115.tar.xz but the compiler failed in the following tests: > > 3/6 Test #3: regression_tests .................***Failed 0.81 sec > Start 4: hello_c > 4/6 Test #4: hello_c .......................... Passed 0.14 sec > Start 5: hello_cpp > 5/6 Test #5: hello_cpp ........................ Passed 0.67 sec > Start 6: path64_bootstrap_test > 6/6 Test #6: path64_bootstrap_test ............***Failed 42.28 sec > > 67% tests passed, 2 tests failed out of 6 > > Total Test time (real) = 44.74 sec > > The following tests FAILED: > 3 - regression_tests (Failed) > 6 - path64_bootstrap_test (Failed) > Errors while running CTest > > Are these known errors for that build? Normally I'd bug you about using vanilla upstream, but in this case I think JK's branch is in better shape. (Apologies about not merging it yet, but we have a QA project we'll be testing it with and open sourcing soon - compiler agnostic fwiw) Specifically about your question - It's probably unexpected and I'm curious what processor and version of FBSD this is. > > I also tested it on a fortran code. Here is the runtime result: > > 0.923u /usr/local/path64/bin/pathf95 -O3 -LANG:copyinout=ON:recursive=ON -OPT:goto=ON > 1.283u gfortran46 -O3 > > I actually compiled gfortran with CLooG-PPL but the optimization flags from GRAPHITE does not change the run time of this code. Am I reading the result correctly that we're faster? You may also want to add/test -ipa to your flags.. Side notes : 1) -ipa == LTO in gcc which I don't know if it works at all on FBSD (We have some linker work that may help this situation in the future) 2) I don't care what others say - Graphite isn't afaik production ready so *if* you ever do see any performance gains from it - ensure that you strongly validate before using in production setup 3) We've added the latest User Guide online - http://www.pathscale.com/EKOPath-User-Guide Thanks a lot for testing! ./C From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 22:08:06 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id D502B106564A for ; Fri, 25 Nov 2011 22:08:06 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 8101114F638; Fri, 25 Nov 2011 22:08:06 +0000 (UTC) Message-ID: <4ED011C6.8060605@FreeBSD.org> Date: Fri, 25 Nov 2011 14:08:06 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Cy Schubert References: <201111251609.pAPG97dT008848@slippy.cwsent.com> In-Reply-To: <201111251609.pAPG97dT008848@slippy.cwsent.com> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Jason Hellenthal , hackers@FreeBSD.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 22:08:06 -0000 On 11/25/2011 08:09, Cy Schubert wrote: > You're right. Sorry. It was late, after a long night of O/T. Actually I was in the same boat, which is why my reply was even grumpier than usual, sorry. Meanwhile I like your suggestion of having cron check that it's within $time_period before running the @reboot jobs. I'm not quite so sure that the current behavior needs to be preserved though ... I doubt people purposely restart cron often enough to be anything but surprised by the current behavior. Doug -- "We could put the whole Internet into a book." "Too practical." Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 25 22:05:30 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88F52106564A for ; Fri, 25 Nov 2011 22:05:30 +0000 (UTC) (envelope-from debian@kvr.at) Received: from mail.kvr.at (83-65-151-179.work.xdsl-line.inode.at [83.65.151.179]) by mx1.freebsd.org (Postfix) with ESMTP id 07DED8FC0C for ; Fri, 25 Nov 2011 22:05:29 +0000 (UTC) Received: from chello062178002186.1.11.univie.teleweb.at ([62.178.2.186]:58049 helo=[10.7.77.7]) by mail.kvr.at with esmtpsa (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RU3Rm-0001DB-PM for freebsd-hackers@freebsd.org; Fri, 25 Nov 2011 22:36:38 +0100 Message-ID: <4ED00A68.4040606@kvr.at> Date: Fri, 25 Nov 2011 22:36:40 +0100 From: Christian Kastner User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111120 Icedove/3.1.16 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20111125070241.GA7915@DataIX.net> In-Reply-To: <20111125070241.GA7915@DataIX.net> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 26 Nov 2011 00:40:11 +0000 Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Nov 2011 22:05:30 -0000 Hi, On 2011-11-25 08:02, Jason Hellenthal wrote: > So with that said... is there a way we could actually make this run @reboot only ? Debian's cron[0] and Fedora's cronie[1] have solved this by touching a file on first startup and running @reboot only when this file does not yet exist. Note that while [0] may point to other patches that might be of interest to FreeBSD, they are still WIP (as evident from the linked patch) as we are still in the process of quiltifying our current code base. Regards, Christian [0] http://anonscm.debian.org/gitweb/?p=pkg-cron/pkg-cron.git;a=blob;f=debian/patches/features/run-on-reboot;h=94bab7dcbc4b34e4686385ca3ba3037453f1f4bb;hb=refs/heads/sf3 [1] http://git.fedorahosted.org/git/?p=cronie.git;a=commitdiff;h=2abb46f60f496e2725333a86ade0f3913981761d From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 00:54:13 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78E271065672 for ; Sat, 26 Nov 2011 00:54:13 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 28CA28FC14 for ; Sat, 26 Nov 2011 00:54:12 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id pAQ0GdwP018231; Sat, 26 Nov 2011 00:16:39 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.119] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id ttzvuk32y8nb4euheugwnesd82; Sat, 26 Nov 2011 00:16:39 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=windows-1252 From: Tim Kientzle In-Reply-To: <4ED011C6.8060605@FreeBSD.org> Date: Fri, 25 Nov 2011 16:16:38 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201111251609.pAPG97dT008848@slippy.cwsent.com> <4ED011C6.8060605@FreeBSD.org> To: Doug Barton X-Mailer: Apple Mail (2.1251.1) Cc: Jason Hellenthal , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 00:54:13 -0000 On Nov 25, 2011, at 2:08 PM, Doug Barton wrote: > On 11/25/2011 08:09, Cy Schubert wrote: >> You're right. Sorry. It was late, after a long night of O/T. >=20 > Actually I was in the same boat, which is why my reply was even = grumpier > than usual, sorry. >=20 > Meanwhile I like your suggestion of having cron check that it's within > $time_period before running the @reboot jobs Hmmm=85 I thought rc.d distinguished between boot-time and non-boot-time starts already. It might be simpler and more accurate to add a crond command-line option (--run-reboot-scripts) and have the rc.d scripts only pass that in when crond is started at boot time. Tim From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 03:16:15 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id DA184106564A for ; Sat, 26 Nov 2011 03:16:15 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 279101519CA; Sat, 26 Nov 2011 03:16:15 +0000 (UTC) Message-ID: <4ED059FE.8090502@FreeBSD.org> Date: Fri, 25 Nov 2011 19:16:14 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Tim Kientzle References: <201111251609.pAPG97dT008848@slippy.cwsent.com> <4ED011C6.8060605@FreeBSD.org> In-Reply-To: X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: Jason Hellenthal , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 03:16:15 -0000 On 11/25/2011 16:16, Tim Kientzle wrote: > > On Nov 25, 2011, at 2:08 PM, Doug Barton wrote: > >> On 11/25/2011 08:09, Cy Schubert wrote: >>> You're right. Sorry. It was late, after a long night of O/T. >> >> Actually I was in the same boat, which is why my reply was even grumpier >> than usual, sorry. >> >> Meanwhile I like your suggestion of having cron check that it's within >> $time_period before running the @reboot jobs > > Hmmm… I thought rc.d distinguished between boot-time > and non-boot-time starts already. Well sure, rc.d does, but /usr/sbin/crond doesn't. > It might be simpler and more accurate to add a crond command-line > option (--run-reboot-scripts) and have the rc.d scripts only pass that > in when crond is started at boot time. No, since that wouldn't help if the user started it without rc.d, and more importantly, the current behavior is broken. :) Doug -- "We could put the whole Internet into a book." "Too practical." Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 05:11:39 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6157B1065670 for ; Sat, 26 Nov 2011 05:11:39 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id D98DC8FC08 for ; Sat, 26 Nov 2011 05:11:38 +0000 (UTC) Received: by wwe5 with SMTP id 5so3160096wwe.31 for ; Fri, 25 Nov 2011 21:11:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=zmcNmTFVd2myF/ztmm1eXLDMqt1uIaBYmTalgmPAlmk=; b=ihTMigU6jrhLr/aUgAP+ZZBzrUMNX6uXjEpknX5JrBGVYhpLPfZfIuNJJ2PfqLM88W GxiTd0vrW5nicv9Ey6gu24fIf2+KB+6kr/iGm1LxjnMT829A8S5VSWiIJdVOj4b/yxtx L5sGpBMmeL8RSAivz/kGH2NG45I0VYwJCiSdk= Received: by 10.227.198.7 with SMTP id em7mr23416423wbb.13.1322284297780; Fri, 25 Nov 2011 21:11:37 -0800 (PST) Received: from DataIX.net (ppp-21.55.dialinfree.com. [209.172.21.55]) by mx.google.com with ESMTPS id z35sm3237110wbm.12.2011.11.25.21.11.33 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 25 Nov 2011 21:11:36 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAQ5BQmu010411 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Nov 2011 00:11:26 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAQ5BLIF010410; Sat, 26 Nov 2011 00:11:21 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 26 Nov 2011 00:11:21 -0500 From: Jason Hellenthal To: Christian Kastner Message-ID: <20111126051121.GA4628@DataIX.net> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LQksG6bCIzRHxTLp" Content-Disposition: inline In-Reply-To: <4ED00A68.4040606@kvr.at> Cc: freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 05:11:39 -0000 --LQksG6bCIzRHxTLp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 25, 2011 at 10:36:40PM +0100, Christian Kastner wrote: > Hi, >=20 > On 2011-11-25 08:02, Jason Hellenthal wrote: > > So with that said... is there a way we could actually make this run @re= boot only ? >=20 > Debian's cron[0] and Fedora's cronie[1] have solved this by touching a > file on first startup and running @reboot only when this file does not > yet exist. >=20 > Note that while [0] may point to other patches that might be of interest > to FreeBSD, they are still WIP (as evident from the linked patch) as we > are still in the process of quiltifying our current code base. >=20 While this sounds like a perfectly sane way to handle the problem at hand t= his also introduces a need to write some cleanup code to take care of the f= ile semantics. I think comparing daemon start times to the time a system wa= s booted or similiar would alleviate the need for all that. Maybe a flag fo= r @reboot "-s " seconds after boottime to allow running @reboot jo= bs. And set the default to 3600 seconds. At least this would allow adjustme= nt for those startup processes that may take some considerable time before = multiuser mode is entered.=20 Just some thought. I really don't think we need to go the route of using files to store inform= ation when there is enough information available already via syscall's. >=20 > [0] > http://anonscm.debian.org/gitweb/?p=3Dpkg-cron/pkg-cron.git;a=3Dblob;f=3D= debian/patches/features/run-on-reboot;h=3D94bab7dcbc4b34e4686385ca3ba303745= 3f1f4bb;hb=3Drefs/heads/sf3 >=20 > [1] > http://git.fedorahosted.org/git/?p=3Dcronie.git;a=3Dcommitdiff;h=3D2abb46= f60f496e2725333a86ade0f3913981761d >=20 --LQksG6bCIzRHxTLp Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJO0HT4AAoJEJBXh4mJ2FR+mooH/iN1y855ce0wwIL3UnJOOYHz U/N7gkPdFbuMJG2BARdApZpwuZ4HNNGnwE0S/aprJKzheHPM5/9vhLUDV8jzGxf9 RcCHvPes4viheH2XV3/ahrhqCodpSFKN7Gb/YwGM0fIcgvUf2Atv/P20Ur+/x2GE ZtdR+IO9a0GgjorKBclvD8j3JnXsZ7mY45dJ/xI5zYDH7k7OMGJhovCkfTOjWkhX zLYxwc4GKKBFkSNGTpzObENdYCHqKh1Bk/Vu7azrzmBX3qizPrKLVOg2eGt0ddm+ dMX+4t1iepd041zS5Q5k+y2uMDJrGfyiKDlYY6s+bOcLyLyNxGntWRiz2b5uYxc= =v2fY -----END PGP SIGNATURE----- --LQksG6bCIzRHxTLp-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 07:09:02 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04BB9106566B for ; Sat, 26 Nov 2011 07:09:02 +0000 (UTC) (envelope-from Cy.Schubert@komquats.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id BD4268FC0C for ; Sat, 26 Nov 2011 07:09:01 +0000 (UTC) Received: from lb7f8hsrpno-svcs.dcs.int.inet (HELO pd5ml3no-ssvc.prod.shaw.ca) ([10.0.144.222]) by pd6mo1no-svcs.prod.shaw.ca with ESMTP; 26 Nov 2011 00:09:01 -0700 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=K6fnQZae8TPX1i0cofjQtTsb/A4CHt4xfMPVU6P219U= c=1 sm=1 a=FNQ37yD0mI0A:10 a=QrugwKR0C_UA:10 a=wAGQQ9Az6v0A:10 a=BLceEmwcHowA:10 a=ICAaq7hcmGcA:10 a=8nJEP1OIZ-IA:10 a=2Er20JxOMs3KTlR2XTlUiQ==:17 a=s1O25tkdAAAA:8 a=BWvPGDcYAAAA:8 a=6I5d2MoRAAAA:8 a=p6W2Vv0txPRS-G8_77cA:9 a=3OlXMcyiwNwU2BbtYn0A:7 a=wPNLvfGTeEIA:10 a=OyOq_G8mXAEA:10 a=V7tsTZBp22UA:10 a=SV7veod9ZcQA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Received: from unknown (HELO spqr.komquats.com) ([24.68.73.211]) by pd5ml3no-dmz.prod.shaw.ca with ESMTP; 26 Nov 2011 00:09:00 -0700 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 0C2CB46B6B; Fri, 25 Nov 2011 23:08:59 -0800 (PST) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.14.5/8.14.5) with ESMTP id pAQ78wvO045883; Fri, 25 Nov 2011 23:08:59 -0800 (PST) (envelope-from Cy.Schubert@komquats.com) Message-Id: <201111260708.pAQ78wvO045883@slippy.cwsent.com> X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.komquats.com/ To: Warren Block In-Reply-To: Message from Warren Block of "Fri, 25 Nov 2011 10:53:39 MST." Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Fri, 25 Nov 2011 23:08:58 -0800 Cc: Tom Evans , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Cy Schubert List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 07:09:02 -0000 In message , Warren Bl= ock=20 writ es: > This message is in MIME format. The first part should be readable te= xt, > while the remaining parts are likely unreadable without MIME-aware to= ols. >=20 > ---902635197-1839580335-1322243619=3D:80691 > Content-Type: TEXT/PLAIN; charset=3DUTF-8; format=3Dflowed > Content-Transfer-Encoding: 8BIT >=20 > On Fri, 25 Nov 2011, Tom Evans wrote: >=20 > > On Fri, Nov 25, 2011 at 4:09 PM, Cy Schubert wro > te: > >> Changing the behaviour by default would change the semantics of =40r= eboot, > >> altering =C2=A0the behaviour of cron jobs which rely on the brokenne= ss. What if > >> both behaviours are wanted on the same system? Unlikely, as I can't = see > >> anyone relying on this broken behaviour. Having said that, I'm sure = there > >> are cron jobs that do rely on the broken behaviour, so it may be bes= t to > >> simply deprecate the broken behaviour and make one or the other a co= mmand > >> line option. > > > > > > The problem is that the behaviour is not broken, it works exactly as > > described in crontab(5) - it is just confusing. >=20 > But crontab(5) just says =22startup=22, when really it means =22cron st= artup=22,=20 > so: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D227981 >=20 > > It's also slightly nonsensical - the command isn't run at reboot, it > > is run at boot. >=20 > It isn't just at boot, even. Really it should be called =40cronstart. > But that ship probably sailed a long time ago. A better alias could be= =20 > added and =40reboot marked as deprecated. (This does not address the=20 > technical problem of really only running something at system startup.=20 > IMHO, rc scripts are a better fit for that.) Agreed, that's what rc scripts are for. OTOH, a non-root user can't create rc scripts. Having said that, any=20 non-root rc scripts I've ever run always required negotiation, e.g. oracl= e=20 and other apps. You want to start other apps in a specified order. When running non-root rc scripts, it's a simple matter of, /bin/su - oracle -c '/home/oracle/product/oracle-8i/bin/startup.sh' If average users really do need to run something at boot they're likely=20 running some kind of service, e.g. some kind of DBMS, on the machine and = that would usually if not always require some kind of cooperation with th= e=20 sysadmin. --=20 Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 07:25:11 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 02B22106566C for ; Sat, 26 Nov 2011 07:25:11 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 4C27D150EB3; Sat, 26 Nov 2011 07:25:03 +0000 (UTC) Message-ID: <4ED0944E.7020709@FreeBSD.org> Date: Fri, 25 Nov 2011 23:25:02 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Cy Schubert References: <201111260708.pAQ78wvO045883@slippy.cwsent.com> In-Reply-To: <201111260708.pAQ78wvO045883@slippy.cwsent.com> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Tom Evans , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 07:25:11 -0000 On 11/25/2011 23:08, Cy Schubert wrote: > If average users really do need to run something at boot they're likely > running some kind of service I don't think second-guessing what users are doing is going to be a useful exercise here. I will also tell you flat out that this is not the only use for an @reboot cron job. Rather than debating whether users *should* be doing it this way or not, can we please focus on fixing it to be non-stupid? -- "We could put the whole Internet into a book." "Too practical." Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 10:49:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D335D10656D0 for ; Sat, 26 Nov 2011 10:49:25 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9FD418FC16 for ; Sat, 26 Nov 2011 10:49:25 +0000 (UTC) Received: by iakl21 with SMTP id l21so8952054iak.13 for ; Sat, 26 Nov 2011 02:49:25 -0800 (PST) Received: by 10.42.151.68 with SMTP id d4mr15842922icw.36.1322304564715; Sat, 26 Nov 2011 02:49:24 -0800 (PST) Received: from [192.168.1.43] (ppp-58-11-68-49.revip2.asianet.co.th. [58.11.68.49]) by mx.google.com with ESMTPS id ew6sm64751450igc.4.2011.11.26.02.49.21 (version=SSLv3 cipher=OTHER); Sat, 26 Nov 2011 02:49:23 -0800 (PST) Message-ID: <4ED0C418.5000307@pathscale.com> Date: Sat, 26 Nov 2011 17:48:56 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Mozilla/5.0 (X11; SunOS i86pc; rv:6.0.2) Gecko/20111003 Thunderbird/6.0.2 MIME-Version: 1.0 To: Tak Pui Lou References: <08E5746B-621E-47D6-AE0E-8D359608284F@LBL.gov> <4ECFF924.9010403@pathscale.com> <5C4A8661-DFF0-4F4A-9E0E-E33083FB1B2D@lbl.gov> In-Reply-To: <5C4A8661-DFF0-4F4A-9E0E-E33083FB1B2D@lbl.gov> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: Porting PathScale's EKOPath Compiler Suite X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 10:49:26 -0000 On 11/26/11 02:26 PM, Tak Pui Lou wrote: > On Nov 25, 2011, at 12:23 PM, C. Bergström wrote: > >> On 11/25/11 04:38 PM, Tak Pui Lou wrote: >>> Hello, >>> >>> I have tested the port from http://people.freebsd.org/~jkim/path64-devel-20111117.tar.bz2 and http://people.freebsd.org/~jkim/path64-20111115.tar.xz but the compiler failed in the following tests: >>> >>> 3/6 Test #3: regression_tests .................***Failed 0.81 sec >>> Start 4: hello_c >>> 4/6 Test #4: hello_c .......................... Passed 0.14 sec >>> Start 5: hello_cpp >>> 5/6 Test #5: hello_cpp ........................ Passed 0.67 sec >>> Start 6: path64_bootstrap_test >>> 6/6 Test #6: path64_bootstrap_test ............***Failed 42.28 sec >>> >>> 67% tests passed, 2 tests failed out of 6 >>> >>> Total Test time (real) = 44.74 sec >>> >>> The following tests FAILED: >>> 3 - regression_tests (Failed) >>> 6 - path64_bootstrap_test (Failed) >>> Errors while running CTest >>> >>> Are these known errors for that build? >> Normally I'd bug you about using vanilla upstream, but in this case I think JK's branch is in better shape. (Apologies about not merging it yet, but we have a QA project we'll be testing it with and open sourcing soon - compiler agnostic fwiw) >> > I did search on the Internet to check if the upstream has got the patches merged or not. But, I did not find too much information about this. So, I tried JK's branch instead. When you feel that I should try the source on github, please let me know. >> Specifically about your question - It's probably unexpected and I'm curious what processor and version of FBSD this is. > The kernel is compiled from 9.0-RC2 (releng/9.0 r227910) with gcc 4.2 that comes with the OS. I cannot give you the 'uname -a' output now because I have just compiled and installed a kernel with clang but I remembered it was updated two days ago before I upgraded from stable/8 to releng/9.0. The CPU is an AMD Athlon II 270u x2 running at 2 GHz. >>> I also tested it on a fortran code. Here is the runtime result: >>> >>> 0.923u /usr/local/path64/bin/pathf95 -O3 -LANG:copyinout=ON:recursive=ON -OPT:goto=ON >>> 1.283u gfortran46 -O3 >>> >>> I actually compiled gfortran with CLooG-PPL but the optimization flags from GRAPHITE does not change the run time of this code. >> Am I reading the result correctly that we're faster? You may also want to add/test -ipa to your flags.. >> > Yes, this code compiled from pathf95 runs faster than that compiled from gfortran46. It may be more interesting to mention that I also have OpenIndiana 151a installed on the same computer and tested the code with Solaris Studio 12.2. The runtime for the same code compiled with Solaris Studio 12.2 is ~1.0xx u. On OpenIndiana, I have only tested the optimization flags that do not require SUNWprivate_1.5 version of libmtsk.so. All results are checked in those run. > > I will try -ipa later and let you know if it makes any difference in runtime. (I think I have already tried that but let me do this again.) I'll get you setup with EKOPath/Path64 on OI as well so you can check that our performance is consistent across OS and your test environments From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 11:25:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8229106564A for ; Sat, 26 Nov 2011 11:25:57 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id A36578FC0C for ; Sat, 26 Nov 2011 11:25:57 +0000 (UTC) Received: by iakl21 with SMTP id l21so9005605iak.13 for ; Sat, 26 Nov 2011 03:25:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=2P3qLysTSr5KBar2TohbpS+lkcaJmNGzPk64KhNByk8=; b=rLMYbrtdwN5zXRynzjwRPb+3Cs8M/N42jg67pFqPtL02aWqqlOmKXYUQgfXVPne/pL J7IUKcayt5yV9Yc+xQ2VtxGPY6Z2+wAgWiD5IOUuJfOz4VWNolAO3YUKk2k5N0JJkuSY 6hGNqjJDZkGprC7UaMyqeepDgEY8VXXnheXdM= Received: by 10.42.29.1 with SMTP id p1mr11421865icc.40.1322304901379; Sat, 26 Nov 2011 02:55:01 -0800 (PST) MIME-Version: 1.0 Sender: utisoft@gmail.com Received: by 10.231.12.139 with HTTP; Sat, 26 Nov 2011 02:54:29 -0800 (PST) In-Reply-To: <4ED0944E.7020709@FreeBSD.org> References: <201111260708.pAQ78wvO045883@slippy.cwsent.com> <4ED0944E.7020709@FreeBSD.org> From: Chris Rees Date: Sat, 26 Nov 2011 10:54:29 +0000 X-Google-Sender-Auth: IjGyN7GWQidGYCKvQ16CUgOF9Gg Message-ID: To: Doug Barton Content-Type: text/plain; charset=ISO-8859-1 Cc: Tom Evans , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 11:25:57 -0000 On 26 November 2011 07:25, Doug Barton wrote: > On 11/25/2011 23:08, Cy Schubert wrote: >> If average users really do need to run something at boot they're likely >> running some kind of service > > I don't think second-guessing what users are doing is going to be a > useful exercise here. I will also tell you flat out that this is not the > only use for an @reboot cron job. > > Rather than debating whether users *should* be doing it this way or not, > can we please focus on fixing it to be non-stupid? +1 I find the idea of using time since boot to decide whether this is cron's first startup repellent in the extreme. Whatever solution is decided (and I'm thinking a variable passed to rc.d/cron or a state file in /var/run would both work fine), PLEASE let's not go down the route of that. How long? What if I need to restart cron shortly after boot? What if boot takes longer? Plenty of competent coders in here, but please consider me interested in helping with a solution if needed. Chris From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 06:01:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 840881065675 for ; Sat, 26 Nov 2011 06:01:03 +0000 (UTC) (envelope-from gmx@ross.cx) Received: from www81.your-server.de (www81.your-server.de [213.133.104.81]) by mx1.freebsd.org (Postfix) with ESMTP id 433118FC14 for ; Sat, 26 Nov 2011 06:01:03 +0000 (UTC) Received: from [188.108.235.211] (helo=michael-think) by www81.your-server.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1RUB3E-0008BC-JB; Sat, 26 Nov 2011 06:43:48 +0100 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: "Christian Kastner" , "Jason Hellenthal" References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> <20111126051121.GA4628@DataIX.net> Date: Sat, 26 Nov 2011 06:43:38 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Michael Ross" Message-ID: In-Reply-To: <20111126051121.GA4628@DataIX.net> User-Agent: Opera Mail/11.52 (Win32) X-Authenticated-Sender: gmx@ross.cx X-Virus-Scanned: Clear (ClamAV 0.97.3/13991/Sat Nov 26 00:07:28 2011) X-Mailman-Approved-At: Sat, 26 Nov 2011 12:24:11 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 06:01:03 -0000 Am 26.11.2011, 06:11 Uhr, schrieb Jason Hellenthal : > > > On Fri, Nov 25, 2011 at 10:36:40PM +0100, Christian Kastner wrote: >> Hi, >> >> On 2011-11-25 08:02, Jason Hellenthal wrote: >> > So with that said... is there a way we could actually make this run >> @reboot only ? >> >> Debian's cron[0] and Fedora's cronie[1] have solved this by touching a >> file on first startup and running @reboot only when this file does not >> yet exist. >> >> Note that while [0] may point to other patches that might be of interest >> to FreeBSD, they are still WIP (as evident from the linked patch) as we >> are still in the process of quiltifying our current code base. >> > > While this sounds like a perfectly sane way to handle the problem at > hand this also introduces a need to write some cleanup code to take care > of the file semantics. I think comparing daemon start times to the time > a system was booted or similiar would alleviate the need for all that. > Maybe a flag for @reboot "-s " seconds after boottime to allow > running @reboot jobs. And set the default to 3600 seconds. At least this > would allow adjustment for those startup processes that may take some > considerable time before multiuser mode is entered. > > Just some thought. > > I really don't think we need to go the route of using files to store > information when there is enough information available already via > syscall's. If system startup were to be unusually delayed (dhcp or nfs trouble eg), $time_period might have passed when cron starts, so there would have to be some notifying mechanism for @reboot jobs not being run, and operator action would be required. One could "cron restart" within $time_period and run @reboot twice. I don't like this idea. In my bikeshed, @reboot should be called @cronstart, but it shouldn't because of uniformity across OSs. The cron manpage should reflect the actual behaviour (already patched in svn), and all "definitely only on boot"-scripts should go in local/etc/rc.d. Michael From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 10:43:03 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5194106566C for ; Sat, 26 Nov 2011 10:43:03 +0000 (UTC) (envelope-from tplou@lbl.gov) Received: from ironport3.lbl.gov (ironport3.lbl.gov [128.3.41.25]) by mx1.freebsd.org (Postfix) with ESMTP id 95B378FC0C for ; Sat, 26 Nov 2011 10:43:03 +0000 (UTC) X-Ironport-SBRS: 5.2 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AsgBAIrC0E7RVdK0mGdsb2JhbABEFhCqUggiAQEBAQEICQ0HFCWBcgEBAQMBEgJYDQULCxguNAEFARwGNYdjCJk2CpxMg1+GIGMEiCGaAT2EFw X-IronPort-AV: E=Sophos;i="4.69,575,1315206000"; d="scan'208";a="58407342" Received: from mail-iy0-f180.google.com ([209.85.210.180]) by ironport3.lbl.gov with ESMTP; 26 Nov 2011 02:42:52 -0800 Received: by iagz35 with SMTP id z35so9093900iag.39 for ; Sat, 26 Nov 2011 02:42:52 -0800 (PST) Received: by 10.43.44.199 with SMTP id uh7mr15642362icb.25.1322304171170; Sat, 26 Nov 2011 02:42:51 -0800 (PST) Received: from kalliste.redcotton.org (kalliste.redcotton.org. [208.106.25.235]) by mx.google.com with ESMTPS id ft1sm64704724igc.3.2011.11.26.02.42.49 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Nov 2011 02:42:50 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=iso-8859-1 From: Tak Pui Lou In-Reply-To: <4ECFF924.9010403@pathscale.com> Date: Fri, 25 Nov 2011 23:26:30 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <5C4A8661-DFF0-4F4A-9E0E-E33083FB1B2D@lbl.gov> References: <08E5746B-621E-47D6-AE0E-8D359608284F@LBL.gov> <4ECFF924.9010403@pathscale.com> To: =?iso-8859-1?Q?=22C=2E_Bergstr=F6m=22?= X-Mailer: Apple Mail (2.1084) X-Mailman-Approved-At: Sat, 26 Nov 2011 12:24:27 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Porting PathScale's EKOPath Compiler Suite X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 10:43:03 -0000 On Nov 25, 2011, at 12:23 PM, C. Bergstr=F6m wrote: > On 11/25/11 04:38 PM, Tak Pui Lou wrote: >> Hello, >>=20 >> I have tested the port from = http://people.freebsd.org/~jkim/path64-devel-20111117.tar.bz2 and = http://people.freebsd.org/~jkim/path64-20111115.tar.xz but the compiler = failed in the following tests: >>=20 >> 3/6 Test #3: regression_tests .................***Failed 0.81 sec >> Start 4: hello_c >> 4/6 Test #4: hello_c .......................... Passed 0.14 sec >> Start 5: hello_cpp >> 5/6 Test #5: hello_cpp ........................ Passed 0.67 sec >> Start 6: path64_bootstrap_test >> 6/6 Test #6: path64_bootstrap_test ............***Failed 42.28 sec >>=20 >> 67% tests passed, 2 tests failed out of 6 >>=20 >> Total Test time (real) =3D 44.74 sec >>=20 >> The following tests FAILED: >> 3 - regression_tests (Failed) >> 6 - path64_bootstrap_test (Failed) >> Errors while running CTest >>=20 >> Are these known errors for that build? > Normally I'd bug you about using vanilla upstream, but in this case I = think JK's branch is in better shape. (Apologies about not merging it = yet, but we have a QA project we'll be testing it with and open sourcing = soon - compiler agnostic fwiw) >=20 I did search on the Internet to check if the upstream has got the = patches merged or not. But, I did not find too much information about = this. So, I tried JK's branch instead. When you feel that I should try = the source on github, please let me know. > Specifically about your question - It's probably unexpected and I'm = curious what processor and version of FBSD this is. The kernel is compiled from 9.0-RC2 (releng/9.0 r227910) with gcc 4.2 = that comes with the OS. I cannot give you the 'uname -a' output now = because I have just compiled and installed a kernel with clang but I = remembered it was updated two days ago before I upgraded from stable/8 = to releng/9.0. The CPU is an AMD Athlon II 270u x2 running at 2 GHz. >>=20 >> I also tested it on a fortran code. Here is the runtime result: >>=20 >> 0.923u /usr/local/path64/bin/pathf95 -O3 = -LANG:copyinout=3DON:recursive=3DON -OPT:goto=3DON >> 1.283u gfortran46 -O3 >>=20 >> I actually compiled gfortran with CLooG-PPL but the optimization = flags from GRAPHITE does not change the run time of this code. > Am I reading the result correctly that we're faster? You may also = want to add/test -ipa to your flags.. >=20 Yes, this code compiled from pathf95 runs faster than that compiled from = gfortran46. It may be more interesting to mention that I also have = OpenIndiana 151a installed on the same computer and tested the code with = Solaris Studio 12.2. The runtime for the same code compiled with Solaris = Studio 12.2 is ~1.0xx u. On OpenIndiana, I have only tested the = optimization flags that do not require SUNWprivate_1.5 version of = libmtsk.so. All results are checked in those run. I will try -ipa later and let you know if it makes any difference in = runtime. (I think I have already tried that but let me do this again.) > Side notes : > 1) -ipa =3D=3D LTO in gcc which I don't know if it works at all on = FBSD (We have some linker work that may help this situation in the = future) > 2) I don't care what others say - Graphite isn't afaik production = ready so *if* you ever do see any performance gains from it - ensure = that you strongly validate before using in production setup > 3) We've added the latest User Guide online - = http://www.pathscale.com/EKOPath-User-Guide >=20 > Thanks a lot for testing! >=20 > ./C Thank you for making PathScale Compilers open source! ---L= From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 15:44:22 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AEB4E106564A; Sat, 26 Nov 2011 15:44:22 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 750838FC12; Sat, 26 Nov 2011 15:44:22 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 5A2F0111B36; Sat, 26 Nov 2011 16:44:21 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Stefan Bethke In-Reply-To: Date: Sat, 26 Nov 2011 16:44:20 +0100 Content-Transfer-Encoding: 7bit Message-Id: <9B3B0E26-F63C-4422-A434-FC9D737BE51A@lassitu.de> References: <201111260708.pAQ78wvO045883@slippy.cwsent.com> <4ED0944E.7020709@FreeBSD.org> To: Chris Rees X-Mailer: Apple Mail (2.1251.1) Cc: Tom Evans , Doug Barton , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 15:44:22 -0000 Am 26.11.2011 um 11:54 schrieb Chris Rees: > PLEASE let's not go down the route of that. How long? What if I need to > restart cron shortly after boot? What if boot takes longer? Plus "interesting" time changes during boot due to ntpdate/ntpd. -- Stefan Bethke Fon +49 151 14070811 From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 19:00:48 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42912106566C for ; Sat, 26 Nov 2011 19:00:48 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id C50DA8FC08 for ; Sat, 26 Nov 2011 19:00:47 +0000 (UTC) Received: by wwe5 with SMTP id 5so4133204wwe.31 for ; Sat, 26 Nov 2011 11:00:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=VdhtrSpBu0KimZBzczrZhJr83/xsHltRoDEM7k5lwH0=; b=jf1tPWrHJnlRJz1UKLnDDwRICO3Ti52NjskTpkwOZlASZrMWhbCN4iEVm/k7CY+vH5 NR2dtA9i9jHAOVucH7E0n29QYLoOXlLvlN34lNSFaJVjyWKSNGSKYt7saWOuu7T4nQuG NbpusD9fllQmIyHUNR/sYyYSaFYCTmJi+656g= Received: by 10.227.206.129 with SMTP id fu1mr24676130wbb.22.1322334046895; Sat, 26 Nov 2011 11:00:46 -0800 (PST) Received: from DataIX.net (ppp-21.81.dialinfree.com. [209.172.21.81]) by mx.google.com with ESMTPS id 6sm19532779wby.22.2011.11.26.11.00.42 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Nov 2011 11:00:45 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAQJ0ZPn058601 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Nov 2011 14:00:36 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAQJ0U7B058600; Sat, 26 Nov 2011 14:00:30 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 26 Nov 2011 14:00:30 -0500 From: Jason Hellenthal To: Michael Ross Message-ID: <20111126190030.GA58253@DataIX.net> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> <20111126051121.GA4628@DataIX.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: Christian Kastner , freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 19:00:48 -0000 On Sat, Nov 26, 2011 at 06:43:38AM +0100, Michael Ross wrote: > Am 26.11.2011, 06:11 Uhr, schrieb Jason Hellenthal : > > > > > > > On Fri, Nov 25, 2011 at 10:36:40PM +0100, Christian Kastner wrote: > >> Hi, > >> > >> On 2011-11-25 08:02, Jason Hellenthal wrote: > >> > So with that said... is there a way we could actually make this run > >> @reboot only ? > >> > >> Debian's cron[0] and Fedora's cronie[1] have solved this by touching a > >> file on first startup and running @reboot only when this file does not > >> yet exist. > >> > >> Note that while [0] may point to other patches that might be of interest > >> to FreeBSD, they are still WIP (as evident from the linked patch) as we > >> are still in the process of quiltifying our current code base. > >> > > > > While this sounds like a perfectly sane way to handle the problem at > > hand this also introduces a need to write some cleanup code to take care > > of the file semantics. I think comparing daemon start times to the time > > a system was booted or similiar would alleviate the need for all that. > > Maybe a flag for @reboot "-s " seconds after boottime to allow > > running @reboot jobs. And set the default to 3600 seconds. At least this > > would allow adjustment for those startup processes that may take some > > considerable time before multiuser mode is entered. > > > > Just some thought. > > > > I really don't think we need to go the route of using files to store > > information when there is enough information available already via > > syscall's. > > If system startup were to be unusually delayed (dhcp or nfs trouble eg), > $time_period might have passed when cron starts, so there would have to be > some notifying mechanism for @reboot jobs not being run, and operator > action would be required. > I agree but also disagree. 1 hour or 3600 seconds is plenty of time to wait for the "@reboot" extension scripts to fire. > One could "cron restart" within $time_period and run @reboot twice. > > I don't like this idea. > > In my bikeshed, @reboot should be called @cronstart, but it shouldn't > because of uniformity across OSs. > The cron manpage should reflect the actual behaviour (already patched in > svn), > and all "definitely only on boot"-scripts should go in local/etc/rc.d. > This has nothing to do with rc.d. This is a cron(8) problem only. From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 19:31:30 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1588E1065672 for ; Sat, 26 Nov 2011 19:31:30 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D469B8FC13 for ; Sat, 26 Nov 2011 19:31:29 +0000 (UTC) Received: by iakl21 with SMTP id l21so9698735iak.13 for ; Sat, 26 Nov 2011 11:31:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=/mOJEm2nkjrj8fIlOytnTP9IP2F4PKoHufwCW2TVvWI=; b=bl5xQV3PgOkuOxNP/XMF1RQUfhFuArgUlvVpjBlz/GmG/E86IJ9TCDy18S83Bhugch PFwU1H3i+Uw4p23YIex1nD47hVLWqT+cqxW+VbKVfXOto6wHo44K+my+a91prYDHx/NG jQ8/eLtv/3n3jocEbgNsy/KiLJB9SSa1rg4HI= Received: by 10.231.82.131 with SMTP id b3mr1343526ibl.74.1322334331086; Sat, 26 Nov 2011 11:05:31 -0800 (PST) MIME-Version: 1.0 Sender: utisoft@gmail.com Received: by 10.231.12.139 with HTTP; Sat, 26 Nov 2011 11:05:00 -0800 (PST) In-Reply-To: <20111126190030.GA58253@DataIX.net> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> <20111126051121.GA4628@DataIX.net> <20111126190030.GA58253@DataIX.net> From: Chris Rees Date: Sat, 26 Nov 2011 19:05:00 +0000 X-Google-Sender-Auth: 5tg7F9vghnN7DI-WFCQDdn9G4sk Message-ID: To: Jason Hellenthal Content-Type: text/plain; charset=ISO-8859-1 Cc: Christian Kastner , Michael Ross , freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 19:31:30 -0000 On 26 November 2011 19:00, Jason Hellenthal wrote: > On Sat, Nov 26, 2011 at 06:43:38AM +0100, Michael Ross wrote: >> Am 26.11.2011, 06:11 Uhr, schrieb Jason Hellenthal : >> > On Fri, Nov 25, 2011 at 10:36:40PM +0100, Christian Kastner wrote: >> >> Hi, >> >> >> >> On 2011-11-25 08:02, Jason Hellenthal wrote: >> >> > So with that said... is there a way we could actually make this run >> >> @reboot only ? >> >> >> >> Debian's cron[0] and Fedora's cronie[1] have solved this by touching a >> >> file on first startup and running @reboot only when this file does not >> >> yet exist. >> >> >> >> Note that while [0] may point to other patches that might be of interest >> >> to FreeBSD, they are still WIP (as evident from the linked patch) as we >> >> are still in the process of quiltifying our current code base. >> >> >> > >> > While this sounds like a perfectly sane way to handle the problem at >> > hand this also introduces a need to write some cleanup code to take care >> > of the file semantics. I think comparing daemon start times to the time >> > a system was booted or similiar would alleviate the need for all that. >> > Maybe a flag for @reboot "-s " seconds after boottime to allow >> > running @reboot jobs. And set the default to 3600 seconds. At least this >> > would allow adjustment for those startup processes that may take some >> > considerable time before multiuser mode is entered. >> > >> > Just some thought. >> > >> > I really don't think we need to go the route of using files to store >> > information when there is enough information available already via >> > syscall's. >> >> If system startup were to be unusually delayed (dhcp or nfs trouble eg), >> $time_period might have passed when cron starts, so there would have to be >> some notifying mechanism for @reboot jobs not being run, and operator >> action would be required. >> > > I agree but also disagree. 1 hour or 3600 seconds is plenty of time to wait for the "@reboot" extension scripts to fire. Yes, and if I restart cron 30 minutes after boot, I'm screwed. Chris From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 19:36:23 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BDA1106564A; Sat, 26 Nov 2011 19:36:23 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3154A8FC14; Sat, 26 Nov 2011 19:36:22 +0000 (UTC) Received: by wwe5 with SMTP id 5so4170508wwe.31 for ; Sat, 26 Nov 2011 11:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=c6Fd92uzgXF0VXp1Ibc9MIyFjfIF2y2Pt6OhswvdiKM=; b=ozbHHX4prKYHLR3r5lEcCPgwY70Consd5L24jsCmARskUOYBARL3ABbLo5RA3Xxqxq 0NQv6+qXjpuNK13LRnjCunVZu5TsM5C6mvYWKyBOVRBiiJs8zr9aSeG+U9heMxVqK867 kNiPojfC+shRnaNNAe2CJ9/zaHBKs6ZokAUfo= Received: by 10.227.205.5 with SMTP id fo5mr1968394wbb.4.1322336181304; Sat, 26 Nov 2011 11:36:21 -0800 (PST) Received: from DataIX.net (ppp-21.232.dialinfree.com. [209.172.21.232]) by mx.google.com with ESMTPS id hq5sm12338598wib.7.2011.11.26.11.36.14 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Nov 2011 11:36:20 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAQJa8iO003033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Nov 2011 14:36:08 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAQJa3hN003032; Sat, 26 Nov 2011 14:36:03 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 26 Nov 2011 14:36:03 -0500 From: Jason Hellenthal To: Chris Rees Message-ID: <20111126193602.GA2368@DataIX.net> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> <20111126051121.GA4628@DataIX.net> <20111126190030.GA58253@DataIX.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6c2NcOVqGQ03X4Wi" Content-Disposition: inline In-Reply-To: Cc: Christian Kastner , Michael Ross , freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 19:36:23 -0000 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Nov 26, 2011 at 07:05:00PM +0000, Chris Rees wrote: > On 26 November 2011 19:00, Jason Hellenthal wrote: > > On Sat, Nov 26, 2011 at 06:43:38AM +0100, Michael Ross wrote: > >> Am 26.11.2011, 06:11 Uhr, schrieb Jason Hellenthal : > >> > On Fri, Nov 25, 2011 at 10:36:40PM +0100, Christian Kastner wrote: > >> >> Hi, > >> >> > >> >> On 2011-11-25 08:02, Jason Hellenthal wrote: > >> >> > So with that said... is there a way we could actually make this r= un > >> >> @reboot only ? > >> >> > >> >> Debian's cron[0] and Fedora's cronie[1] have solved this by touchin= g a > >> >> file on first startup and running @reboot only when this file does = not > >> >> yet exist. > >> >> > >> >> Note that while [0] may point to other patches that might be of int= erest > >> >> to FreeBSD, they are still WIP (as evident from the linked patch) a= s we > >> >> are still in the process of quiltifying our current code base. > >> >> > >> > > >> > While this sounds like a perfectly sane way to handle the problem at > >> > hand this also introduces a need to write some cleanup code to take = care > >> > of the file semantics. I think comparing daemon start times to the t= ime > >> > a system was booted or similiar would alleviate the need for all tha= t. > >> > Maybe a flag for @reboot "-s " seconds after boottime to al= low > >> > running @reboot jobs. And set the default to 3600 seconds. At least = this > >> > would allow adjustment for those startup processes that may take some > >> > considerable time before multiuser mode is entered. > >> > > >> > Just some thought. > >> > > >> > I really don't think we need to go the route of using files to store > >> > information when there is enough information available already via > >> > syscall's. > >> > >> If system startup were to be unusually delayed (dhcp or nfs trouble eg= ), > >> $time_period might have passed when cron starts, so there would have t= o be > >> some notifying mechanism for @reboot jobs not being run, and operator > >> action would be required. > >> > > > > I agree but also disagree. 1 hour or 3600 seconds is plenty of time to = wait for the "@reboot" extension scripts to fire. >=20 > Yes, and if I restart cron 30 minutes after boot, I'm screwed. >=20 Very true. But it would still be keeping its same behavior that it has had = for the last .... some odd years. Also it should be noted that my references to using time as way to figure o= ut if cron has been run @reboot only does have another fallback. If a syste= m is brought into single-user mode then you would obviously want the @reboo= t scripts to run again since essentially everything has been brought down s= imiliar to a reboot. what would be nice is if the init and kernel would keep kern.init.times and= create a syscall for thes so the time gets set when init goes mult-iuser. = Then at least there would be something real to compare to. --6c2NcOVqGQ03X4Wi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJO0T+iAAoJEJBXh4mJ2FR+T8wH/RJJWMP77V67XWMIAvGJ0On5 Dw/qbLJkI7zKB1YiNHR6sknp7r7+7BT18OPYLFCnvUxaoYkO/Nzkv8UfHSGo3bTB KRzDbQTjRaasbWHqE6yTAJW1e2nwfJ/f93HLmLF5GfZKmptBATbmHeBPavov/Fz/ bZTJrkGEaAE+ZU+ifYrpbVfrdh2c75pg3T0EbEWT2F+QgPvkhcNxZnjwaolUOOfi 8mBNJGJk4NHRrXcUHzBDgnjWVNuRsXhdZORF8Mf6XLgVUJmQTWIUSMTougZiN+aK wucM/DZmiuGayqqGks1jeSWJyKWZWlK0CBZq0cuVOKmzVsn4Hbk2yREWi9TWQ8k= =py9R -----END PGP SIGNATURE----- --6c2NcOVqGQ03X4Wi-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 19:43:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3716106566B; Sat, 26 Nov 2011 19:43:57 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id C2F8A8FC13; Sat, 26 Nov 2011 19:43:56 +0000 (UTC) Received: by wwe5 with SMTP id 5so4178317wwe.31 for ; Sat, 26 Nov 2011 11:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=PtLaDOnzPP6i2Dtrrq7ZeGad8Uejy/udQxgS1ek+9Fs=; b=Mf6jU4e+9NTTyNcRcSB3OzsJvb542ewouWc+WBQ/xEnHNtS66aH7zbqj4FKuNBiIe1 n+OgJpskxq9JNWDjr1mWbbvTFnk5p/f29VYY3xLUXJcil7s3tKcMJvLYEoa0p3hSjlmq AI4DD+I4x4Dfs0GXCweFWYfUw3Riwb7466AOg= Received: by 10.180.87.199 with SMTP id ba7mr39250725wib.27.1322336635992; Sat, 26 Nov 2011 11:43:55 -0800 (PST) Received: from DataIX.net (ppp-21.232.dialinfree.com. [209.172.21.232]) by mx.google.com with ESMTPS id d17sm32528096wbh.19.2011.11.26.11.43.52 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Nov 2011 11:43:55 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAQJhlGG003220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Nov 2011 14:43:47 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAQJhdMq003219; Sat, 26 Nov 2011 14:43:39 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 26 Nov 2011 14:43:39 -0500 From: Jason Hellenthal To: Stefan Bethke Message-ID: <20111126194339.GB2368@DataIX.net> References: <201111260708.pAQ78wvO045883@slippy.cwsent.com> <4ED0944E.7020709@FreeBSD.org> <9B3B0E26-F63C-4422-A434-FC9D737BE51A@lassitu.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Pd0ReVV5GZGQvF3a" Content-Disposition: inline In-Reply-To: <9B3B0E26-F63C-4422-A434-FC9D737BE51A@lassitu.de> Cc: Chris Rees , Tom Evans , Doug Barton , hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 19:43:57 -0000 --Pd0ReVV5GZGQvF3a Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Nov 26, 2011 at 04:44:20PM +0100, Stefan Bethke wrote: > Am 26.11.2011 um 11:54 schrieb Chris Rees: >=20 > > PLEASE let's not go down the route of that. How long? What if I need to > > restart cron shortly after boot? What if boot takes longer? >=20 > Plus "interesting" time changes during boot due to ntpdate/ntpd. >=20 Though it is related to when cron(8) runs jobs, in itself is not the proble= m. If you have interesting time changes that far off, then you have serious= 'other' problems on your hands than what we are discussing. That would be = for another thread. --Pd0ReVV5GZGQvF3a Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJO0UFrAAoJEJBXh4mJ2FR+3O4H/jHAFjke9T1ilj1XZNNMDDfu m07PLJDy7pI1m+12znRhFm/TYMqqaptlJawCcAF/65TL/ByFVRtaITUkL5yycvx0 COE4XzemMtj3whkj2BmnvLr/xPJClmq42YPXOQRLMHWc1MlAZ9uqK8q9hslMwGUz 2NceOADRJvAC6JwqGGNejMUKRc48egBLbsAx+f1jTTtUoqUTpHjUCQ8hN1BLPsiY ePZ6REYWC9E9l7NBmjN9KqVpczmURLDMZZMGiQlyCnfWMpV3YJCLPTaLwojWP5AH 0KzO0icd8z1t8wAs2StH+4BG0DEjL3NloXurAyas6Myl94k1fG6FafBeOSCfyU4= =d6fn -----END PGP SIGNATURE----- --Pd0ReVV5GZGQvF3a-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 19:53:46 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 318421065675 for ; Sat, 26 Nov 2011 19:53:46 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id B31D48FC1D for ; Sat, 26 Nov 2011 19:53:45 +0000 (UTC) Received: by wwe5 with SMTP id 5so4187982wwe.31 for ; Sat, 26 Nov 2011 11:53:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=QkQa25RBo6/608lt32ez0eUtJwPZQTAnbv+q2jwlFzI=; b=MlziwgF5bv3aLqZ3K1DBppXIYx5pY4DBaEEz+nNjUgA4AyN5n4ASRQNHVSfDPu9hYu 1NdTLBCiclZXUW1Y2rZ/CIR5ZqlP0V2+7A84WV8aZSRID4ncbO6SkLQ3EU/rb/4vdFAG s907porplabcS/OE+KPJhJZ535dhJQJX6zsIo= Received: by 10.180.103.131 with SMTP id fw3mr39012597wib.57.1322337224378; Sat, 26 Nov 2011 11:53:44 -0800 (PST) Received: from DataIX.net (ppp-21.232.dialinfree.com. [209.172.21.232]) by mx.google.com with ESMTPS id y3sm12356618wiy.3.2011.11.26.11.53.40 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Nov 2011 11:53:43 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pAQJrYNR003626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 26 Nov 2011 14:53:34 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pAQJr53W003620; Sat, 26 Nov 2011 14:53:05 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Sat, 26 Nov 2011 14:53:05 -0500 From: Jason Hellenthal To: Dieter BSD Message-ID: <20111126195305.GC2368@DataIX.net> References: <20111125175344.233450@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H8ygTp4AXg6deix2" Content-Disposition: inline In-Reply-To: <20111125175344.233450@gmx.com> Cc: hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 19:53:46 -0000 --H8ygTp4AXg6deix2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 25, 2011 at 12:53:43PM -0500, Dieter BSD wrote: > The system doesn't go multiuser until the rc jobs complete, > even if you attempt to background them with '&'. ??This can be > a problem with long running jobs. ??I started using cron @reboot > for this reason. >=20 > I haven't run into the problem since I've never needed to run > /etc/rc.d/cron restart. Yeah I have never ran into this either throughout all my use cases. cron(8)= has always ran and stayed running until my OP where I found out the hardwa= y. Would make someone proud to be called Paul Vixie. ;) >=20 > > Add an option to cron to check lastlog and if within 5 or 10 minutes > > of the last reboot, then call run_reboot_jobs(). >=20 > Depending on timestamps might be okay as a temporary quick-and-dirty > workaround, but there is likely to be a case where it will also do the > wrong thing. ??What if you need to restart cron within the 5-10 minutes? >=20 > Maybe something like: rc script sets a flag, cron_reboot script checks > and resets flag. ??The flag could be a file ("> /tmp/rebooting_system"). > Better yet, a run-the-reboot-script command line option could be added > to cron. cron(8) already creates a pid file. Maybe it wouldn't be so bad to just che= ck the timestamp of that before creating its new pid file along with compar= ing to system boottime so we could make an accurate measure whether it shou= ld or should not run the @reboot extension at least down to 1 minute. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --H8ygTp4AXg6deix2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJO0UOhAAoJEJBXh4mJ2FR+GREH+wVcZUtj8FjknGLpY1IzO6QF BDOC5X1dDtxka0TnUdvCcvmGfFM7lx4AqTnRKljfUeEMfRuDA8myrdv7pAUEbNSp wERAKFRG3zm8cUw7Mh2KZNc+9VD0rQjmo6KVtsqgByx8KfvPbY5EriZN2gaFkgf7 mwjg1dEdWOAMC75V16c2T1UeIUaM4AtfdeZxi1Br67Oi3bwmuR+8T/Z82hxZ/Lbo ss7gqV9WpACZK7JbZeTJvS9V3oQGk5VyH618KfJn6aewuYo2Smn6Zb9VBD1iDP9u 0iafQBEcvYtxzALodv+iuZPwPR9hcR8JtgetKqybBvmZXGM9gUi+YqoRMBZYsr4= =0R4k -----END PGP SIGNATURE----- --H8ygTp4AXg6deix2-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 20:28:05 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6084106566C for ; Sat, 26 Nov 2011 20:28:05 +0000 (UTC) (envelope-from apseudoutopia@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 544B38FC13 for ; Sat, 26 Nov 2011 20:28:04 +0000 (UTC) Received: by bkbzs8 with SMTP id zs8so7732145bkb.13 for ; Sat, 26 Nov 2011 12:28:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=29L6ulJlbIP/8GuSB3EIG/c0qeKg90zy0THeYhBaFq8=; b=il+kftWO10C3wFTdhS5eXA9lbFhWtX2c91NBymCy4cpFNQUF+/R/49MHLLb3GKN510 woPvA6CBVSnf/8lFKnTmEqPcWdZO3/JZv2051miNFMhfDlMe9E3GXRPUbXBS1pLCZ4r2 r9UiO7FpvTvXLOmRmVaeBPtjMoGZdNq0YMCs8= MIME-Version: 1.0 Received: by 10.204.157.154 with SMTP id b26mr39512734bkx.52.1322337526860; Sat, 26 Nov 2011 11:58:46 -0800 (PST) Received: by 10.204.184.8 with HTTP; Sat, 26 Nov 2011 11:58:46 -0800 (PST) In-Reply-To: <4ED00A68.4040606@kvr.at> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> Date: Sat, 26 Nov 2011 14:58:46 -0500 Message-ID: From: APseudoUtopia To: Christian Kastner Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 20:28:05 -0000 On Fri, Nov 25, 2011 at 4:36 PM, Christian Kastner wrote: > Hi, > > On 2011-11-25 08:02, Jason Hellenthal wrote: >> So with that said... is there a way we could actually make this run @reboot only ? > > Debian's cron[0] and Fedora's cronie[1] have solved this by touching a > file on first startup and running @reboot only when this file does not > yet exist. > I like this idea, however it has a major caveat: Assuming the shutdown scripts remove said file (and the boot scripts create said file), what happens in the event that the disk was umount'ed uncleanly? For example, a power failure (I know, that's what UPSs are for, but lets ignore that for a second). If the system is configured to automatically boot after a power failure, the @reboot cron script wont run (since the said file still exists...). From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 20:33:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4403C1065670 for ; Sat, 26 Nov 2011 20:33:11 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) by mx1.freebsd.org (Postfix) with ESMTP id D4A7C8FC0A for ; Sat, 26 Nov 2011 20:33:10 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id C35381DD64F; Sat, 26 Nov 2011 21:33:09 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 9D07628468; Sat, 26 Nov 2011 21:33:09 +0100 (CET) Date: Sat, 26 Nov 2011 21:33:09 +0100 From: Jilles Tjoelker To: APseudoUtopia Message-ID: <20111126203309.GB89541@stack.nl> References: <20111125070241.GA7915@DataIX.net> <4ED00A68.4040606@kvr.at> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Christian Kastner , freebsd-hackers@freebsd.org Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 20:33:11 -0000 On Sat, Nov 26, 2011 at 02:58:46PM -0500, APseudoUtopia wrote: > On Fri, Nov 25, 2011 at 4:36 PM, Christian Kastner wrote: > > On 2011-11-25 08:02, Jason Hellenthal wrote: > >> So with that said... is there a way we could actually make this run > >> @reboot only ? > > Debian's cron[0] and Fedora's cronie[1] have solved this by touching a > > file on first startup and running @reboot only when this file does not > > yet exist. > I like this idea, however it has a major caveat: Assuming the shutdown > scripts remove said file (and the boot scripts create said file), what > happens in the event that the disk was umount'ed uncleanly? For > example, a power failure (I know, that's what UPSs are for, but lets > ignore that for a second). If the system is configured to > automatically boot after a power failure, the @reboot cron script wont > run (since the said file still exists...). The file can be stored in /var/run, which is cleared at boot. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 20:51:43 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 149E61065670 for ; Sat, 26 Nov 2011 20:51:43 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id E74168FC12 for ; Sat, 26 Nov 2011 20:51:42 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id pAQKpcSu023728; Sat, 26 Nov 2011 20:51:38 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.119] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id cdpg9t5h85aym8447adjzwzaew; Sat, 26 Nov 2011 20:51:37 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Tim Kientzle In-Reply-To: <20111126195305.GC2368@DataIX.net> Date: Sat, 26 Nov 2011 12:51:36 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <7FD0AD28-B509-4E1F-BE82-713F85F17C1D@kientzle.com> References: <20111125175344.233450@gmx.com> <20111126195305.GC2368@DataIX.net> To: Jason Hellenthal X-Mailer: Apple Mail (2.1251.1) Cc: hackers@freebsd.org, Dieter BSD Subject: Re: cron(8) mis-feature with @reboot long after system startup X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 20:51:43 -0000 >>> Add an option to cron to check lastlog and if within 5 or 10 minutes >>> of the last reboot, then call run_reboot_jobs(). >>=20 >> Depending on timestamps might be okay as a temporary quick-and-dirty >> workaround, but there is likely to be a case where it will also do = the >> wrong thing. ??What if you need to restart cron within the 5-10 = minutes? >>=20 >> Maybe something like: rc script sets a flag, cron_reboot script = checks >> and resets flag. ??The flag could be a file ("> = /tmp/rebooting_system"). >> Better yet, a run-the-reboot-script command line option could be = added >> to cron. >=20 > cron(8) already creates a pid file. Maybe it wouldn't be so bad to = just check the timestamp of that before creating its new pid file along = with comparing to system boottime so we could make an accurate measure = whether it should or should not run the @reboot extension at least down = to 1 minute. Timing-based approaches like this tend to be brittle and fail in strange ways. Remember that "boot time" basically means "run by /etc/rc". Is there any way for /etc/rc.d/cron to notice that it's being run by /etc/rc and pass an extra flag to cron in that case? (If not, then perhaps /etc/rc should set some kind of marker when it starts --- such as an environment variable or a=20 file on disk --- that scripts such as /etc/rc.d/cron can use to distinguish boot-time startup from manual startup.) Tim From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 26 23:30:31 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E2831065670 for ; Sat, 26 Nov 2011 23:30:31 +0000 (UTC) (envelope-from kline@thought.org) Received: from thought.org (plato.thought.org [209.180.213.209]) by mx1.freebsd.org (Postfix) with ESMTP id 228918FC0A for ; Sat, 26 Nov 2011 23:30:31 +0000 (UTC) Received: by thought.org (Postfix, from userid 1001) id 92314E80A82; Sat, 26 Nov 2011 15:13:12 -0800 (PST) Date: Sat, 26 Nov 2011 15:13:12 -0800 From: Gary Kline To: FreeBSD Mailing List , Hackers Mailing List Message-ID: <20111126231309.GA9562@thought.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: Thought Unlimited. Public service Unix since 1986. Of_Interest: With 25 years of service to the Unix community. User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: gtk help, anybody? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2011 23:30:31 -0000 hi guys, can any one on questions or hackers give me a clue how to straighten out a 50-line gtk program? or suggest a good on-line tutorial? because this is seruiously OT, please send your home email. thanks much [either way], gary -- Gary Kline kline@thought.org http://www.thought.org Public Service Unix Journey Toward the Dawn, E-Book: http://www.thought.org The 8.57a release of Jottings: http://jottings.thought.org Twenty-five years of service to the Unix community.