From owner-freebsd-arch@FreeBSD.ORG Sun Dec 25 15:09:59 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B93BD106564A; Sun, 25 Dec 2011 15:09:59 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.freebsd.org (Postfix) with ESMTP id 3A1B68FC15; Sun, 25 Dec 2011 15:09:58 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail03.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id pBPF9su1010312 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 26 Dec 2011 02:09:56 +1100 Date: Mon, 26 Dec 2011 02:09:54 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Alexander Best In-Reply-To: <20111224121425.GA31084@freebsd.org> Message-ID: <20111226003337.Y805@besplex.bde.org> References: <20111223235642.GA37495@freebsd.org> <20111224160050.T1141@besplex.bde.org> <20111224093753.GA12377@freebsd.org> <20111224222040.Q2266@besplex.bde.org> <20111224121425.GA31084@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: [rfc] removing -mpreferred-stack-boundary=2 flag for i386? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Dec 2011 15:09:59 -0000 On Sat, 24 Dec 2011, Alexander Best wrote: > On Sat Dec 24 11, Bruce Evans wrote: >> On Sat, 24 Dec 2011, Alexander Best wrote: >> >>> On Sat Dec 24 11, Bruce Evans wrote: >>>> On Fri, 23 Dec 2011, Alexander Best wrote: >>> ... >>>>> the gcc(1) man page states the following: >>>>> >>>>> " >>>>> This extra alignment does consume extra stack space, and generally >>>>> increases code size. Code that is sensitive to stack space usage, >>>>> such as embedded systems and operating system kernels, may want to >>>>> reduce the preferred alignment to -mpreferred-stack-boundary=2. >>>>> " >>>>> >>>>> the comment in sys/conf/kern.mk however sorta suggests that the default >>>>> alignment of 4 bytes might improve performance. >>>> >>>> The default stack alignment is 16 bytes, which unimproves performance. >>> >>> maybe the part of the comment in sys/conf/kern.mk, which mentions that a >>> stack >>> alignment of 16 bytes might improve micro benchmark results should be >>> removed. >>> this would prevent people (like me) from thinking, using a stack alignment >>> of >>> 4 bytes is a compromise between size and efficiently. it isn't! currently a >>> stack alignment of 16 bytes has no advantages towards one with 4 bytes on >>> i386. >> >> I think the comment is clear enough. It it mentions all the tradeoffs. >> It is only slightly cryptic in saying that these are tradeoffs and that >> the configuration is our best guess at the best tradeoff -- it just says >> "while" for both. It goes without saying that we don't use our worst >> guess. Anyone wanting to change this should run benchmarks and beware >> that micro-benchmarks are especially useless. The changed comment is not >> so good since it no longer mentions micro-bencharmarks or says "while". > > if micro benchmark results aren't of any use, why should the claim that the > default stack alignment of 16 bytes might produce better outcome stay? Because: - the actual claim is the opposite of that (it is that the default 16-byte alignments is probably a loss overall) - the claim that the default 16-byte alignment may benefit micro-benchmarks is true, even without the weaselish miswording of "might" in it. There is always at least 1 micro-benchmark that will benefit from almost any change, and here we expect a benefit in many microbenchmarks that don't bust the caches. Except, 16-byte alignment isn't supported (*) in the kernel, so we actually expect a loss from many microbenchmarks that don't bust the caches. - the second claim warns inexperienced benchmarkers not to claim that the default is better because it is better in microbenchmarks. > it doesn't seem as if anybody has micro benchmarked 16 bytes vs. 4 bytes stack > alignment, until now. so the micro benchmark statement in the comment seems to > be pure speculation. No, it is obviously true. > even worse...it indicates that by removing the > -mpreferred-stack-boundary=2 flag, one can gain a performance boost by > sacrifying a few more bytes of kernel (and module) size. No, it is part of the sentence explaining why removing the -mpreferred-stack-boundary=2 flag will probably regain the "overall loss" that is avoided by using the flag. > this suggests that the behavior -mpreferred-stack-boundary=2 vs. not specyfing > it, losely equals the semantics of -Os vs. -O2. No, -Os guarantees slower execution by forcing optimization to prefer space savings over time savings in more ways. Except, -Os is completely broken in -current (in the kernel), and gives very large negative space savings (about 50%). It last worked with gcc-3. Its brokenness with gcc-4 is related to kern.pre.mk still specifying -finline-limit flags that are more suitable for gcc-3 (gcc has _many_ flags for giving more delicate control over inlining, and better defaults for them) and excessive inlining in gcc-4 given by -funit-at-a-time -finline-functions-called-once. These apparently cause gcc's inliner to go insane with -Os. When I tried to fix this by reducing inlining, I couldn't find any threshold that fixed -Os without breaking inlining of functions that are declared inline. (*) A primary part of the lack of support for 16-byte stack alignment in the kernel no special stack alignment for the main kernel entry point, namely syscall(). From i386/exception.s: % SUPERALIGN_TEXT % IDTVEC(int0x80_syscall) At this point, the stack has 5 words on it (it was 16-byte aligned before that). % pushl $2 /* sizeof "int 0x80" */ % subl $4,%esp /* skip over tf_trapno */ % pushal % pushl %ds % pushl %es % pushl %fs % SET_KERNEL_SREGS % cld % FAKE_MCOUNT(TF_EIP(%esp)) % pushl %esp We "push" 14 more words. This gives perfect misaligment to the worst odd word boundary (perfect if only word boundaries are allowed). gcc wants the stack to be aligned to a 4*n word boundary before function calls, but here we have a 4*n+3 word boundary. (4*n+3 is worse than 4*n+1 since 2 more words instead of 4 will cross the next 16-byte boundary). % call syscall Using the default -mpreferred-stack-boundary will preserve the perfect misaligment across all C functions called by syscall(). % add $4, %esp % MEXITCOUNT % jmp doreti Old versions didn't have the pessimization of pushing the frame pointer. This is a minor pessimization, except it uses more stack, unless you use the default -mpreferred-stack-boundary. Without this, only 18 words were pushed, so the misalignment was imperfect (to a 4*n+2 word boundary). If the default stack alignment is any use at all (in the kernel), then it is mainly to prevent 64-bit data types being laid out across cache line boundaries. Alignment to a 4*n+2 word boundary gives that just as well as alignment to a 4*n+0 word boundary. I tested using the default -mpreferred-stack-boundary in FreeBSD-~5.2, which doesn't push the frame pointer. This gave the expected results, except the optimization for a microbenchmark was surprisingly large. For a macro-benchmark, I built some kernels. This seemed to take a little longer (about 0.2%, and not statistically significant). But the time for a clock_gettime() microbenchmark was reduced from 271 ns per call to 263.5 ns per call. That's with the stack for clock_gettime() imperfectly misaligned to a 4*n+2 word boundary. But changing the stack alignment by subtracting more from the stack in syscons made little difference, unless it was changed to an odd byte boundary (then clock_gettime() took about 324 ns). amd64 is of course more careful about this (since its ABI requires 16-byte alignment). According to log messages, the initial %rsp (before anything is pushed onto it in the above) is offset by 8 bytes or so, as necessary to make the final %rsp come out aligned. Pushing the frame pointer would have broken this. However, on amd64, the first arg is passed in %rdi, so there is no push to pass the frame pointer and the stack remains aligned. When the frame pointer was passed "by reference", adjusting the stack after the pushes would have broken the reference, so the offset method was essential. Now it is not needed (unless we want or need frame to be aligned, since %rdi can pass the frame pointer wherever the frame is, and the offset method becomes a minor optimization. If you remove the -mpreferred-stack-boundary=2 optimization, be sure to remove this one too, since it is tinier. Bruce From owner-freebsd-arch@FreeBSD.ORG Mon Dec 26 10:10:40 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 981ED1065673; Mon, 26 Dec 2011 10:10:40 +0000 (UTC) Date: Mon, 26 Dec 2011 10:10:40 +0000 From: Alexander Best To: current@freebsd.org Message-ID: <20111226101040.GA6361@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline Cc: freebsd-toolchain@freebsd.org, freebsd-arch@freebsd.org Subject: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 10:10:40 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline hi there, i grep'ed through src/sys and found several places where WERROR= was set in order to get rid of the default -Werror setting. i tried to remove those WERROR= overrides from any Makefile, where doing so did not break tinderbox. in those cases, where it couldn't be completely removed, i added conditions to only set WERROR= for the particular achitecture or compiler, where tinderbox did not suceed without the WERROR=. i talked to dim@ on #freebsd-clang@OFTC and he was against enclosing WERROR= in a architecture or compiler condition. his statement was: " dim : I'm not going to riddle all those makefiles with endless comments, people have to learn to look in the commit history for a file. " however the problem is that most of the time the commit message by the person who added WERROR= to a Makefile doesn't state *why* the person did it and most importantly it doesn't state in *which* case tinderbox failed without the WERROR= (see *). so my suggestion would be that instead of setting WERROR= unconditionally in Makefiles, it should be enclosed in a condition to only set it in a particular case, where tinderbox *will* fail. an example is r228861. here dim@ set WERROR= in sys/modules/nve/Makefile unconditionally. however tinderbox will only fail when A) clang is used as compiler B) when building for i386 these facts aren't clear from the commit message. so i'd suggest (please see the attached patch) to conditionalise such settings. another point i'd like to make is that WERROR= might have been set in a Makefile in the past to unbreak tinderbox for a certain arch that is no longer supported. since most of the time the commit logs simply state something like: *) "...remove -Werror for now to unbreak tinderbox..." nobody will notice that after support for a certain arch was removed, the WERROR= line can be removed from that particular Makefile. that's why i'd like to propose the following patch. i ran a full tinderbox run against r228878 and it suceeded. cheers. alex --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="remove-werror.diff" Index: sys/modules/xfs/Makefile =================================================================== --- sys/modules/xfs/Makefile (revision 228878) +++ sys/modules/xfs/Makefile (working copy) @@ -6,8 +6,6 @@ KMOD= xfs -WERROR= - SRCS = vnode_if.h \ xfs_alloc.c \ xfs_alloc_btree.c \ Index: sys/modules/sound/driver/maestro/Makefile =================================================================== --- sys/modules/sound/driver/maestro/Makefile (revision 228878) +++ sys/modules/sound/driver/maestro/Makefile (working copy) @@ -5,6 +5,5 @@ KMOD= snd_maestro SRCS= device_if.h bus_if.h pci_if.h SRCS+= maestro.c -WERROR= .include Index: sys/modules/aic7xxx/ahd/Makefile =================================================================== --- sys/modules/aic7xxx/ahd/Makefile (revision 228878) +++ sys/modules/aic7xxx/ahd/Makefile (working copy) @@ -4,7 +4,6 @@ .PATH: ${.CURDIR}/../../../dev/aic7xxx KMOD= ahd -WERROR= GENSRCS= aic79xx_seq.h aic79xx_reg.h REG_PRINT_OPT= AHD_REG_PRETTY_PRINT=1 Index: sys/modules/ie/Makefile =================================================================== --- sys/modules/ie/Makefile (revision 228878) +++ sys/modules/ie/Makefile (working copy) @@ -6,6 +6,8 @@ KMOD= if_ie SRCS= if_ie.c if_ie_isa.c \ isa_if.h bus_if.h device_if.h +.if ${MACHINE_CPUARCH} == "i386" WERROR= +.endif .include Index: sys/modules/agp/Makefile =================================================================== --- sys/modules/agp/Makefile (revision 228878) +++ sys/modules/agp/Makefile (working copy) @@ -20,7 +20,6 @@ SRCS+= device_if.h bus_if.h agp_if.h pci_if.h SRCS+= opt_agp.h opt_bus.h MFILES= kern/device_if.m kern/bus_if.m dev/agp/agp_if.m dev/pci/pci_if.m -WERROR= EXPORT_SYMS= agp_find_device \ agp_state \ Index: sys/modules/bios/smapi/Makefile =================================================================== --- sys/modules/bios/smapi/Makefile (revision 228878) +++ sys/modules/bios/smapi/Makefile (working copy) @@ -6,7 +6,6 @@ KMOD= smapi SRCS= smapi.c smapi_bios.S \ bus_if.h device_if.h -WERROR= .if ${CC:T:Mclang} == "clang" # XXX: clang integrated-as doesn't grok 16-bit assembly yet CFLAGS+= ${.IMPSRC:T:Msmapi_bios.S:C/^.+$/-no-integrated-as/} Index: sys/modules/nve/Makefile =================================================================== --- sys/modules/nve/Makefile (revision 228878) +++ sys/modules/nve/Makefile (working copy) @@ -7,7 +7,9 @@ device_if.h bus_if.h pci_if.h miibus_if.h \ os+%DIKED-nve.h OBJS+= nvenetlib.o +.if ${MACHINE_CPUARCH} == "i386" && ${CC:T:Mclang} == "clang" WERROR= +.endif CLEANFILES+= nvenetlib.o os+%DIKED-nve.h nvenetlib.o: ${.CURDIR}/../../contrib/dev/nve/${MACHINE}/${.TARGET}.bz2.uu --W/nzBZO5zC0uMSeA-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 26 22:08:13 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7ADEC106566B for ; Mon, 26 Dec 2011 22:08:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 09F9C8FC08 for ; Mon, 26 Dec 2011 22:08:12 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBQM7voj008492 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 27 Dec 2011 00:07:57 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBQM7uWi001825; Tue, 27 Dec 2011 00:07:56 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBQM7ugE001824; Tue, 27 Dec 2011 00:07:56 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 27 Dec 2011 00:07:56 +0200 From: Kostik Belousov To: amd64@freebsd.org Message-ID: <20111226220756.GR50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YzFme3vxTykXnmM8" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: AVX X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Dec 2011 22:08:13 -0000 --YzFme3vxTykXnmM8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, The patch at http://people.freebsd.org/~kib/misc/avx.1.patch is the preliminary version, i.e. WIP, of the implementation of variable-sized extended FPU states, as defined by Intel SDM. The architecture (sometimes referred to as XSAVE) allows CPU to provide additional context state data and report the details of context save area layout to OS in the extendable form. AVX with it 256-bit YMM register file is the first application of the architecture. Due to extensibility and self-description of XSAVE area, by default kernel allows all CPU extensions, i.e. also possible future features of FPU. The tunable 'hw.xsave_allow_unknown' is provided, which disables CPU features not known to the kernel. The tunable 'hw.use_xsave' may be used to disable XSAVE support at all. The 'struct fpusave fpu_user_save' member was removed from amd64 pcb, the save area is chopped from the top of the thread kernel kernel stack (fpu_user_save was located there previously). I moved thread0 pcb set up after first stage of FPU initialization is done. Also, there is a silent change in fpu init code, now we explicitely clear XMM register file in the initial FPU state. At least on my SandyBridge, XMM init state is garbage, could be some vacuum tubes they stuff into chip were broken when machine crashed from 1 meter down. Since YMM file is part of the FPU state, it must be added to mcontext_t. Unfortunately, it is impossible to extend mcontext_t in-place and not break present ABI without duplicating all syscalls and libraries entries that use ucontext_t. The cause is that ucontext_t layout puts mcontext_t into the middle of the structure, frozing its size. The solution in the patch is to put extended save area outside the mcontext_t, and place a pointer to area in mcontext_t. Pointer validity is specified by _MC_HASFPXSTATE mcontext_t flag. Signal post provides the extended area on the signal stack frame automagically. If pointers to area are in context, setcontext(2) uses it. For getcontext(2), no easy solution exists. I tried to play with providing libc wrapper for getcontext(2) that hides mcontext_t size increase, but gave up due to ucontext_t embedding. Instead, getcontextx(3) API is implemented that allocates the needed memory. The patch defines ABI for both i386 and amd64 FreeBSD processes, but the implementation is provided for amd64 kernel only for both ABIs. I do not see any reason to run i386 kernel on machines capable of AVX. The fpu_kern KPI has to be changed to accomodate for variable size TODO items: - I think we are eligible for XSAVEOPT use, but did not looked into details yet; - benchmark the cost for context switches and signal handlers entry/sigreturn; - makecontextx(3) ?; - padlock(4) testing; - make i386 fpu_kern KPI changes; - implement (trivial) getcontextx(3) for other architectures. Patch in the present form only usable on amd64 machine for both amd64 and i386 userspace. --YzFme3vxTykXnmM8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk748DwACgkQC3+MBN1Mb4j0QQCfXTUIz73F08AmyLo0mppP0+1y VZMAoMLckeDmraBPb47HKhirrMxGJOiM =PzoA -----END PGP SIGNATURE----- --YzFme3vxTykXnmM8-- From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 10:42:03 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B82E106564A; Tue, 27 Dec 2011 10:42:03 +0000 (UTC) (envelope-from philip@paeps.cx) Received: from rincewind.paeps.cx (rincewind.paeps.cx [IPv6:2002:596a:f092::149]) by mx1.freebsd.org (Postfix) with ESMTP id F00338FC20; Tue, 27 Dec 2011 10:42:02 +0000 (UTC) Received: from twoflower.paeps.cx (unknown [176.99.70.219]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "twoflower.paeps.cx", Issuer "CAcert Class 3 Root" (verified OK)) by rincewind.paeps.cx (Postfix) with ESMTPS id 0CE3BD74495; Tue, 27 Dec 2011 11:41:53 +0100 (CET) Received: by twoflower.paeps.cx (Postfix, from userid 1001) id F13EB3E2106; Tue, 27 Dec 2011 02:04:49 +0100 (CET) Date: Tue, 27 Dec 2011 02:04:49 +0100 From: Philip Paeps To: freebsd-current@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Message-ID: <20111227010449.GA6244@twoflower.paeps.cx> Mail-Followup-To: freebsd-current@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org References: <20111226101040.GA6361@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20111226101040.GA6361@freebsd.org> X-PGP-Fingerprint: 356B AE02 4763 F739 2FA2 E438 2649 E628 C5D3 4D05 X-Date: Today is Sweetmorn, the 69th day of The Aftermath in the YOLD 3177 X-Phase-of-Moon: The Moon is Waxing Crescent (7% of Full) X-Philip-Conspiracy: There is no conspiracy User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 10:42:03 -0000 On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: > i grep'ed through src/sys and found several places where WERROR= was set in > order to get rid of the default -Werror setting. i tried to remove those > WERROR= overrides from any Makefile, where doing so did not break tinderbox. > > in those cases, where it couldn't be completely removed, i added conditions to > only set WERROR= for the particular achitecture or compiler, where tinderbox > did not suceed without the WERROR=. Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the following kind of bugs which break tinderbox". Possibly wrapped in an architecture-test where appropriate. - Philip -- Philip Paeps Senior Reality Engineer Ministry of Information From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 11:27:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 2183D1065678; Tue, 27 Dec 2011 11:27:43 +0000 (UTC) Date: Tue, 27 Dec 2011 11:27:43 +0000 From: Alexander Best To: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Message-ID: <20111227112743.GA154@freebsd.org> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Dxnq1zWXvFF0Q93v" Content-Disposition: inline In-Reply-To: <20111227010449.GA6244@twoflower.paeps.cx> Cc: Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 11:27:43 -0000 --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue Dec 27 11, Philip Paeps wrote: > On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: > > i grep'ed through src/sys and found several places where WERROR= was set in > > order to get rid of the default -Werror setting. i tried to remove those > > WERROR= overrides from any Makefile, where doing so did not break tinderbox. > > > > in those cases, where it couldn't be completely removed, i added conditions to > > only set WERROR= for the particular achitecture or compiler, where tinderbox > > did not suceed without the WERROR=. > > Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this > code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the > following kind of bugs which break tinderbox". > > Possibly wrapped in an architecture-test where appropriate. well there are a few issues here: 1) Jan Beich informed me via a private email that enclosing WERROR in arch specific conditions is a bad idea. if the code gets ported to a new architecture WERROR doesn't get set and so for every new architecture one has to evaluate again, whether WERROR needs to be set or not. so i'm inclined to agree with dim@ that we should not add architecture specific conditions -- however i think adding compiler specific conditions is a good idea. 2) the problem with settings WARNS= or specific -Wno-X or -Wno-error=X is that expecially GCC doesn't have specific -WX flags for certain warnings. some warnings are implied by -Wall and cannot be turned off seperately. so in order to get rid of these warnings (which are being handled as errors), we would need to disable -Wall. and i think setting WERROR= in order to handle all warnings for specific code as warnings rather than as errors is the better solution. i've reworked the patch to only remove WERROR=, where it is not needed anymore for any supported arch, or where it can be wrapped in a compiler condition. cheers. alex > > - Philip > > -- > Philip Paeps > Senior Reality Engineer > Ministry of Information --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="remove-werror.diff2" Index: sys/modules/xfs/Makefile =================================================================== --- sys/modules/xfs/Makefile (revision 228911) +++ sys/modules/xfs/Makefile (working copy) @@ -6,8 +6,6 @@ KMOD= xfs -WERROR= - SRCS = vnode_if.h \ xfs_alloc.c \ xfs_alloc_btree.c \ Index: sys/modules/sound/driver/maestro/Makefile =================================================================== --- sys/modules/sound/driver/maestro/Makefile (revision 228911) +++ sys/modules/sound/driver/maestro/Makefile (working copy) @@ -5,6 +5,5 @@ KMOD= snd_maestro SRCS= device_if.h bus_if.h pci_if.h SRCS+= maestro.c -WERROR= .include Index: sys/modules/aic7xxx/ahd/Makefile =================================================================== --- sys/modules/aic7xxx/ahd/Makefile (revision 228911) +++ sys/modules/aic7xxx/ahd/Makefile (working copy) @@ -4,7 +4,6 @@ .PATH: ${.CURDIR}/../../../dev/aic7xxx KMOD= ahd -WERROR= GENSRCS= aic79xx_seq.h aic79xx_reg.h REG_PRINT_OPT= AHD_REG_PRETTY_PRINT=1 Index: sys/modules/agp/Makefile =================================================================== --- sys/modules/agp/Makefile (revision 228911) +++ sys/modules/agp/Makefile (working copy) @@ -20,7 +20,6 @@ SRCS+= device_if.h bus_if.h agp_if.h pci_if.h SRCS+= opt_agp.h opt_bus.h MFILES= kern/device_if.m kern/bus_if.m dev/agp/agp_if.m dev/pci/pci_if.m -WERROR= EXPORT_SYMS= agp_find_device \ agp_state \ Index: sys/modules/bios/smapi/Makefile =================================================================== --- sys/modules/bios/smapi/Makefile (revision 228911) +++ sys/modules/bios/smapi/Makefile (working copy) @@ -6,7 +6,6 @@ KMOD= smapi SRCS= smapi.c smapi_bios.S \ bus_if.h device_if.h -WERROR= .if ${CC:T:Mclang} == "clang" # XXX: clang integrated-as doesn't grok 16-bit assembly yet CFLAGS+= ${.IMPSRC:T:Msmapi_bios.S:C/^.+$/-no-integrated-as/} Index: sys/modules/nve/Makefile =================================================================== --- sys/modules/nve/Makefile (revision 228911) +++ sys/modules/nve/Makefile (working copy) @@ -7,7 +7,9 @@ device_if.h bus_if.h pci_if.h miibus_if.h \ os+%DIKED-nve.h OBJS+= nvenetlib.o +.if ${CC:T:Mclang} == "clang" WERROR= +.endif CLEANFILES+= nvenetlib.o os+%DIKED-nve.h nvenetlib.o: ${.CURDIR}/../../contrib/dev/nve/${MACHINE}/${.TARGET}.bz2.uu --Dxnq1zWXvFF0Q93v-- From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 11:47:32 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47E9C106564A; Tue, 27 Dec 2011 11:47:32 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id E07F48FC13; Tue, 27 Dec 2011 11:47:31 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 658C17300A; Tue, 27 Dec 2011 13:04:09 +0100 (CET) Date: Tue, 27 Dec 2011 13:04:09 +0100 From: Luigi Rizzo To: Alexander Best Message-ID: <20111227120409.GA64631@onelab2.iet.unipi.it> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> <20111227112743.GA154@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111227112743.GA154@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 11:47:32 -0000 On Tue, Dec 27, 2011 at 11:27:43AM +0000, Alexander Best wrote: > On Tue Dec 27 11, Philip Paeps wrote: > > On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: > > > i grep'ed through src/sys and found several places where WERROR= was set in > > > order to get rid of the default -Werror setting. i tried to remove those > > > WERROR= overrides from any Makefile, where doing so did not break tinderbox. > > > > > > in those cases, where it couldn't be completely removed, i added conditions to > > > only set WERROR= for the particular achitecture or compiler, where tinderbox > > > did not suceed without the WERROR=. > > > > Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this > > code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the > > following kind of bugs which break tinderbox". > > > > Possibly wrapped in an architecture-test where appropriate. > > well there are a few issues here: > > 1) Jan Beich informed me via a private email that enclosing WERROR in arch > specific conditions is a bad idea. if the code gets ported to a new > architecture WERROR doesn't get set and so for every new architecture one > has to evaluate again, whether WERROR needs to be set or not. > so i'm inclined to agree with dim@ that we should not add architecture > specific conditions -- however i think adding compiler specific conditions > is a good idea. > > 2) the problem with settings WARNS= or specific -Wno-X or -Wno-error=X is that > expecially GCC doesn't have specific -WX flags for certain warnings. some > warnings are implied by -Wall and cannot be turned off seperately. so in > order to get rid of these warnings (which are being handled as errors), we > would need to disable -Wall. and i think setting WERROR= in order to handle > all warnings for specific code as warnings rather than as errors is the > better solution. > > i've reworked the patch to only remove WERROR=, where it is not needed anymore > for any supported arch, or where it can be wrapped in a compiler condition. It seems to me that the removal of unnecessary WERROR= needed no discussion since day one so why don't you go ahead and commit it. I don't understand the comment on issue #1 above. There is a minuscule (six, before your patch ?) number of Makefiles with WERROR= . If you make the assignment architecture-specific, the worst it can happen is that the variable is not cleared, and if the build breaks, all you need is to add the extra architecture in these few places. cheers luigi > cheers. > alex > > > > > - Philip > > > > -- > > Philip Paeps > > Senior Reality Engineer > > Ministry of Information > Index: sys/modules/xfs/Makefile > =================================================================== > --- sys/modules/xfs/Makefile (revision 228911) > +++ sys/modules/xfs/Makefile (working copy) > @@ -6,8 +6,6 @@ > > KMOD= xfs > > -WERROR= > - > SRCS = vnode_if.h \ > xfs_alloc.c \ > xfs_alloc_btree.c \ > Index: sys/modules/sound/driver/maestro/Makefile > =================================================================== > --- sys/modules/sound/driver/maestro/Makefile (revision 228911) > +++ sys/modules/sound/driver/maestro/Makefile (working copy) > @@ -5,6 +5,5 @@ > KMOD= snd_maestro > SRCS= device_if.h bus_if.h pci_if.h > SRCS+= maestro.c > -WERROR= > > .include > Index: sys/modules/aic7xxx/ahd/Makefile > =================================================================== > --- sys/modules/aic7xxx/ahd/Makefile (revision 228911) > +++ sys/modules/aic7xxx/ahd/Makefile (working copy) > @@ -4,7 +4,6 @@ > .PATH: ${.CURDIR}/../../../dev/aic7xxx > KMOD= ahd > > -WERROR= > GENSRCS= aic79xx_seq.h aic79xx_reg.h > REG_PRINT_OPT= > AHD_REG_PRETTY_PRINT=1 > Index: sys/modules/agp/Makefile > =================================================================== > --- sys/modules/agp/Makefile (revision 228911) > +++ sys/modules/agp/Makefile (working copy) > @@ -20,7 +20,6 @@ > SRCS+= device_if.h bus_if.h agp_if.h pci_if.h > SRCS+= opt_agp.h opt_bus.h > MFILES= kern/device_if.m kern/bus_if.m dev/agp/agp_if.m dev/pci/pci_if.m > -WERROR= > > EXPORT_SYMS= agp_find_device \ > agp_state \ > Index: sys/modules/bios/smapi/Makefile > =================================================================== > --- sys/modules/bios/smapi/Makefile (revision 228911) > +++ sys/modules/bios/smapi/Makefile (working copy) > @@ -6,7 +6,6 @@ > KMOD= smapi > SRCS= smapi.c smapi_bios.S \ > bus_if.h device_if.h > -WERROR= > .if ${CC:T:Mclang} == "clang" > # XXX: clang integrated-as doesn't grok 16-bit assembly yet > CFLAGS+= ${.IMPSRC:T:Msmapi_bios.S:C/^.+$/-no-integrated-as/} > Index: sys/modules/nve/Makefile > =================================================================== > --- sys/modules/nve/Makefile (revision 228911) > +++ sys/modules/nve/Makefile (working copy) > @@ -7,7 +7,9 @@ > device_if.h bus_if.h pci_if.h miibus_if.h \ > os+%DIKED-nve.h > OBJS+= nvenetlib.o > +.if ${CC:T:Mclang} == "clang" > WERROR= > +.endif > > CLEANFILES+= nvenetlib.o os+%DIKED-nve.h > nvenetlib.o: ${.CURDIR}/../../contrib/dev/nve/${MACHINE}/${.TARGET}.bz2.uu > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 19:27:08 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 1D47E1065676; Tue, 27 Dec 2011 19:27:08 +0000 (UTC) Date: Tue, 27 Dec 2011 19:27:08 +0000 From: Alexander Best To: Luigi Rizzo Message-ID: <20111227192708.GA58521@freebsd.org> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> <20111227112743.GA154@freebsd.org> <20111227120409.GA64631@onelab2.iet.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111227120409.GA64631@onelab2.iet.unipi.it> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 19:27:08 -0000 On Tue Dec 27 11, Luigi Rizzo wrote: > On Tue, Dec 27, 2011 at 11:27:43AM +0000, Alexander Best wrote: > > On Tue Dec 27 11, Philip Paeps wrote: > > > On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: > > > > i grep'ed through src/sys and found several places where WERROR= was set in > > > > order to get rid of the default -Werror setting. i tried to remove those > > > > WERROR= overrides from any Makefile, where doing so did not break tinderbox. > > > > > > > > in those cases, where it couldn't be completely removed, i added conditions to > > > > only set WERROR= for the particular achitecture or compiler, where tinderbox > > > > did not suceed without the WERROR=. > > > > > > Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this > > > code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the > > > following kind of bugs which break tinderbox". > > > > > > Possibly wrapped in an architecture-test where appropriate. > > > > well there are a few issues here: > > > > 1) Jan Beich informed me via a private email that enclosing WERROR in arch > > specific conditions is a bad idea. if the code gets ported to a new > > architecture WERROR doesn't get set and so for every new architecture one > > has to evaluate again, whether WERROR needs to be set or not. > > so i'm inclined to agree with dim@ that we should not add architecture > > specific conditions -- however i think adding compiler specific conditions > > is a good idea. > > > > 2) the problem with settings WARNS= or specific -Wno-X or -Wno-error=X is that > > expecially GCC doesn't have specific -WX flags for certain warnings. some > > warnings are implied by -Wall and cannot be turned off seperately. so in > > order to get rid of these warnings (which are being handled as errors), we > > would need to disable -Wall. and i think setting WERROR= in order to handle > > all warnings for specific code as warnings rather than as errors is the > > better solution. > > > > i've reworked the patch to only remove WERROR=, where it is not needed anymore > > for any supported arch, or where it can be wrapped in a compiler condition. > > It seems to me that the removal of unnecessary WERROR= needed no > discussion since day one so why don't you go ahead and commit it. anybody is free to commit this part, since i don't own a commit bit. ;) > > I don't understand the comment on issue #1 above. There is a minuscule > (six, before your patch ?) > number of Makefiles with WERROR= . If you make the assignment > architecture-specific, the worst it can happen is that the variable > is not cleared, and if the build breaks, all you need is to > add the extra architecture in these few places. good point. basically the question with WERROR is: should it be a big hammer to disable turning warnings into errors for all archs or do we want to set WERROR in a more specific manor, where it's absolutely necessary. cheers. alex > > cheers > luigi > > > cheers. > > alex > > > > > > > > - Philip > > > > > > -- > > > Philip Paeps > > > Senior Reality Engineer > > > Ministry of Information > > > Index: sys/modules/xfs/Makefile > > =================================================================== > > --- sys/modules/xfs/Makefile (revision 228911) > > +++ sys/modules/xfs/Makefile (working copy) > > @@ -6,8 +6,6 @@ > > > > KMOD= xfs > > > > -WERROR= > > - > > SRCS = vnode_if.h \ > > xfs_alloc.c \ > > xfs_alloc_btree.c \ > > Index: sys/modules/sound/driver/maestro/Makefile > > =================================================================== > > --- sys/modules/sound/driver/maestro/Makefile (revision 228911) > > +++ sys/modules/sound/driver/maestro/Makefile (working copy) > > @@ -5,6 +5,5 @@ > > KMOD= snd_maestro > > SRCS= device_if.h bus_if.h pci_if.h > > SRCS+= maestro.c > > -WERROR= > > > > .include > > Index: sys/modules/aic7xxx/ahd/Makefile > > =================================================================== > > --- sys/modules/aic7xxx/ahd/Makefile (revision 228911) > > +++ sys/modules/aic7xxx/ahd/Makefile (working copy) > > @@ -4,7 +4,6 @@ > > .PATH: ${.CURDIR}/../../../dev/aic7xxx > > KMOD= ahd > > > > -WERROR= > > GENSRCS= aic79xx_seq.h aic79xx_reg.h > > REG_PRINT_OPT= > > AHD_REG_PRETTY_PRINT=1 > > Index: sys/modules/agp/Makefile > > =================================================================== > > --- sys/modules/agp/Makefile (revision 228911) > > +++ sys/modules/agp/Makefile (working copy) > > @@ -20,7 +20,6 @@ > > SRCS+= device_if.h bus_if.h agp_if.h pci_if.h > > SRCS+= opt_agp.h opt_bus.h > > MFILES= kern/device_if.m kern/bus_if.m dev/agp/agp_if.m dev/pci/pci_if.m > > -WERROR= > > > > EXPORT_SYMS= agp_find_device \ > > agp_state \ > > Index: sys/modules/bios/smapi/Makefile > > =================================================================== > > --- sys/modules/bios/smapi/Makefile (revision 228911) > > +++ sys/modules/bios/smapi/Makefile (working copy) > > @@ -6,7 +6,6 @@ > > KMOD= smapi > > SRCS= smapi.c smapi_bios.S \ > > bus_if.h device_if.h > > -WERROR= > > .if ${CC:T:Mclang} == "clang" > > # XXX: clang integrated-as doesn't grok 16-bit assembly yet > > CFLAGS+= ${.IMPSRC:T:Msmapi_bios.S:C/^.+$/-no-integrated-as/} > > Index: sys/modules/nve/Makefile > > =================================================================== > > --- sys/modules/nve/Makefile (revision 228911) > > +++ sys/modules/nve/Makefile (working copy) > > @@ -7,7 +7,9 @@ > > device_if.h bus_if.h pci_if.h miibus_if.h \ > > os+%DIKED-nve.h > > OBJS+= nvenetlib.o > > +.if ${CC:T:Mclang} == "clang" > > WERROR= > > +.endif > > > > CLEANFILES+= nvenetlib.o os+%DIKED-nve.h > > nvenetlib.o: ${.CURDIR}/../../contrib/dev/nve/${MACHINE}/${.TARGET}.bz2.uu > > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 20:32:14 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B11DE106567A; Tue, 27 Dec 2011 20:32:14 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 636D88FC0C; Tue, 27 Dec 2011 20:32:14 +0000 (UTC) Received: from [10.0.0.63] (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBRKSisK039899 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Tue, 27 Dec 2011 13:28:46 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20111227010449.GA6244@twoflower.paeps.cx> Date: Tue, 27 Dec 2011 13:28:44 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8DD9BAA5-78E0-491A-9E4E-6915B0A9FBAA@bsdimp.com> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> To: Philip Paeps X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Tue, 27 Dec 2011 13:28:46 -0700 (MST) Cc: freebsd-current@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 20:32:14 -0000 On Dec 26, 2011, at 6:04 PM, Philip Paeps wrote: > On 2011-12-26 10:10:40 (+0000), Alexander Best = wrote: >> i grep'ed through src/sys and found several places where WERROR=3D = was set in >> order to get rid of the default -Werror setting. i tried to remove = those >> WERROR=3D overrides from any Makefile, where doing so did not break = tinderbox. >>=20 >> in those cases, where it couldn't be completely removed, i added = conditions to >> only set WERROR=3D for the particular achitecture or compiler, where = tinderbox >> did not suceed without the WERROR=3D. >=20 > Wouldn't it be better to set WARNS=3Dx rather than WERROR=3D? WERROR=3D= says "this > code has bugs, it breaks tinderbox" whereas WARNS=3Dx says "this code = has the > following kind of bugs which break tinderbox". Agreed... > Possibly wrapped in an architecture-test where appropriate. Not so much... When you make architecture-specific tests, experience = has shown that we don't fix bugs and they languish for a long time. = Many times, these warnings are real. Sadly, we've found no way to tag = the ones that aren't real yet as safe to ignore... Warner From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 22:54:28 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 60B701065673; Tue, 27 Dec 2011 22:54:28 +0000 (UTC) Date: Tue, 27 Dec 2011 22:54:28 +0000 From: Alexander Best To: Warner Losh Message-ID: <20111227225428.GA12771@freebsd.org> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> <8DD9BAA5-78E0-491A-9E4E-6915B0A9FBAA@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8DD9BAA5-78E0-491A-9E4E-6915B0A9FBAA@bsdimp.com> Cc: freebsd-arch@freebsd.org, freebsd-current@freebsd.org, Philip Paeps Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 22:54:28 -0000 On Tue Dec 27 11, Warner Losh wrote: > > On Dec 26, 2011, at 6:04 PM, Philip Paeps wrote: > > > On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: > >> i grep'ed through src/sys and found several places where WERROR= was set in > >> order to get rid of the default -Werror setting. i tried to remove those > >> WERROR= overrides from any Makefile, where doing so did not break tinderbox. > >> > >> in those cases, where it couldn't be completely removed, i added conditions to > >> only set WERROR= for the particular achitecture or compiler, where tinderbox > >> did not suceed without the WERROR=. > > > > Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this > > code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the > > following kind of bugs which break tinderbox". > > Agreed... in this case it would have to be WARNS=1 then, because anything > 1 will enable -Wall, which is the warning that breaks sys/modules/ie. cheers. alex > > > Possibly wrapped in an architecture-test where appropriate. > > Not so much... When you make architecture-specific tests, experience has shown that we don't fix bugs and they languish for a long time. Many times, these warnings are real. Sadly, we've found no way to tag the ones that aren't real yet as safe to ignore... > > Warner > From owner-freebsd-arch@FreeBSD.ORG Tue Dec 27 23:49:21 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D49EB106566B; Tue, 27 Dec 2011 23:49:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 935BE8FC0A; Tue, 27 Dec 2011 23:49:21 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:2838:1eb:929f:da87] (unknown [IPv6:2001:7b8:3a7:0:2838:1eb:929f:da87]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id B2E4A5C37; Wed, 28 Dec 2011 00:49:19 +0100 (CET) Message-ID: <4EFA597D.3000508@FreeBSD.org> Date: Wed, 28 Dec 2011 00:49:17 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org, freebsd-arch@freebsd.org References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> In-Reply-To: <20111227010449.GA6244@twoflower.paeps.cx> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Dec 2011 23:49:21 -0000 On 2011-12-27 02:04, Philip Paeps wrote: > On 2011-12-26 10:10:40 (+0000), Alexander Best wrote: >> i grep'ed through src/sys and found several places where WERROR= was set in >> order to get rid of the default -Werror setting. i tried to remove those >> WERROR= overrides from any Makefile, where doing so did not break tinderbox. >> >> in those cases, where it couldn't be completely removed, i added conditions to >> only set WERROR= for the particular achitecture or compiler, where tinderbox >> did not suceed without the WERROR=. > > Wouldn't it be better to set WARNS=x rather than WERROR=? WERROR= says "this > code has bugs, it breaks tinderbox" whereas WARNS=x says "this code has the > following kind of bugs which break tinderbox". In my opinion, WERROR= says: there are warnings in this code which cannot be fixed right now, due to varying reasons, but we don't want to muffle them entirely, so somebody will eventually fix them in the future (or just delete the code, if it is unmaintained, or unmaintainable, like nve). If you set WARNS to a low level, you can be sure nobody ever sees the warnings, and they will never be fixed. That may be appropriate in some cases, but not the ones I just added WERROR= to. Those are just crufty drivers, that nobody wants to burn their fingers on. :) From owner-freebsd-arch@FreeBSD.ORG Wed Dec 28 02:47:53 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 233EA106566B for ; Wed, 28 Dec 2011 02:47:53 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 06DEE8FC0A for ; Wed, 28 Dec 2011 02:47:53 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id B1EDC17649; Tue, 27 Dec 2011 18:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1325040472; bh=PQX8c1dflrLsHf2j8FHgU00zzeQ8Fmdow7J+1kKTq/Y=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: Content-Type:Content-Transfer-Encoding; b=xXAQjDYNOeYA+f6xT3aXtCYHmrMr3EEAgCC+XqqpG4htTUavRxuqHg6DKYK5UEbpL ASMWn0hCKJ1ObB2Z8HleBlzRDlNPeUB+sj+NK7o2Gv6A8AFHv+oq8tcoeL2GG8amRw BzCkysx6sk5A6xDTgkRf95mcJVNLkPgycKJDK2bI= Message-ID: <4EFA8357.5040101@delphij.net> Date: Tue, 27 Dec 2011 18:47:51 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net Subject: Preferred place for MD code for new libraries? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 02:47:53 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, It seems that we have two different style of arranging files. One is the libc way, where MD code are put directly under src/lib/libc, another is libthr way, where MD code are put under src/lib/libthr/arch, which is the preferred for new libraries? Do we want to convert from one to another by the way? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk76g1cACgkQOfuToMruuMClLwCeI+5RrruTdq7DGXp5Ysvh1182 EhEAnjCzbl4YE9yhu1hCAc4x6BCtHLeU =h4Ug -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Wed Dec 28 04:34:28 2011 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6EF281065676 for ; Wed, 28 Dec 2011 04:34:28 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 29F048FC1B for ; Wed, 28 Dec 2011 04:34:28 +0000 (UTC) Received: from [10.0.0.63] (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBS4U6GZ042831 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Tue, 27 Dec 2011 21:30:08 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <4EFA8357.5040101@delphij.net> Date: Tue, 27 Dec 2011 21:29:50 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <650DAD7F-FB70-4335-98A6-326A3C18DF10@bsdimp.com> References: <4EFA8357.5040101@delphij.net> To: d@delphij.net X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Tue, 27 Dec 2011 21:30:08 -0700 (MST) Cc: freebsd-arch@FreeBSD.org Subject: Re: Preferred place for MD code for new libraries? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Dec 2011 04:34:28 -0000 On Dec 27, 2011, at 7:47 PM, Xin Li wrote: > It seems that we have two different style of arranging files. One is > the libc way, where MD code are put directly under src/lib/libc, > another is libthr way, where MD code are put under = src/lib/libthr/arch, > which is the preferred for new libraries? Do we want to convert from > one to another by the way? New code generally has been adding arch/ where this sort of thing is = necessary. Old code retains the old layout. There's been some heated = debates about whether or not it makes sense to move things en mass. = With subversion, many of the old arguments are not valid (since they = centered around cvs repo copy suckage). However, in addition to the = technical issue, there was the 'social' issue that some loud and = outspoken people hated it, while more, less forceful, folks liked it. = Thankfully many of the loudest yellers have mellowed with the decade or = so that's passed since the issue was last openly litigated. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 08:43:14 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3737106564A; Thu, 29 Dec 2011 08:43:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 058B68FC14; Thu, 29 Dec 2011 08:43:13 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBT8h8cx015178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 29 Dec 2011 10:43:09 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBT8h8A1016518; Thu, 29 Dec 2011 10:43:08 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBT8h8IW016517; Thu, 29 Dec 2011 10:43:08 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Dec 2011 10:43:08 +0200 From: Kostik Belousov To: arch@freebsd.org Message-ID: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6PhDeOF0ICWr3i50" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: kan@freebsd.org, des@freebsd.org Subject: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 08:43:14 -0000 --6PhDeOF0ICWr3i50 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, below is the patch to add fdlopen(3) function to rtld. It allows to load shared object referenced by the opened file descriptor. Intent is to allow the caller to do some additional checks on the object before committing to load it, in race-free manner. The facility was requested by des@, who also tested the patch. I will commit this shortly unless somebody objects. diff --git a/include/dlfcn.h b/include/dlfcn.h index 794fde1..c508843 100644 --- a/include/dlfcn.h +++ b/include/dlfcn.h @@ -118,6 +118,7 @@ void *dlopen(const char *, int); void *dlsym(void * __restrict, const char * __restrict); =20 #if __BSD_VISIBLE +void *fdlopen(int, int); int dladdr(const void * __restrict, Dl_info * __restrict); dlfunc_t dlfunc(void * __restrict, const char * __restrict); int dlinfo(void * __restrict, int, void * __restrict); diff --git a/lib/libc/gen/Makefile.inc b/lib/libc/gen/Makefile.inc index 1e068a5..11d1a58 100644 --- a/lib/libc/gen/Makefile.inc +++ b/lib/libc/gen/Makefile.inc @@ -95,8 +95,8 @@ MLINKS+=3Ddirectory.3 closedir.3 directory.3 dirfd.3 dire= ctory.3 opendir.3 \ directory.3 fdopendir.3 \ directory.3 readdir.3 directory.3 readdir_r.3 directory.3 rewinddir.3 \ directory.3 seekdir.3 directory.3 telldir.3 -MLINKS+=3Ddlopen.3 dlclose.3 dlopen.3 dlerror.3 dlopen.3 dlfunc.3 \ - dlopen.3 dlsym.3 +MLINKS+=3Ddlopen.3 fdlopen.3 dlopen.3 dlclose.3 dlopen.3 dlerror.3 \ + dlopen.3 dlfunc.3 dlopen.3 dlsym.3 MLINKS+=3Derr.3 err_set_exit.3 err.3 err_set_file.3 err.3 errc.3 err.3 err= x.3 \ err.3 verr.3 err.3 verrc.3 err.3 verrx.3 err.3 vwarn.3 err.3 vwarnc.3 \ err.3 vwarnx.3 err.3 warnc.3 err.3 warn.3 err.3 warnx.3 diff --git a/lib/libc/gen/Symbol.map b/lib/libc/gen/Symbol.map index adc5964..e00e746 100644 --- a/lib/libc/gen/Symbol.map +++ b/lib/libc/gen/Symbol.map @@ -382,6 +382,7 @@ FBSD_1.2 { }; =20 FBSD_1.3 { + fdlopen; __FreeBSD_libc_enter_restricted_mode; }; =20 diff --git a/lib/libc/gen/dlfcn.c b/lib/libc/gen/dlfcn.c index b109cc9..7be9f87 100644 --- a/lib/libc/gen/dlfcn.c +++ b/lib/libc/gen/dlfcn.c @@ -147,6 +147,15 @@ dl_iterate_phdr(int (*callback)(struct dl_phdr_info *,= size_t, void *), return 0; } =20 +#pragma weak fdlopen +void * +fdlopen(int fd, int mode) +{ + + _rtld_error(sorry); + return NULL; +} + #pragma weak _rtld_atfork_pre void _rtld_atfork_pre(int *locks) diff --git a/lib/libc/gen/dlopen.3 b/lib/libc/gen/dlopen.3 index 3da9b6e..871808b 100644 --- a/lib/libc/gen/dlopen.3 +++ b/lib/libc/gen/dlopen.3 @@ -32,11 +32,12 @@ .\" @(#) dlopen.3 1.6 90/01/31 SMI .\" $FreeBSD$ .\" -.Dd July 7, 2009 +.Dd December 21, 2011 .Dt DLOPEN 3 .Os .Sh NAME .Nm dlopen , +.Nm fdlopen , .Nm dlsym , .Nm dlfunc , .Nm dlerror , @@ -49,6 +50,8 @@ .Ft void * .Fn dlopen "const char *path" "int mode" .Ft void * +.Fn fdlopen "int fd" "int mode" +.Ft void * .Fn dlsym "void * restrict handle" "const char * restrict symbol" .Ft dlfunc_t .Fn dlfunc "void * restrict handle" "const char * restrict symbol" @@ -164,6 +167,32 @@ be interrogated with .Fn dlerror . .Pp The +.Fn fdlopen +function is similar to +.Fn dlopen , +but it takes the file descriptor argument +.Fa fd , +which is used for the file operations needed to load an object +into the address space. +The file descriptor +.Fa fd +is not closed by the function regardless a result of execution. +The +.Fa fd +argument -1 is interpreted as a reference to the main +executable of the process, similar to +.Va NULL +value for +.Fa name +for +.Fn dlopen . +The +.Fn fdlopen +function can be used by the code that needs to perform +additional checks on the loaded objects, to prevent races with +symlinking or renames. +.Pp +The .Fn dlsym function returns the address binding of the symbol described in the null-terminated @@ -354,6 +383,7 @@ option to the C language compiler. .Sh ERRORS The .Fn dlopen , +.Fn fdlopen , .Fn dlsym , and .Fn dlfunc diff --git a/libexec/rtld-elf/Symbol.map b/libexec/rtld-elf/Symbol.map index 28b24a1..9ad6251 100644 --- a/libexec/rtld-elf/Symbol.map +++ b/libexec/rtld-elf/Symbol.map @@ -18,6 +18,10 @@ FBSD_1.0 { __tls_get_addr; }; =20 +FBSD_1.3 { + fdlopen; +}; + FBSDprivate_1.0 { _rtld_thread_init; _rtld_allocate_tls; diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c index 6cddd15..e1b813a 100644 --- a/libexec/rtld-elf/rtld.c +++ b/libexec/rtld-elf/rtld.c @@ -83,7 +83,7 @@ static void digest_dynamic2(Obj_Entry *, const Elf_Dyn *,= const Elf_Dyn *); static void digest_dynamic(Obj_Entry *, int); static Obj_Entry *digest_phdr(const Elf_Phdr *, int, caddr_t, const char *= ); static Obj_Entry *dlcheck(void *); -static Obj_Entry *dlopen_object(const char *name, Obj_Entry *refobj, +static Obj_Entry *dlopen_object(const char *name, int fd, Obj_Entry *refob= j, int lo_flags, int mode); static Obj_Entry *do_load_object(int, const char *, char *, struct stat *,= int); static int do_search_info(const Obj_Entry *obj, int, struct dl_serinfo *); @@ -103,7 +103,7 @@ static void load_filtees(Obj_Entry *, int flags, RtldLo= ckState *); static void unload_filtees(Obj_Entry *); static int load_needed_objects(Obj_Entry *, int); static int load_preload_objects(void); -static Obj_Entry *load_object(const char *, const Obj_Entry *, int); +static Obj_Entry *load_object(const char *, int fd, const Obj_Entry *, int= ); static void map_stacks_exec(RtldLockState *); static Obj_Entry *obj_from_addr(const void *); static void objlist_call_fini(Objlist *, Obj_Entry *, RtldLockState *); @@ -120,6 +120,7 @@ static int resolve_objects_ifunc(Obj_Entry *first, bool= bind_now, RtldLockState *lockstate); static int rtld_dirname(const char *, char *); static int rtld_dirname_abs(const char *, char *); +static void *rtld_dlopen(const char *name, int fd, int mode); static void rtld_exit(void); static char *search_library_path(const char *, const char *); static const void **get_program_var_addr(const char *, RtldLockState *); @@ -1543,7 +1544,7 @@ load_filtee1(Obj_Entry *obj, Needed_Entry *needed, in= t flags) { =20 for (; needed !=3D NULL; needed =3D needed->next) { - needed->obj =3D dlopen_object(obj->strtab + needed->name, obj, + needed->obj =3D dlopen_object(obj->strtab + needed->name, -1, obj, flags, ((ld_loadfltr || obj->z_loadfltr) ? RTLD_NOW : RTLD_LAZY) | RTLD_LOCAL); } @@ -1567,7 +1568,7 @@ process_needed(Obj_Entry *obj, Needed_Entry *needed, = int flags) Obj_Entry *obj1; =20 for (; needed !=3D NULL; needed =3D needed->next) { - obj1 =3D needed->obj =3D load_object(obj->strtab + needed->name, obj, + obj1 =3D needed->obj =3D load_object(obj->strtab + needed->name, -1, obj, flags & ~RTLD_LO_NOLOAD); if (obj1 =3D=3D NULL && !ld_tracing && (flags & RTLD_LO_FILTEES) =3D=3D 0) return (-1); @@ -1614,7 +1615,7 @@ load_preload_objects(void) =20 savech =3D p[len]; p[len] =3D '\0'; - if (load_object(p, NULL, 0) =3D=3D NULL) + if (load_object(p, -1, NULL, 0) =3D=3D NULL) return -1; /* XXX - cleanup */ p[len] =3D savech; p +=3D len; @@ -1624,6 +1625,13 @@ load_preload_objects(void) return 0; } =20 +static const char * +printable_path(const char *path) +{ + + return (path =3D=3D NULL ? "" : path); +} + /* * Load a shared object into memory, if it is not already loaded. * @@ -1631,36 +1639,51 @@ load_preload_objects(void) * on failure. */ static Obj_Entry * -load_object(const char *name, const Obj_Entry *refobj, int flags) +load_object(const char *name, int fd1, const Obj_Entry *refobj, int flags) { Obj_Entry *obj; - int fd =3D -1; + int fd; struct stat sb; char *path; =20 - for (obj =3D obj_list->next; obj !=3D NULL; obj =3D obj->next) - if (object_match_name(obj, name)) - return obj; + if (name !=3D NULL) { + for (obj =3D obj_list->next; obj !=3D NULL; obj =3D obj->next) { + if (object_match_name(obj, name)) + return (obj); + } =20 - path =3D find_library(name, refobj); - if (path =3D=3D NULL) - return NULL; + path =3D find_library(name, refobj); + if (path =3D=3D NULL) + return (NULL); + } else + path =3D NULL; =20 /* - * If we didn't find a match by pathname, open the file and check - * again by device and inode. This avoids false mismatches caused - * by multiple links or ".." in pathnames. + * If we didn't find a match by pathname, or the name is not + * supplied, open the file and check again by device and inode. + * This avoids false mismatches caused by multiple links or ".." + * in pathnames. * * To avoid a race, we open the file and use fstat() rather than * using stat(). */ - if ((fd =3D open(path, O_RDONLY)) =3D=3D -1) { - _rtld_error("Cannot open \"%s\"", path); - free(path); - return NULL; + fd =3D -1; + if (fd1 =3D=3D -1) { + if ((fd =3D open(path, O_RDONLY)) =3D=3D -1) { + _rtld_error("Cannot open \"%s\"", path); + free(path); + return (NULL); + } + } else { + fd =3D dup(fd1); + if (fd =3D=3D -1) { + _rtld_error("Cannot dup fd"); + free(path); + return (NULL); + } } if (fstat(fd, &sb) =3D=3D -1) { - _rtld_error("Cannot fstat \"%s\"", path); + _rtld_error("Cannot fstat \"%s\"", printable_path(path)); close(fd); free(path); return NULL; @@ -1668,7 +1691,7 @@ load_object(const char *name, const Obj_Entry *refobj= , int flags) for (obj =3D obj_list->next; obj !=3D NULL; obj =3D obj->next) if (obj->ino =3D=3D sb.st_ino && obj->dev =3D=3D sb.st_dev) break; - if (obj !=3D NULL) { + if (obj !=3D NULL && name !=3D NULL) { object_add_name(obj, name); free(path); close(fd); @@ -1702,20 +1725,25 @@ do_load_object(int fd, const char *name, char *path= , struct stat *sbp, */ if (dangerous_ld_env) { if (fstatfs(fd, &fs) !=3D 0) { - _rtld_error("Cannot fstatfs \"%s\"", path); - return NULL; + _rtld_error("Cannot fstatfs \"%s\"", printable_path(path)); + return NULL; } if (fs.f_flags & MNT_NOEXEC) { _rtld_error("Cannot execute objects on %s\n", fs.f_mntonname); return NULL; } } - dbg("loading \"%s\"", path); - obj =3D map_object(fd, path, sbp); + dbg("loading \"%s\"", printable_path(path)); + obj =3D map_object(fd, printable_path(path), sbp); if (obj =3D=3D NULL) return NULL; =20 - object_add_name(obj, name); + /* + * If DT_SONAME is present in the object, digest_dynamic2 already + * added it to the object names. + */ + if (name !=3D NULL) + object_add_name(obj, name); obj->path =3D path; digest_dynamic(obj, 0); if (obj->z_noopen && (flags & (RTLD_LO_DLOPEN | RTLD_LO_TRACE)) =3D=3D @@ -2211,6 +2239,20 @@ dllockinit(void *context, void * dlopen(const char *name, int mode) { + + return (rtld_dlopen(name, -1, mode)); +} + +void * +fdlopen(int fd, int mode) +{ + + return (rtld_dlopen(NULL, fd, mode)); +} + +static void * +rtld_dlopen(const char *name, int fd, int mode) +{ RtldLockState lockstate; int lo_flags; =20 @@ -2231,7 +2273,7 @@ dlopen(const char *name, int mode) if (ld_tracing !=3D NULL) lo_flags |=3D RTLD_LO_TRACE; =20 - return (dlopen_object(name, obj_main, lo_flags, + return (dlopen_object(name, fd, obj_main, lo_flags, mode & (RTLD_MODEMASK | RTLD_GLOBAL))); } =20 @@ -2246,7 +2288,8 @@ dlopen_cleanup(Obj_Entry *obj) } =20 static Obj_Entry * -dlopen_object(const char *name, Obj_Entry *refobj, int lo_flags, int mode) +dlopen_object(const char *name, int fd, Obj_Entry *refobj, int lo_flags, + int mode) { Obj_Entry **old_obj_tail; Obj_Entry *obj; @@ -2261,11 +2304,11 @@ dlopen_object(const char *name, Obj_Entry *refobj, = int lo_flags, int mode) =20 old_obj_tail =3D obj_tail; obj =3D NULL; - if (name =3D=3D NULL) { + if (name =3D=3D NULL && fd =3D=3D -1) { obj =3D obj_main; obj->refcount++; } else { - obj =3D load_object(name, refobj, lo_flags); + obj =3D load_object(name, fd, refobj, lo_flags); } =20 if (obj) { --6PhDeOF0ICWr3i50 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk78KBwACgkQC3+MBN1Mb4jyugCeJPLComnZDVYsVEPKsnKoHi97 H6IAnAlVIlJFT+OJdot/n9I7EH+701lZ =Sgxd -----END PGP SIGNATURE----- --6PhDeOF0ICWr3i50-- From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 11:38:33 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB1B01065679; Thu, 29 Dec 2011 11:38:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 621E38FC08; Thu, 29 Dec 2011 11:38:32 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBTBcT12048584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 29 Dec 2011 13:38:30 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBTBcTn5017055; Thu, 29 Dec 2011 13:38:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBTBcTbJ017054; Thu, 29 Dec 2011 13:38:29 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 29 Dec 2011 13:38:29 +0200 From: Kostik Belousov To: Xin LI Message-ID: <20111229113829.GF50300@deviant.kiev.zoral.com.ua> References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="e6MacjH9Q9s0loUc" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: kan@freebsd.org, arch@freebsd.org, des@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 11:38:34 -0000 --e6MacjH9Q9s0loUc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 29, 2011 at 03:15:09AM -0800, Xin LI wrote: > Hi, >=20 > On Thu, Dec 29, 2011 at 12:43 AM, Kostik Belousov w= rote: > > Hello, > > below is the patch to add fdlopen(3) function to rtld. It allows to load > > shared object referenced by the opened file descriptor. Intent is to al= low > > the caller to do some additional checks on the object before committing > > to load it, in race-free manner. > > > > The facility was requested by des@, who also tested the patch. > > > > I will commit this shortly unless somebody objects. >=20 > Will this prevent e.g. writes to the .so file after open, but before fdlo= pen()? How can it ? Even in theory ? --e6MacjH9Q9s0loUc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk78UTUACgkQC3+MBN1Mb4hM4gCgmsfNZYsBJfDhb/99S3E6lrZi SwwAn24CEu9AFTyhdNjkr0spYFrJ64L0 =WUjy -----END PGP SIGNATURE----- --e6MacjH9Q9s0loUc-- From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 11:41:32 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E69051065678; Thu, 29 Dec 2011 11:41:32 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 936EA8FC1E; Thu, 29 Dec 2011 11:41:32 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so13975833obb.13 for ; Thu, 29 Dec 2011 03:41:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RaMXO6qu2S/7xD54q2FBYoQ9cXztBePMWc7WzLsW2Ss=; b=IfmBmE+1OZ26fMaxVho3vKsHpCmXN75zMk7c/X2m+/BX/Zif9yGgBljynoBS2C3o98 0uDZXozdGqhlx6tDX5P/7wiwd9zojqSz6LvHkXUSZTGB5qHkGhEq9UfnQa24OFKgSCK1 zPbzbJg424KqKcQ+z5GQ5igoG+Z95p8vK5Moc= MIME-Version: 1.0 Received: by 10.182.42.37 with SMTP id k5mr31046226obl.40.1325157309882; Thu, 29 Dec 2011 03:15:09 -0800 (PST) Received: by 10.182.67.163 with HTTP; Thu, 29 Dec 2011 03:15:09 -0800 (PST) In-Reply-To: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> Date: Thu, 29 Dec 2011 03:15:09 -0800 Message-ID: From: Xin LI To: Kostik Belousov Content-Type: text/plain; charset=UTF-8 Cc: kan@freebsd.org, arch@freebsd.org, des@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 11:41:33 -0000 Hi, On Thu, Dec 29, 2011 at 12:43 AM, Kostik Belousov wrote: > Hello, > below is the patch to add fdlopen(3) function to rtld. It allows to load > shared object referenced by the opened file descriptor. Intent is to allow > the caller to do some additional checks on the object before committing > to load it, in race-free manner. > > The facility was requested by des@, who also tested the patch. > > I will commit this shortly unless somebody objects. Will this prevent e.g. writes to the .so file after open, but before fdlopen()? Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 15:31:33 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BC541065704; Thu, 29 Dec 2011 15:31:33 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from mx0.hoeg.nl (mx0.hoeg.nl [IPv6:2a01:4f8:101:5343::aa]) by mx1.freebsd.org (Postfix) with ESMTP id E595E8FC12; Thu, 29 Dec 2011 15:31:32 +0000 (UTC) Received: by mx0.hoeg.nl (Postfix, from userid 1000) id 5B94A2A28CC3; Thu, 29 Dec 2011 16:31:32 +0100 (CET) Date: Thu, 29 Dec 2011 16:31:32 +0100 From: Ed Schouten To: freebsd-current@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Message-ID: <20111229153132.GI1895@hoeg.nl> References: <20111226101040.GA6361@freebsd.org> <20111227010449.GA6244@twoflower.paeps.cx> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3W6riq+uV6J42Ub" Content-Disposition: inline In-Reply-To: <20111227010449.GA6244@twoflower.paeps.cx> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 15:31:33 -0000 --u3W6riq+uV6J42Ub Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Philip Paeps , 20111227 02:04: > Wouldn't it be better to set WARNS=3Dx rather than WERROR=3D? Not entirely related to the discussion, but worth mentioning: WARNS is only used by -- not . --=20 Ed Schouten WWW: http://80386.nl/ --u3W6riq+uV6J42Ub Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQIcBAEBAgAGBQJO/IfUAAoJEG5e2P40kaK7RiQP/3ki4pMwhoej8ptchr8p6MGX QBNW0qsbwzXShl8h4iHU9CDg7y5Nx1npfkT7aMqtDRCuX0E6rxovsGo77Em1amwG LilQT8SYoxBtEPB9fLXH/ul3GCzvBM3gjDJHZKB+zLILHLCfn8vUYJww0orfrDFF WCH2wBQhKD04nd0rzauU2LQiIAj145N10emvHmH6poDDlbObJX1jSox6nEq1+gu6 NNSqOKuFInCsjc6iRI8QWbmDgoMWhPAlHY6SfTXKOug8dg9CKCIO6noBURIsNa37 FNAk6DXbMIStf4uMtQIb1mDQ4H66qKLDzgDHOX8TnKN4bnK26H6me0yKe2XCBVxv x0+MaWQ87zwRu4Xz4EZmP89rfawufIHSleL6/MVc/9i6OIRC1IGRsXRRh6ppkkmb /AWb1vsY+jwFFN7NJt4Y2DNSuMpj3pGnJEjqtiLGHjTODWa4nUXnqCX/fm2nZy9b b6U1G+F51olPke/9MnrqgO7F1qaBNIWnyAozZ0Vn/MBFLaySB15LIyedKwQNjYuG PRo2FTs69zEJeIOahguyjz7Wgj+nmNp4xFI5K7aECCriJUUAnoi7SOWxYfyQi0eu jGWjA6/2Okjg3rk0nOC1nG3cfYNb8trFx2G7fnHu07V1M56eGvntwiJ48mYp6I5t Lte52ACz1O2urUCJgDAw =LjXm -----END PGP SIGNATURE----- --u3W6riq+uV6J42Ub-- From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 18:04:41 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9CC4106566B; Thu, 29 Dec 2011 18:04:41 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 4A5FC8FC13; Thu, 29 Dec 2011 18:04:41 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 086C56F42; Thu, 29 Dec 2011 18:04:34 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id F349E877C; Thu, 29 Dec 2011 19:04:33 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Xin LI References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> Date: Thu, 29 Dec 2011 19:04:33 +0100 In-Reply-To: (Xin LI's message of "Thu, 29 Dec 2011 03:15:09 -0800") Message-ID: <86fwg3ntji.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , arch@freebsd.org, kan@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 18:04:41 -0000 Xin LI writes: > Will this prevent e.g. writes to the .so file after open, but before > fdlopen()? The latest version of OpenPAM checks the ownership and permissions of modules before it loads them; it will not load modules that are writable by anyone except root and the process's euid. This patch prevents an attacker from switching the .so file between the ownership checks and the dlopen(3) call. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 18:20:42 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A748A106566B; Thu, 29 Dec 2011 18:20:42 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 55A9A8FC19; Thu, 29 Dec 2011 18:20:42 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so14563067obb.13 for ; Thu, 29 Dec 2011 10:20:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6UFGoWNPCkywx7Z5WV8JXZ9Rj+cHEt16ELED2FDuZbk=; b=MwPPxH6Kno9ElMz7fWFtrH7UY3glfJkstJmb+VlLV78XocfcXAOFn0OWR/cDhu2h3C QvzInZkVgAWcNuYTQ2JItFF03PIHha8R1Nv1O4j3jRFHwFiGfF9rFcwacPlwuZzF6I8c rtNCwhHXPHtlzl+urVfvK++dxesOfUz2TPYLM= MIME-Version: 1.0 Received: by 10.182.76.134 with SMTP id k6mr23707183obw.10.1325182841890; Thu, 29 Dec 2011 10:20:41 -0800 (PST) Received: by 10.182.67.163 with HTTP; Thu, 29 Dec 2011 10:20:41 -0800 (PST) In-Reply-To: <20111229113829.GF50300@deviant.kiev.zoral.com.ua> References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> <20111229113829.GF50300@deviant.kiev.zoral.com.ua> Date: Thu, 29 Dec 2011 10:20:41 -0800 Message-ID: From: Xin LI To: Kostik Belousov Content-Type: text/plain; charset=UTF-8 Cc: kan@freebsd.org, arch@freebsd.org, des@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 18:20:42 -0000 On Thu, Dec 29, 2011 at 3:38 AM, Kostik Belousov wrote: > On Thu, Dec 29, 2011 at 03:15:09AM -0800, Xin LI wrote: >> Hi, >> >> On Thu, Dec 29, 2011 at 12:43 AM, Kostik Belousov wrote: >> > Hello, >> > below is the patch to add fdlopen(3) function to rtld. It allows to load >> > shared object referenced by the opened file descriptor. Intent is to allow >> > the caller to do some additional checks on the object before committing >> > to load it, in race-free manner. >> > >> > The facility was requested by des@, who also tested the patch. >> > >> > I will commit this shortly unless somebody objects. >> >> Will this prevent e.g. writes to the .so file after open, but before fdlopen()? > > How can it ? Even in theory ? e.g. process 1 open(), check for permissions, etc., then process 2 open() for write, write something, then process proceed with fdlopen()? (We do not have mandatory "lock" on file handles to prevent writing I think?). But no, this is not a "major" concern and yes, I think fdlopen() is something good to have. Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die From owner-freebsd-arch@FreeBSD.ORG Thu Dec 29 18:31:37 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD254106564A for ; Thu, 29 Dec 2011 18:31:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id B6CD68FC17 for ; Thu, 29 Dec 2011 18:31:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 70B6C46B1A for ; Thu, 29 Dec 2011 13:31:37 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 02304B91E for ; Thu, 29 Dec 2011 13:31:37 -0500 (EST) From: John Baldwin To: arch@freebsd.org Date: Thu, 29 Dec 2011 13:31:34 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201112291331.34671.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 29 Dec 2011 13:31:37 -0500 (EST) Cc: Subject: Update 'make cscope' to handle MACHINE_CPUARCH and 'x86' X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Dec 2011 18:31:37 -0000 This patch updates 'make cscope' to automatically add MACHINE_CPUARCH to CSCOPE_ARCHDIR if it differs from MACHINE. It also adds a special case to include the 'x86' directory for targets that use sys/x86. Index: Makefile =================================================================== --- Makefile (revision 228954) +++ Makefile (working copy) @@ -13,11 +13,19 @@ CSCOPEDIRS= boot bsm cam cddl compat conf contrib netgraph netinet netinet6 netipsec netipx netnatm netncp \ netsmb nfs nfsclient nfsserver nlm opencrypto \ pci rpc security sys ufs vm xdr xen ${CSCOPE_ARCHDIR} +.if !defined(CSCOPE_ARCHDIR) .if defined(ALL_ARCH) -CSCOPE_ARCHDIR ?= amd64 arm i386 ia64 mips pc98 powerpc sparc64 x86 +CSCOPE_ARCHDIR = amd64 arm i386 ia64 mips pc98 powerpc sparc64 x86 .else -CSCOPE_ARCHDIR ?= ${MACHINE} +CSCOPE_ARCHDIR = ${MACHINE} +.if ${MACHINE} != ${MACHINE_CPUARCH} +CSCOPE_ARCHDIR += ${MACHINE_CPUARCH} .endif +.if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64" +CSCOPE_ARCHDIR += x86 +.endif +.endif +.endif # Loadable kernel modules -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 13:06:50 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D70E1065672; Fri, 30 Dec 2011 13:06:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 092498FC13; Fri, 30 Dec 2011 13:06:50 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:408b:3bdd:8466:f24f] (unknown [IPv6:2001:7b8:3a7:0:408b:3bdd:8466:f24f]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 48BF75C37; Fri, 30 Dec 2011 14:06:49 +0100 (CET) Message-ID: <4EFDB76C.1030901@FreeBSD.org> Date: Fri, 30 Dec 2011 14:06:52 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: Alexander Best References: <20111226101040.GA6361@freebsd.org> In-Reply-To: <20111226101040.GA6361@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-toolchain@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 13:06:50 -0000 On 2011-12-26 11:10, Alexander Best wrote: > that's why i'd like to propose the following patch. i ran a full tinderbox run > against r228878 and it suceeded. Did you also try this with clang? For example the xfs module alone gets a whole slew of warnings, which would be fatal if WERROR= was removed: sys/gnu/fs/xfs/xfs_dir2_block.c:1149:17: warning: array index of '1' indexes past the end of an array (that contains 1 element) [-Warray-bounds] dep->name[0] = dep->name[1] = '.'; ^ ~ sys/gnu/fs/xfs/xfs_dir2_data.h:90:2: note: array 'name' declared here __uint8_t name[1]; /* name bytes, no null */ ^ 1 warning generated. sys/gnu/fs/xfs/xfs_dir2_sf.c:117:27: warning: array index of '1' indexes past the end of an array (that contains 1 element) [-Warray-bounds] dep->name[0] == '.' && dep->name[1] == '.'; ^ ~ sys/gnu/fs/xfs/xfs_dir2_data.h:90:2: note: array 'name' declared here __uint8_t name[1]; /* name bytes, no null */ ^ sys/gnu/fs/xfs/xfs_dir2_sf.c:237:28: warning: array index of '1' indexes past the end of an array (that contains 1 element) [-Warray-bounds] dep->name[0] == '.' && dep->name[1] == '.') ^ ~ sys/gnu/fs/xfs/xfs_dir2_data.h:90:2: note: array 'name' declared here __uint8_t name[1]; /* name bytes, no null */ ^ 2 warnings generated. sys/gnu/fs/xfs/xfs_vfsops.c:1958:19: warning: format string is not a string literal (potentially insecure) [-Wformat-security] sbuf_printf(m, xfs_infop->str); ^~~~~~~~~~~~~~ 1 warning generated. sys/gnu/fs/xfs/FreeBSD/xfs_super.c:257:9: warning: format string is not a string literal (potentially insecure) [-Wformat-security] printf(message); ^~~~~~~ 1 warning generated. sys/gnu/fs/xfs/FreeBSD/xfs_stats.c:68:32: warning: format string is not a string literal (potentially insecure) [-Wformat-security] len += sprintf(buffer + len, xstats[i].desc); ^~~~~~~~~~~~~~ sys/gnu/fs/xfs/FreeBSD/xfs_stats.c:99:33: warning: if statement has empty body [-Wempty-body] if (&xfs_read_xfsstats != NULL); ^ 2 warnings generated. sys/gnu/fs/xfs/FreeBSD/xfs_ioctl.c:1252:10: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] error = error; ~~~~~ ^ ~~~~~ sys/gnu/fs/xfs/FreeBSD/xfs_ioctl.c:1290:9: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] error = error; ~~~~~ ^ ~~~~~ sys/gnu/fs/xfs/FreeBSD/xfs_ioctl.c:1299:10: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] error = error; ~~~~~ ^ ~~~~~ sys/gnu/fs/xfs/FreeBSD/xfs_ioctl.c:1350:9: warning: explicitly assigning a variable of type 'int' to itself [-Wself-assign] error = error; ~~~~~ ^ ~~~~~ 4 warnings generated. Most of the warnings should probably be fixed, but as this is mainly contrib code, and GPL to boot, it makes not much sense to do so. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 13:12:32 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D39F106566C; Fri, 30 Dec 2011 13:12:32 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 270598FC08; Fri, 30 Dec 2011 13:12:31 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id D03626172; Fri, 30 Dec 2011 13:12:29 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 78B258822; Fri, 30 Dec 2011 14:12:29 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Xin LI References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> <20111229113829.GF50300@deviant.kiev.zoral.com.ua> Date: Fri, 30 Dec 2011 14:12:29 +0100 In-Reply-To: (Xin LI's message of "Thu, 29 Dec 2011 10:20:41 -0800") Message-ID: <867h1enqyq.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Kostik Belousov , arch@freebsd.org, kan@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 13:12:32 -0000 Xin LI writes: > Kostik Belousov writes: > > Xin LI writes: > > > Will this prevent e.g. writes to the .so file after open, but > > > before fdlopen()? > > How can it ? Even in theory ? > e.g. process 1 open(), check for permissions, etc., then process 2 > open() for write, write something, then process proceed with > fdlopen()? (We do not have mandatory "lock" on file handles to > prevent writing I think?). I think Kostik meant "how could you, even in theory, prevent this from happening", and the answer is that you can't. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 14:08:35 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5270106564A; Fri, 30 Dec 2011 14:08:35 +0000 (UTC) (envelope-from theraven@theravensnest.org) Received: from theravensnest.org (theravensnest.org [109.169.23.128]) by mx1.freebsd.org (Postfix) with ESMTP id 7C5F08FC0C; Fri, 30 Dec 2011 14:08:35 +0000 (UTC) Received: from [192.168.0.2] (cpc2-cwma5-0-0-cust875.7-3.cable.virginmedia.com [86.11.39.108]) (authenticated bits=0) by theravensnest.org (8.14.4/8.14.4) with ESMTP id pBUDSpdF032333 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Fri, 30 Dec 2011 13:28:51 GMT (envelope-from theraven@theravensnest.org) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: David Chisnall In-Reply-To: <4EFDB76C.1030901@FreeBSD.org> Date: Fri, 30 Dec 2011 13:28:51 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20111226101040.GA6361@freebsd.org> <4EFDB76C.1030901@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1251.1) Cc: Alexander Best , freebsd-toolchain@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 14:08:36 -0000 On 30 Dec 2011, at 13:06, Dimitry Andric wrote: > sys/gnu/fs/xfs/xfs_dir2_block.c:1149:17: warning: array index of '1' = indexes past the end of an array (that contains 1 element) = [-Warray-bounds] I recall some discussion of this warning on the clang list a few months = ago, and I believe that it should now only appear if you are compiling = in a C99 or C11 dialect mode (the rationale is that any variable-length = structures in C99 should be using a zero-sized array as the final = element, while C89 lacked any ability to do this). =20 I suspect a lot of similar warnings are caused by the difference in = default dialects between clang and gcc. Adding an explicit -std=3Dc89 = to the cflags any modules that are using an archaic dialect of C may = silence a lot of these... David= From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 14:28:04 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E43C106566B; Fri, 30 Dec 2011 14:28:04 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 032448FC08; Fri, 30 Dec 2011 14:28:03 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBUEQeM1028085 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 30 Dec 2011 16:26:40 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBUEQeeZ021326; Fri, 30 Dec 2011 16:26:40 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBUEQeDn021325; Fri, 30 Dec 2011 16:26:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 30 Dec 2011 16:26:40 +0200 From: Kostik Belousov To: Dag-Erling Sm??rgrav Message-ID: <20111230142640.GO50300@deviant.kiev.zoral.com.ua> References: <20111229084308.GD50300@deviant.kiev.zoral.com.ua> <20111229113829.GF50300@deviant.kiev.zoral.com.ua> <867h1enqyq.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Ly7k/MJGvmX3BJ+w" Content-Disposition: inline In-Reply-To: <867h1enqyq.fsf@ds4.des.no> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: kan@freebsd.org, arch@freebsd.org, Xin LI Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 14:28:04 -0000 --Ly7k/MJGvmX3BJ+w Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 30, 2011 at 02:12:29PM +0100, Dag-Erling Sm??rgrav wrote: > Xin LI writes: > > Kostik Belousov writes: > > > Xin LI writes: > > > > Will this prevent e.g. writes to the .so file after open, but > > > > before fdlopen()? > > > How can it ? Even in theory ? > > e.g. process 1 open(), check for permissions, etc., then process 2 > > open() for write, write something, then process proceed with > > fdlopen()? (We do not have mandatory "lock" on file handles to > > prevent writing I think?). >=20 > I think Kostik meant "how could you, even in theory, prevent this from > happening", and the answer is that you can't. Sorry if it was not clear enough. BSD is not SysV, we do not have mandatory locking. Some people consider it a feature. In principle, fdlopen() can take additional argument struct timespec mtim, that would be compared with st_mtim from fstat(2) right before calling constructors from the dso. The passed mtim shall be taken from fstat(2) before the application code starts its checks. But this is completely ridiculous and pointless. Presented use case for fdlopen(3) is valid and useful IMO. --Ly7k/MJGvmX3BJ+w Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk79yiAACgkQC3+MBN1Mb4g5fQCfVOFIgXa/g6S0gJKj2Snc+bZ8 y4QAn1CxDPM3DrF93dhyiJG5GNC2zXJp =R/tZ -----END PGP SIGNATURE----- --Ly7k/MJGvmX3BJ+w-- From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 16:00:43 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B0E6106564A for ; Fri, 30 Dec 2011 16:00:43 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2BBAD8FC08 for ; Fri, 30 Dec 2011 16:00:41 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 6B0AC5DD1; Fri, 30 Dec 2011 16:00:40 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBUG0dH7029242; Fri, 30 Dec 2011 16:00:39 GMT (envelope-from phk@phk.freebsd.dk) To: Kostik Belousov From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 30 Dec 2011 16:26:40 +0200." <20111230142640.GO50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 30 Dec 2011 16:00:39 +0000 Message-ID: <29241.1325260839@critter.freebsd.dk> Cc: kan@freebsd.org, Dag-Erling Sm??rgrav , Xin LI , arch@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 16:00:43 -0000 In message <20111230142640.GO50300@deviant.kiev.zoral.com.ua>, Kostik Belousov writes: >Presented use case for fdlopen(3) is valid and useful IMO. I agree, I even have a similar use-case in Varnish. Has anybody racked their brains to make sure this doesn't have security implications ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 16:48:23 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79DC6106564A; Fri, 30 Dec 2011 16:48:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id DAE218FC08; Fri, 30 Dec 2011 16:48:22 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBUGksXn040752 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 30 Dec 2011 18:46:54 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBUGksAR021712; Fri, 30 Dec 2011 18:46:54 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBUGkrCN021711; Fri, 30 Dec 2011 18:46:53 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 30 Dec 2011 18:46:53 +0200 From: Kostik Belousov To: Poul-Henning Kamp Message-ID: <20111230164653.GQ50300@deviant.kiev.zoral.com.ua> References: <20111230142640.GO50300@deviant.kiev.zoral.com.ua> <29241.1325260839@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JIZzZL2Ew2M9i6mq" Content-Disposition: inline In-Reply-To: <29241.1325260839@critter.freebsd.dk> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: kan@freebsd.org, Dag-Erling Sm??rgrav , Xin LI , arch@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 16:48:23 -0000 --JIZzZL2Ew2M9i6mq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 30, 2011 at 04:00:39PM +0000, Poul-Henning Kamp wrote: > In message <20111230142640.GO50300@deviant.kiev.zoral.com.ua>, Kostik Bel= ousov=20 > writes: >=20 > >Presented use case for fdlopen(3) is valid and useful IMO. >=20 > I agree, I even have a similar use-case in Varnish. >=20 > Has anybody racked their brains to make sure this doesn't have security > implications ? I am wondering what kind of security consequences you have in mind ? My initial concern with the patch was the lack of the name supplied for the loaded dso. But, the rtld already adds DT_SONAME to the names of the object, and later it checks for duplicates using vnode identity returned by fstat(2), so I removed the name argument from API. --JIZzZL2Ew2M9i6mq Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk796v0ACgkQC3+MBN1Mb4g8UQCglkXyv0/QLWbQqkumCNGImiHo 5LYAoJBgMhb7R8U9I0n+709OzlLCxwCj =U4Bd -----END PGP SIGNATURE----- --JIZzZL2Ew2M9i6mq-- From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 16:52:07 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD107106564A; Fri, 30 Dec 2011 16:52:07 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 5C5F58FC0A; Fri, 30 Dec 2011 16:52:07 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 28F015DAC; Fri, 30 Dec 2011 16:52:06 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBUGq5Ab029459; Fri, 30 Dec 2011 16:52:05 GMT (envelope-from phk@phk.freebsd.dk) To: Kostik Belousov From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 30 Dec 2011 18:46:53 +0200." <20111230164653.GQ50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 30 Dec 2011 16:52:05 +0000 Message-ID: <29458.1325263925@critter.freebsd.dk> Cc: kan@freebsd.org, Dag-Erling Sm??rgrav , Xin LI , arch@freebsd.org Subject: Re: fdlopen(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 16:52:08 -0000 In message <20111230164653.GQ50300@deviant.kiev.zoral.com.ua>, Kostik Belousov writes: >> Has anybody racked their brains to make sure this doesn't have security >> implications ? > >I am wondering what kind of security consequences you have in mind ? I don't have anything specifically in mind, but adding a new way to communicate executable code always trigger my "is that really a good idea" detector... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Dec 30 17:04:58 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D0E51065670; Fri, 30 Dec 2011 17:04:58 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 283728FC19; Fri, 30 Dec 2011 17:04:58 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:408b:3bdd:8466:f24f] (unknown [IPv6:2001:7b8:3a7:0:408b:3bdd:8466:f24f]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id F0C3C5C37; Fri, 30 Dec 2011 18:04:56 +0100 (CET) Message-ID: <4EFDEF3B.4090200@FreeBSD.org> Date: Fri, 30 Dec 2011 18:04:59 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: David Chisnall References: <20111226101040.GA6361@freebsd.org> <4EFDB76C.1030901@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Best , freebsd-toolchain@freebsd.org, current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [rfc] removing/conditionalising WERROR= in Makefiles X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Dec 2011 17:04:58 -0000 On 2011-12-30 14:28, David Chisnall wrote: > On 30 Dec 2011, at 13:06, Dimitry Andric wrote: > >> sys/gnu/fs/xfs/xfs_dir2_block.c:1149:17: warning: array index of '1' indexes past the end of an array (that contains 1 element) [-Warray-bounds] > > I recall some discussion of this warning on the clang list a few months ago, and I believe that it should now only appear if you are compiling in a C99 or C11 dialect mode (the rationale is that any variable-length structures in C99 should be using a zero-sized array as the final element, while C89 lacked any ability to do this). Yes, that is perfectly fine, but the xfs code defines the struct in question as follows: /* * Active entry in a data block. Aligned to 8 bytes. * Tag appears as the last 2 bytes. */ typedef struct xfs_dir2_data_entry { xfs_ino_t inumber; /* inode number */ __uint8_t namelen; /* name length */ __uint8_t name[1]; /* name bytes, no null */ /* variable offset */ xfs_dir2_data_off_t tag; /* starting offset of us */ } xfs_dir2_data_entry_t; E.g there *is* an overrun, but maybe it was really supposed to be like that. Meanwhile, upstream has apparently caught on: http://oss.sgi.com/archives/xfs/2011-07/msg00024.html From owner-freebsd-arch@FreeBSD.ORG Sat Dec 31 23:39:43 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8C07106564A for ; Sat, 31 Dec 2011 23:39:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 67DF18FC0C for ; Sat, 31 Dec 2011 23:39:43 +0000 (UTC) Received: by vcbfk1 with SMTP id fk1so20130087vcb.13 for ; Sat, 31 Dec 2011 15:39:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=t4yAQ5itg2kcphdNAfKpPCL9g+DN+Mon7n3GEIN5R4M=; b=aD2KZ2YRJun6NE8rhIvdRD0sJNHSsDVpsm9zsoqmGKhJGL0FaWeZUP5jYw7yquP7Kp lE7hBHT03biQDn/CzltmU0kv00V5S8a4R+vItKF1tnNdYIcn8qAnPqskV7LcgZyTdaAh XOlkYAPO9PyaqHIfh5ygeZi4KX6d8Sxwqy5Zk= MIME-Version: 1.0 Received: by 10.220.213.137 with SMTP id gw9mr25285061vcb.3.1325374782629; Sat, 31 Dec 2011 15:39:42 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.36.5 with HTTP; Sat, 31 Dec 2011 15:39:42 -0800 (PST) Date: Sat, 31 Dec 2011 15:39:42 -0800 X-Google-Sender-Auth: AbFrr9DGqyqpOsPlGi6JzC-IXc8 Message-ID: From: Adrian Chadd To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Request for help: how do teach module building about kernel options? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2011 23:39:43 -0000 Hi, I need a bit of a hand with this. I'd like to be able to make the wlan and ath modules aware of kernel configuration options. For example, a kernel configuration with IEEE80211_SUPPORT_TDMA won't build a wlan module that'll run successfully, as wlan/Makefile doesn't know to suck in ieee80211_tdma.c . I could just wrap the whole file up in an #ifdef, but I'd like to try and instead only build / link that object in if it's needed. Similarly, the ath module currently builds everything, regardless of what options are currently enabled in the kernel configuration file. So it'll always build ar5210, ar5211, ar5212, ar5416, ar9001, ar9002 support, along with ath_rate_sample. Instead, I'd like to be able to specify which HAL objects to link in, much like how you can do this with "device ath_rfX" for the RF backends and "device ath_arX" for the chipset support. For the integrated SoC stuff, it'd be nice to only build a HAL which supports the relevant hardware, rather than having to suck it _all_ in. So, what kinds of evil ways can people dream up to achieve this? :) Adrian