From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 00:18:57 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E9D8014D; Sun, 3 Nov 2013 00:18:56 +0000 (UTC) (envelope-from ben@b1c1l1.com) Received: from lancer.b1c1l1.com (lancer.b1c1l1.com [IPv6:2607:f358:1a:1a:1000::]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D13F820B5; Sun, 3 Nov 2013 00:18:56 +0000 (UTC) Received: by lancer.b1c1l1.com (Postfix) with ESMTPSA id 1AD645C34; Sat, 2 Nov 2013 17:18:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=b1c1l1.com; s=default; t=1383437925; bh=j57xrpILG1Z6ZlopruGax9AavbtsxAezMPq2po7+MC8=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=64pg/LbOrx08boHNJ2pW5oHwynMh8r4peQ8Qtz0TY/Bxq0eLnA7oIuKufc7DNHz3M uWJwJ8i5S0EAeTav3k3VAttF7Z2MokWod8TeFoi1+VvR3pPL9Uy+IyPMA5mnovvIRZ LKIQ05WXfxcF7FCZb7EOeKvISE6pIcawcMwkT7p4= Date: Sat, 2 Nov 2013 17:18:40 -0700 From: Benjamin Lee To: Colin Percival Subject: Re: Automated submission of kernel panic reports Message-ID: <20131102171840.4505cb02@b1c1l1.com> In-Reply-To: <526F8EB3.1040205@freebsd.org> References: <526F8EB3.1040205@freebsd.org> X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.22; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/sWu=gIRINNgEnv.Emfppnvg"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 00:18:57 -0000 --Sig_/sWu=gIRINNgEnv.Emfppnvg Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 29 Oct 2013 03:32:19 -0700, Colin Percival w= rote: > Hi all, >=20 > I've written some code for automatically submitting kernel panic reports, > and I'd like some feedback before I place it into the ports tree. Just a style question, any reason not to place the file names into variables? Something like: --- panicmail.orig 2013-11-02 16:47:42.117858435 -0700 +++ panicmail 2013-11-02 17:06:15.388951568 -0700 @@ -38,19 +38,23 @@ panicmail_gather() { local tmpfile=3D`mktemp` || exit 1 + local kernel_file=3D`sysctl -n kern.bootfile` || exit 1 + local info_file=3D"${dumpdir}/info.$1" + local vmcore_file=3D"${dumpdir}/vmcore.$1" + local panicmail_file=3D"${dumpdir}/panicmail.$1" =20 # We want the dump header. - cat "${dumpdir}/info.$1" > "${dumpdir}/panicmail.$1" - echo >> "${dumpdir}/panicmail.$1" + cat "${info_file}" > "${panicmail_file}" + echo >> "${panicmail_file}" =20 # And we want a backtrace (we should be able to pipe the commands # directly into kgdb, but that doesn't work with our /bin/sh): - echo "Backtrace:" >> "${dumpdir}/panicmail.$1" + echo "Backtrace:" >> "${panicmail_file}" echo bt > ${tmpfile} echo quit >> ${tmpfile} - kgdb -q `sysctl -n kern.bootfile` "${dumpdir}/vmcore.$1" \ - < ${tmpfile} >> "${dumpdir}/panicmail.$1" 2> /dev/null - echo >> "${dumpdir}/panicmail.$1" + kgdb -q "${kernel_file}" "${vmcore_file}" \ + < ${tmpfile} >> "${panicmail_file}" 2> /dev/null + echo >> "${panicmail_file}" rm ${tmpfile} } =20 --=20 Benjamin Lee http://www.b1c1l1.com/ --Sig_/sWu=gIRINNgEnv.Emfppnvg Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJSdZZjAAoJEIdV4+NBZRmFQwwP/RdXWHiBst38Ir2vJ8LErOMl BPNaWUez7QP/UXvTtThNFgTngPFnetbDxh0leMiT7+RnYp1oDkyBtEjAedywbtZN +BvgMYXkJ2I4IxGrHS07yeVDcNVMo0JmrnPdbvRvAMRducF1cAQ4aCjhlmNSIXaA uGxrDANNssljxGYsY1aTeGS9PkMfacST7ApQkrrHeMVxdQe8FmMdi9+GxiTcBfac gJWLSUi6sRUxAjko9lxKXm1v2LhjalPgPAC9RJIHkGMbV6f521MVy3Bz3pCPNBz4 f4TqBW5HGurv8UDCbMJFX7AhL88D/kTbHxHBs905BKA+91OSpfTF1ocBJ9OjmzRU CyabnFiUNtofAoKRrIJzemptS5E1j3yH8gv3Cw+/dmp5y0fps6i9zilyXl43nxLJ VgEBJdDUF+jqf2xK7F7pAKIdZuy3aS9beTJHlQt+rnMmJYROXyonuDbtDZOilRDQ SuZFoZc+YNVmZHK+l4kbgaTQJml8sH5DqjBLNHxD/PylSODRpI5ZgpMPWFPmkCij YlOcwoZPODDdLc15hxDysgv9/j2MD5SG2B6ItONWaZBe60RbSYdQUjO1fVkLm/cR I3gHEgQd53C+dpvsSThAKPgt/BnFoUT7SOc1/HR3mlUX47Dl5FNkscpMtsUAG1Ho 3ExHIpF4t9SXsRx69xCp =WlVA -----END PGP SIGNATURE----- --Sig_/sWu=gIRINNgEnv.Emfppnvg-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 00:20:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA433240 for ; Sun, 3 Nov 2013 00:20:18 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 57D6220CA for ; Sun, 3 Nov 2013 00:20:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=fxyJe9XytQfYSfTVo+B3r4By7k4=; b=XjO5zWNXLQ/0/xQapY Vy3LDOsVLFAy/PV1Pv9vLpspoMCvITzu5imgDvF0KI5Y6f0zrx4Pw2hA1yF3AqdF ur/g8Er8FCg6yHc3SpiarKuw+27l15f0jgp+KIb0hfsn+FZKRDHp/Iq7lpwC/NrJ Z6vG5fgbMWlhpdo7ZtQYF6Lq0= Received: by mf71.sendgrid.net with SMTP id mf71.1834.527596C06 Sun, 03 Nov 2013 00:20:16 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi24 (SG) with ESMTP id 1421b54dfe3.1346.1940a5 for ; Sat, 02 Nov 2013 19:20:16 -0500 (CST) Received: (qmail 82487 invoked from network); 3 Nov 2013 00:20:15 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 3 Nov 2013 00:20:15 -0000 Received: (qmail 33216 invoked from network); 3 Nov 2013 00:18:44 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 3 Nov 2013 00:18:44 -0000 Message-ID: <52759664.50800@freebsd.org> Date: Sat, 02 Nov 2013 17:18:44 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Benjamin Lee Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <20131102171840.4505cb02@b1c1l1.com> In-Reply-To: <20131102171840.4505cb02@b1c1l1.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cprgq76Bt3n6czqls2a7aJ0XzCToaXHADwRVGnH6Hg3XHfLZsBPGo1Dc1vXA7rHSMfLEnfUxkc3T6lAvUr5R/jqXrJNsWzjRJ2lnGywJS64I= Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 00:20:18 -0000 On 11/02/13 17:18, Benjamin Lee wrote: > On Tue, 29 Oct 2013 03:32:19 -0700, Colin Percival wrote: >> Hi all, >> >> I've written some code for automatically submitting kernel panic reports, >> and I'd like some feedback before I place it into the ports tree. > > Just a style question, any reason not to place the file names into > variables? Something like: No reason to avoid it, I just don't see any need. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 00:22:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B3ABB3D3 for ; Sun, 3 Nov 2013 00:22:53 +0000 (UTC) (envelope-from royce.williams@gmail.com) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 415482110 for ; Sun, 3 Nov 2013 00:22:53 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ev20so2721332lab.11 for ; Sat, 02 Nov 2013 17:22:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:content-type; bh=Farp5h9LTAsnPIyH8yR/s29i+gX1z85jP4eRikQJ8+o=; b=AcEGwUndE9uM5cFU5Aw/TzGf+5swa2lqb8AFfN15wopGf8gshm0Bf6+7Uq+seXr8xc tuBnsg74uLerrmRLWG09uLYu5EV4EhQQrd47R5EtMWd46K+F+x2NATcg8JRRfYsVPlJa q+lLEsHXrw1cVJ43GPOGwCPV0DG3/LGyQRZFXLQsb1RT5rcuPq4I5M8+aMBeCI8bexGs cJEXHT823RIrS5M8JEMfl8pgxEvEQYQJKl8V+zyX0h8hEqsp+/1sfZKs2gZcFGO/9yZZ bUqu3CtQmUX9y/QBJMBO19D5QnDYZbwuxHpGiY8aeHfUFBrUSYy8eU8LzWdyrRN+boJC rO4Q== X-Received: by 10.152.5.162 with SMTP id t2mr6225463lat.1.1383438171055; Sat, 02 Nov 2013 17:22:51 -0700 (PDT) MIME-Version: 1.0 Sender: royce.williams@gmail.com Received: by 10.112.119.169 with HTTP; Sat, 2 Nov 2013 17:22:30 -0700 (PDT) In-Reply-To: <52759664.50800@freebsd.org> References: <526F8EB3.1040205@freebsd.org> <20131102171840.4505cb02@b1c1l1.com> <52759664.50800@freebsd.org> From: Royce Williams Date: Sat, 2 Nov 2013 16:22:30 -0800 X-Google-Sender-Auth: 4YH4YYo5gXeIGFJYqcJPtZaea4E Message-ID: Subject: Re: Automated submission of kernel panic reports To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 00:22:53 -0000 On Sat, Nov 2, 2013 at 4:18 PM, Colin Percival wrote: > On 11/02/13 17:18, Benjamin Lee wrote: > > On Tue, 29 Oct 2013 03:32:19 -0700, Colin Percival > wrote: > >> Hi all, > >> > >> I've written some code for automatically submitting kernel panic > reports, > >> and I'd like some feedback before I place it into the ports tree. > > > > Just a style question, any reason not to place the file names into > > variables? Something like: > > No reason to avoid it, I just don't see any need. It's definitely more "quickly legible" to the human trying to grok the code. Royce From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 01:15:59 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6510CCE for ; Sun, 3 Nov 2013 01:15:59 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E69B9231E for ; Sun, 3 Nov 2013 01:15:58 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0M4002-1Vur4p0ymo-00rbAZ for ; Sun, 03 Nov 2013 02:15:57 +0100 Message-ID: <5275A3AD.2020501@gmx.com> Date: Sun, 03 Nov 2013 02:15:25 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: Colin Percival , freebsd-hackers@freebsd.org Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <5271A465.2030206@gmx.com> <5275888E.6010806@freebsd.org> <52758BCD.1000307@gmx.com> <52758C1F.9080601@freebsd.org> In-Reply-To: <52758C1F.9080601@freebsd.org> Content-Type: multipart/mixed; boundary="------------090203010501040905000505" X-Provags-ID: V03:K0:dILavHSPN8lkQ1vkN3K0aw95hc7Cx9cEFPLvYAiCcGSCC8Pyrm0 vD6CQ10LM6Zrorm1Z9igfjbqUmflGOXagCatMt+NsAoOgkSrH6JKyChYdwOBn8G2ROoWQxK E1aUdsWApKOOA8IRbaXHXsQJnuGVdVmPvbKgMa+9V28ok2YrG+04A+9k57dbv7eqjLDLj3n knkTsWDopcZT0q2+HhilQ== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 01:15:59 -0000 This is a multi-part message in MIME format. --------------090203010501040905000505 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Colin Percival wrote, On 11/03/2013 00:34: > Let me know if you see anything I missed. Patch 1: mostly bikeshed stuff. Patch 2: double quote stuff. I didn't know anything about dump number rotation. How does that work? By default, upon the 11th dump, the bounds file will contain ``0'', but the last dump will be stored at /var/crash/{info,vmcore}.9 ? Because in that case, the ``bounds - 1'' method won't work. --------------090203010501040905000505 Content-Type: text/x-patch; name="1_pedantic.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="1_pedantic.patch" --- panicmail.old 2013-11-03 01:15:03.000000000 +0100 +++ panicmail.mid 2013-11-03 02:05:20.000000000 +0100 @@ -30,7 +30,7 @@ . /etc/rc.subr name="panicmail" -rcvar=panicmail_enable +rcvar="panicmail_enable" start_cmd="panicmail_run" stop_cmd=":" @@ -51,6 +51,8 @@ kgdb -q `sysctl -n kern.bootfile` "${dumpdir}/vmcore.$1" \ < ${tmpfile} >> "${dumpdir}/panicmail.$1" 2> /dev/null echo >> "${dumpdir}/panicmail.$1" + + # Remove the temporary file. rm ${tmpfile} } @@ -67,7 +69,7 @@ lam -s '|' ${tmpfile} -s '|' >> "${dumpdir}/panicmail.$1.enc" echo "-----ENCRYPTED FREEBSD PANIC DATA ENDS HERE-----------------------" >> "${dumpdir}/panicmail.$1.enc" - # Remove temporary file. + # Remove the temporary file. rm ${tmpfile} } @@ -97,7 +99,7 @@ ${panicmail_sendto}; you should be able to do this by hitting "Reply" in your mail client and removing everything up to this point. - + EOF cat "${dumpdir}/panicmail.$1.enc" } @@ -124,7 +126,7 @@ # Quit if we have no dumps. if ! [ -f "${dumpdir}/bounds" ]; then - return 0; + return 0 fi # If we have info.last, use that to figure out the last dump number. @@ -132,20 +134,20 @@ nr=`readlink ${dumpdir}/info.last` nr=${nr##*.} else - # Otherwise get the number from bounds. + # Otherwise, get the number from bounds. nr=$((`cat ${dumpdir}/bounds` - 1)) fi # Make sure the dump actually exists. if ! [ -f "${dumpdir}/info.${nr}" ] || ! [ -f "${dumpdir}/vmcore.${nr}" ]; then - return 0; + return 0 fi # Have we already sent an email about this one? We compare times in # order to catch the case where dump numbers repeat. if [ "${dumpdir}/panicmail.${nr}" -nt "${dumpdir}/vmcore.${nr}" ]; then - return 0; + return 0 fi # Gather information about this panic. --------------090203010501040905000505 Content-Type: text/x-patch; name="2_dqingmyassoff.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="2_dqingmyassoff.patch" --- panicmail.mid 2013-11-03 02:05:20.000000000 +0100 +++ panicmail.new 2013-11-03 02:11:38.000000000 +0100 @@ -46,14 +46,14 @@ # And we want a backtrace (we should be able to pipe the commands # directly into kgdb, but that doesn't work with our /bin/sh): echo "Backtrace:" >> "${dumpdir}/panicmail.$1" - echo bt > ${tmpfile} - echo quit >> ${tmpfile} - kgdb -q `sysctl -n kern.bootfile` "${dumpdir}/vmcore.$1" \ - < ${tmpfile} >> "${dumpdir}/panicmail.$1" 2> /dev/null + echo bt > "${tmpfile}" + echo quit >> "${tmpfile}" + kgdb -q "`sysctl -n kern.bootfile`" "${dumpdir}/vmcore.$1" \ + < "${tmpfile}" >> "${dumpdir}/panicmail.$1" 2> /dev/null echo >> "${dumpdir}/panicmail.$1" # Remove the temporary file. - rm ${tmpfile} + rm "${tmpfile}" } # Encrypt the information in the panic report. @@ -62,15 +62,15 @@ local tmpfile=`mktemp` || exit 1 # Encrypt using pkesh. - /usr/local/bin/pkesh enc "$2" "${dumpdir}/panicmail.$1" ${tmpfile} || exit 1 + /usr/local/bin/pkesh enc "$2" "${dumpdir}/panicmail.$1" "${tmpfile}" || exit 1 # Add extra armour. echo "-----ENCRYPTED FREEBSD PANIC DATA STARTS HERE---------------------" > "${dumpdir}/panicmail.$1.enc" - lam -s '|' ${tmpfile} -s '|' >> "${dumpdir}/panicmail.$1.enc" + lam -s '|' "${tmpfile}" -s '|' >> "${dumpdir}/panicmail.$1.enc" echo "-----ENCRYPTED FREEBSD PANIC DATA ENDS HERE-----------------------" >> "${dumpdir}/panicmail.$1.enc" # Remove the temporary file. - rm ${tmpfile} + rm "${tmpfile}" } # Construct an email destined for root to review and forward. @@ -131,11 +131,11 @@ # If we have info.last, use that to figure out the last dump number. if [ -e "${dumpdir}/info.last" ]; then - nr=`readlink ${dumpdir}/info.last` + nr=`readlink "${dumpdir}/info.last"` nr=${nr##*.} else # Otherwise, get the number from bounds. - nr=$((`cat ${dumpdir}/bounds` - 1)) + nr=$((`cat "${dumpdir}/bounds"` - 1)) fi # Make sure the dump actually exists. @@ -151,16 +151,16 @@ fi # Gather information about this panic. - panicmail_gather ${nr} + panicmail_gather "${nr}" # Encrypt the panic information. - panicmail_encrypt ${nr} "${panicmail_key}" + panicmail_encrypt "${nr}" "${panicmail_key}" # Generate and send an email. if checkyesno panicmail_autosubmit; then - panicmail_auto ${nr} | sendmail -t + panicmail_auto "${nr}" | sendmail -t else - panicmail_root ${nr} | sendmail -t + panicmail_root "${nr}" | sendmail -t fi } --------------090203010501040905000505-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 01:20:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 237C22A7 for ; Sun, 3 Nov 2013 01:20:53 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 990FF235D for ; Sun, 3 Nov 2013 01:20:52 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MhQxO-1VHQ0E2zxa-00MavO for ; Sun, 03 Nov 2013 02:20:44 +0100 Message-ID: <5275A4CD.70406@gmx.com> Date: Sun, 03 Nov 2013 02:20:13 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: Colin Percival , freebsd-hackers@freebsd.org Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <5271A465.2030206@gmx.com> <5275888E.6010806@freebsd.org> <52758BCD.1000307@gmx.com> <52758C1F.9080601@freebsd.org> In-Reply-To: <52758C1F.9080601@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:Q5yBbkAIIXB4GC95hyA1nIfV/1qf6RTVmzzBcKf0OmshuemZcSE W6W7XCINDjKBtfEwDxgEKfYJqhNY1ZdjI4uqxB9928AiKvtl9XD4vX6QRjzYlZgjwe0YHlU iW7rynQeMi+qyh662S8jaZOCrXZEc+yXUpwY5moElAWR40lSqHBNzBYU2Th/2IkKAdWaSf+ udEMtjv/j2Zd+KN3kk0QA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 01:20:53 -0000 S00N(TM), the configuration variable defaults (panicmail_enable, panicmail_sendto, etc.) will need to be moved to /etc/defaults/rc.conf. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 07:09:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 13B66D77 for ; Sun, 3 Nov 2013 07:09:48 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id B6D2820A1 for ; Sun, 3 Nov 2013 07:09:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=iHmvlRWYgFMk50UlU/ZERXldrLc=; b=FNuKc0FUuje/3KuudO AtyW4gSaxtlDfGAy38xH1yhtkn+ZNypAIABFR1kUGiUtyWsuJBABi4FBp1Muahmt YbnOKHeLgFeIcj7gWV1y/oOhmYAgutrsbGIxxDtpvNtnWzRL8VLsD+WWGZOZshOE wM3tm/7J6D3O43rMR1nu7p9MQ= Received: by mf69 with SMTP id mf69.23501.5275F6B36 Sun, 03 Nov 2013 07:09:39 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi32 (SG) with ESMTP id 1421ccbad64.17bf.1a48ba for ; Sun, 03 Nov 2013 01:09:39 -0600 (CST) Received: (qmail 97874 invoked from network); 3 Nov 2013 07:09:38 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 3 Nov 2013 07:09:38 -0000 Received: (qmail 35241 invoked from network); 3 Nov 2013 07:08:06 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 3 Nov 2013 07:08:06 -0000 Message-ID: <5275F656.2090804@freebsd.org> Date: Sun, 03 Nov 2013 00:08:06 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: dt71@gmx.com, freebsd-hackers@freebsd.org Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <5271A465.2030206@gmx.com> <5275888E.6010806@freebsd.org> <52758BCD.1000307@gmx.com> <52758C1F.9080601@freebsd.org> <5275A3AD.2020501@gmx.com> In-Reply-To: <5275A3AD.2020501@gmx.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cyuePT7KD/24H/Dl0Ji22OO3mGKO+KphuKCb7jBOxmXqVjnAmMHbDtRNw+LtH7RzSDnr2iaVgJQnYk7ErcKjjfWtjmJTUg+zwAKbjeI5PaEA= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 07:09:48 -0000 On 11/02/13 18:15, dt71@gmx.com wrote: > Colin Percival wrote, On 11/03/2013 00:34: >> Let me know if you see anything I missed. > > Patch 1: mostly bikeshed stuff. > Patch 2: double quote stuff. Thanks. > I didn't know anything about dump number rotation. How does that work? By > default, upon the 11th dump, the bounds file will contain ``0'', but the last > dump will be stored at /var/crash/{info,vmcore}.9 ? Because in that case, the > ``bounds - 1'' method won't work. Aside from a 3 minute window on HEAD, the versions of savecore which can rotate back to using /var/crash/*.0 also create .last symlinks. So I'm not worried about that; if we don't have a .last symlink to rely on, bounds - 1 is good enough. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 07:13:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A8590E72 for ; Sun, 3 Nov 2013 07:13:19 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 4812020DC for ; Sun, 3 Nov 2013 07:13:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=ujyS3vBD8ruH6Yj+F2VzwVYMWFQ=; b=Qo5QMj2RvyWMuXnN6k 9qC0sgPmCC45JmmrWysb5kKvWmzAySKGQKq7wjbwqoROt726TofpujYneMtR4Bxt YiH9obZ9OMF7tQqmQCOvVZirIo6Ay+V34lYgWPLGQ597Hksew2JVbWcpOuJ/xO97 vdqgURL8e1xfMM0evOAU1akTQ= Received: by filter-171.sjc1.sendgrid.net with SMTP id filter-171.3490.5275F7883 Sun, 03 Nov 2013 07:13:12 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi30 (SG) with ESMTP id 1421cceeb98.1ad6.1f7ac1 for ; Sun, 03 Nov 2013 01:13:12 -0600 (CST) Received: (qmail 97998 invoked from network); 3 Nov 2013 07:13:11 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 3 Nov 2013 07:13:11 -0000 Received: (qmail 35278 invoked from network); 3 Nov 2013 07:11:39 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 3 Nov 2013 07:11:39 -0000 Message-ID: <5275F72B.7040808@freebsd.org> Date: Sun, 03 Nov 2013 00:11:39 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: dt71@gmx.com, freebsd-hackers@freebsd.org Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <5271A465.2030206@gmx.com> <5275888E.6010806@freebsd.org> <52758BCD.1000307@gmx.com> <52758C1F.9080601@freebsd.org> <5275A4CD.70406@gmx.com> In-Reply-To: <5275A4CD.70406@gmx.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cnqC7kSw/qcvCChoCt9x4OFMP09EIugkVeKH4Bf0z/1DtdxAIon1bYAXBmAufDum9BxgMKsuD5P1a4HXQoYRhKom0Vq6A3bS2Q+HQKCsaXE4= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 07:13:19 -0000 On 11/02/13 18:20, dt71@gmx.com wrote: > S00N(TM), the configuration variable defaults (panicmail_enable, > panicmail_sendto, etc.) will need to be moved to /etc/defaults/rc.conf. Not soon -- only when (if) this goes into the base system. My intention is to use the ports tree as a "testing ground" for this (like I did with freebsd-update and portsnap). -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 09:03:26 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5190FFF6 for ; Sun, 3 Nov 2013 09:03:26 +0000 (UTC) (envelope-from symbolics@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DEA2524D3 for ; Sun, 3 Nov 2013 09:03:25 +0000 (UTC) Received: from lemon ([80.7.17.14]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MFuWk-1VO4Py0CfO-00EwjI for ; Sun, 03 Nov 2013 10:03:24 +0100 Received: by lemon (Postfix, from userid 1001) id 6A5ACEB2DA; Sun, 3 Nov 2013 09:03:23 +0000 (GMT) Date: Sun, 3 Nov 2013 09:03:23 +0000 From: symbolics@gmx.com To: Warren Block Subject: Re: GEOM mentor request Message-ID: <20131103090323.GA1744@lemon> References: <20131101103158.GA35397@lemon> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Provags-ID: V03:K0:rW2soKbRotZJoiX6pFlXxnKqVoEsXj4i2ilDzNn6qSAf0doRPyC N97jcu4+CxTOeGBHCl/Mp+bnWZAFQfN3tKj+a5ipjrIqBxMAK6FXdc8ZYZs0u99tugkO/9w VnY8vxjZ5JIPWx6aK1xiheKbFloc3muRwYFERnAgj70rGxomumLn86xePa5KOcBam5gJERb +zdtF/7Iu2B7nHDgckTtA== Cc: geom@freebsd.org, hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 09:03:26 -0000 On Fri, Nov 01, 2013 at 01:23:12PM -0600, Warren Block wrote: > On Fri, 1 Nov 2013, symbolics@gmx.com wrote: > > > + Implement new things. Some ideas I have had: > > + GEOM "ERASE" - Rewrite deletes into random writes. > > + GEOM "PLUG" - Persistent version of the connect/disconnect verbs > > where the flag sits in the class metadata. This might be a cleaner > > approach, rather than adding the verbs to all the existing > > providers. > > + GEOM "TAP" - Allow userspace processes to hook into the GEOM > > API. Intended for debugging and development. > > + GEOM "WCACHE" - Allow you to use small, fast provider as a buffer > > for a larger, slower provider. > > + GEOM DTrace provider. Provide GEOM specific probes to complement > > the IO provider. > > + Probably other bits I can't remember right now. > > How about an explicit geom retaste command? "true > /dev/ada0" is > misleading to the reader. Yes, that would be good. It's on my list. > Also, a RAM-cached version of gmirror that would report writes finished > as soon as the faster drive finishes. Kind of the opposite of the > WCACHE above. This would permit creating mirrors of an SSD and hard > drive without performance loss, at least up until available write > buffer space runs out. This one may not be so easy. I can see the benefit. This would be like a mirror with a journal. As long as it has a different name from mirror, 'lazy mirror' ?, I think it would be interesting. The only concern I have would be that some users could use it and assume the normal mirror semantics, e.g. that all discs are equally redundant, which wouldn't be true. --sym From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 15:32:06 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2C1C9236; Sun, 3 Nov 2013 15:32:06 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BE42B23D4; Sun, 3 Nov 2013 15:32:05 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.7/8.14.7) with ESMTP id rA3FW4Sd038854; Sun, 3 Nov 2013 08:32:04 -0700 (MST) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.7/8.14.7/Submit) with ESMTP id rA3FW3sB038851; Sun, 3 Nov 2013 08:32:04 -0700 (MST) (envelope-from wblock@wonkity.com) Date: Sun, 3 Nov 2013 08:32:03 -0700 (MST) From: Warren Block To: symbolics@gmx.com Subject: Re: GEOM mentor request In-Reply-To: <20131103090323.GA1744@lemon> Message-ID: References: <20131101103158.GA35397@lemon> <20131103090323.GA1744@lemon> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Sun, 03 Nov 2013 08:32:04 -0700 (MST) Cc: geom@freebsd.org, hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 15:32:06 -0000 On Sun, 3 Nov 2013, symbolics@gmx.com wrote: > On Fri, Nov 01, 2013 at 01:23:12PM -0600, Warren Block wrote: >> On Fri, 1 Nov 2013, symbolics@gmx.com wrote: >> >>> + Implement new things. Some ideas I have had: >>> + GEOM "ERASE" - Rewrite deletes into random writes. >>> + GEOM "PLUG" - Persistent version of the connect/disconnect verbs >>> where the flag sits in the class metadata. This might be a cleaner >>> approach, rather than adding the verbs to all the existing >>> providers. >>> + GEOM "TAP" - Allow userspace processes to hook into the GEOM >>> API. Intended for debugging and development. >>> + GEOM "WCACHE" - Allow you to use small, fast provider as a buffer >>> for a larger, slower provider. >>> + GEOM DTrace provider. Provide GEOM specific probes to complement >>> the IO provider. >>> + Probably other bits I can't remember right now. >> >> How about an explicit geom retaste command? "true > /dev/ada0" is >> misleading to the reader. > > Yes, that would be good. It's on my list. > >> Also, a RAM-cached version of gmirror that would report writes finished >> as soon as the faster drive finishes. Kind of the opposite of the >> WCACHE above. This would permit creating mirrors of an SSD and hard >> drive without performance loss, at least up until available write >> buffer space runs out. This one may not be so easy. > > I can see the benefit. This would be like a mirror with a journal. As > long as it has a different name from mirror, 'lazy mirror' ?, I think it > would be interesting. The only concern I have would be that some users > could use it and assume the normal mirror semantics, e.g. that all discs > are equally redundant, which wouldn't be true. I've been calling it a "slow" mirror. Come to think of it, that's a little misleading. "Async mirror"? There may be an existing term. As pointed out, it's probably non-trivial to implement. The WCACHE you suggest above (the Linux guys have "bcache") is probably more benefit to more people. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 20:49:29 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 83031A61 for ; Sun, 3 Nov 2013 20:49:29 +0000 (UTC) (envelope-from Kamil.Choudhury@anserinae.net) Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.225]) by mx1.freebsd.org (Postfix) with ESMTP id 49968219B for ; Sun, 3 Nov 2013 20:49:28 +0000 (UTC) Received: from [74.73.121.187] ([74.73.121.187:62661] helo=janus.anserinae.net) by cdptpa-oedge02 (envelope-from ) (ecelerity 3.5.0.35861 r(Momo-dev:tip)) with ESMTP id 98/11-27821-1D6B6725; Sun, 03 Nov 2013 20:49:21 +0000 Received: from JANUS.anserinae.net ([fe80::192c:4b89:9fe9:dc6d]) by janus.anserinae.net ([fe80::192c:4b89:9fe9:dc6d%11]) with mapi id 14.03.0123.003; Sun, 3 Nov 2013 15:49:21 -0500 From: Kamil Choudhury To: "hackers@freebsd.org" Subject: pkgng for configuration management? Thread-Topic: pkgng for configuration management? Thread-Index: Ac7Y1N0HhuY2bDV4QwqNyPNCJZ8Ziw== Date: Sun, 3 Nov 2013 20:49:20 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.21] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RR-Connecting-IP: 107.14.168.130:25 X-Cloudmark-Score: 0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 20:49:29 -0000 I've been setting up a private pkgng repository to push software to a famil= y of about 20 different hosts.=20 One command software deployment is pretty awesome, so I got to thinking: wh= y not go one step further and start pushing configurations for each of thes= e hosts via pkgng as well (either by putting the config files into the init= ial software pkg, or via a separate pkg that installs only the configuratio= ns)?=20 Has anyone else tried going down this rabbit hole? If so, what has your exp= erience with the system been? =20 Thanks Kamil=20 From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 3 21:30:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E24CEADF; Sun, 3 Nov 2013 21:30:08 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A1F52239A; Sun, 3 Nov 2013 21:30:08 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id 764AC120206; Sun, 3 Nov 2013 22:29:50 +0100 (CET) Received: by turtle.stack.nl (Postfix, from userid 1677) id 16918CB4E; Sun, 3 Nov 2013 22:29:50 +0100 (CET) Date: Sun, 3 Nov 2013 22:29:50 +0100 From: Jilles Tjoelker To: Colin Percival Subject: Re: Automated submission of kernel panic reports Message-ID: <20131103212950.GA22571@stack.nl> References: <526F8EB3.1040205@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <526F8EB3.1040205@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 21:30:09 -0000 On Tue, Oct 29, 2013 at 03:32:19AM -0700, Colin Percival wrote: > The code is in > http://svnweb.freebsd.org/base/user/cperciva/panicmail/ > and it uses my FreeBSD-base-system-only public-key encryption code: > http://svnweb.freebsd.org/base/user/cperciva/pkesh/ Some remarks about panicmail: > local tmpfile=`mktemp` || exit 1 This kind of thing does not do what you expect. The 'local' utility returns 0 because it successfully created the local variable, ignoring the status from the command substitution. Use local tmpfile tmpfile=`mktemp` || exit 1 in both occurrences. > # And we want a backtrace (we should be able to pipe the commands > # directly into kgdb, but that doesn't work with our /bin/sh): It looks like the problem is, in fact, that gdb will not read from a pipe. When the pipe is at EOF, poll() returns POLLHUP status and gdb aborts with "Hangup detected on fd 0" even though there is unread data in the kernel buffer. Earlier kernels incorrectly did not return POLLHUP, hiding the gdb bug. Some other shells (but not ash-derived ones such as all versions of FreeBSD sh) provide here-document input as a temporary file, avoiding the gdb bug. > return 0; It would be better style to omit the redundant semicolon here (occurs several times). Some remarks about pkesh: > D=`mktemp -d "${TMP:-/tmp}/pkesh.XXXXXX"` I think the usual environment variable for the directory for temporary files is TMPDIR, not TMP. The rest of the script uses $D mostly unquoted, unlike the change that was made to panicmail. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 03:01:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C7AB4198 for ; Mon, 4 Nov 2013 03:01:04 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 57A8B22E1 for ; Mon, 4 Nov 2013 03:01:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=qN4Z+/O2BmyjCHX8EbbSh9Qqans=; b=CmpgytlF7zScs60FVL 5x9cypTAGmA2cxCMT0R5SkZFHZ9M4NHfnc/vOZI4fYtqKCYGYJOWxhjG7C3oFzMl MUYkx1zmmYDax2FUNPDKL2LBkooSzg6e6lNekMAIfZWw5GMA7kXZhWFlH7qB4lQA zsnEohi6UbjA5uhZoC9yttiTQ= Received: by mf94 with SMTP id mf94.26985.52770DEF1 Mon, 04 Nov 2013 03:01:03 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi43 (SG) with ESMTP id 142210e6dc7.7341.22d624 for ; Sun, 03 Nov 2013 21:01:03 -0600 (CST) Received: (qmail 34228 invoked from network); 4 Nov 2013 03:01:01 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 03:01:01 -0000 Received: (qmail 41104 invoked from network); 4 Nov 2013 02:59:27 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 02:59:27 -0000 Message-ID: <52770D8E.2030906@freebsd.org> Date: Sun, 03 Nov 2013 18:59:26 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Jilles Tjoelker Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <20131103212950.GA22571@stack.nl> In-Reply-To: <20131103212950.GA22571@stack.nl> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cdFiRW89jQpnLRxdkpFJNihm/Cpmh/rCB8lfJbn7XaRz3faFO9vOC0rQs1OjTDBwf+4JPFZoaaxX7hkqIdp/XooftDifm5C4phSJ3fYvBef0= Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 03:01:04 -0000 On 11/03/13 13:29, Jilles Tjoelker wrote: > Some remarks about panicmail: > >> local tmpfile=`mktemp` || exit 1 > > This kind of thing does not do what you expect. The 'local' utility > returns 0 because it successfully created the local variable, ignoring > the status from the command substitution. Use > local tmpfile > tmpfile=`mktemp` || exit 1 > in both occurrences. Wow, I had no idea. I guess it never occurred to me that 'local' would even *have* an exit status... >> return 0; > > It would be better style to omit the redundant semicolon here (occurs > several times). Yes, that's already fixed (pointed out by dt71). > Some remarks about pkesh: > >> D=`mktemp -d "${TMP:-/tmp}/pkesh.XXXXXX"` > > I think the usual environment variable for the directory for temporary > files is TMPDIR, not TMP. Fixed. > The rest of the script uses $D mostly unquoted, unlike the change that > was made to panicmail. Fixed. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 03:14:17 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A0C9C46F for ; Mon, 4 Nov 2013 03:14:17 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6122323D1 for ; Mon, 4 Nov 2013 03:14:16 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id s19so3728079qcw.35 for ; Sun, 03 Nov 2013 19:14:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=jTQvQDQ4n7q5DgFAwKqnzGoIUb3enRoUZn/tZjhef1E=; b=jnFsxAzOBeYKSuuuXmmwUzMxg1VNBgKZ4KdfzqEFCp1vnaxf6KmZp5W+gLdnvrDoJH d798HdXU8t0pX2/wDo/jzl3tD58SeKivMUtsd5D2D1PVyG+FUQbIef2C+/ibqGX2niFj fcqHX/tl3Z2YVCPQn9b58PsgCyvyPwtp8CS7M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=jTQvQDQ4n7q5DgFAwKqnzGoIUb3enRoUZn/tZjhef1E=; b=m743zLk+VeEnuneVwtvRvslj+tQKO6CGXlLkSh10SaOlaAfE6QJLSs+R4MolkayL9x 8acu6PB65/XbPEpmnwQGVlGpbKrJ576vyjHnOeeCMP/fPaKhoFJzTecv7UrImfmgCh+Y 3SWi3xM1ecEgjS7WQjqJFgSnZ77CH3cUQC1zH2BuAm1/2amlCn0bkjF4s4k9/3NX7lXq eNsrOPq8MKBQeCSEFsxy+tXdVpBfEKWiaM5DF270QHg/O/5pAWIUMPDvGEzr7HMj9MGD bigL4Hzi4BPW55GMt4QLSka/yeQP8boydV3GAW2s6/LU6L2FWJELwP84D/tWauF83I+5 qAXA== X-Gm-Message-State: ALoCoQmDm7K8rrp1ziQo2aIat4frCNSSxhnyDJqh1ejoJXvEzO4cY+yT3BQkitkCYrM98s+8P2mm X-Received: by 10.229.172.3 with SMTP id j3mr19489363qcz.10.1383534856149; Sun, 03 Nov 2013 19:14:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.63.101 with HTTP; Sun, 3 Nov 2013 19:13:45 -0800 (PST) In-Reply-To: <52770D8E.2030906@freebsd.org> References: <526F8EB3.1040205@freebsd.org> <20131103212950.GA22571@stack.nl> <52770D8E.2030906@freebsd.org> From: Eitan Adler Date: Sun, 3 Nov 2013 22:13:45 -0500 Message-ID: Subject: Re: Automated submission of kernel panic reports To: Colin Percival Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Hackers , Jilles Tjoelker X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 03:14:17 -0000 On Sun, Nov 3, 2013 at 9:59 PM, Colin Percival wrote: > On 11/03/13 13:29, Jilles Tjoelker wrote: >> Some remarks about panicmail: >> >>> local tmpfile=`mktemp` || exit 1 >> >> This kind of thing does not do what you expect. The 'local' utility >> returns 0 because it successfully created the local variable, ignoring >> the status from the command substitution. Use >> local tmpfile >> tmpfile=`mktemp` || exit 1 >> in both occurrences. > > Wow, I had no idea. I guess it never occurred to me that 'local' would > even *have* an exit status... It can even exit non-zero: $ foo() { readonly bar; local bar="f"; echo $?; }; foo; local: bar: is read only 2 -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 03:54:49 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A53C1B34 for ; Mon, 4 Nov 2013 03:54:49 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 421EE2571 for ; Mon, 4 Nov 2013 03:54:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=lv8EvQIc7AwIgA2DWBxmYYfOZ6g=; b=ux71CNDwr4iWc605w8 yaePnKch2BeUmicKRNC9/ep0LIx/EvSV6/Hp1Jmh+LPWDHLk0b8Zo5+4vE4m7br9 dQ6ZLLWIZA6VWaAj5pbmtstqGQDnijiyf7YZs3+JD5qNcbFPDhpTNNXfRtassv0f wZKFPOfCVssltpfc9EiIn4oNs= Received: by filter-188.sjc1.sendgrid.net with SMTP id filter-188.8732.52771A871 Mon, 04 Nov 2013 03:54:47 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi35 (SG) with ESMTP id 142213fa19c.5224.1d5d6b for ; Sun, 03 Nov 2013 21:54:47 -0600 (CST) Received: (qmail 36232 invoked from network); 4 Nov 2013 03:54:46 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 03:54:46 -0000 Received: (qmail 41270 invoked from network); 4 Nov 2013 03:53:10 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 03:53:10 -0000 Message-ID: <52771A26.1020102@freebsd.org> Date: Sun, 03 Nov 2013 19:53:10 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Eitan Adler Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <20131103212950.GA22571@stack.nl> <52770D8E.2030906@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0c8Up126TZIW6AEQOLpK8FO0BfXRyOmVHfmA0e6u8e8Kq1a0sQvHaPWIRs8HrVDMSDSKVOcjBw74yyVk17dMWsI1BQBokjz7FbJVBa2ZUW3GU= Cc: FreeBSD Hackers , Jilles Tjoelker X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 03:54:49 -0000 On 11/03/13 19:13, Eitan Adler wrote: > On Sun, Nov 3, 2013 at 9:59 PM, Colin Percival wrote: >> Wow, I had no idea. I guess it never occurred to me that 'local' would >> even *have* an exit status... > > It can even exit non-zero: > $ foo() { readonly bar; local bar="f"; echo $?; }; foo; > local: bar: is read only > 2 Aha! I was thinking "but how could it ever fail?", and read-only variables never occurred to me. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 10:43:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C51A4830 for ; Mon, 4 Nov 2013 10:43:24 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 4F9342A72 for ; Mon, 4 Nov 2013 10:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:content-type:content-transfer-encoding; s=smtpapi; bh=EITjIOwvVnCTBEbG1UvbMqRPY5o=; b=DKL65Vo2uekKUr2hV/ TGVCkvco4i/+zq/y3E6dPVWDvn+fATpwDsXnSjbIfvZEfZuwcU20lSUCgirkrWSn xPhgv2vBn5ngUBcv+MU08f1k6cr/q34T6tMel2CKnnWDVUoc9Qx4HZ4qBL9Ah1cw tKrOBUI1cfUR/NMRW3sPNYCKg= Received: by filter-173.sjc1.sendgrid.net with SMTP id filter-173.2963.52777A471 Mon, 04 Nov 2013 10:43:19 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi39 (SG) with ESMTP id 14222b5a7fe.650b.180503 for ; Mon, 04 Nov 2013 04:43:19 -0600 (CST) Received: (qmail 55081 invoked from network); 4 Nov 2013 10:43:18 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 10:43:18 -0000 Received: (qmail 2835 invoked from network); 4 Nov 2013 10:41:49 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 10:41:49 -0000 Message-ID: <527779ED.9040303@freebsd.org> Date: Mon, 04 Nov 2013 02:41:49 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: FreeBSD Hackers , FreeBSD current Subject: Automated submission of kernel panic reports: sysutils/panicmail X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cH6pUGr2/vIk8L1w0+GGYcVRK8rE0fEqR7ZPY1EJ8a/pkB/kDZSyDhklwi8UiA9IPczqExd9nsMdnIF87n0rFAzYt74YP4/Sl8kYz/s2bv/I= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 10:43:24 -0000 Hi all, After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have now added sysutils/panicmail to the FreeBSD ports tree. If you install this and add panicmail_enable="YES" to your /etc/rc.conf, a panic report will be generated and sent to root@ for you to review and submit (via email). You can skip the reviewing step and submit panics automatically by setting panicmail_autosubmit="YES". The panics submitted are encrypted to an RSA key which I hold in order to keep them secure in transit; and I intend to keep the raw panic reports confidential except to the minimum extent necessary for other developers to help me process the incoming reports. If I receive enough panic reports to be useful, I hope to provide developers with aggregate statistics. This may include: * regular email reports listing the "top panics", to help guide developers towards the most fertile areas for stability improvements; * email to specific developers alerting them to recurring panics in code they maintain (especially if it becomes clear that the panic has been recently introduced); and * guidance to re@ and secteam@ about how often a particular panic occurs if an errata notice is being considered as well as other yet-to-be-imagined reports of a similarly aggregate and anonymized nature. So please install the sysutils/panicmail port and enable it in rc.conf! This all depends on getting useful data, and I can't do that without your help. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 10:48:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A38A3BAD; Mon, 4 Nov 2013 10:48:02 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3311A2ABF; Mon, 4 Nov 2013 10:48:01 +0000 (UTC) Received: from [192.168.0.247] (46-65-50-29.zone16.bethere.co.uk [46.65.50.29]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id rA4AlrJn092717; Mon, 4 Nov 2013 10:47:53 GMT (envelope-from rb@gid.co.uk) Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Bob Bishop In-Reply-To: <527779ED.9040303@freebsd.org> Date: Mon, 4 Nov 2013 10:47:48 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <2DED1CF2-9486-4AB1-A443-B4E276856903@gid.co.uk> References: <527779ED.9040303@freebsd.org> To: Colin Percival X-Mailer: Apple Mail (2.1283) Cc: FreeBSD Hackers , FreeBSD current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 10:48:02 -0000 Hi, On 4 Nov 2013, at 10:41, Colin Percival wrote: > Hi all, >=20 > After considerable review on freebsd-hackers (thanks dt71 and jilles!) = I have > now added sysutils/panicmail to the FreeBSD ports tree. [etc] Nice. Is this applicable to all supported branches? -- Bob Bishop rb@gid.co.uk From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 10:55:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A90F4125 for ; Mon, 4 Nov 2013 10:55:10 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 56BC42B2E for ; Mon, 4 Nov 2013 10:55:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=cfseBBGDVBzpfopF62jgS2rrCpU=; b=fq5QA9dokMGxUmR5Ek ZNTHja5Gttwc7ugFCXj0LFzd+VGL9JF8B4+jKpOr4lkeHzUi68mZiqZ+z/mxg/gB iXBq1dL/BFrCfOmAUZqhROHbmjZDMRiHjiBKf/SZjIKjjAWGrjNJqW5lxg06qWiH WDPCcRiJUaPNr7WGB3h0DZFI8= Received: by mf82.sendgrid.net with SMTP id mf82.441.52777D0D1 Mon, 04 Nov 2013 10:55:09 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi46 (SG) with ESMTP id 14222c07a87.52b3.1cb15b for ; Mon, 04 Nov 2013 04:55:08 -0600 (CST) Received: (qmail 55509 invoked from network); 4 Nov 2013 10:55:08 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 10:55:08 -0000 Received: (qmail 2975 invoked from network); 4 Nov 2013 10:53:37 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 10:53:37 -0000 Message-ID: <52777CB1.3010001@freebsd.org> Date: Mon, 04 Nov 2013 02:53:37 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Bob Bishop Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> <2DED1CF2-9486-4AB1-A443-B4E276856903@gid.co.uk> In-Reply-To: <2DED1CF2-9486-4AB1-A443-B4E276856903@gid.co.uk> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cmlRq/v18kQCyDo8XQRP/pstbbskQuuOexV2U5FaLrpZdfvb+92x86qKurBhFUrkUhz7Q5lKledgszqEszR+0VfF+IWE9RpQH/dqL1CL66do= Cc: FreeBSD Hackers , FreeBSD current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 10:55:10 -0000 On 11/04/13 02:47, Bob Bishop wrote: > On 4 Nov 2013, at 10:41, Colin Percival wrote: >> After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have >> now added sysutils/panicmail to the FreeBSD ports tree. [etc] > > Nice. Is this applicable to all supported branches? Yes... the code should work all the way back to 5.0 (it's an rc.d script), although I doubt ports infrastructure will allow you to install anything from today's ports tree on a system running FreeBSD 5.0. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 12:49:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 521468F5; Mon, 4 Nov 2013 12:49:24 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 048852351; Mon, 4 Nov 2013 12:49:24 +0000 (UTC) Received: from Alfreds-MacBook-Air.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id C463E1A3C19; Mon, 4 Nov 2013 04:49:23 -0800 (PST) Message-ID: <527797D8.5040404@freebsd.org> Date: Mon, 04 Nov 2013 04:49:28 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Colin Percival , FreeBSD Hackers , FreeBSD current , Jordan Hubbard Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> In-Reply-To: <527779ED.9040303@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 12:49:24 -0000 On 11/4/13, 2:41 AM, Colin Percival wrote: > Hi all, > > After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have > now added sysutils/panicmail to the FreeBSD ports tree. If you install this > and add > panicmail_enable="YES" > to your /etc/rc.conf, a panic report will be generated and sent to root@ for > you to review and submit (via email). You can skip the reviewing step and > submit panics automatically by setting panicmail_autosubmit="YES". > > The panics submitted are encrypted to an RSA key which I hold in order to keep > them secure in transit; and I intend to keep the raw panic reports confidential > except to the minimum extent necessary for other developers to help me process > the incoming reports. > > If I receive enough panic reports to be useful, I hope to provide developers > with aggregate statistics. This may include: > > * regular email reports listing the "top panics", to help guide developers > towards the most fertile areas for stability improvements; > > * email to specific developers alerting them to recurring panics in code they > maintain (especially if it becomes clear that the panic has been recently > introduced); and > > * guidance to re@ and secteam@ about how often a particular panic occurs if > an errata notice is being considered > > as well as other yet-to-be-imagined reports of a similarly aggregate and > anonymized nature. > > So please install the sysutils/panicmail port and enable it in rc.conf! This > all depends on getting useful data, and I can't do that without your help. > Colin, have you had a few minutes to check out the crash reporting facilities in FreeNAS? The reason I ask is that: 1) we would like to share code. 2) we have this running for a few months now and have a huge corpus of information. 3) we are building a nice UI (screenshots attached) over it, we have a couple of thousands of lines of code we can share for this. Our scripts can be found here: 1) A startup script that sends us the crashes on system start: https://github.com/freenas/freenas/blob/master/nanobsd/Files/etc/rc.d/ix_textdump 2) A script to submit data at boot OR from command line that sends more comprehensive system information "ixdiagnose": https://github.com/freenas/freenas/blob/master/nanobsd/Files/usr/local/bin/ixdiagnose 3) A very simple script to upload that report: https://github.com/freenas/freenas/blob/master/nanobsd/Files/usr/local/bin/crashuploader We send a minimal set of information: kernel stack trace, ddb buffer and hardware. Just enough to get some very, very handy stuff. I can share with you offline the crash server code, it's django and relatively straight forward. The screenshots can also be seen at: http://people.freebsd.org/~alfred/crashreporter/ We could modify our framework for FreeBSD to do so by checking for a sentinel file depending on the host type and only auto-sending if we see that. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 14:48:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B76BF70F for ; Mon, 4 Nov 2013 14:48:10 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8AD052A09 for ; Mon, 4 Nov 2013 14:48:10 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 22BE12136A for ; Mon, 4 Nov 2013 09:48:07 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute5.internal (MEProxy); Mon, 04 Nov 2013 09:48:08 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=1QcS8bqorT6cMhJSJ4kzo7CAtU0=; b=YRR 22kjU/z+vXHOq8BfBx9VNAt7q2d0YJ22aslpHYpOwQEYdprSbbGPxMbniARFdmfD AuH9RdROTktgn97rryZl97r436blH7L4Jsl7hm1SoGYqVfLqkdK+7AbsvLRP1Al/ uHJKRn4ydLZw/trooultpzvquWgwQE6hCsjUydPI= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id F3CA810B5A6; Mon, 4 Nov 2013 09:48:06 -0500 (EST) Message-Id: <1383576486.2000.42722261.6F9C8CF8@webmail.messagingengine.com> X-Sasl-Enc: plIaqfyi55ae6VdGsNEQa+YCVN6bsp6o/w6tBKY3AxGP 1383576486 From: Mark Felder To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-d4893488 In-Reply-To: References: Subject: Re: pkgng for configuration management? Date: Mon, 04 Nov 2013 08:48:06 -0600 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 14:48:10 -0000 On Sun, Nov 3, 2013, at 14:49, Kamil Choudhury wrote: > I've been setting up a private pkgng repository to push software to a > family of about 20 different hosts. > > One command software deployment is pretty awesome, so I got to thinking: > why not go one step further and start pushing configurations for each of > these hosts via pkgng as well (either by putting the config files into > the initial software pkg, or via a separate pkg that installs only the > configurations)? > > Has anyone else tried going down this rabbit hole? If so, what has your > experience with the system been? > I suppose you could create metapackages that write out configuration files. I'm not sure I like that approach, though. People tend to gravitate towards things like puppet/chef/ansible/etc. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 17:09:10 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 03F4BDE3 for ; Mon, 4 Nov 2013 17:09:10 +0000 (UTC) (envelope-from ben@b1c1l1.com) Received: from lancer.b1c1l1.com (lancer.b1c1l1.com [72.13.86.100]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E04372426 for ; Mon, 4 Nov 2013 17:09:09 +0000 (UTC) Received: by lancer.b1c1l1.com (Postfix) with ESMTPSA id 197055C34; Mon, 4 Nov 2013 09:09:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=b1c1l1.com; s=default; t=1383584943; bh=jZ1x8TrlCTEEaRIAhhO5nT4Sd+wPyk/+HPbpDF6Bh8U=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=4+/N7DGFgUfJDBk8N4bcnNNYjMasD8OLKE3/ed0rNQqYzGgdknxlFgWwYKrU6feXC j9FXT9o0QA0m3dNetHW7tOyn6GmQdsFpwHvGLVFIufRVsRXiamD+h2LXjpf1HZfvJ1 Io5haLhoNWqkI/FsUX/PJYRgaFCVvCsnspj0AQdE= Date: Mon, 4 Nov 2013 09:08:57 -0800 From: Benjamin Lee To: Kamil Choudhury Subject: Re: pkgng for configuration management? Message-ID: <20131104090857.6e174d66@b1c1l1.com> In-Reply-To: References: X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.22; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/SZcs7.Upj63ClBqh_SM+Vr."; protocol="application/pgp-signature" Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 17:09:10 -0000 --Sig_/SZcs7.Upj63ClBqh_SM+Vr. Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sun, 3 Nov 2013 20:49:20 +0000, Kamil Choudhury wrote: > I've been setting up a private pkgng repository to push software to a fam= ily of about 20 different hosts.=20 >=20 > One command software deployment is pretty awesome, so I got to thinking: = why not go one step further and start pushing configurations for each of th= ese hosts via pkgng as well (either by putting the config files into the in= itial software pkg, or via a separate pkg that installs only the configurat= ions)?=20 >=20 > Has anyone else tried going down this rabbit hole? If so, what has your e= xperience with the system been? =20 Stick to UNIX philosophy: do one thing and do it well. Package management and configuration management have different problem domains. Consider, for example: do you upgrade software packages at the same rate that you modify configuration files? pkg(8) is a great package management tool, but it isn't (and shouldn't be) a configuration management tool. --=20 Benjamin Lee http://www.b1c1l1.com/ --Sig_/SZcs7.Upj63ClBqh_SM+Vr. Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJSd9StAAoJEIdV4+NBZRmFObEQAKatfI1Vu4INHqMfgEvSrBmW zhEoSKn9HOYJhCEeA22dn7R7vmAecIcTrfr3ls88e1bzHUBzAU1nueNnqk4V3yqm txQUX5OiilECIU4W1Dv0VlIZl3tMD8g5jQIPk/71CwgaeMmhZDK2t6B2Km1RbG8V 6Xj+SdqJ/ynmz0t+RFPMgzyCEKWPsq6uEvg+xzwcBQRP4mhQ6uaJE/7k2G/aU2/H nNkQs+UM/3qwULmGp3JF6VbrremGbcqi1UZCldDMkssiyzqEWRTGjry/0qVvlx4X m9hyjktWeUtjmOUI98v5ijNioioe8T9svfCnL/ZjJap1WI94oM9zRR182OQ5d1Wq kygor0LWBU9sAR+AwgO7/mc2GFvejYUxWWETgSlpXd+eyxYSF99qxe4YPNksk48f tG7ChVpWwlyBwqA30axFWosoqTva/R4smoPlpQOLaq3uShjiB8CMkGXX12ytHEEE TidLXbq1va2WbmKRi6RRb+0X5/s8ZK3ul1ghWXLIO3z7ZhyN4HNDPImTyl0MvphD dCTvEGW4swd1lgOUOLoMfbo5arOt0JXLtSlhNteO7keNpp30QHOxdeMSjt2yuwDk lcVnEX+mrRJmu6zARz6k80KqO4wT5oCBihBFezm25HLLKtB8cECBirp8M9BOwWih gI829IC0GiQktTjmuz+Y =Aiue -----END PGP SIGNATURE----- --Sig_/SZcs7.Upj63ClBqh_SM+Vr.-- From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 18:20:08 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 83A57584 for ; Mon, 4 Nov 2013 18:20:08 +0000 (UTC) (envelope-from jamebus@gmail.com) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 232962862 for ; Mon, 4 Nov 2013 18:20:07 +0000 (UTC) Received: by mail-wi0-f180.google.com with SMTP id ey11so954789wid.7 for ; Mon, 04 Nov 2013 10:20:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PP/bOIb6930GVYqMP73jdMyW/KU5a2QIjjbazz+B62k=; b=oP2VrdiSQrU+se5u3b4QxqzmqMmgzmNZSYe5JdpG7aR6AOXSHS8D3kvv9fU68XxzaI ECJAsbYvelH7kvM8cEoSR+3yCF7cJIBZSFvUa3hqjo7Oe2mHUJHKPspKDaTE61gmL3I4 Mj6cHuTO5tqGVamjchw72xBAaKEMSx6Jgir77g75yPrKAEKaSIvhcMrdZA/DO+Lg/ejf xKrxuEMMJsalJabMPXxrGzj+1Xd9kbGBqg1s4HusIOKbdPo4hOaKlfYUtvvbweechrMA rFUWjq89QWYCPQ9/P+WaERy5xBOgPLAIgbMNE6kfG86y03AL2vYWxffwGu4L+oGLC5mJ o0Ww== MIME-Version: 1.0 X-Received: by 10.180.184.112 with SMTP id et16mr13485201wic.4.1383589205073; Mon, 04 Nov 2013 10:20:05 -0800 (PST) Sender: jamebus@gmail.com Received: by 10.227.171.142 with HTTP; Mon, 4 Nov 2013 10:20:05 -0800 (PST) In-Reply-To: References: Date: Mon, 4 Nov 2013 12:20:05 -0600 X-Google-Sender-Auth: SAE8vNL9LMD2uuW9qr0FxTGcUbU Message-ID: Subject: Re: pkgng for configuration management? From: James To: Kamil Choudhury Content-Type: text/plain; charset=UTF-8 Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 18:20:08 -0000 I do this and it works really well. One primary motivation to try such a thing is because in my situation a lot of config files are tightly coupled with the system. I need everything to update at once. Granted, this is just for configuration files that don't change from system-to-system and/or can be auto-generated. Things that change a lot are maintained differently. -- James. http://jameb.us/ From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 18:48:17 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CCC6FDE3 for ; Mon, 4 Nov 2013 18:48:17 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5FE2F2A3A for ; Mon, 4 Nov 2013 18:48:17 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MbPPQ-1VKdrC1H4G-00IkpI for ; Mon, 04 Nov 2013 19:48:10 +0100 Message-ID: <5277EBCA.3010807@gmx.com> Date: Mon, 04 Nov 2013 19:47:38 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: Jilles Tjoelker Subject: Re: Automated submission of kernel panic reports References: <526F8EB3.1040205@freebsd.org> <20131103212950.GA22571@stack.nl> In-Reply-To: <20131103212950.GA22571@stack.nl> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:M9rtwMXUyDsGBr8v3qibc1lw6n63PAfA66s8gJbB1V+CfWaW+Lm MS0GYCkwglGFibC+TMfp6BCvMSt2q25VYgt7fVC4CR+RL2fxlStC9Wa/gcM1L+RwSASo22F Wrfcsn4GYpD9Om9MdSwkC5GZx4rRIlirzxAMzg9EqmjyTZntnMTNvu61H1QZcYfjUCkYtvp B06Y4UJfagnwN8wzIzK9w== Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 18:48:17 -0000 Jilles Tjoelker wrote, On 11/03/2013 22:29: > The 'local' utility > returns 0 because it successfully created the local variable, ignoring > the status from the command substitution. Use > local tmpfile > tmpfile=`mktemp` || exit 1 > in both occurrences. Does it make sense to use local tmpfile || exit 1 tmpfile=`mktemp` || exit 1 ? Bah, let's just use ``set -e''... From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 18:50:13 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0A6CAEF5 for ; Mon, 4 Nov 2013 18:50:13 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7AAC82A5B for ; Mon, 4 Nov 2013 18:50:12 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MfEMs-1VJ0000Yv4-00OnGj for ; Mon, 04 Nov 2013 19:50:10 +0100 Message-ID: <5277EC42.1020709@gmx.com> Date: Mon, 04 Nov 2013 19:49:38 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: Colin Percival , FreeBSD Hackers , FreeBSD current Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> In-Reply-To: <527779ED.9040303@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:C6i0BbjbN/B+bMEycynCa7X/QbOVdxLSoW2ZW9EcCQYadbQyqZ5 QobtAx3yNlnCTvARaaQYXd47NDSSlUmSef+vUGorBFWoU3gaayAFFGvV1t2dNm+48Q1EfFc nAkxNateAyuPEkC/2pal2kI0MbHHWnJBMz8VJgVxMPX5pozUcTP5P2xX78nPwofgNJ0MQHr PPE4A2f6/gLh3cDI04wUA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 18:50:13 -0000 Colin Percival wrote, On 11/04/2013 11:41: > After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have > now added sysutils/panicmail to the FreeBSD ports tree. The pkesh script is probably still in need of a big review (S00N(TM)...). From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 21:31:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1505ABFE for ; Mon, 4 Nov 2013 21:31:10 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 9DD6B248C for ; Mon, 4 Nov 2013 21:31:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=/rwPuf2MaNsL1wSQtgryW8JJEX0=; b=eFXjWTfgbESiRJKgsp Mh1Cxe0+4u6PIKCZDWlJzh4nr+OBynzbop1HBddUkelNObFlOkB86xeZD7QcpnNN Hr+j9S8FENAZlw6KYkPMI9Ug0GV186nU+pXGvXjrLQnQPM05MhokDagz/MxFRuhj gdOugoNXvmXDiQAYCEdX9QBgg= Received: by mf91 with SMTP id mf91.3807.527812175 Mon, 04 Nov 2013 21:31:03 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi32 (SG) with ESMTP id 1422506aaa5.17c6.2441e8 for ; Mon, 04 Nov 2013 15:31:03 -0600 (CST) Received: (qmail 75532 invoked from network); 4 Nov 2013 21:31:00 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 21:31:00 -0000 Received: (qmail 5554 invoked from network); 4 Nov 2013 21:29:27 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 21:29:27 -0000 Message-ID: <527811B7.5090102@freebsd.org> Date: Mon, 04 Nov 2013 13:29:27 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Alfred Perlstein , FreeBSD Hackers , FreeBSD current , Jordan Hubbard Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> <527797D8.5040404@freebsd.org> In-Reply-To: <527797D8.5040404@freebsd.org> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0c3oNCjDEvTPokX+lNcd8VQtBvDBYQVfcIgOx+xWPVwhZ9NU/Zijcff++wlMx6t+VOhpU2iogNsFw+8XSI/SglcNflPGTj+sw8Johw6ojvLvo= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 21:31:10 -0000 On 11/04/13 04:49, Alfred Perlstein wrote: > Colin, have you had a few minutes to check out the crash reporting facilities in > FreeNAS? Yes. > The reason I ask is that: > > 1) we would like to share code. > 2) we have this running for a few months now and have a huge corpus of information. > 3) we are building a nice UI (screenshots attached) over it, we have a couple of > thousands of lines of code we can share for this. Once I have a useful number of panics collected, I was hoping to take the best pieces from FreeNAS's processing, from the SoC project, and from the processing I've been doing of automatic panic reports from EC2 instances. > We send a minimal set of information: kernel stack trace, ddb buffer and > hardware. Just enough to get some very, very handy stuff. I'm currently sending the dump header and what I get from kgdb 'bt'. If I find that I'm missing something important, I can always add it to a new version of the panicmail port. ;-) > I can share with you offline the crash server code, it's django and relatively > straight forward. I'll come back to you about this once I have some data. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 4 21:31:51 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 94498DF2 for ; Mon, 4 Nov 2013 21:31:51 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 45F8B24A8 for ; Mon, 4 Nov 2013 21:31:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=BnxJuGbFMwFIyExZA7qIz7BRsaU=; b=jYFRxQTOyKYT3BPhNP HPAjIwO4HHIkJWu2i//d2ihsvT5Efmiw8tULeYOfuN2grw86ICqdBFnesIffFe7s LEk+YTCzIaCvUYhpmYb/7KL53KQYy9HtG5fJ4r7z9lVROJ31sLb0KfAfTmbwREPd C/6Rh+rlXRnUg34U2QPCP7XTQ= Received: by mf66.sendgrid.net with SMTP id mf66.31948.527812448 Mon, 04 Nov 2013 21:31:49 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.15]) by mi40 (SG) with ESMTP id 14225075cd2.12f5.30e43b for ; Mon, 04 Nov 2013 15:31:48 -0600 (CST) Received: (qmail 75567 invoked from network); 4 Nov 2013 21:31:47 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 4 Nov 2013 21:31:47 -0000 Received: (qmail 5562 invoked from network); 4 Nov 2013 21:30:14 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 4 Nov 2013 21:30:14 -0000 Message-ID: <527811E6.3070408@freebsd.org> Date: Mon, 04 Nov 2013 13:30:14 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: dt71@gmx.com, FreeBSD Hackers , FreeBSD current Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> <5277EC42.1020709@gmx.com> In-Reply-To: <5277EC42.1020709@gmx.com> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cO2KNgwBztJF2poLVyX7dLogOmfUkJuon+77JUExhoBWYTA2BrMkcoOBoFoXoZX68zxDKK0asGs2uzIQALrJjSYCAkFik2sgO1D/tmUdPoOc= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Nov 2013 21:31:51 -0000 On 11/04/13 10:49, dt71@gmx.com wrote: > Colin Percival wrote, On 11/04/2013 11:41: >> After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have >> now added sysutils/panicmail to the FreeBSD ports tree. > > The pkesh script is probably still in need of a big review (S00N(TM)...). Go for it! It's a very simple script. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 03:24:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0CBCCE0C for ; Tue, 5 Nov 2013 03:24:34 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id 95578273A for ; Tue, 5 Nov 2013 03:24:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=nIEGU6EcoVXcLiUvk25QdqaQOzk=; b=qdbg3+nrnnypMMaMUp 70Yy0IRxc+U5Uc3POAGhwsttApGeeeEJuZz6v6eHgJ0b8TRWVdp9rFMA3BCR9UH+ Up2ILh8lncoFhBNHtoP+OKoCcQzEJdHNvZnIuyxDjulyyDgnIXykFK436MJBzcMT 3bTIE4zkG8RHWnjsrArNxLvyQ= Received: by mf74.sendgrid.net with SMTP id mf74.8350.527864F07 Tue, 05 Nov 2013 03:24:32 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi30 (SG) with ESMTP id 142264a4c82.1acd.305298 for ; Mon, 04 Nov 2013 21:24:32 -0600 (CST) Received: (qmail 86814 invoked from network); 5 Nov 2013 03:24:30 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 5 Nov 2013 03:24:30 -0000 Received: (qmail 2278 invoked from network); 5 Nov 2013 03:22:56 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 5 Nov 2013 03:22:56 -0000 Message-ID: <5278648F.4070904@freebsd.org> Date: Mon, 04 Nov 2013 19:22:55 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Thomas Mueller , freebsd-current@freebsd.org Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: In-Reply-To: X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0c2OLEpC94DeFoAwMHV0DEFvpH0v5vMwnIcX188zugTx+TVNENDqeJyHILHSJwVcANYKK1O4dfT+hACWgPnZyms3JLV76InhjicHIYXkiN9oA= Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 03:24:34 -0000 On 11/04/13 18:26, Thomas Mueller wrote: > Question that arises is how does the system know where to send the email, and through what SMTP server, especially if panicmail_autosubmit="YES". The code assumes that your system knows how to deliver email. An out-of-the-box FreeBSD install has sendmail and can do this. If you don't enable panicmail_autosubmit then it also assumes you're reading or forwarding root's email -- which you should be doing anyway. > In the case of a kernel panic, wouldn't the system crash/freeze, and would it then be able to compose an email message? The email is generated from the crashdump when the system next boots. > I use mail/mpop and mail/msmtp rather than messing with sendmail or postfix; have multiple email accounts and inboxes. > > Now come to think of it, I don't think I ever sent an email from FreeBSD as root, only as nonroot. Don't you get "daily run output" and "security run output" emails? > Something like panicmail ought to be ported to NetBSD pkgsrc, considering that NetBSD seems so much more unstable and crash-prone than FreeBSD on my hardware. Go right ahead. It's a small shell script -- might even work fine without any changes. It's BSD licensed, of course. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 02:26:09 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6E1B634E for ; Tue, 5 Nov 2013 02:26:09 +0000 (UTC) (envelope-from mueller6721@twc.com) Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.225]) by mx1.freebsd.org (Postfix) with ESMTP id 2BFD22449 for ; Tue, 5 Nov 2013 02:26:09 +0000 (UTC) Received: from [74.130.200.176] ([74.130.200.176:65127] helo=localhost) by cdptpa-oedge02 (envelope-from ) (ecelerity 3.5.0.35861 r(Momo-dev:tip)) with ESMTP id A4/EA-27821-93758725; Tue, 05 Nov 2013 02:26:01 +0000 Date: Tue, 05 Nov 2013 02:26:01 +0000 Message-ID: From: "Thomas Mueller" To: freebsd-current@freebsd.org Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail X-RR-Connecting-IP: 107.14.168.130:25 X-Cloudmark-Score: 0 X-Mailman-Approved-At: Tue, 05 Nov 2013 03:32:22 +0000 Cc: FreeBSD Hackers , Colin Percival X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 02:26:09 -0000 > Hi all, > After considerable review on freebsd-hackers (thanks dt71 and jilles!) I have > now added sysutils/panicmail to the FreeBSD ports tree. If you install this > and add > panicmail_enable="YES" > to your /etc/rc.conf, a panic report will be generated and sent to root@ for > you to review and submit (via email). You can skip the reviewing step and > submit panics automatically by setting panicmail_autosubmit="YES". > The panics submitted are encrypted to an RSA key which I hold in order to keep > them secure in transit; and I intend to keep the raw panic reports confidential > except to the minimum extent necessary for other developers to help me process > the incoming reports. > If I receive enough panic reports to be useful, I hope to provide developers > with aggregate statistics. This may include: > * regular email reports listing the "top panics", to help guide developers > towards the most fertile areas for stability improvements; > * email to specific developers alerting them to recurring panics in code they > maintain (especially if it becomes clear that the panic has been recently > introduced); and > * guidance to re@ and secteam@ about how often a particular panic occurs if > an errata notice is being considered > as well as other yet-to-be-imagined reports of a similarly aggregate and > anonymized nature. > So please install the sysutils/panicmail port and enable it in rc.conf! This > all depends on getting useful data, and I can't do that without your help. -- > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid Question that arises is how does the system know where to send the email, and through what SMTP server, especially if panicmail_autosubmit="YES". In the case of a kernel panic, wouldn't the system crash/freeze, and would it then be able to compose an email message? I use mail/mpop and mail/msmtp rather than messing with sendmail or postfix; have multiple email accounts and inboxes. Now come to think of it, I don't think I ever sent an email from FreeBSD as root, only as nonroot. Something like panicmail ought to be ported to NetBSD pkgsrc, considering that NetBSD seems so much more unstable and crash-prone than FreeBSD on my hardware. Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 10:01:52 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0CC5363A; Tue, 5 Nov 2013 10:01:52 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EEFF52A6F; Tue, 5 Nov 2013 10:01:50 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA01767; Tue, 05 Nov 2013 12:01:49 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VddSG-000ATG-PH; Tue, 05 Nov 2013 12:01:48 +0200 Message-ID: <5278C1D4.1000601@FreeBSD.org> Date: Tue, 05 Nov 2013 12:00:52 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Mark Johnston Subject: Re: sdt "sname" removal References: <5270246B.6070105@FreeBSD.org> <20131031033636.GC9355@raichu> <5271FD6B.1040807@FreeBSD.org> In-Reply-To: <5271FD6B.1040807@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, dtrace@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 10:01:52 -0000 on 31/10/2013 08:49 Andriy Gapon said the following: > on 31/10/2013 05:36 Mark Johnston said the following: >> On Tue, Oct 29, 2013 at 11:11:07PM +0200, Andriy Gapon wrote: >>> >>> I never understood why FreeBSD SDT as opposed to upstream SDT requires the same >>> or almost the same probe name to be specified twice. This seems to be silly and >>> a little bit error-prone. >>> In other words, I do not see any reason not to re-use the original upstream >>> trick where double underscore in a providers name in the C code gets converted >>> to a single dash in a DTrace provider name. [*] >>> >>> So here is my take at that: >>> http://people.freebsd.org/~avg/sdt-sname-removal.diff >>> >>> An inline preview of the change: >>> -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_ok, priv-ok, "int"); >>> -SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv_err, priv-err, "int"); >>> +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__ok, "int"); >>> +SDT_PROBE_DEFINE1(priv, kernel, priv_check, priv__err, "int"); >>> >>> It's possible that I missed some places where old style SDT_PROBE_DEFINE macros >>> are used or where an old probe name is used with SDT_PROBE_ARGTYPE or SDT_PROBE. >> >> A good way to test this is to compare the output of 'dtrace -lv' with and >> without your change. If nothing changes, I'd be pretty confident that >> the diff is correct. > > Provided that my kernel has all of the SDT probes :-) > >>> >>> Please test, review, comment, etc. >> >> I don't think this diff will apply cleanly to head - I've made some changes >> that will cause conflicts, and the diff doesn't touch netinet/in_kdtrace.c >> or kern/subr_devstat.c. > > Oh, yes, my head is from ~ 2 month ago. Need to update ASAP and will rebase the > change then. > >> Could you also update the SDT(9) man page? Also >> the "strlcpy(name, ..." immediately before the loop you added to sdt.c >> becomes redundant. > > Good points. Will fix. I have rebased my changes and addressed your comments. The updated patch is in the same place: http://people.freebsd.org/~avg/sdt-sname-removal.diff I am also merging the other thread into this one, so here is an update patch for DTRACE_PROBE* macros: http://people.freebsd.org/~avg/dtrace-probe-macros.diff The most notable change is that the argument types are now recorded. Thank you for reviewing and your suggestions! -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 18:23:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 160F5BA; Tue, 5 Nov 2013 18:23:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E3A022C31; Tue, 5 Nov 2013 18:23:23 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 026B3B988; Tue, 5 Nov 2013 13:23:23 -0500 (EST) From: John Baldwin To: freebsd-current@freebsd.org Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail Date: Tue, 5 Nov 2013 12:27:14 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <527779ED.9040303@freebsd.org> <527797D8.5040404@freebsd.org> <527811B7.5090102@freebsd.org> In-Reply-To: <527811B7.5090102@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201311051227.14157.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 05 Nov 2013 13:23:23 -0500 (EST) Cc: FreeBSD Hackers , Alfred Perlstein , Jordan Hubbard , Colin Percival X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 18:23:24 -0000 On Monday, November 04, 2013 4:29:27 pm Colin Percival wrote: > On 11/04/13 04:49, Alfred Perlstein wrote: > > Colin, have you had a few minutes to check out the crash reporting facilities in > > FreeNAS? > > Yes. > > > The reason I ask is that: > > > > 1) we would like to share code. > > 2) we have this running for a few months now and have a huge corpus of information. > > 3) we are building a nice UI (screenshots attached) over it, we have a couple of > > thousands of lines of code we can share for this. > > Once I have a useful number of panics collected, I was hoping to take the best > pieces from FreeNAS's processing, from the SoC project, and from the processing > I've been doing of automatic panic reports from EC2 instances. > > > We send a minimal set of information: kernel stack trace, ddb buffer and > > hardware. Just enough to get some very, very handy stuff. > > I'm currently sending the dump header and what I get from kgdb 'bt'. If I find > that I'm missing something important, I can always add it to a new version of > the panicmail port. ;-) One of my previous employers maintained a database of panics and I added ways to recognize "known panics" and tag them. I ended up relying a lot on stack trace details from specific OS versions to mark a panic as an instance of a specific bug. Also, you may have very different stack traces even on the same build version for a single bug. In the case of my employer we had a constrained set of kernel configs and specific build versions to work with. It might be harder to correctly match panics in the wild what with patched trees and random kernel configs. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 21:01:54 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 48F4664D for ; Tue, 5 Nov 2013 21:01:54 +0000 (UTC) (envelope-from bounces+73574-4a99-freebsd-hackers=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id C6D16267A for ; Tue, 5 Nov 2013 21:01:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=4gVMKdsg3RpsJj/QrPk447Ju66Q=; b=yhLPWKOiXnNLjGTS0N 1OAcI7V73fy0LhTeC9hRJygFFdqiqJlguX0iJR+cN7exTY0dQm9LISbwJYOipa4z rtMX4e50RBjB8lA4pyjgZ31G3XJbM8gEl3tUQg5iaLRH9z+IVri1+NeXMvdBCOjI t7O/L2wFh8HQogCZ3HttryJ9Q= Received: by mf33.sendgrid.net with SMTP id mf33.29110.52795CC13 Tue, 05 Nov 2013 21:01:53 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi40 (SG) with ESMTP id 1422a125280.12f2.3682c0 for ; Tue, 05 Nov 2013 15:01:53 -0600 (CST) Received: (qmail 28420 invoked from network); 5 Nov 2013 21:01:52 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 5 Nov 2013 21:01:52 -0000 Received: (qmail 9076 invoked from network); 5 Nov 2013 21:00:15 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 5 Nov 2013 21:00:15 -0000 Message-ID: <52795C5F.7060008@freebsd.org> Date: Tue, 05 Nov 2013 13:00:15 -0800 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: John Baldwin , freebsd-current@freebsd.org Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail References: <527779ED.9040303@freebsd.org> <527797D8.5040404@freebsd.org> <527811B7.5090102@freebsd.org> <201311051227.14157.jhb@freebsd.org> In-Reply-To: <201311051227.14157.jhb@freebsd.org> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: W2XBZA0V/n0voZZ6SjDkgjXvzGvkLIaljy40FLIRIHTVMXCc7ynl2WKQUz0qqp0cFB/d/8c+5QtH1ix9B1GswpgGU1zg8ZX8Ic4E6qeC+qhcrVWNfHftDZRysh6T6ZxGIsCWkyaSpIk6GoNJ4U8qyTwdU5hh1ke3c01RNxaoZnc= Cc: FreeBSD Hackers , Alfred Perlstein , Jordan Hubbard X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 21:01:54 -0000 On 11/05/13 09:27, John Baldwin wrote: > One of my previous employers maintained a database of panics and I added ways > to recognize "known panics" and tag them. I ended up relying a lot on stack > trace details from specific OS versions to mark a panic as an instance of a > specific bug. Also, you may have very different stack traces even on the same > build version for a single bug. In the case of my employer we had a > constrained set of kernel configs and specific build versions to work with. > It might be harder to correctly match panics in the wild what with patched > trees and random kernel configs. Right, I'm sure there will be panics I can't match up against anything else -- but this is fine. If I get enough panic reports, I can still get useful data out even if some of them aren't immediately usable. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 21:53:57 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AB84A94C; Tue, 5 Nov 2013 21:53:57 +0000 (UTC) (envelope-from cjpugmed@gmail.com) Received: from mail-vb0-x230.google.com (mail-vb0-x230.google.com [IPv6:2607:f8b0:400c:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 32FC429EE; Tue, 5 Nov 2013 21:53:57 +0000 (UTC) Received: by mail-vb0-f48.google.com with SMTP id o19so2768214vbm.7 for ; Tue, 05 Nov 2013 13:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=3RSWT7mJUBnNJ4Ji44MhEJNlme2P1yx3mN51qW/yT+s=; b=xLXHXji2kLgUbKx7Q7ylK2FFFMJChC8EbVvY+Q6g0OhvomMTevAGCzSnTkR6udXJZG r3IGEpLS1bZ5AysxIIIpA2rJXOjJ6NC92SbU+at0vm8b0ss0f7ritORvIg+3wTAlsfeC 8nFL/JPsYhObdGob1SRdfVAkw/c59ObmXbC56EiJZW9c5I4+UNP2S1yEFP/Y26MA3aCx 5E/MB275IeSdxq6PK71+lKMJIKkLlNlFl+cbfkywI8u08NJBgASgfn3CzpAmR/sEKNyG tpxOhMB9pfXER9T6uAguMLSz29QFToNfIBAqFbcOONFIHMCOWE9V8kZvrHNAK7y8VVe/ j27Q== MIME-Version: 1.0 X-Received: by 10.221.64.17 with SMTP id xg17mr17171606vcb.5.1383688436221; Tue, 05 Nov 2013 13:53:56 -0800 (PST) Received: by 10.220.1.17 with HTTP; Tue, 5 Nov 2013 13:53:56 -0800 (PST) Date: Tue, 5 Nov 2013 22:53:56 +0100 Message-ID: Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail From: Carlos Jacobo Puga Medina To: freebsd-hackers@freebsd.org, cperciva@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 21:53:57 -0000 Seems that rc.d scripts order doesn't allow geli-encrypted swap and dumpdev to cooperate together. For this reason, I wonder what happens in such cases, is not possible to use panicmail unless Pawel's proposal [1] which argued to obtain and save crash info in memory somewhere before we configure swap and copy it to /var/crash/ once we mount it. Is sustainable this proposal or is there yet another alternative to consider for those who use an encrypted GELI swap? --CJPM From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 21:55:33 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 165ABA47; Tue, 5 Nov 2013 21:55:33 +0000 (UTC) (envelope-from cjpugmed@gmail.com) Received: from mail-vb0-x232.google.com (mail-vb0-x232.google.com [IPv6:2607:f8b0:400c:c02::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B98852A05; Tue, 5 Nov 2013 21:55:32 +0000 (UTC) Received: by mail-vb0-f50.google.com with SMTP id x16so2916397vbf.9 for ; Tue, 05 Nov 2013 13:55:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=wue4mHW8FmxuGoxeevZozKvkIH6fzGoSpgJzJLoGzOc=; b=qb4Ajtm2EvcWrJVxasMViTul8kIsh0YR3vuLkYMd3bLEdSDxXmtoQO6SEpS9eqO1eq O8EAUYlm3JeNREDVfUxFffFh950hokiQcICbWTspRnbQRAvkt7oLvCdMNUy9ku+rWEZY UoZcuaVoDETsY522pYGmDPzb6S4MQoMO00JOsOxqeWQd1BXLpUgnEx/88+nIj4RAHrLe OQsZ2a9hIxUAPn+eSOE3MYgkGyZfsrrN+pevj1MHFCRqQv5opV38ze5y42PdH6zeU+9Y dFkqIWjrBlgkz3r/iP+s2R17Ikr+xvs9UQbPIXmMJ6I0eme3moiySeGLj9RhR7Gxz7A/ P9fA== MIME-Version: 1.0 X-Received: by 10.52.120.78 with SMTP id la14mr14829161vdb.9.1383688531925; Tue, 05 Nov 2013 13:55:31 -0800 (PST) Received: by 10.220.1.17 with HTTP; Tue, 5 Nov 2013 13:55:31 -0800 (PST) In-Reply-To: References: Date: Tue, 5 Nov 2013 22:55:31 +0100 Message-ID: Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail From: Carlos Jacobo Puga Medina To: freebsd-hackers@freebsd.org, cperciva@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 21:55:33 -0000 Add missing thread related to Pawel's proposal [1] [1] http://lists.freebsd.org/pipermail/freebsd-fs/2010-September/009256.html 2013/11/5 Carlos Jacobo Puga Medina > Seems that rc.d scripts order doesn't allow geli-encrypted swap and > dumpdev to cooperate together. For this reason, I wonder what happens in > such cases, is not possible to use panicmail unless Pawel's proposal [1] > which argued to obtain and save crash info in memory somewhere before we > configure swap and copy it to /var/crash/ once we mount it. > > Is sustainable this proposal or is there yet another alternative to > consider for those who use an encrypted GELI swap? > > --CJPM > From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 5 22:02:34 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1F894F26; Tue, 5 Nov 2013 22:02:34 +0000 (UTC) (envelope-from jlh@FreeBSD.org) Received: from caravan.chchile.org (caravan.chchile.org [178.32.125.136]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B83602A8F; Tue, 5 Nov 2013 22:02:33 +0000 (UTC) Received: by caravan.chchile.org (Postfix, from userid 1000) id 4757EC2D26; Tue, 5 Nov 2013 22:02:26 +0000 (UTC) Date: Tue, 5 Nov 2013 23:02:26 +0100 From: Jeremie Le Hen To: Baptiste Daroussin Subject: Re: Importing netbsd cdb Message-ID: <20131105220225.GF37879@caravan.chchile.org> Mail-Followup-To: Baptiste Daroussin , hackers@FreeBSD.org References: <20131027232628.GB74512@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131027232628.GB74512@ithaqua.etoilebsd.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Nov 2013 22:02:34 -0000 Hi Baptiste, On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote: > > Here is a code that imports the cdbrw from netbsd into a new lib/libcdbrw > library, the read part is also added to libc but not exposed. > > As an example of using that library I also got the service_mkdb patches from > netbsd that makes it by default emit a cdb database and add a switch to allow it > to still create the old .db database format. > > in the libc, getservent has been modified to first try to read the .cdb files > and fallback on reading the old .db file. (I'm not sure if it is worth keeping > reading the old db format.) > > http://people.freebsd.org/~bapt/cdbrw.diff > > The plan after that is to get pw_util(3) directly generating a cdb file for > pwd.db and spwd.db and modifiy pwd_mkdb(8) so that by default it uses the API > from pw_util(3) and have a switch to fallback on creating in bdb format. > > I also plan to do the same for cap_mkdb(1) and getcap(3). > > With cdb querying is way faster (I don't have number but I can get some it > needed) > the size of the db is also way smaller: > 64K /var/db/services.cdb > 2,1M /var/db/services.db > > Any objection? What are the benefits of this, beside generating a much smaller database? Do you know about benchmarks that compare it against the current database engine used? It would be unfortunate to notice a regression afterward :). Also, if you go ahead, do you have an idea of the migration procedure we could offer to our users, when they will upgrade to 11.0-RELEASE? Cheers, -- Jeremie Le Hen Scientists say the world is made up of Protons, Neutrons and Electrons. They forgot to mention Morons. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 01:43:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B547D888 for ; Wed, 6 Nov 2013 01:43:10 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5F71C2577 for ; Wed, 6 Nov 2013 01:43:10 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0LhOSG-1VzkYx0smu-00mbRr for ; Wed, 06 Nov 2013 02:43:03 +0100 Message-ID: <52799E85.4050002@gmx.com> Date: Wed, 06 Nov 2013 02:42:29 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: FreeBSD Hackers , jasone@freebsd.org Subject: alignment of thread-local storage Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:c2+iTcFPHKObqtyUri/PxnV9rFifdr8ObV5TVajoRFY6WRXXy/f UhvzAUktV+SIEn6RvdCYxPupA8rfuu2uSZ5OSKJZHccyOwRwux8CM7sTgfygdpz7fR/59oa Jgao/9Di+5ur2XPdnXd+mDMrpvTXxTyEXRiSDFJRZgb710i/o42lBkdQbxF+v6CZI6PIRvM NAFEj7Hlsun3hmC4g8Nuw== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 01:43:10 -0000 Starting with revision 191847 of Clang/LLVM, a bus error tends to happen in realloc() under special circumstances. To reproduce: (1) Compile the following program and link it with the cURL library (or see (3b)): #include #include int main(void) { getpwuid(0); } (2) Compile libc (I use -CURRENT), but when compiling jemalloc.c, specifically use Clang, revision >=191847, and use -march=prescott (or similar) and at least -O1. (3) Run the program with the libc just built. The program will hopefully stop with a bus error. (3b) If choosing not to link with any library, then run the program through gdb(1). The program will hopefully also hit the bus error. In other words: # echo 'CPUTYPE=prescott' >> /etc/make.conf # echo 'CFLAGS=-g -O1' >> /etc/make.conf # cd /usr/src/lib/libc && make # cd # cat > x.c <<-EOF #include #include int main(void) { getpwuid(0); } EOF # clang x.c -L /usr/local/lib -lcurl # env LD_LIBRARY_PATH=/usr/src/lib/libc ./a.out The last 2 command lines can also be: # clang x.c # env LD_LIBRARY_PATH=/usr/src/lib/libc gdb ./a.out (gdb) run The backtrace is: 0x281d4235 in __realloc (ptr=0x282db7d0, size=) at jemalloc_jemalloc.c:1249 1249 ta->allocated += usize; (gdb) bt #0 0x281d4235 in __realloc (ptr=0x282db7d0, size=) at jemalloc_jemalloc.c:1249 #1 0x2826c119 in yygrowstack (data=0x282f5144) at nsparser.c:411 #2 0x2826b7e6 in _nsyyparse () at nsparser.c:470 #3 0x28276d34 in nss_configure () at /usr/src/lib/libc/net/nsdispatch.c:372 #4 0x28276301 in _nsdispatch (retval=0xbfbfdbe4, disp_tab=0x282dafb4, database=0x282d4982 "passwd", method_name=0x282d49b1 "getpwuid_r", defaults=0x282da594) at /usr/src/lib/libc/net/nsdispatch.c:645 #5 0x28254e9d in getpwuid_r (uid=0, pwd=0x282f4f50, buffer=0x28c0c400 "", bufsize=1024, result=0xbfbfdbe4) at /usr/src/lib/libc/gen/getpwent.c:609 #6 0x28255208 in wrap_getpwuid_r (key=..., pwd=0x282f4f50, buffer=0x28c0c400 "", bufsize=1024, res=0xbfbfdbe4) at /usr/src/lib/libc/gen/getpwent.c:686 #7 0x28254fda in getpw (fn=0x282551b0 , key=...) at /usr/src/lib/libc/gen/getpwent.c:654 #8 0x282551a3 in getpwuid (uid=0) at /usr/src/lib/libc/gen/getpwent.c:714 #9 0x0804860a in main () The current understanding (sort of) of the problem is: - The __jemalloc_thread_allocated_tls variable is updated using a processor instruction that requires alignment (paddq), that is, as of Clang/LLVM r191847. The variable is defined as: __thread thread_allocated_t __attribute__((tls_model("initial-exec"))) thread_allocated_tls = {0,0}; - The variable turns out to be insufficiently aligned, having only 4-byte alignment. the thread_allocated_tls object *should* be 16-byte aligned is it? (if not, that's the bug; the generated code looks correct and good) in the IR we have @thread_allocated_tls = global ..., align 16 how is storage for TLS variables allocated in the first place? compilers like to pretend that they exist magically, but they do not yeah, Clang emits the TLS variable as a 16-byte aligned symbol zygoloid: how are TLS variable actually allocated though? it can't be done once at load time like for globals so I'm guessing it must be malloc()ed or something if so, suspect your malloc So, what could be the bottom line of this? That is, which one is WRONG -- FreeBSD or Clang (or both)? From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 02:33:44 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E00A47B; Wed, 6 Nov 2013 02:33:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7E0A5278F; Wed, 6 Nov 2013 02:33:44 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id rA62XZS7032562; Wed, 6 Nov 2013 04:33:35 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua rA62XZS7032562 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id rA62XZk7032561; Wed, 6 Nov 2013 04:33:35 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Nov 2013 04:33:35 +0200 From: Konstantin Belousov To: dt71@gmx.com Subject: Re: alignment of thread-local storage Message-ID: <20131106023335.GZ59496@kib.kiev.ua> References: <52799E85.4050002@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hLtrwzwPDOHHtPAe" Content-Disposition: inline In-Reply-To: <52799E85.4050002@gmx.com> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: FreeBSD Hackers , jasone@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 02:33:44 -0000 --hLtrwzwPDOHHtPAe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 02:42:29AM +0100, dt71@gmx.com wrote: > Starting with revision 191847 of Clang/LLVM, a bus error tends to happen = in realloc() under special circumstances. >=20 As I understand, the rev. of clang referenced is higher than what we have in src/contrib, right ? >=20 > To reproduce: >=20 > (1) Compile the following program and link it with the cURL library (or s= ee (3b)): > #include > #include > int main(void) { getpwuid(0); } > (2) Compile libc (I use -CURRENT), but when compiling jemalloc.c, specifi= cally use Clang, revision >=3D191847, and use -march=3Dprescott (or similar= ) and at least -O1. > (3) Run the program with the libc just built. The program will hopefully = stop with a bus error. > (3b) If choosing not to link with any library, then run the program throu= gh gdb(1). The program will hopefully also hit the bus error. >=20 Provide the readily build binaries which are neccessary to reproduce the problem, i.e. libc, the main program and any other dso. > In other words: > # echo 'CPUTYPE=3Dprescott' >> /etc/make.conf > # echo 'CFLAGS=3D-g -O1' >> /etc/make.conf > # cd /usr/src/lib/libc && make > # cd > # cat > x.c <<-EOF > #include > #include > int main(void) { getpwuid(0); } > EOF > # clang x.c -L /usr/local/lib -lcurl > # env LD_LIBRARY_PATH=3D/usr/src/lib/libc ./a.out >=20 > The last 2 command lines can also be: > # clang x.c > # env LD_LIBRARY_PATH=3D/usr/src/lib/libc gdb ./a.out > (gdb) run >=20 >=20 > The backtrace is: >=20 > 0x281d4235 in __realloc (ptr=3D0x282db7d0, size=3D) > at jemalloc_jemalloc.c:1249 > 1249 ta->allocated +=3D usize; > (gdb) bt > #0 0x281d4235 in __realloc (ptr=3D0x282db7d0, size=3D) > at jemalloc_jemalloc.c:1249 > #1 0x2826c119 in yygrowstack (data=3D0x282f5144) at nsparser.c:411 > #2 0x2826b7e6 in _nsyyparse () at nsparser.c:470 > #3 0x28276d34 in nss_configure () at /usr/src/lib/libc/net/nsdispatch.c:= 372 > #4 0x28276301 in _nsdispatch (retval=3D0xbfbfdbe4, disp_tab=3D0x282dafb4, > database=3D0x282d4982 "passwd", method_name=3D0x282d49b1 "getpwuid_r= ", > defaults=3D0x282da594) at /usr/src/lib/libc/net/nsdispatch.c:645 > #5 0x28254e9d in getpwuid_r (uid=3D0, pwd=3D0x282f4f50, buffer=3D0x28c0c= 400 "", > bufsize=3D1024, result=3D0xbfbfdbe4) at /usr/src/lib/libc/gen/getpwe= nt.c:609 > #6 0x28255208 in wrap_getpwuid_r (key=3D..., pwd=3D0x282f4f50, > buffer=3D0x28c0c400 "", bufsize=3D1024, res=3D0xbfbfdbe4) > at /usr/src/lib/libc/gen/getpwent.c:686 > #7 0x28254fda in getpw (fn=3D0x282551b0 , key=3D...) > at /usr/src/lib/libc/gen/getpwent.c:654 > #8 0x282551a3 in getpwuid (uid=3D0) at /usr/src/lib/libc/gen/getpwent.c:= 714 > #9 0x0804860a in main () >=20 >=20 > The current understanding (sort of) of the problem is: >=20 > - The __jemalloc_thread_allocated_tls variable is updated using a process= or instruction that requires alignment (paddq), that is, as of Clang/LLVM r= 191847. The variable is defined as: > __thread thread_allocated_t __attribute__((tls_model("initial-exec"))) th= read_allocated_tls =3D {0,0}; >=20 > - The variable turns out to be insufficiently aligned, having only 4-byte= alignment. >=20 > the thread_allocated_tls object *should* be 16-byte aligned > is it? > (if not, that's the bug; the generated code looks correct and = good) > in the IR we have @thread_allocated_tls =3D global ..., align = 16 >=20 > how is storage for TLS variables allocated in the first place? > compilers like to pretend that they exist magically, but they do n= ot >=20 > yeah, Clang emits the TLS variable as a 16-byte aligned symbol >=20 > zygoloid: how are TLS variable actually allocated though? > it can't be done once at load time like for globals > so I'm guessing it must be malloc()ed or something > if so, suspect your malloc >=20 >=20 > So, what could be the bottom line of this? That is, which one is WRONG --= FreeBSD or Clang (or both)? I do not see anything in the jemalloc sources which indicate that thread_allocated must be 16-bytes aligned. The natural aligment for the structure is 8 bytes. --hLtrwzwPDOHHtPAe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSeap+AAoJEJDCuSvBvK1BIxAP/iAc4repK8yBbOK+A6qd/NLl dumHFHpq4O2q7MnfrfwAxe6MLiWnQUnyTtX+H/LCDIH1DW9bxVlhX55F9/6wixCG NiGrr4nhhgzD/k9bCgCaVN3xDPkAly2X5z0t5H0ywdWySo4JpYjtlEUO8PY3W38q E9xOFla2RQtPyU9Qu4OprksiPUEMCibZzcwRQLLZs1R2e+bxZIm6d2WTLxqvH6ul RQTZ4wood16Eal8ggIrd+xc1A3muVSR9uEP5yq16tJukcN+aNmgHjpHDAGdTAM7V Wd4M8lx2CqluDpKWeDrNosbtUEmLn4LANhBzq4CvRmxvmvExTMPCEntu72Cvcciu qE04bCS9vxVO5RJwzQtdm2Kxvox65b8whU0fqfwxqRmpX8MP9JsWuRpsHumy7veZ bHYwIOOEH5MKFlh+itOmPgUDCrLSYH14gA+oYxmuf7Bc3YVe19wlEc/IpyEzRa+O SmQNu1ReEoDTc/XTKZsVa0woYWoK+LWIFJeWgGN3L1qLoCyyPfUv4t0QCDc5ye5A 5d/2QKmvdBxWfPZev6yzfLEVoU9Z+dT7e2ABrskqTMMDzqw0zCxT9rZW8vwKUneC KQjBMmL7bAp5oYg6a7WK1wVl4xdbRE+We3UI80niCnY1G6mlt4yuvdXDjythISpz iiJNJAX7z6MM5qmP1/d9 =1yEy -----END PGP SIGNATURE----- --hLtrwzwPDOHHtPAe-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 05:20:13 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F297CB32 for ; Wed, 6 Nov 2013 05:20:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A770D225F for ; Wed, 6 Nov 2013 05:20:13 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id e14so16734650iej.36 for ; Tue, 05 Nov 2013 21:20:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=mD8ZNPshuJnIUrxNdDl/PqZZoIRd84QXz0V/sU9IqT0=; b=n8Xbplc5QA8NPZcusxqJk4Q07e2mvcaPDWP1Czeb2wAby2gKIGmeKdOVIQjVAm8umv NwET/t01qPi5qbEr5cfR8Vc8ibtdDJ26uDV/bJOGFc6Pyb/TMc1zKUQ42pwm3ff6E9Z0 Rs86umS88ZgDYFKwu7WjTOJoV/gBlT3JhtdCoqFZ8sdPi5qS5Lp8baWl5FWauuY8V/nU +C0zBVydU/eCJvo+3g378lI6O5bCAST+J997Hz53wauOggbm2jCOqHH0TImMmENKYgCY eqvic2ugpNvL+iBEcpC8ZzKD1tF02GDeOqv+cEyku4taEpNbCNllDtgUf20l6y/igsSb kWSg== X-Received: by 10.50.40.37 with SMTP id u5mr18897650igk.29.1383715213006; Tue, 05 Nov 2013 21:20:13 -0800 (PST) Received: from raichu (24-212-218-13.cable.teksavvy.com. [24.212.218.13]) by mx.google.com with ESMTPSA id qi3sm12366645igc.8.2013.11.05.21.20.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 05 Nov 2013 21:20:12 -0800 (PST) Sender: Mark Johnston Date: Wed, 6 Nov 2013 00:20:10 -0500 From: Mark Johnston To: freebsd-hackers@freebsd.org Subject: dl_iterate_phdr() omits ld-elf.so Message-ID: <20131106052010.GB2826@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 05:20:14 -0000 Hello, While experimenting with dl_iterate_phdr(3), I noticed that it doesn't include the runtime linker itself in the list of objects. This is inconsistent with related interfaces such as /map in procfs, and kinfo_getvmmap(3), so it seems incorrect to me that rtld is excluded from the list of callback arguments. Is there a reason for this behaviour? If not, does anyone have thoughts on the diff below which fixes this? Thanks, -Mark diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c index fa6dc2a..b55effa 100644 --- a/libexec/rtld-elf/rtld.c +++ b/libexec/rtld-elf/rtld.c @@ -3269,6 +3269,11 @@ dl_iterate_phdr(__dl_iterate_hdr_callback callback, void *param) break; } + if (error == 0) { + rtld_fill_dl_phdr_info(&obj_rtld, &phdr_info); + error = callback(&phdr_info, sizeof(phdr_info), param); + } + lock_release(rtld_bind_lock, &bind_lockstate); lock_release(rtld_phdr_lock, &phdr_lockstate); From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 06:50:22 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F1F5AF6C for ; Wed, 6 Nov 2013 06:50:22 +0000 (UTC) (envelope-from dt71@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8A67525EB for ; Wed, 6 Nov 2013 06:50:22 +0000 (UTC) Received: from [157.181.98.186] ([157.181.98.186]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MTSmp-1VEolt2H0p-00SSLF for ; Wed, 06 Nov 2013 07:50:14 +0100 Message-ID: <5279E686.5090508@gmx.com> Date: Wed, 06 Nov 2013 07:49:42 +0100 From: dt71@gmx.com User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:24.0) Gecko/20100101 Firefox/24.0 SeaMonkey/2.21 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: alignment of thread-local storage References: <52799E85.4050002@gmx.com> <20131106023335.GZ59496@kib.kiev.ua> In-Reply-To: <20131106023335.GZ59496@kib.kiev.ua> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:rf98ioX5Qw7d2EQT+xX8EiAAPIwWKLMZiayeLYCWX8OADepgNhr nOmmom4KTuh8Hm0AVDL7p+yyqoo/p0K8EZsTqsvY8pn7fTvoxRQ2dO/k8RZ6AEapfCZl+pw jIB3vLh5ykVyUQVwrb9k6Js34j148xFI8ekjTft2hBOdjbdMTwTkWOlU6lw/9BCQM5r1F9q 6IQ2dSfzgWlwulxcTuMwQ== Cc: FreeBSD Hackers , jasone@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 06:50:23 -0000 Konstantin Belousov wrote, On 11/06/2013 03:33: > On Wed, Nov 06, 2013 at 02:42:29AM +0100, dt71@gmx.com wrote: >> Starting with revision 191847 of Clang/LLVM, a bus error tends to happen in realloc() under special circumstances. >> > As I understand, the rev. of clang referenced is higher than what we have > in src/contrib, right ? Correct. >> - The __jemalloc_thread_allocated_tls variable is updated using a processor instruction that requires alignment (paddq), that is, as of Clang/LLVM r191847. The variable is defined as: >> __thread thread_allocated_t __attribute__((tls_model("initial-exec"))) thread_allocated_tls = {0,0}; >> >> - The variable turns out to be insufficiently aligned, having only 4-byte alignment. > > I do not see anything in the jemalloc sources which indicate that > thread_allocated must be 16-bytes aligned. The natural aligment for the > structure is 8 bytes. Which is still strictly more than the actual alignment (8 vs 4). Clang even attempts to use certain vector-instructions or what, so it supposedly increases its alignment-request appropriately. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 08:46:04 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 80C1E339; Wed, 6 Nov 2013 08:46:04 +0000 (UTC) (envelope-from atte.peltomaki@iki.fi) Received: from filtteri1.pp.htv.fi (filtteri1.pp.htv.fi [213.243.153.184]) by mx1.freebsd.org (Postfix) with ESMTP id 1F09C2BB2; Wed, 6 Nov 2013 08:46:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by filtteri1.pp.htv.fi (Postfix) with ESMTP id F399721BC49; Wed, 6 Nov 2013 10:45:55 +0200 (EET) X-Virus-Scanned: Debian amavisd-new at pp.htv.fi Received: from smtp6.welho.com ([213.243.153.40]) by localhost (filtteri1.pp.htv.fi [213.243.153.184]) (amavisd-new, port 10024) with ESMTP id V8VN4P3WDBWW; Wed, 6 Nov 2013 10:45:51 +0200 (EET) Received: from kameli.org (cs181030192.pp.htv.fi [82.181.30.192]) by smtp6.welho.com (Postfix) with SMTP id E7ABE5BC003; Wed, 6 Nov 2013 10:45:50 +0200 (EET) Received: by kameli.org (sSMTP sendmail emulation); Wed, 06 Nov 2013 10:45:50 +0200 From: "Atte Peltomaki" Date: Wed, 6 Nov 2013 10:45:50 +0200 To: Baptiste Daroussin , hackers@FreeBSD.org Subject: Re: Importing netbsd cdb Message-ID: <20131106084550.GA21489@ass.pp.htv.fi> References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20131105220225.GF37879@caravan.chchile.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 08:46:04 -0000 On Tue, Nov 05, 2013 at 11:02:26PM +0100, Jeremie Le Hen wrote: > On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote: > > > > Here is a code that imports the cdbrw from netbsd into a new lib/libcdbrw > > library, the read part is also added to libc but not exposed. > > > > As an example of using that library I also got the service_mkdb patches from > > netbsd that makes it by default emit a cdb database and add a switch to allow it > > to still create the old .db database format. > > > > in the libc, getservent has been modified to first try to read the .cdb files > > and fallback on reading the old .db file. (I'm not sure if it is worth keeping > > reading the old db format.) > > > > http://people.freebsd.org/~bapt/cdbrw.diff > > > > The plan after that is to get pw_util(3) directly generating a cdb file for > > pwd.db and spwd.db and modifiy pwd_mkdb(8) so that by default it uses the API > > from pw_util(3) and have a switch to fallback on creating in bdb format. > > > > I also plan to do the same for cap_mkdb(1) and getcap(3). > > > > With cdb querying is way faster (I don't have number but I can get some it > > needed) > > the size of the db is also way smaller: > > 64K /var/db/services.cdb > > 2,1M /var/db/services.db > > > > Any objection? > > What are the benefits of this, beside generating a much smaller > database? Do you know about benchmarks that compare it against the > current database engine used? It would be unfortunate to notice a > regression afterward :). At least randomly googled third-party benchmarks suggest that CDB is vastly superior to BDB: http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/ -- Atte Peltomäki atte.peltomaki@iki.fi <> http://kameli.org "Your effort to remain what you are is what limits you" From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 10:39:54 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 47752A81 for ; Wed, 6 Nov 2013 10:39:54 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D41B1231C for ; Wed, 6 Nov 2013 10:39:53 +0000 (UTC) Received: by mail-wg0-f48.google.com with SMTP id b13so4740096wgh.27 for ; Wed, 06 Nov 2013 02:39:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=KQbVP1+iunScqREY4IPXoEYUo8X8JHDZZ6io43ipd7E=; b=HcO9A/sCCCT6H+pFt/KQk3Rtc6nhgk2Df+xoE9MUtNdkc7wMf4594VmtK/I6CPQj+O S5oAVkwUMzdZIEybTeE9poonhGRWSRV5/ZzzcsOwNJfmYBW06oB1E1jnm+glTCX/46cy HLwGrdc7PAgGwF8GnzxPg9N3B/eEdDQph6BXxTWQIcyRslB0DO2uvtS8s8/9RF7sDVzj tCgIo3lD9hwDSGtzqHrJSEoMszOEjUHNul+O6t7dsi4EHel1wYPpJoJBpVU/mZ/X+puo thLlHj8+spf1el7rHiURf8Y5XNZUF3Q5TUU6kMhTjtlKJBnpPSiQtC8wn3fB9hBgpaFd 5D3A== X-Received: by 10.180.73.70 with SMTP id j6mr20287292wiv.47.1383734391868; Wed, 06 Nov 2013 02:39:51 -0800 (PST) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPSA id qc10sm23058162wic.9.2013.11.06.02.39.50 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 06 Nov 2013 02:39:50 -0800 (PST) Sender: Baptiste Daroussin Date: Wed, 6 Nov 2013 11:39:48 +0100 From: Baptiste Daroussin To: Atte Peltomaki Subject: Re: Importing netbsd cdb Message-ID: <20131106103947.GI56315@ithaqua.etoilebsd.net> References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> <20131106084550.GA21489@ass.pp.htv.fi> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TA4f0niHM6tHt3xR" Content-Disposition: inline In-Reply-To: <20131106084550.GA21489@ass.pp.htv.fi> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 10:39:54 -0000 --TA4f0niHM6tHt3xR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 10:45:50AM +0200, Atte Peltomaki wrote: > On Tue, Nov 05, 2013 at 11:02:26PM +0100, Jeremie Le Hen wrote: > > On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote: > > >=20 > > > Here is a code that imports the cdbrw from netbsd into a new lib/libc= dbrw > > > library, the read part is also added to libc but not exposed. > > >=20 > > > As an example of using that library I also got the service_mkdb patch= es from > > > netbsd that makes it by default emit a cdb database and add a switch = to allow it > > > to still create the old .db database format. > > >=20 > > > in the libc, getservent has been modified to first try to read the .c= db files > > > and fallback on reading the old .db file. (I'm not sure if it is wort= h keeping > > > reading the old db format.) > > >=20 > > > http://people.freebsd.org/~bapt/cdbrw.diff > > >=20 > > > The plan after that is to get pw_util(3) directly generating a cdb fi= le for > > > pwd.db and spwd.db and modifiy pwd_mkdb(8) so that by default it uses= the API > > > from pw_util(3) and have a switch to fallback on creating in bdb form= at. > > >=20 > > > I also plan to do the same for cap_mkdb(1) and getcap(3). > > >=20 > > > With cdb querying is way faster (I don't have number but I can get so= me it > > > needed) > > > the size of the db is also way smaller: > > > 64K /var/db/services.cdb > > > 2,1M /var/db/services.db > > >=20 > > > Any objection? > >=20 > > What are the benefits of this, beside generating a much smaller > > database? Do you know about benchmarks that compare it against the > > current database engine used? It would be unfortunate to notice a > > regression afterward :). >=20 > At least randomly googled third-party benchmarks suggest that CDB is > vastly superior to BDB: > http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/ >=20 =46rom the netbsd commit log, it says that in case of services it is at lea= st as slow, and at best faster, from my testing it is way faster, (sorry no numbe= rs) Concerning the migration, the way I did it for now, is that getservent is t= rying to open the cdb database and fallsback on the bdb one, so migrating to 11.0-RELEASE should be ootb. Running: services_mkdb Will generate the cdb version of the db services_mkdb -V db Will generate the old format db. regards, Bapt --TA4f0niHM6tHt3xR Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlJ6HHMACgkQ8kTtMUmk6Ex53wCfXkcvKJVQD2+UFNYk4TENeH3P Z2MAnjAxndTFcClDUVPYiOFXbQQjfePf =G0I4 -----END PGP SIGNATURE----- --TA4f0niHM6tHt3xR-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 13:13:57 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DFE5B14C for ; Wed, 6 Nov 2013 13:13:57 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 768C32E3B for ; Wed, 6 Nov 2013 13:13:57 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/afgnrylsiWy0bTV9rgkJ X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-109-45-222-180.web.vodafone.de [109.45.222.180]) by smtp.strato.de (RZmta 32.11 DYNA|AUTH) with (TLSv1.0:DHE-RSA-AES256-SHA encrypted) ESMTPSA id L03aedpA6CpbdF ; Wed, 6 Nov 2013 14:13:44 +0100 (CET) Received: by britannica.bec.de (sSMTP sendmail emulation); Wed, 06 Nov 2013 14:13:43 +0100 Date: Wed, 6 Nov 2013 14:13:43 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, hackers@FreeBSD.org Subject: Re: Importing netbsd cdb Message-ID: <20131106131343.GA28180@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, hackers@FreeBSD.org References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> <20131106084550.GA21489@ass.pp.htv.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131106084550.GA21489@ass.pp.htv.fi> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 13:13:57 -0000 On Wed, Nov 06, 2013 at 10:45:50AM +0200, Atte Peltomaki wrote: > At least randomly googled third-party benchmarks suggest that CDB is > vastly superior to BDB: > http://www.dmo.ca/blog/benchmarking-hash-databases-on-large-data/ Please note that the term "CDB" is somewhat overloaded, e.g. there are a number of completely different implementations and file formats. Joerg From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 13:53:31 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 19D33175 for ; Wed, 6 Nov 2013 13:53:31 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A269B2136 for ; Wed, 6 Nov 2013 13:53:30 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/afgnrylsiWy0bTV9rgkJ X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-109-45-222-180.web.vodafone.de [109.45.222.180]) by smtp.strato.de (RZmta 32.12 DYNA|AUTH) with (TLSv1.0:DHE-RSA-AES256-SHA encrypted) ESMTPSA id 000b45pA6DrR0FA ; Wed, 6 Nov 2013 14:53:27 +0100 (CET) Received: by britannica.bec.de (sSMTP sendmail emulation); Wed, 06 Nov 2013 14:53:26 +0100 Date: Wed, 6 Nov 2013 14:53:26 +0100 From: Joerg Sonnenberger To: cfe-commits@cs.uiuc.edu, hackers@FreeBSD.org Subject: Re: SSE2 intrinsics: gcc46 vs. clang contradiction Message-ID: <20131106135326.GB29109@britannica.bec.de> Mail-Followup-To: cfe-commits@cs.uiuc.edu, hackers@FreeBSD.org References: <20131101124645.GA73456@regency.nsu.ru> <20131101154320.GA11359@regency.nsu.ru> <52742115.9010404@pathscale.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 13:53:31 -0000 On Sat, Nov 02, 2013 at 01:07:47AM +0100, Dimitry Andric wrote: > Index: tools/clang/lib/Headers/emmintrin.h > =================================================================== > --- tools/clang/lib/Headers/emmintrin.h (revision 193039) > +++ tools/clang/lib/Headers/emmintrin.h (working copy) > @@ -1366,7 +1366,7 @@ _mm_movepi64_pi64(__m128i __a) > } > > static __inline__ __m128i __attribute__((__always_inline__, __nodebug__)) > -_mm_movpi64_pi64(__m64 __a) > +_mm_movpi64_epi64(__m64 __a) > { > return (__m128i){ (long long)__a, 0 }; > } > > Is this OK? Comparing the naming schema of the others, this looks good. Joerg From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 16:45:33 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8FC312FE for ; Wed, 6 Nov 2013 16:45:33 +0000 (UTC) (envelope-from tyler@monkeypox.org) Received: from starfish.geekisp.com (starfish.geekisp.com [216.168.135.166]) by mx1.freebsd.org (Postfix) with SMTP id 21F3B2C3A for ; Wed, 6 Nov 2013 16:45:32 +0000 (UTC) Received: (qmail 26995 invoked by uid 1003); 6 Nov 2013 16:45:31 -0000 Received: from unknown (HELO kiwi.coupleofllamas.com) (tyler@monkeypox.org@172.56.16.12) by mail.geekisp.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 6 Nov 2013 16:45:31 -0000 Date: Wed, 6 Nov 2013 08:45:26 -0800 From: "R. Tyler Croy" To: hackers@FreeBSD.org Subject: Using bsd.prog.mk in a project with multiple binaries Message-ID: <20131106164526.GV11443@kiwi.coupleofllamas.com> Mail-Followup-To: hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="D/26AIznG/rf8cgR" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 16:45:33 -0000 --D/26AIznG/rf8cgR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Howdy, I'm trying to find some documentation or examples of using bsd.prog.mk inside of a project with multiple binaries that need to get created. I'm able to shuffle the manpages and source code into directories to where each binary can have a Makefile, but I cannot find anything in the Developer's Handbook discussing such a project layout. E.g.: -daemon.git/ -daemon/ daemon.man.1 daemon.c -daemontool/ daemontool.man.1 daemontool.c Does one simply use a parent Makefile which calls into each directory? Or is there some other .mk file with helpers for this? A related question would be, does this even make snese as far as a project layout is concerned? This project I intend on creating a port for once it's further along, and I'd like to avoid painting myself into any cornners with the makefiles now if I can avoid it. Any advice/examples/pointers would be appreciated :) Cheers - R. Tyler Croy -------------------------------------- Code: https://github.com/rtyler Chatter: https://twitter.com/agentdero rtyler@jabber.org --D/26AIznG/rf8cgR Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlJ6ciYACgkQFCbH3D9R4W/b0ACglJ8KqqIqj9u84/JhPj8jVBo0 YeEAn3igMGtnqXRapMpqT7C6ZZg2EacI =lZDf -----END PGP SIGNATURE----- --D/26AIznG/rf8cgR-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 16:48:15 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 57942416 for ; Wed, 6 Nov 2013 16:48:15 +0000 (UTC) (envelope-from tyler@monkeypox.org) Received: from starfish.geekisp.com (starfish.geekisp.com [216.168.135.166]) by mx1.freebsd.org (Postfix) with SMTP id 067F52C58 for ; Wed, 6 Nov 2013 16:48:14 +0000 (UTC) Received: (qmail 1074 invoked by uid 1003); 6 Nov 2013 16:48:14 -0000 Received: from unknown (HELO kiwi.coupleofllamas.com) (tyler@monkeypox.org@172.56.16.12) by mail.geekisp.com with (DHE-RSA-AES256-SHA encrypted) SMTP; 6 Nov 2013 16:48:13 -0000 Date: Wed, 6 Nov 2013 08:48:07 -0800 From: "R. Tyler Croy" To: Kamil Choudhury Subject: Re: pkgng for configuration management? Message-ID: <20131106164807.GW11443@kiwi.coupleofllamas.com> Mail-Followup-To: Kamil Choudhury , "hackers@freebsd.org" References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="z8OPzjJiggy3JOV2" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 16:48:15 -0000 --z8OPzjJiggy3JOV2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, 03 Nov 2013, Kamil Choudhury wrote: > I've been setting up a private pkgng repository to push software to a fam= ily of about 20 different hosts.=20 >=20 > One command software deployment is pretty awesome, so I got to thinking: = why not go one step further and start pushing configurations for each of th= ese hosts via pkgng as well (either by putting the config files into the in= itial software pkg, or via a separate pkg that installs only the configurat= ions)?=20 >=20 > Has anyone else tried going down this rabbit hole? If so, what has your e= xperience with the system been? =20 Zach Leslie has written a good pkgng provider for Puppet (https://github.com/xaque208/puppet-pkgng) which worked on FreeBSD9, I've y= et to test it on 10 though. I highly recommend going the Puppet route instead of attempting to use the packaging system for configuration. There's lots of horror stories in the L= inux community of people wrapping everything in the world into debs or rpms, and regretting it later. If you do go down the Puppet route, you may consider joining the Puppet BSD mailing list: Cheers - R. Tyler Croy -------------------------------------- Code: https://github.com/rtyler Chatter: https://twitter.com/agentdero rtyler@jabber.org --z8OPzjJiggy3JOV2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlJ6cscACgkQFCbH3D9R4W/mjgCgkIvHXLP9jyzFVlxvYQVgl8GC 7gkAoJC8EUVwnI7ZQZkh1K2pZblObgKS =dAf1 -----END PGP SIGNATURE----- --z8OPzjJiggy3JOV2-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 17:13:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4FC53D22; Wed, 6 Nov 2013 17:13:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 637DD2DEC; Wed, 6 Nov 2013 17:13:07 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id rA6HD1h7095348; Wed, 6 Nov 2013 19:13:01 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua rA6HD1h7095348 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id rA6HD1Cx095347; Wed, 6 Nov 2013 19:13:01 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 6 Nov 2013 19:13:01 +0200 From: Konstantin Belousov To: Mark Johnston Subject: Re: dl_iterate_phdr() omits ld-elf.so Message-ID: <20131106171301.GG59496@kib.kiev.ua> References: <20131106052010.GB2826@raichu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i2jtRfQnnEqHm4mE" Content-Disposition: inline In-Reply-To: <20131106052010.GB2826@raichu> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 17:13:08 -0000 --i2jtRfQnnEqHm4mE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 12:20:10AM -0500, Mark Johnston wrote: > Hello, >=20 > While experimenting with dl_iterate_phdr(3), I noticed that it doesn't > include the runtime linker itself in the list of objects. This is > inconsistent with related interfaces such as /map in procfs, and > kinfo_getvmmap(3), so it seems incorrect to me that rtld is excluded > from the list of callback arguments. >=20 > Is there a reason for this behaviour? If not, does anyone have thoughts > on the diff below which fixes this? >=20 > Thanks, > -Mark >=20 > diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c > index fa6dc2a..b55effa 100644 > --- a/libexec/rtld-elf/rtld.c > +++ b/libexec/rtld-elf/rtld.c > @@ -3269,6 +3269,11 @@ dl_iterate_phdr(__dl_iterate_hdr_callback callback= , void *param) > break; > =20 > } > + if (error =3D=3D 0) { > + rtld_fill_dl_phdr_info(&obj_rtld, &phdr_info); > + error =3D callback(&phdr_info, sizeof(phdr_info), param); > + } > + > lock_release(rtld_bind_lock, &bind_lockstate); > lock_release(rtld_phdr_lock, &phdr_lockstate); I cannot make a case where this patch would be problematic, but rtld is very special object in the process address space indeed. The patch is needed exactly because rtld is not included into the list of the loaded objects, and more, symbol resolution from rtld is a special case. Doing dlopen() on rtld path would probably break things in funny way. Still, I think the patch is worth committing, but be prepared to handle the broken cases, which could come out in quite indirect ways. BTW, why do you need this ? --i2jtRfQnnEqHm4mE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSenidAAoJEJDCuSvBvK1B73sP+gIcruqWZrtdXa9QmeBKIQUF RFeurKNDCgx1qx9GNQa5KmzqkXPeAxyZG0D7Tm7hkIyA9sWc4/QU6/0hEYQj5GW/ ZmzVv5ryG6zxdMsxsiKwqL80OjlEFUxMkws5ZxmcEOT+LJXQAGRKSekZCpHtAV7v 7nCMzlwXsCJFT12Dg4qRmCsm119x7BcjdO8mTC88Gtrl0cWijrllz/1bXslGghsn 7OjBozr/QqZ5rs1qSZd4M8Qkw/DudFAW2OqToCvsRm/XHVg0QtAGWF+bgCIngYtX 2vXiYnQituaYcXK4mx8GZBKE6zPy8KjaHPso/MSXukAN1GC7PAVAHbVbUSE10KZo 1MiVWcDjLHsG1iidMXRmVXHUqfigvLCDRPvr+HHvwvMQ5lJ4uQDSFRwOteWpSN5h r/pGlGwZ77FyAJJX9zUbPCsbtfYnGw37OuHwx49+77rKKAqee6IyC7WI6W9n+ElW uPtgf2Gi6qkZKQg/XcObj2XAmg2Usw1E80EdO7RNyM3OMBbU9clb+6JTTMdom5oM xJOg+L3y0KgmgvYhQ6vKm3jNIGuJV+nRguQv1PicgpKKPY3bWphmKa8OeYdsto49 Aul9vtXAJnh58TrbABqz1+GPBouevJ3EKc8z5cQDCMGH8LAIKLj18s2f9UGD+yJm 8i2qhhY7HtOzJ4ah94lR =f+ZW -----END PGP SIGNATURE----- --i2jtRfQnnEqHm4mE-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 17:24:16 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2942ABB for ; Wed, 6 Nov 2013 17:24:16 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-qe0-x234.google.com (mail-qe0-x234.google.com [IPv6:2607:f8b0:400d:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB81C2E90 for ; Wed, 6 Nov 2013 17:24:15 +0000 (UTC) Received: by mail-qe0-f52.google.com with SMTP id w7so6532917qeb.39 for ; Wed, 06 Nov 2013 09:24:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=SP0rR78/zJkhxqhzzWfRyDpvBY9RpmKTCEWZLJZ7lr8=; b=AVOvnnrKesZTaBB/XsI8+WYqjFlY4LB4HY+iYI2kww6Xo31a4HYQGbdtI5FAFEfhlt yyT1X+M4DdIIc5e7ON8nLs9C8oTWRkl/GIRd9yf4jPWXdUHYDSsvkZWwo0e9A3G58MxR SD0YrDqjSTkd6Hf6ueyir3AhrNotgvH+l6JOA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=SP0rR78/zJkhxqhzzWfRyDpvBY9RpmKTCEWZLJZ7lr8=; b=PyVXFFj6/YK5yqvz/160gs3RaKJwE1LVeMbC6d/yU6WqSVowH7YW/PANv4vMZnFprn vjA2BlQ648lCxgxWU7zUpsJcdUzl3HmAmujdDMhdDt3LmUSvNATEw5YTyKwbSjGpfNXw xISsUvbDLE8gLOgWUwZqWVwoCcO0TgDlnZV9nlKIjKUbreQI2W6yc1nhC1QX8cGnATNZ 6KEWTIsbfq4qGiG7/l9/BIQag+ogr9HI/xZotfInrCIupyYmShriVkSAf0MKvvayVti8 p8JVVm0lUfZ5PY/BDW1LuFYwWeJQrvPzKJwsNpetsyEsuFUqRtp/j+ZjzhwUFbTWbqAW 7jmA== X-Gm-Message-State: ALoCoQlcS/NYUTTRSCWaijpC5xDfEj/eybIHjo5pm4kzJRHNuU52ZFK/n1yf5zyr0BkKYcSUvG56 X-Received: by 10.224.69.132 with SMTP id z4mr7455809qai.78.1383758655075; Wed, 06 Nov 2013 09:24:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.63.101 with HTTP; Wed, 6 Nov 2013 09:23:44 -0800 (PST) In-Reply-To: <20131106084550.GA21489@ass.pp.htv.fi> References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> <20131106084550.GA21489@ass.pp.htv.fi> From: Eitan Adler Date: Wed, 6 Nov 2013 12:23:44 -0500 Message-ID: Subject: Re: Importing netbsd cdb To: Atte Peltomaki Content-Type: text/plain; charset=UTF-8 Cc: Baptiste Daroussin , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 17:24:16 -0000 On Wed, Nov 6, 2013 at 3:45 AM, Atte Peltomaki wrote: > On Tue, Nov 05, 2013 at 11:02:26PM +0100, Jeremie Le Hen wrote: >> On Mon, Oct 28, 2013 at 12:26:28AM +0100, Baptiste Daroussin wrote: >> > >> > Here is a code that imports the cdbrw from netbsd into a new lib/libcdbrw >> > library, the read part is also added to libc but not exposed. ... This change seems reasonable to me. Would the compatibility code be removed eventually? After the next major release? -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 19:12:53 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EE27F4B7 for ; Wed, 6 Nov 2013 19:12:53 +0000 (UTC) (envelope-from Kamil.Choudhury@anserinae.net) Received: from cdptpa-oedge-vip.email.rr.com (cdptpa-outbound-snat.email.rr.com [107.14.166.225]) by mx1.freebsd.org (Postfix) with ESMTP id AF1812602 for ; Wed, 6 Nov 2013 19:12:53 +0000 (UTC) Received: from [74.73.121.187] ([74.73.121.187:57865] helo=janus.anserinae.net) by cdptpa-oedge02 (envelope-from ) (ecelerity 3.5.0.35861 r(Momo-dev:tip)) with ESMTP id 84/0D-27821-DA49A725; Wed, 06 Nov 2013 19:12:46 +0000 Received: from JANUS.anserinae.net ([fe80::192c:4b89:9fe9:dc6d]) by janus.anserinae.net ([fe80::192c:4b89:9fe9:dc6d%11]) with mapi id 14.03.0123.003; Wed, 6 Nov 2013 14:12:45 -0500 From: Kamil Choudhury To: "R. Tyler Croy" Subject: RE: pkgng for configuration management? Thread-Topic: pkgng for configuration management? Thread-Index: AQHO2yOq/7mvNW7G5EudJegpEdwvh5oYkXb/ Date: Wed, 6 Nov 2013 19:12:44 +0000 Message-ID: References: , <20131106164807.GW11443@kiwi.coupleofllamas.com> In-Reply-To: <20131106164807.GW11443@kiwi.coupleofllamas.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [12.47.208.50] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-RR-Connecting-IP: 107.14.168.130:25 X-Cloudmark-Score: 0 Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 19:12:54 -0000 =0A= > Zach Leslie has written a good pkgng provider for Puppet=0A= > (https://github.com/xaque208/puppet-pkgng) which worked on FreeBSD9, I've= yet=0A= > to test it on 10 though.=0A= =0A= Thanks for the pointer -- I'll play around with it. =0A= =0A= > There's lots of horror stories in the Linux=0A= > community of people wrapping everything in the world into debs or rpms, a= nd=0A= > regretting it later.=0A= =0A= Out of morbid curiosity, do you have links to any of these horror stories? = They sound =0A= like they'd make for some good campfire horror stories :) =0A= =0A= Thanks! =0A= =0A= From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 22:14:00 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 634A7B5D for ; Wed, 6 Nov 2013 22:14:00 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 33BB62301 for ; Wed, 6 Nov 2013 22:14:00 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id u16so252787iet.21 for ; Wed, 06 Nov 2013 14:13:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=sjwPYaD1GOtrev8HTqaJyPwOYubcgzz1tDt/2gFNbWQ=; b=qDcTH0vOkawRiKDbboIspE2GXuUWZ0z0E8Vu9NHYizbMx55x3CkmtcFPFZL8evFH2r R01nzf6gFnPkTfKv9W/MxQ03PkQmGK/cGbT87RSYLMothYkcygG98gHBWP1aZIGyKl6D mWkkp7vLrIp6u/BHJdaRqVo6/c87lyg6//sS9b0m2TKxAbGeT00NMWdkTR7RWpl891It rSpoEY6pgagnMVMU73leMqz6i+Bd84PpPJYAROZUkU7kjZmj+57f2ljU3zpAJryt78be ZNeA132hsUlLDe6/gCZU+Xh4qCg13PA4z6G8zmyRc9QLlNvX/n6DZIux5Eq0zmRwMWBu 9Ykg== X-Received: by 10.42.128.207 with SMTP id n15mr3536647ics.7.1383776039538; Wed, 06 Nov 2013 14:13:59 -0800 (PST) Received: from charmander.sandvine.com ([64.7.137.182]) by mx.google.com with ESMTPSA id x5sm455908iga.6.2013.11.06.14.13.58 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 06 Nov 2013 14:13:59 -0800 (PST) Sender: Mark Johnston Date: Wed, 6 Nov 2013 18:13:52 -0500 From: Mark Johnston To: Konstantin Belousov Subject: Re: dl_iterate_phdr() omits ld-elf.so Message-ID: <20131106231352.GB86666@charmander.sandvine.com> References: <20131106052010.GB2826@raichu> <20131106171301.GG59496@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131106171301.GG59496@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 22:14:00 -0000 On Wed, Nov 06, 2013 at 07:13:01PM +0200, Konstantin Belousov wrote: > On Wed, Nov 06, 2013 at 12:20:10AM -0500, Mark Johnston wrote: > > Hello, > > > > While experimenting with dl_iterate_phdr(3), I noticed that it doesn't > > include the runtime linker itself in the list of objects. This is > > inconsistent with related interfaces such as /map in procfs, and > > kinfo_getvmmap(3), so it seems incorrect to me that rtld is excluded > > from the list of callback arguments. > > > > Is there a reason for this behaviour? If not, does anyone have thoughts > > on the diff below which fixes this? > > > > Thanks, > > -Mark > > > > diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c > > index fa6dc2a..b55effa 100644 > > --- a/libexec/rtld-elf/rtld.c > > +++ b/libexec/rtld-elf/rtld.c > > @@ -3269,6 +3269,11 @@ dl_iterate_phdr(__dl_iterate_hdr_callback callback, void *param) > > break; > > > > } > > + if (error == 0) { > > + rtld_fill_dl_phdr_info(&obj_rtld, &phdr_info); > > + error = callback(&phdr_info, sizeof(phdr_info), param); > > + } > > + > > lock_release(rtld_bind_lock, &bind_lockstate); > > lock_release(rtld_phdr_lock, &phdr_lockstate); > > I cannot make a case where this patch would be problematic, but rtld is > very special object in the process address space indeed. The patch is > needed exactly because rtld is not included into the list of the loaded > objects, and more, symbol resolution from rtld is a special case. Doing > dlopen() on rtld path would probably break things in funny way. > > Still, I think the patch is worth committing, but be prepared to handle > the broken cases, which could come out in quite indirect ways. > > BTW, why do you need this ? I was just trying to find a portable way to figure out the address at which a given object was located, and noticed the omission because the runtime linker is included in the list on Linux. It seemed like a bug to me just based on what dl_iterate_phdr(3) is supposed to do: invoke a callback for each loaded ELF object, which includes rtld even though it's special. So I don't really need this change. In this case, do you still think it's worth committing? Or should I just leave it alone? -Mark From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 22:25:33 2013 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BD3FCFEA for ; Wed, 6 Nov 2013 22:25:33 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 731DA23BF for ; Wed, 6 Nov 2013 22:25:32 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id rA6MPVYF024074 for ; Wed, 6 Nov 2013 16:25:31 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id rA6MPVlM024073 for hackers@FreeBSD.org; Wed, 6 Nov 2013 16:25:31 -0600 (CST) (envelope-from brooks) Date: Wed, 6 Nov 2013 16:25:31 -0600 From: Brooks Davis To: hackers@FreeBSD.org Subject: Re: Using bsd.prog.mk in a project with multiple binaries Message-ID: <20131106222531.GF91063@lor.one-eyed-alien.net> References: <20131106164526.GV11443@kiwi.coupleofllamas.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oFbHfjnMgUMsrGjO" Content-Disposition: inline In-Reply-To: <20131106164526.GV11443@kiwi.coupleofllamas.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 22:25:33 -0000 --oFbHfjnMgUMsrGjO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 08:45:26AM -0800, R. Tyler Croy wrote: > Howdy, I'm trying to find some documentation or examples of using bsd.pro= g.mk > inside of a project with multiple binaries that need to get created. I'm = able > to shuffle the manpages and source code into directories to where each bi= nary > can have a Makefile, but I cannot find anything in the Developer's Handbo= ok > discussing such a project layout. E.g.: >=20 > -daemon.git/ > -daemon/ > daemon.man.1 > daemon.c > -daemontool/ > daemontool.man.1 > daemontool.c >=20 >=20 > Does one simply use a parent Makefile which calls into each directory? Or= is > there some other .mk file with helpers for this? You can use bsd.subdir.mk. See usr.bin/clang for a somewhat complicated example. > A related question would be, does this even make snese as far as a > project layout is concerned? This project I intend on creating a port > for once it's further along, and I'd like to avoid painting myself > into any cornners with the makefiles now if I can avoid it. This seems plusable as a layout. Note that while you need one Makefile per program, you don't necessicairly need to spread the code out if you don't want to. If you'd rather keep the source files in one place you can use .PATH: directives to access them from a central location. -- Brooks --oFbHfjnMgUMsrGjO Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFSesHaXY6L6fI4GtQRAh8iAKDJ04pNf+UQC+CM83KVRgJGfSoZnwCeKDqR NEa0W0QkHicSWPZFe/aFuyg= =9bxp -----END PGP SIGNATURE----- --oFbHfjnMgUMsrGjO-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 22:33:04 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8BE0D2F7; Wed, 6 Nov 2013 22:33:04 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E0698244B; Wed, 6 Nov 2013 22:33:03 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so267289lbj.17 for ; Wed, 06 Nov 2013 14:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=y9SVNLAQWH4rVN0fk+Vb4ili5PSu0Q7qaj+wpz5G718=; b=JdOIueE2nATcjHSkslxyHYs45RM6sMwTSa4kFHsjhf779+eMBEC4lEMGc7NEkwyh/3 JTmEc2YkeEan4Ze8yIdOEQXRrBolUn4aQblaGicXkW5MNMA/gCnUmG3ejE8qLuIjIGHm b+q+eJfsLpP9eB1sSNbLcx7PVTbvotEKFO9WRsJjmpWbFja/NwFjHfW0PzlBokacCS/x tZYolItclEqWr9Z/CAyWjcJR3FTmQAfkaqMRv+ikSWl5HWxW2UfjjjgnukRfgaagSzDO RZQo01O8eNPjvqxuFdOu8L3BGEYu4fqhxJfoPtozuuN2EY8gZ6Yk8FoXwAOKu3JaTx8U DFSg== MIME-Version: 1.0 X-Received: by 10.152.184.198 with SMTP id ew6mr3995777lac.34.1383777181851; Wed, 06 Nov 2013 14:33:01 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.23.35 with HTTP; Wed, 6 Nov 2013 14:33:01 -0800 (PST) In-Reply-To: <20131106222531.GF91063@lor.one-eyed-alien.net> References: <20131106164526.GV11443@kiwi.coupleofllamas.com> <20131106222531.GF91063@lor.one-eyed-alien.net> Date: Wed, 6 Nov 2013 14:33:01 -0800 X-Google-Sender-Auth: asQITtxhl5nIgUNQxNlJ8bkXMUg Message-ID: Subject: Re: Using bsd.prog.mk in a project with multiple binaries From: Luigi Rizzo To: Brooks Davis Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 22:33:04 -0000 On Wed, Nov 6, 2013 at 2:25 PM, Brooks Davis wrote: > On Wed, Nov 06, 2013 at 08:45:26AM -0800, R. Tyler Croy wrote: > > Howdy, I'm trying to find some documentation or examples of using > bsd.prog.mk > > inside of a project with multiple binaries that need to get created. I'm > able > ... > This seems plusable as a layout. Note that while you need one Makefile > per program, you don't necessicairly need to spread the code out if you > don't want to. If you'd rather keep the source files in one place you > can use .PATH: directives to access them from a central location. > Since I have had a similar problem in the past... i am also under the impression that those Makefile must be in different directories -- am i wrong ? (in this case having all the sources in a single place does not help too much; and I'd rather have a single Makefile to handle a small set of closely related programs) cheers luigi From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 23:06:39 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 9109AC1 for ; Wed, 6 Nov 2013 23:06:39 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 46293262E for ; Wed, 6 Nov 2013 23:06:38 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id rA6N6btl024298; Wed, 6 Nov 2013 17:06:37 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id rA6N6ahm024297; Wed, 6 Nov 2013 17:06:36 -0600 (CST) (envelope-from brooks) Date: Wed, 6 Nov 2013 17:06:36 -0600 From: Brooks Davis To: Luigi Rizzo Subject: Re: Using bsd.prog.mk in a project with multiple binaries Message-ID: <20131106230636.GG91063@lor.one-eyed-alien.net> References: <20131106164526.GV11443@kiwi.coupleofllamas.com> <20131106222531.GF91063@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oOB74oR0WcNeq9Zb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 23:06:39 -0000 --oOB74oR0WcNeq9Zb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 02:33:01PM -0800, Luigi Rizzo wrote: > On Wed, Nov 6, 2013 at 2:25 PM, Brooks Davis wrote: >=20 > > On Wed, Nov 06, 2013 at 08:45:26AM -0800, R. Tyler Croy wrote: > > > Howdy, I'm trying to find some documentation or examples of using > > bsd.prog.mk > > > inside of a project with multiple binaries that need to get created. = I'm > > able > > ... > > This seems plusable as a layout. Note that while you need one Makefile > > per program, you don't necessicairly need to spread the code out if you > > don't want to. If you'd rather keep the source files in one place you > > can use .PATH: directives to access them from a central location. > > >=20 > Since I have had a similar problem in the past... > i am also under the impression that those Makefile must be > in different directories -- am i wrong ? > (in this case having all the sources in a single place does > not help too much; and I'd rather have a single Makefile > to handle a small set of closely related programs) The Makefiles do need to be in different directories since they need to be called Makefile. I suppose you could hand roll a toplevel Makefile that called a series of Makefile.prog1, Makefile.prog2, etc files, but that seems not very useful. -- Brooks --oOB74oR0WcNeq9Zb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFSest7XY6L6fI4GtQRAjevAJsEKRYhOs053H+mwW1YQ00Jp2jBrACgz5zO cG8mvjmNbkBGFdPd4JznZ+4= =dz+M -----END PGP SIGNATURE----- --oOB74oR0WcNeq9Zb-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 23:30:40 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2D39CCB6; Wed, 6 Nov 2013 23:30:40 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id E406C27C8; Wed, 6 Nov 2013 23:30:38 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 3AF677300A; Thu, 7 Nov 2013 00:32:29 +0100 (CET) Date: Thu, 7 Nov 2013 00:32:29 +0100 From: Luigi Rizzo To: Brooks Davis Subject: Re: Using bsd.prog.mk in a project with multiple binaries Message-ID: <20131106233229.GA44585@onelab2.iet.unipi.it> References: <20131106164526.GV11443@kiwi.coupleofllamas.com> <20131106222531.GF91063@lor.one-eyed-alien.net> <20131106230636.GG91063@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131106230636.GG91063@lor.one-eyed-alien.net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 23:30:40 -0000 On Wed, Nov 06, 2013 at 05:06:36PM -0600, Brooks Davis wrote: > On Wed, Nov 06, 2013 at 02:33:01PM -0800, Luigi Rizzo wrote: > > On Wed, Nov 6, 2013 at 2:25 PM, Brooks Davis wrote: > > > > > On Wed, Nov 06, 2013 at 08:45:26AM -0800, R. Tyler Croy wrote: > > > > Howdy, I'm trying to find some documentation or examples of using > > > bsd.prog.mk > > > > inside of a project with multiple binaries that need to get created. I'm > > > able > > > ... > > > This seems plusable as a layout. Note that while you need one Makefile > > > per program, you don't necessicairly need to spread the code out if you > > > don't want to. If you'd rather keep the source files in one place you > > > can use .PATH: directives to access them from a central location. > > > > > > > Since I have had a similar problem in the past... > > i am also under the impression that those Makefile must be > > in different directories -- am i wrong ? > > (in this case having all the sources in a single place does > > not help too much; and I'd rather have a single Makefile > > to handle a small set of closely related programs) > > The Makefiles do need to be in different directories since they need to > be called Makefile. I suppose you could hand roll a toplevel Makefile > that called a series of Makefile.prog1, Makefile.prog2, etc files, but > that seems not very useful. right. I was hoping that someday we could have support in bsd.prog.mk to handle multiple program names in ${PROG}, perhaps expanding the dependencies using OBJS.prog1 OBJS.prog2 ... etc .for i in ${PROG} ${i}: ${OBJS}.$i .endfor (i know it is more complex than this, with PROG_FULL and LDADD, LDFLAGS and CFLAGS which may also be per-program) cheers luigi From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 23:42:53 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 588D13AD; Wed, 6 Nov 2013 23:42:53 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BB31828A4; Wed, 6 Nov 2013 23:42:52 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/afgnrylsiWy0bTV9rgkJ X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-109-45-222-180.web.vodafone.de [109.45.222.180]) by smtp.strato.de (RZmta 32.11 DYNA|AUTH) with (TLSv1.0:DHE-RSA-AES256-SHA encrypted) ESMTPSA id z013d5pA6LbuO1 ; Thu, 7 Nov 2013 00:42:39 +0100 (CET) Received: by britannica.bec.de (sSMTP sendmail emulation); Thu, 07 Nov 2013 00:42:38 +0100 Date: Thu, 7 Nov 2013 00:42:38 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, "freebsd-hackers@freebsd.org" Subject: Re: Using bsd.prog.mk in a project with multiple binaries Message-ID: <20131106234238.GA2604@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, "freebsd-hackers@freebsd.org" References: <20131106164526.GV11443@kiwi.coupleofllamas.com> <20131106222531.GF91063@lor.one-eyed-alien.net> <20131106230636.GG91063@lor.one-eyed-alien.net> <20131106233229.GA44585@onelab2.iet.unipi.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131106233229.GA44585@onelab2.iet.unipi.it> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 23:42:53 -0000 On Thu, Nov 07, 2013 at 12:32:29AM +0100, Luigi Rizzo wrote: > I was hoping that someday we could have support in bsd.prog.mk > to handle multiple program names in ${PROG}, perhaps expanding > the dependencies using OBJS.prog1 OBJS.prog2 ... etc NetBSD supports PROGS for this purpose. Joerg From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 6 23:43:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ABA7A492; Wed, 6 Nov 2013 23:43:18 +0000 (UTC) (envelope-from cjpugmed@gmail.com) Received: from mail-ve0-x22c.google.com (mail-ve0-x22c.google.com [IPv6:2607:f8b0:400c:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 57C3028B0; Wed, 6 Nov 2013 23:43:18 +0000 (UTC) Received: by mail-ve0-f172.google.com with SMTP id cz12so165903veb.17 for ; Wed, 06 Nov 2013 15:43:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xuCAbi8lwgoUly9V4BWy5GytjK/5b88UeW5CZcxnrcM=; b=sep5rLFkV8TerpDFA3jQXOY1qJaGh5cunEU9p04rYqsJtdFLipfwlnFgwIROEz5Yyn BVjhMEzgpjDOATcxCYl1eZr04SYdbQOkQTC5XZx0UjmwUuqmtyBp43GYUWTsZpoteUbx /KxrSz3qtYo1L7iyWy0QBaJ3EfJwuLP8Xc3CpDmUe5xJlXj/r/N++1s+Juta0KofCU7I 583j0IdtsRwePhT9qRashR2m6v+j3wz9zUs0ShcwdmFVzhYihltJSKwF78YKs5jLf5Tp 5+TxeW/B9glEsUQyusC+NpB0FqpsUvl7G7TBuUZp8oTlR/shWZmzqfGlw7Two0Ft+7sL W+CQ== MIME-Version: 1.0 X-Received: by 10.58.178.239 with SMTP id db15mr4434021vec.9.1383781397379; Wed, 06 Nov 2013 15:43:17 -0800 (PST) Received: by 10.220.1.17 with HTTP; Wed, 6 Nov 2013 15:43:17 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Nov 2013 00:43:17 +0100 Message-ID: Subject: Re: Automated submission of kernel panic reports: sysutils/panicmail From: Carlos Jacobo Puga Medina To: freebsd-hackers@freebsd.org, cperciva@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2013 23:43:18 -0000 At last, I realized how to solved this inconvenient. Digging a bit I found what I needed. All is summarized in the pr124747 which describes perfectly this problem and attached a patch to fix it. I add the modified patch according to current rc.d scripts included. diff -ur encswap.orig encswap --- encswap.orig 2013-11-06 23:31:43.000000000 +0100 +++ encswap 2013-11-07 00:14:03.000000000 +0100 @@ -3,8 +3,8 @@ # $FreeBSD: release/9.2.0/etc/rc.d/encswap 180563 2008-07-16 19:22:48Z dougb $ # -# PROVIDE: disks -# REQUIRE: initrandom +# PROVIDE: encswap +# REQUIRE: initrandom disks # KEYWORD: nojail . /etc/rc.subr diff -ur savecore.orig savecore --- savecore.orig 2013-11-06 23:32:21.000000000 +0100 +++ savecore 2013-11-07 00:14:18.000000000 +0100 @@ -4,7 +4,7 @@ # # PROVIDE: savecore -# REQUIRE: dumpon ddb syslogd +# BEFORE: encswap # KEYWORD: nojail . /etc/rc.subr diff -ur swap1.orig swap1 --- swap1.orig 2013-11-06 23:32:28.000000000 +0100 +++ swap1 2013-11-07 00:14:29.000000000 +0100 @@ -4,7 +4,7 @@ # # PROVIDE: localswap -# REQUIRE: disks +# REQUIRE: encswap disks # KEYWORD: nojail shutdown . /etc/rc.subr I tested it and now it works like a charm. 2013/11/5 Carlos Jacobo Puga Medina > Add missing thread related to Pawel's proposal [1] > > [1] > http://lists.freebsd.org/pipermail/freebsd-fs/2010-September/009256.html > > > 2013/11/5 Carlos Jacobo Puga Medina > >> Seems that rc.d scripts order doesn't allow geli-encrypted swap and >> dumpdev to cooperate together. For this reason, I wonder what happens in >> such cases, is not possible to use panicmail unless Pawel's proposal [1] >> which argued to obtain and save crash info in memory somewhere before we >> configure swap and copy it to /var/crash/ once we mount it. >> >> Is sustainable this proposal or is there yet another alternative to >> consider for those who use an encrypted GELI swap? >> >> --CJPM >> > > From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 03:13:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 60338FE3; Thu, 7 Nov 2013 03:13:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-we0-x234.google.com (mail-we0-x234.google.com [IPv6:2a00:1450:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CC69F2512; Thu, 7 Nov 2013 03:13:23 +0000 (UTC) Received: by mail-we0-f180.google.com with SMTP id q59so353846wes.25 for ; Wed, 06 Nov 2013 19:13:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=I1uFSyXy1UODP0m3/m+pruqlZaP/LC4mZ64ez+NIB5I=; b=OPaCObxlKJldEDBw8CY2WnXKlOmThxFGq760lxRnNmQVXjsNHlm75YAl832X5n1ETK q0fJfcByw0On5T1WFKciDjCUD4iUNkp5H+LQ5JpdibbGa9UMNXvo1nVa1W3CmywfxGDs QAUgjQpOYPxOgAATAFaWfdJE4yGctqxFO39zxbSPb+iVlQE10qmyHHDxWV/WLiTI0+D/ Dzq+nync+qwvf7AuuPMJeqLimWv2AmhbFWNSLNXCKlz3yK2J7eaNTxnrIgAMlqnLw4mB nF/ZlvacKZQX+Kp5HBWlPlk6OoFSk9475lWpA6JrCrKghxSmX4uiWMILlwCasxZAso7U mUvg== MIME-Version: 1.0 X-Received: by 10.180.185.10 with SMTP id ey10mr557068wic.29.1383794001924; Wed, 06 Nov 2013 19:13:21 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Wed, 6 Nov 2013 19:13:21 -0800 (PST) In-Reply-To: <20131106234238.GA2604@britannica.bec.de> References: <20131106164526.GV11443@kiwi.coupleofllamas.com> <20131106222531.GF91063@lor.one-eyed-alien.net> <20131106230636.GG91063@lor.one-eyed-alien.net> <20131106233229.GA44585@onelab2.iet.unipi.it> <20131106234238.GA2604@britannica.bec.de> Date: Wed, 6 Nov 2013 20:13:21 -0700 X-Google-Sender-Auth: x58SXdws_EqC0U3on7MBXeIQoVM Message-ID: Subject: Re: Using bsd.prog.mk in a project with multiple binaries From: Alan Somers To: freebsd-hackers@freebsd.org, "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 03:13:24 -0000 On Wed, Nov 6, 2013 at 4:42 PM, Joerg Sonnenberger wrote: > On Thu, Nov 07, 2013 at 12:32:29AM +0100, Luigi Rizzo wrote: >> I was hoping that someday we could have support in bsd.prog.mk >> to handle multiple program names in ${PROG}, perhaps expanding >> the dependencies using OBJS.prog1 OBJS.prog2 ... etc > > NetBSD supports PROGS for this purpose. In fact, Simon and Garrett recently ported NetBSD's mechanism. It's called bsd.progs.mk and it's in FreeBSD/head and stable/10. It works much like bsd.prog.mk. Basic usage is like this: PROGS= foo bar baz SRCS.foo= foo.c common.c SRCS.bar= bar.c common.c SRCS.baz= baz.c common.c It's still something of a work in progress. The exact usage is likely to change for awhile. -Alan From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 03:57:39 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 53241AE7 for ; Thu, 7 Nov 2013 03:57:39 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com [209.85.217.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3C1F2754 for ; Thu, 7 Nov 2013 03:57:38 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id x18so458454lbi.30 for ; Wed, 06 Nov 2013 19:57:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=fhjlt3OIqeaUPNlaRaPJx2Cs5a/zso7hanUIPxaWdgY=; b=Mk9aLuGp5jI0ir/cf3N2q97I+PUmo+YW7F4DW1gy/ghsRTaEFl7RZr4cJvOqRUtM01 ITCkZg490b4QkdO70PBttABCDvjM/2vwSKaK0ojvytYxmeqpf01bHSN562Ytl1cYBrVP cijkmn4yFehA93dbfkjieT9WsQXCO7MjfcvdcTIAczeRCHEizRYRNhtJVG12AEIp7xjA 0gsx+3qaejJnkTMelF6yAcKf//k1owJ0ig7BoJF6Z4Jr3AqDc50wJplKWZcx+K88RLxC XRVb+oFgpDlK13yT4/0jDDpCl+muYV8wHDLFWL0favYTiDGgPf0FsNgstjwcMiNiBJ2x NDrw== X-Gm-Message-State: ALoCoQlUNIdaPW93H3hCGwg3raAGB7RbYDkkQNYNxevpJlJRf8b99TJqJ2jyAcUon63tbHtOjSk5 X-Received: by 10.112.136.65 with SMTP id py1mr4861448lbb.4.1383796651030; Wed, 06 Nov 2013 19:57:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.132.135 with HTTP; Wed, 6 Nov 2013 19:57:10 -0800 (PST) X-Originating-IP: [172.26.38.44] In-Reply-To: <20131106103947.GI56315@ithaqua.etoilebsd.net> References: <20131027232628.GB74512@ithaqua.etoilebsd.net> <20131105220225.GF37879@caravan.chchile.org> <20131106084550.GA21489@ass.pp.htv.fi> <20131106103947.GI56315@ithaqua.etoilebsd.net> From: Julio Merino Date: Wed, 6 Nov 2013 22:57:10 -0500 Message-ID: Subject: Re: Importing netbsd cdb To: Baptiste Daroussin Content-Type: text/plain; charset=ISO-8859-1 Cc: hackers@freebsd.org, Atte Peltomaki X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 03:57:39 -0000 On Wed, Nov 6, 2013 at 5:39 AM, Baptiste Daroussin wrote: > From the netbsd commit log, it says that in case of services it is at least as > slow, and at best faster, from my testing it is way faster, (sorry no numbers) > > Concerning the migration, the way I did it for now, is that getservent is trying > to open the cdb database and fallsback on the bdb one, so migrating to > 11.0-RELEASE should be ootb. > > Running: > services_mkdb > > Will generate the cdb version of the db Hmm... but who would be in charge of running this conversion process during an upgrade to 11? The administrator by hand, or would this be automated in some manner? Also, if this is expected to be done by hand: it's possible for somebody to forget doing the conversion when upgrading to 11. Would there be any notices announcing that the old format is deprecated and would go away by 12? I'm having in mind a failure case where you upgrade from 11 to 12 and things break because you forgot to rebuild the database with 11 (which is the release that allows you to do the conversion). -- Julio Merino / @jmmv From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 06:12:40 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 88C6CE4C; Thu, 7 Nov 2013 06:12:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0D03B2D65; Thu, 7 Nov 2013 06:12:39 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id rA76CYbA067753; Thu, 7 Nov 2013 08:12:34 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua rA76CYbA067753 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id rA76CYPs067752; Thu, 7 Nov 2013 08:12:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 7 Nov 2013 08:12:34 +0200 From: Konstantin Belousov To: Mark Johnston Subject: Re: dl_iterate_phdr() omits ld-elf.so Message-ID: <20131107061234.GO59496@kib.kiev.ua> References: <20131106052010.GB2826@raichu> <20131106171301.GG59496@kib.kiev.ua> <20131106231352.GB86666@charmander.sandvine.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xQILpJsL5cebVBoz" Content-Disposition: inline In-Reply-To: <20131106231352.GB86666@charmander.sandvine.com> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 06:12:40 -0000 --xQILpJsL5cebVBoz Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 06, 2013 at 06:13:52PM -0500, Mark Johnston wrote: > On Wed, Nov 06, 2013 at 07:13:01PM +0200, Konstantin Belousov wrote: > > On Wed, Nov 06, 2013 at 12:20:10AM -0500, Mark Johnston wrote: > > > Hello, > > >=20 > > > While experimenting with dl_iterate_phdr(3), I noticed that it doesn't > > > include the runtime linker itself in the list of objects. This is > > > inconsistent with related interfaces such as /map in procfs, and > > > kinfo_getvmmap(3), so it seems incorrect to me that rtld is excluded > > > from the list of callback arguments. > > >=20 > > > Is there a reason for this behaviour? If not, does anyone have though= ts > > > on the diff below which fixes this? > > >=20 > > > Thanks, > > > -Mark > > >=20 > > > diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c > > > index fa6dc2a..b55effa 100644 > > > --- a/libexec/rtld-elf/rtld.c > > > +++ b/libexec/rtld-elf/rtld.c > > > @@ -3269,6 +3269,11 @@ dl_iterate_phdr(__dl_iterate_hdr_callback call= back, void *param) > > > break; > > > =20 > > > } > > > + if (error =3D=3D 0) { > > > + rtld_fill_dl_phdr_info(&obj_rtld, &phdr_info); > > > + error =3D callback(&phdr_info, sizeof(phdr_info), param); > > > + } > > > + > > > lock_release(rtld_bind_lock, &bind_lockstate); > > > lock_release(rtld_phdr_lock, &phdr_lockstate); > >=20 > > I cannot make a case where this patch would be problematic, but rtld is > > very special object in the process address space indeed. The patch is > > needed exactly because rtld is not included into the list of the loaded > > objects, and more, symbol resolution from rtld is a special case. Doing > > dlopen() on rtld path would probably break things in funny way. > >=20 > > Still, I think the patch is worth committing, but be prepared to handle > > the broken cases, which could come out in quite indirect ways. > >=20 > > BTW, why do you need this ? >=20 > I was just trying to find a portable way to figure out the address at > which a given object was located, and noticed the omission because the > runtime linker is included in the list on Linux. It seemed like a bug to > me just based on what dl_iterate_phdr(3) is supposed to do: invoke a > callback for each loaded ELF object, which includes rtld even though > it's special. >=20 > So I don't really need this change. In this case, do you still think > it's worth committing? Or should I just leave it alone? It seems to make the function more logical. As I said, I think that the patch is worth committing. --xQILpJsL5cebVBoz Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSey9RAAoJEJDCuSvBvK1BCZUQAKEdzw4UFVa/0oVbv977JK5U E4P295B3YvMEJuIEijWhrdPacrhzS0xGY+EvLZj5KME7wNduBV/dSFCxlFjtnMvm zubNW4YQQbW4+ZCDUEUKuIR9vcdvle8cVoo+RfoXGKPu4MtfZ5Fpj8cL1wmzu3cr B3cZy2XM0RI7GhJfxXgLvFDW71Qjuxw/AYrod9hbI0rHEsXGbhF5ROjcUMzIe4VF v4FJJdYd4mqxZ6lHCdHOrkhMwL59BXFl2ANXAZZNZGi8J2cB+ZktCI7g/8w4ooTo nI7DlSrLb36Lp5N5KJoKJwmHYUpDYBeFnB62yu2xbKX4Y4QbWRQKlbuQtSTn/4iw eTLUu4zr0bHcLihRGnGumKOY2vT/9alOmAHSJfI64Acmsli06rHOiRdMY/Jc8rr7 325Ifmeuu5IkWltKvYJ4vYBIBTkis8hT2lHif2IFcFxKk4KzvhVGLhT22A2qVrO8 aafxyuWZJxRMHlPYVfzSv8JDZvvCJqZW6HeOvjW1P5efyrlPCXmB/2FYoprJL5rT uqrQ/YycIKCVBjuatau37oo2uIVAVvvmPK+ZQHoR/smrGi1JxMr7ru5haAYWZL2N 9azmaKxE5KJQVA/X1w7IsWrqS+ll4lPZ53TCcRxAljupdQdb2ry/UQ7fvViuys9L DFHCY7PmtLAnZ95M+0dC =Uke4 -----END PGP SIGNATURE----- --xQILpJsL5cebVBoz-- From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 12:56:41 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 517BD202 for ; Thu, 7 Nov 2013 12:56:41 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 147852D11 for ; Thu, 7 Nov 2013 12:56:41 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id x12so348726qcv.14 for ; Thu, 07 Nov 2013 04:56:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:content-type; bh=5buDwcvj6QGyZTsDJwfKTCPbJapjUEQf4vb/nWXZgWs=; b=cpfB/jaR/hnNl2CnfZ1+wXR+xw6syvj2XBWiK+B6C8ETgCRl62tEYkkaRYJJ/5m52j 0lRj6CQK94Kyih7lRBKbviFHiA4yGS3A09oG0XvOonrbaoSqi4odhalbzoS02Pgvf5Kp w7baEbLiGCb7TNybgz7vR4ZBXarS1fMiiYDP0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=5buDwcvj6QGyZTsDJwfKTCPbJapjUEQf4vb/nWXZgWs=; b=QC5zo2zzSiMdJKIJk99629MKfUb55jV7XNjbbrCS8cVXFW1W880QM7R1OcankWpiLK 8ESLsmQpXZZsGn7azuDi2DUrC3llhv97WUsEU9TTki3sPBtnSj3RFZIf89h5+IuWHMhR linad/qdh80SafBsEVBuVCMIAp2Kzrj1xCpoK20ZtCM76HiFoiO1frIt3M+0++VGIAfh yUGTqlrwlUZV8va5aauUZ59fwJWU7ixl0dF3WkqtBydGSfmMF1eVR00SqAsJaQaimwi8 JWn1Om3nuUSZpdstlJPhmDu8yjtoisJpiEPh1L4ok6rLUP/mrbB59KZ8mEVbVYWPDJUP /vwA== X-Gm-Message-State: ALoCoQmKKooaNAoeCfSr/M0655L6tfgcUXVqHJr0H93NeK1eNC+rfHJzH1umskGehlwBAm/rziM5 X-Received: by 10.229.185.71 with SMTP id cn7mr12501515qcb.3.1383828999748; Thu, 07 Nov 2013 04:56:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.63.101 with HTTP; Thu, 7 Nov 2013 04:56:09 -0800 (PST) From: Eitan Adler Date: Thu, 7 Nov 2013 07:56:09 -0500 Message-ID: Subject: Adding dehumanize_number to libutil To: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 12:56:41 -0000 Hi all, NetBSD and DragonflyBSD have a function dehumanize_number which is the inverse of humanize_number. I'd like to import this function and switch some base utilities to use it. Is there any objection? Patch is here: http://people.freebsd.org/~eadler/files/dehumanize.diff This implemtation is a direct copy of DragonFlyBSD's version. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 15:35:59 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0666EC91 for ; Thu, 7 Nov 2013 15:35:59 +0000 (UTC) (envelope-from julio@meroh.net) Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com [209.85.216.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BBA9C287C for ; Thu, 7 Nov 2013 15:35:58 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so558828qae.17 for ; Thu, 07 Nov 2013 07:35:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-type:content-disposition:user-agent; bh=pJo0GvKgjB+oaaA75fyyTJqhXo2y0+RGauGYs2SIaOM=; b=m7NGlMw104Ad3lNdRr7SZtm8M3FOxuPJuKccasZbRs+YxX3HivxH8iOTDov3467DHQ WwBH1VkGENRqEj5EyA61JQ+e43MVkFJTvAVPeM0X4N/0UuEWsUsV6NB6qZHtLxckV44I GSGenjXTChgC8SM5rACR8m27k/6GCc8R428ZLd7/l2onecUJTOz3ZC/BF5gqjddkt+ja 2x48fzdAUlxHW5pPXFC/gWWRppFb9IWqX/YgGKLnLIDpip1f6HeYbo5McP0ablaoF8mF l6HwLeI4sljJI4Biso4RFjLZIuCEoP1VAjPjMFcJgVTgDjIMbHAoyxC34zf8JC1wBX6t EdyQ== X-Gm-Message-State: ALoCoQlQB/zrpJsCN1e4kcGJb7e75AV4TWVXj1mNdnrxV7wnv2PdmhqN0ufeqdGr4iQs7WFn1zgd X-Received: by 10.224.171.196 with SMTP id i4mr15285180qaz.38.1383838221792; Thu, 07 Nov 2013 07:30:21 -0800 (PST) Received: from fbair.virtual.network (dhcp-172-26-102-247.nyc.corp.google.com [172.26.102.247]) by mx.google.com with ESMTPSA id r5sm8595125qeh.1.2013.11.07.07.30.20 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Nov 2013 07:30:20 -0800 (PST) Date: Wed, 6 Nov 2013 03:48:49 -0500 From: Julio Merino To: freebsd-hackers@freebsd.org Subject: Is WITH_*=yes in make's command line expected to work? Message-ID: <20131106084849.GA45949@fbair.virtual.network> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 15:35:59 -0000 Hello all, I'm currently working on fixing the build when the TESTS knob is enabled and I have encountered some odd behavior that I'm not sure is expected. Basically, it seems that passing a WITH_*=yes option (note, no -D) through make's command line conflicts with a similar NO_* option. Consider this sample Makefile which is derived from the contents of bsd.own.mk: ----- .if defined(SIMULATE_WITH_FOO_IN_SRC_CONF) WITH_FOO=yes .endif # Same code as in bsd.own.mk. .for var in FOO .if defined(NO_${var}) .if defined(WITH_${var}) .undef WITH_${var} .endif WITHOUT_${var}= .endif .endfor # End code from bsd.own.mk. all: .if defined(WITH_FOO) @echo WITH_FOO .endif .if defined(WITHOUT_FOO) @echo WITHOUT_FOO .endif ----- Now look at the following invocations: $ make -DSIMULATE_WITH_FOO_IN_SRC_CONF WITH_FOO # OK. $ make -DSIMULATE_WITH_FOO_IN_SRC_CONF -DNO_FOO WITHOUT_FOO # OK. $ make -DWITH_FOO -DNO_FOO WITHOUT_FOO # OK. $ make WITH_FOO=yes -DNO_FOO WITH_FOO # OOPS! WITHOUT_FOO Is this expected behavior? It seems to me that the .undef is not working properly in the way it's used by bsd.own.mk. The src.conf(5) manpage says that the WITH_* and WITHOUT_* variables can be provided to make via the command line with -D. There is no mention of providing explicit overrides with WITH_*=yes as I did above. Is it OK to rely on this assumption and consider invocations with WITH_FOO=yes in the command line to be broken? Thank you! From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 15:44:25 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DD6A826B for ; Thu, 7 Nov 2013 15:44:25 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7BEEB2907 for ; Thu, 7 Nov 2013 15:44:25 +0000 (UTC) Received: by mail-wg0-f53.google.com with SMTP id y10so710138wgg.8 for ; Thu, 07 Nov 2013 07:44:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cfZxzRfIuWxD4O5Q3SXm3FjccdqQaqVTr21MQD5yP2s=; b=NnIi51uZR9DV4hmPmJaACY6zRxXmm/yodvV3y3fjBvCxjumjZQW+ZMs9bQU9Kc0mt9 Rzgxl4zvB3Qe+fAUkey1f0FfWu65hcbyc3kUSyJEVKlUuuNwrlSQq7JmIJ0Dsz7p0Xnw coBA7rq0A+mAlB4wDoHRuW0IzQbCaN8S2byukCy29nVHW2UGLA1jWk7IWPkLiw0GELOJ +ekV8Pj1xkrbOWK4gRLCI1tD5/WjM5ge3FitRP9Rbm9ZtYdRwVTBVDAa5NJqSLnE48/r sCF6rzjXK/a82Ocm+ORtN76esvibb5sEKUVCMGNtQrFh1oytxQYZC5rAkbZOofLAqtVD bQvg== MIME-Version: 1.0 X-Received: by 10.194.185.73 with SMTP id fa9mr8019970wjc.29.1383839063598; Thu, 07 Nov 2013 07:44:23 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Thu, 7 Nov 2013 07:44:23 -0800 (PST) In-Reply-To: <20131106084849.GA45949@fbair.virtual.network> References: <20131106084849.GA45949@fbair.virtual.network> Date: Thu, 7 Nov 2013 08:44:23 -0700 X-Google-Sender-Auth: lY6KkKbUrLoRMv3bY6mViQ5y-Tc Message-ID: Subject: Re: Is WITH_="yes"in make's command line expected to work? From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 15:44:25 -0000 On Wed, Nov 6, 2013 at 1:48 AM, Julio Merino wrote: > Hello all, > > I'm currently working on fixing the build when the TESTS knob is enabled > and I have encountered some odd behavior that I'm not sure is expected. > > Basically, it seems that passing a WITH_*=yes option (note, no -D) > through make's command line conflicts with a similar NO_* option. > > Consider this sample Makefile which is derived from the contents of > bsd.own.mk: > > ----- > .if defined(SIMULATE_WITH_FOO_IN_SRC_CONF) > WITH_FOO=yes > .endif > > # Same code as in bsd.own.mk. > .for var in FOO > .if defined(NO_${var}) > .if defined(WITH_${var}) > .undef WITH_${var} > .endif > WITHOUT_${var}= > .endif > .endfor > # End code from bsd.own.mk. > > all: > .if defined(WITH_FOO) > @echo WITH_FOO > .endif > .if defined(WITHOUT_FOO) > @echo WITHOUT_FOO > .endif > ----- > > Now look at the following invocations: > > $ make -DSIMULATE_WITH_FOO_IN_SRC_CONF > WITH_FOO # OK. > > $ make -DSIMULATE_WITH_FOO_IN_SRC_CONF -DNO_FOO > WITHOUT_FOO # OK. > > $ make -DWITH_FOO -DNO_FOO > WITHOUT_FOO # OK. > > $ make WITH_FOO=yes -DNO_FOO > WITH_FOO # OOPS! > WITHOUT_FOO > > Is this expected behavior? It seems to me that the .undef is not working > properly in the way it's used by bsd.own.mk. > > The src.conf(5) manpage says that the WITH_* and WITHOUT_* variables can > be provided to make via the command line with -D. There is no mention of > providing explicit overrides with WITH_*=yes as I did above. Is it OK > to rely on this assumption and consider invocations with WITH_FOO=yes in > the command line to be broken? > > Thank you! Whether or not it's the correct thing to do, I usually set WITH_* as an environment variable. eg "WITH_TESTS=1 make install". It seems to work. I also frequently use a src.conf that is not in the usual location by setting SRCCONF. eg " SRCCONF=/path/to/src.conf make install" -Alan From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 17:25:44 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 94BF4EC2 for ; Thu, 7 Nov 2013 17:25:44 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-ee0-x229.google.com (mail-ee0-x229.google.com [IPv6:2a00:1450:4013:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B5082FC2 for ; Thu, 7 Nov 2013 17:25:44 +0000 (UTC) Received: by mail-ee0-f41.google.com with SMTP id e53so478731eek.28 for ; Thu, 07 Nov 2013 09:25:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=vfX9zSvKYfrroUrlZwsgu70snwIpo9KkveqwgCoytFo=; b=sHwj1yJNxOId1U97HEic3ZwIx6urBjSjS2VTR4lRoz3BNzCu7CZLUbj8TrcZZQTOD3 bAVW6Ieo1PF135Xm5VTMrMLTRjbGj0tezSmzYl8UclvyZkSU3JUiuZgnY2efswMfCy+W arLoc6VLmLucbL3rzUQTHrnhCVCDOumS4kKHbG4U1+MLlThJJxhvVoxh1jlETY71Cs1B 34IzD/Okfp5+cBHXgCT7AUjyUtDHSCESb62elkttCQVk7YWXt13ASsdiyYrGdPxyfBLN Obf+PTfq2Emf0cE5OQrWcUTTPFpeGGFT0Arv8DfqbGWNg+SmCZa72luatIKjMwMScz/v SvWA== X-Received: by 10.15.94.201 with SMTP id bb49mr10545110eeb.23.1383845141913; Thu, 07 Nov 2013 09:25:41 -0800 (PST) Received: from [192.168.1.105] (45.81.datacomsa.pl. [195.34.81.45]) by mx.google.com with ESMTPSA id v45sm11711231eef.11.2013.11.07.09.25.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 07 Nov 2013 09:25:41 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (11B511) From: =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Adding dehumanize_number to libutil Date: Thu, 7 Nov 2013 18:25:34 +0100 To: Eitan Adler Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 17:25:44 -0000 Dnia 7 lis 2013 o godz. 13:56 Eitan Adler napisa=C5=82= (a): > Hi all, >=20 > NetBSD and DragonflyBSD have a function dehumanize_number which is the > inverse of humanize_number. >=20 > I'd like to import this function and switch some base utilities to use > it. Is there any objection? What's the difference between this and expand_number(3)? From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 7 17:35:52 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0AF5B244 for ; Thu, 7 Nov 2013 17:35:52 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BC3072078 for ; Thu, 7 Nov 2013 17:35:51 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so706048qcx.34 for ; Thu, 07 Nov 2013 09:35:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=ODxObwQ6e+vG7HuqB1fnQ/MNcEBE4uNaL50oU3cpsU4=; b=C4CDqkndQv7CT3Dat3UZ+hJMw29NkLQB7LdI+Kml0pEARPIXwKetBgL6FD86ZEHzC3 cTtLFGRCM9hIr/tKeYfI4w6cJfcZy6POw/9DLp8SF8JYETiH517x/sUrjJqn6TmarUWA o054Duru8icKOxtOONyn2B9m/O7XgYV8HTUCE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=ODxObwQ6e+vG7HuqB1fnQ/MNcEBE4uNaL50oU3cpsU4=; b=GpfRWLdqf+2Qr8z9I+Pa6TC33dejPGurEjdzPcHtErXPmHyTrqU9sBUzpT9SJPwYUb 90ynQeXfk5jdHKYRMhnjy254fKFiMXRzPM+nV64mDOjzvKP5RdkJEneaix/633zsk8n3 An5hElm4tdaws2cN4kb+A9pFtlRO3sj5JbeZ/GZtSJ9lGP7IhdnAeeDwFR6b1I7aVwTC CT0eR0hryrXziVjeKUV91382WT2+lH5OOcsIAjP1Z4rKEtIqBxRKOoMe9ELUrJI4TPGK XvsngYU6QoT9g9uavbMKEMGsV/1+xEFTVYDVXP5PqqRJY9FKEq4ph1mB5n1/Qi9xn+qr yJpg== X-Gm-Message-State: ALoCoQn2DbidQlAUyx0I5GE2wOYyO4800Qjeo6vZp6Xct/vECIR3CMx7Pxg77/RUE1cITAa3JMRh X-Received: by 10.49.103.161 with SMTP id fx1mr14930011qeb.68.1383845750858; Thu, 07 Nov 2013 09:35:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.96.63.101 with HTTP; Thu, 7 Nov 2013 09:35:20 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Thu, 7 Nov 2013 12:35:20 -0500 Message-ID: Subject: Re: Adding dehumanize_number to libutil To: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Nov 2013 17:35:52 -0000 On Thu, Nov 7, 2013 at 12:25 PM, Edward Tomasz Napiera=C5=82a wrote: > Dnia 7 lis 2013 o godz. 13:56 Eitan Adler napisa= =C5=82(a): > >> Hi all, >> >> NetBSD and DragonflyBSD have a function dehumanize_number which is the >> inverse of humanize_number. >> >> I'd like to import this function and switch some base utilities to use >> it. Is there any objection? > > What's the difference between this and expand_number(3)? I forgot about / didn't know we had this. Forget the idea then. --=20 Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 8 22:35:00 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F3C9815A for ; Fri, 8 Nov 2013 22:34:59 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71D6724DB for ; Fri, 8 Nov 2013 22:34:58 +0000 (UTC) Received: from server.rulingia.com (c220-239-250-249.belrs5.nsw.optusnet.com.au [220.239.250.249]) by vps.rulingia.com (8.14.7/8.14.5) with ESMTP id rA8MYnOe030008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Sat, 9 Nov 2013 09:34:50 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.7/8.14.7) with ESMTP id rA8MYinY043533 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 9 Nov 2013 09:34:44 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.7/8.14.7/Submit) id rA8MYhEI043532 for freebsd-hackers@freebsd.org; Sat, 9 Nov 2013 09:34:44 +1100 (EST) (envelope-from peter) Date: Sat, 9 Nov 2013 09:34:43 +1100 From: Peter Jeremy To: freebsd-hackers@freebsd.org Subject: Root logins under failure conditions Message-ID: <20131108223443.GB8321@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2/5bycvrmDh4d1IB" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Nov 2013 22:35:00 -0000 --2/5bycvrmDh4d1IB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I have a system where /usr/local is on a different filesystem to / & /usr. On a number of occasions, I have had /usr/local lockup and block root logins - which means a hard reset with no possibility to troubleshoot. I've tried ktrace'ing login sequences and the only access to /usr/local is a result of login(8) or sshd(8) using pam(2) and nsdispatch(3) - both of which search /usr/local - the former looking for /usr/local/etc/pam.conf and /usr/local/etc/pam.d/system, and the latter searching for nss_{files,dns}.so.1 along the paths in ld-elf.so.hints. These files are all optional and (at least for me) don't exist. The problem only occurs when /usr/local is mounted and not responding. If it's not mounted then everything errors out and all is well. Can anyone offer a simple solution so that at least root logins don't need to unnecessarily access /usr/local? --=20 Peter Jeremy --2/5bycvrmDh4d1IB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iKYEARECAGYFAlJ9ZwNfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDBCRjc3QTcyNTg5NEVCRTY0RjREN0VFRUZF OEE0N0JGRjAwRkI4ODcACgkQ/opHv/APuIca5QCdH7jyw5QUZMc+dOjsicSoDzk6 regAoLmzbufW35IDZO9wjX73zAgERg5E =kzTk -----END PGP SIGNATURE----- --2/5bycvrmDh4d1IB-- From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 8 23:45:15 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 456E5473 for ; Fri, 8 Nov 2013 23:45:15 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CFD502887 for ; Fri, 8 Nov 2013 23:45:13 +0000 (UTC) Received: from server.rulingia.com (c220-239-250-249.belrs5.nsw.optusnet.com.au [220.239.250.249]) by vps.rulingia.com (8.14.7/8.14.5) with ESMTP id rA8NjABW030185 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Sat, 9 Nov 2013 10:45:11 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.7/8.14.7) with ESMTP id rA8Nj5UO069142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 9 Nov 2013 10:45:05 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.7/8.14.7/Submit) id rA8Nj5Xq069141 for freebsd-hackers@freebsd.org; Sat, 9 Nov 2013 10:45:05 +1100 (EST) (envelope-from peter) Date: Sat, 9 Nov 2013 10:45:05 +1100 From: Peter Jeremy To: freebsd-hackers@freebsd.org Subject: Are extended attributes data or meta-data? Message-ID: <20131108234505.GC8321@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WChQLJJJfbwij+9x" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Nov 2013 23:45:15 -0000 --WChQLJJJfbwij+9x Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've been getting regular error messages logged by afpd: Nov 9 00:00:19 server afpd[1966]: sys_getextattr_size: error: Permission d= enied I have spent some time digging into it and it's triggered by extattr_get_link(2) returning EACCESS because a file is not readable, but stat(2) on the file succeeded. According to extattr(2), "[n]amed extended attributes are meta-data associated with vnodes" but the actual code for VOP_GETEXTATTR() (at least for ufs & zfs) checks for VREAD access, whereas the VOP_GETATTR() call (used by stat(2)) doesn't include any access checks (so stat(2) will succeed unless namei() fails). IMHO, this behaviour is inconsistent: The extended attributes are documented as "meta-data" and but the access checks are for "data". Which is correct? --=20 Peter Jeremy --WChQLJJJfbwij+9x Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iKYEARECAGYFAlJ9d4FfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDBCRjc3QTcyNTg5NEVCRTY0RjREN0VFRUZF OEE0N0JGRjAwRkI4ODcACgkQ/opHv/APuIdsXwCgifYKpNQPZjbFwD9/ShF0+Od9 TcQAni/HkJh9NkXA4zYgIhBZigDr6PHy =hsuY -----END PGP SIGNATURE----- --WChQLJJJfbwij+9x-- From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 9 18:05:49 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7577D3DF; Sat, 9 Nov 2013 18:05:49 +0000 (UTC) (envelope-from boris.astardzhiev@gmail.com) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E2BF02B65; Sat, 9 Nov 2013 18:05:48 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id ez12so694045wid.15 for ; Sat, 09 Nov 2013 10:05:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=UOPEzkGn/gnvREHCEMg2Bhf1bMcgQ5iIXnURSTeWfi8=; b=IeUTY9LGmavPx2vPDNGSqtzx0ksFVv0s6+kNFeY/AMPgr90KNI81OZsHgHG6iEsB5i Obcc16REATbEqNFdMGhi4uvNK7SeHSsc+4Y1M5ySWT1UjHTxqBDdKCoFr4iAtk3wRZ1o 7uYOmUim3HScgVN9j9Hr6ufBUZSJL4kG5L1HIp0Qu/zT8pj2qGDmJ4zdQNKVWyhBq7XC JJBoz7bFFaWHTKS393bF80eH3yWtZXO7FanNi0vHyUwLwaa8mpoAjigM/woXIzxius9Z wAwnmLuhOrHnaYfmo6qg3RV9JMLg72a0P2ffg+Ua8QBvs6xumsikQsY8eWj0pOFrFsrX 9i3A== MIME-Version: 1.0 X-Received: by 10.180.108.131 with SMTP id hk3mr6655827wib.10.1384020347277; Sat, 09 Nov 2013 10:05:47 -0800 (PST) Received: by 10.216.66.131 with HTTP; Sat, 9 Nov 2013 10:05:47 -0800 (PST) Date: Sat, 9 Nov 2013 20:05:47 +0200 Message-ID: Subject: ARM kgdb remote debugging over USB serial From: Boris Astardzhiev To: freebsd-arm@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Nov 2013 18:05:49 -0000 Hi, I have a question regarding the FreeBSD kernel debugging facilities. Has anyone succeeded in using kgdb with a target ARM machine over a USB serial. I've managed to build kgdb-arm http://bsdimp.blogspot.com/2007/10/cross-debugger.html (This tutorial also applies to building kgdb-arm in binutils) So on the arm machine I enter kdb: db> gdb db> s .. On the debugging machine I easily get into kgdb: kgdb-arm ${KERNEL_PATH}/kernel.debug kgdb> set remotebaud 115200 kgdb> target remote /dev/cuaU0 (the USB interface) So far so good but.. It seems to connect to the ARM device but it stops on an address kgdb has no reference about. I thought I was missing some symbols and.. kgdb> set solib-search-path ${KERNEL_PATH} This seems to load lots of symbols but still I can't backtrace or do anything. Attempting to do a 'bt' it tells me I got a SIGTRAP. I don't seem to understand. Any ideas or materials? Greetings, Boris From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 9 21:06:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8D024312 for ; Sat, 9 Nov 2013 21:06:34 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 69390234B for ; Sat, 9 Nov 2013 21:06:33 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id rA9L6PAk030065; Sat, 9 Nov 2013 21:06:25 GMT (envelope-from kientzle@freebsd.org) Received: from [192.168.2.123] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id sgh2kmq63kn3g2w6edd38vfijw; Sat, 09 Nov 2013 21:06:25 +0000 (UTC) (envelope-from kientzle@freebsd.org) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1822\)) Subject: Re: Are extended attributes data or meta-data? From: Tim Kientzle In-Reply-To: <20131108234505.GC8321@server.rulingia.com> Date: Sat, 9 Nov 2013 13:06:24 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <502A2D02-6AB3-42FC-94D8-261A208751ED@freebsd.org> References: <20131108234505.GC8321@server.rulingia.com> To: Peter Jeremy X-Mailer: Apple Mail (2.1822) Cc: freebsd-hackers Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Nov 2013 21:06:34 -0000 On Nov 8, 2013, at 3:45 PM, Peter Jeremy wrote: > I've been getting regular error messages logged by afpd: > Nov 9 00:00:19 server afpd[1966]: sys_getextattr_size: error: = Permission denied > I have spent some time digging into it and it's triggered by > extattr_get_link(2) returning EACCESS because a file is not readable, > but stat(2) on the file succeeded. >=20 > According to extattr(2), "[n]amed extended attributes are meta-data > associated with vnodes" but the actual code for VOP_GETEXTATTR() (at = least > for ufs & zfs) checks for VREAD access, whereas the VOP_GETATTR() call > (used by stat(2)) doesn't include any access checks (so stat(2) will > succeed unless namei() fails). >=20 > IMHO, this behaviour is inconsistent: The extended attributes are > documented as "meta-data" and but the access checks are for "data". > Which is correct? Practically speaking, extended attributes are used both for data and metadata. I would consider the existing behavior (extattr calls fail on non-readable files) to be correct in the absence of NFSv4 ACLs (which include a specific permission for extattr readability). The extattr(2) manpage should probably document that the calls fail on non-readable files. Tim From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 9 22:29:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AEAC213D; Sat, 9 Nov 2013 22:29:16 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8D02694; Sat, 9 Nov 2013 22:29:16 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEABW2flKDaFve/2dsb2JhbABZgz9TgnS7W0qBPnSCJQEBAQMBAQEBIAQnIAsFFhgCAg0ZAikBCSYGCAcEARwEh1oGDapEkhuBKY0FgQU0B4JrgUUDiUKLfoN+kFuDRB4xgQQ5 X-IronPort-AV: E=Sophos;i="4.93,669,1378872000"; d="scan'208";a="68484756" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 09 Nov 2013 17:28:07 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 417FAB402B; Sat, 9 Nov 2013 17:28:07 -0500 (EST) Date: Sat, 9 Nov 2013 17:28:07 -0500 (EST) From: Rick Macklem To: Tim Kientzle Message-ID: <704471213.12196154.1384036087256.JavaMail.root@uoguelph.ca> In-Reply-To: <502A2D02-6AB3-42FC-94D8-261A208751ED@freebsd.org> Subject: Re: Are extended attributes data or meta-data? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-hackers Hackers , Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Nov 2013 22:29:16 -0000 Tim Kientzle wrote: > > On Nov 8, 2013, at 3:45 PM, Peter Jeremy wrote: > > > I've been getting regular error messages logged by afpd: > > Nov 9 00:00:19 server afpd[1966]: sys_getextattr_size: error: > > Permission denied > > I have spent some time digging into it and it's triggered by > > extattr_get_link(2) returning EACCESS because a file is not > > readable, > > but stat(2) on the file succeeded. > > > > According to extattr(2), "[n]amed extended attributes are meta-data > > associated with vnodes" but the actual code for VOP_GETEXTATTR() > > (at least > > for ufs & zfs) checks for VREAD access, whereas the VOP_GETATTR() > > call > > (used by stat(2)) doesn't include any access checks (so stat(2) > > will > > succeed unless namei() fails). > > > > IMHO, this behaviour is inconsistent: The extended attributes are > > documented as "meta-data" and but the access checks are for "data". > > Which is correct? > > Practically speaking, extended attributes are used both > for data and metadata. > > I would consider the existing behavior (extattr calls fail on > non-readable files) to be correct in the absence of NFSv4 > ACLs (which include a specific permission for extattr readability). > Actually, NFSv4 doesn't support the notion of extended attributes at this time. It is being discussed for a future minor version. It does support the notion of fork files/subfiles, but I think they get their own ACLs. I believe the Read_attribute and Write_attribute flags in NFSv4 ACLs refer to the regular attributes and not extended ones. (I've cc'd trasz, since he'll know.) The Fedoro/Linux "man attr" states that extended attributes in their "user" name space are access controlled via the normal file mechanisms, so I believe Linux does check for read/write permissions. Since Linux distros are the lion's (not referring to an OS X version;-) share of what's out there, I'd say their semantics are defacto standard. --> I think that checking for read (or write) access for extended attributes is correct. rick > The extattr(2) manpage should probably document that > the calls fail on non-readable files. > > Tim > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" >