From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 00:39:42 2007 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.org Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 184B116A41F for ; Sun, 1 Jul 2007 00:39:42 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (pool-71-117-237-217.ptldor.fios.verizon.net [71.117.237.217]) by mx1.freebsd.org (Postfix) with ESMTP id D3AE513C469 for ; Sun, 1 Jul 2007 00:39:39 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (localhost.home.localnet [127.0.0.1]) by schitzo.solgatos.com (8.13.8/8.13.8) with ESMTP id l610dcvZ002765; Sat, 30 Jun 2007 17:39:39 -0700 Received: from sopwith.solgatos.com (uucp@localhost) by schitzo.solgatos.com (8.13.8/8.13.4/Submit) with UUCP id l610dcgj002745; Sat, 30 Jun 2007 17:39:38 -0700 Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id AAA08870; Sun, 1 Jul 2007 00:37:00 GMT Message-Id: <200707010037.AAA08870@sopwith.solgatos.com> To: freebsd-hackers@FreeBSD.org Date: Sat, 30 Jun 2007 17:37:00 +0100 From: Dieter X-Mailman-Approved-At: Sun, 01 Jul 2007 02:14:33 +0000 Cc: pb@ludd.ltu.se Subject: Re: SII3512 rev0 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd@sopwith.solgatos.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 00:39:42 -0000 > pcirev > { ATA_SII3512, 0x02, SIIMEMIO, 0, ATA_SA150, "SiI 3512" }, > { ATA_SII3512, 0x00, SIIMEMIO, SIIBUG, ATA_SA150, "SiI 3512" }, What happened to rev 1? > Any input on the reliability of SII3512 is also welcomed. dmesg | grep 3512 satalink0: Silicon Image SATALink 3512 (rev. 0x01) No reliability problems from the 3512. (up 74 days) This is on an Alpha box running NetBSD. I haven't tried the 3512 with FreeBSD. From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 03:08:33 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9311916A41F for ; Sun, 1 Jul 2007 03:08:33 +0000 (UTC) (envelope-from pb@ludd.ltu.se) Received: from mother.ludd.ltu.se (mother.ludd.ltu.se [130.240.16.3]) by mx1.freebsd.org (Postfix) with ESMTP id 1DE5B13C468 for ; Sun, 1 Jul 2007 03:08:32 +0000 (UTC) (envelope-from pb@ludd.ltu.se) Received: from brother.ludd.ltu.se (root@brother.ludd.ltu.se [130.240.16.78]) by mother.ludd.ltu.se (8.13.6+Sun/8.12.10) with ESMTP id l6138Uv6003027; Sun, 1 Jul 2007 05:08:30 +0200 (MEST) Received: from brother.ludd.ltu.se (pb@localhost [127.0.0.1]) by brother.ludd.ltu.se (8.13.6+Sun/8.12.2) with ESMTP id l6138UDE009651; Sun, 1 Jul 2007 05:08:30 +0200 (MEST) Received: (from pb@localhost) by brother.ludd.ltu.se (8.13.6+Sun/8.13.6/Submit) id l6138U24009649; Sun, 1 Jul 2007 05:08:30 +0200 (MEST) From: Peter B Message-Id: <200707010308.l6138U24009649@brother.ludd.ltu.se> To: freebsd@sopwith.solgatos.com Date: Sun, 1 Jul 2007 05:08:30 +0200 (MEST) In-Reply-To: <200707010037.AAA08870@sopwith.solgatos.com> from "Dieter" at Jun 30, 2007 05:37:00 PM X-Mailer: ELM [version 2.5 PL6] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Jul 2007 06:07:49 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: SII3512 rev0 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 03:08:33 -0000 Dieter wrote: > >> pcirev >> { ATA_SII3512, 0x02, SIIMEMIO, 0, ATA_SA150, "SiI 3512" }, >> { ATA_SII3512, 0x00, SIIMEMIO, SIIBUG, ATA_SA150, "SiI 3512" }, > >What happened to rev 1? > >> Any input on the reliability of SII3512 is also welcomed. > >dmesg | grep 3512 >satalink0: Silicon Image SATALink 3512 (rev. 0x01) > >No reliability problems from the 3512. (up 74 days) > >This is on an Alpha box running NetBSD. I haven't tried the 3512 >with FreeBSD. Could you try SII3512 (r1?) with FreeBSD-6.2 .. ? From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 08:42:33 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3687016A41F for ; Sun, 1 Jul 2007 08:42:33 +0000 (UTC) (envelope-from jgrosch@mooseriver.com) Received: from gdead.mooseriver.com (gdead.mooseriver.com [205.166.121.45]) by mx1.freebsd.org (Postfix) with ESMTP id 2114813C44C for ; Sun, 1 Jul 2007 08:42:33 +0000 (UTC) (envelope-from jgrosch@mooseriver.com) Received: by gdead.mooseriver.com (Postfix, from userid 2010) id 2CB013B709A; Sun, 1 Jul 2007 01:26:15 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on gdead.mooseriver.com X-Spam-Level: X-Spam-Status: No, score=0.3 required=5.0 tests=AWL,BAYES_40, DNS_FROM_RFC_WHOIS autolearn=no version=3.1.8 Received: from mooseriver.com (berkeley.mooseriver.com [75.61.201.134]) by gdead.mooseriver.com (Postfix) with ESMTP id D193A3B70AA for ; Sun, 1 Jul 2007 01:26:09 -0700 (PDT) Received: by mooseriver.com (Postfix, from userid 200) id B4FCA2E5C0C; Sun, 1 Jul 2007 01:26:09 -0700 (PDT) Date: Sun, 1 Jul 2007 01:26:09 -0700 From: Josef Grosch To: freebsd-hackers@freebsd.org Message-ID: <20070701082609.GA1058@mooseriver.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Organization: Moose River, LLC Subject: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: jgrosch@MooseRiver.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 08:42:33 -0000 --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable There are 3 drivers that I have been using with FreeBSD 6.2-RELEASE and 6.2-RELEASE-p5. They are=20 1) Intel's new driver for em 2) Adaptec's new driver for the 3805 3) 3ware's new SATA RAID controller (9500) The drivers can be found : Intel: =20 http://downloadcenter.intel.com/T8Clearance.aspx?sType=3D&agr=3DY&ProductID= =3D839&DwnldID=3D10957&url=3D/10957/eng/em-6.4.1.tar.gz&PrdMap=3D&strOSs=3D= 52&OSFullName=3DFreeBSD*&lang=3Deng Adaptec: http://www.adaptec.com/en-US/speed/raid/aac/unix/aacraid_drv_freebsd6_x86_b= 11669_tgz.htm http://www.adaptec.com/en-US/speed/raid/aac/unix/aacraid_drv_freebsd6_x64_b= 11669_tgz.htm 3ware: http://www.3ware.com/KB/attachments/twa96SE.ko-32bit-freebsd_6.2-GUIDfc6bd7= e1b3194d67b031c9c8277584eb.zip http://www.3ware.com/KB/attachments/twa96SE.ko-64bit-freebsd_6.2-GUID5a8405= 8ef94843898a297faa7a282a19.zip My question is will we be seeing these drivers in 6.3? Josef --=20 Josef Grosch | Another day closer to a | FreeBSD 6.2 jgrosch@MooseRiver.com | Micro$oft free world | Berkeley, Ca. --qMm9M+Fa2AknHoGS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFGh2Uhy8prLS1GYSERAkjOAJ99LDulCHy920NmCY7VVgZdgMMejgCgtlIb Jdhe+iTcyJ0B8hsqzhZa4uE= =BTxm -----END PGP SIGNATURE----- --qMm9M+Fa2AknHoGS-- From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 09:30:04 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 21BD316A421 for ; Sun, 1 Jul 2007 09:30:04 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from cydem.org (S01060060977141e2.vc.shawcable.net [24.87.15.201]) by mx1.freebsd.org (Postfix) with ESMTP id 0935C13C480 for ; Sun, 1 Jul 2007 09:30:01 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from soralx (soralx [192.168.0.240]) by cydem.org (Postfix/FreeBSD) with ESMTP id 2880E7F266; Sun, 1 Jul 2007 02:30:15 -0700 (PDT) Date: Sun, 1 Jul 2007 02:30:00 -0700 From: To: freebsd-hackers@freebsd.org Message-ID: <20070701023000.5b08947e@soralx> In-Reply-To: <20070701082609.GA1058@mooseriver.com> References: <20070701082609.GA1058@mooseriver.com> X-Mailer: Claws Mail 2.9.1 (GTK+ 2.10.12; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: jgrosch@MooseRiver.com Subject: Re: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 09:30:04 -0000 On Sun, 1 Jul 2007 01:26:09 -0700 Josef Grosch wrote: > There are 3 drivers that I have been using with FreeBSD 6.2-RELEASE > and 6.2-RELEASE-p5. They are > > 1) Intel's new driver for em > > 2) Adaptec's new driver for the 3805 > > 3) 3ware's new SATA RAID controller (9500) 04. myk by Marvell (88E8056 -- gigabit NIC) [SorAlx] ridin' VN1500-B2 From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 09:58:39 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 41A8B16A421 for ; Sun, 1 Jul 2007 09:58:39 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from ch-smtp01.sth.basefarm.net (ch-smtp01.sth.basefarm.net [80.76.149.212]) by mx1.freebsd.org (Postfix) with ESMTP id F161613C483 for ; Sun, 1 Jul 2007 09:58:38 +0000 (UTC) (envelope-from erikt@midgard.homeip.net) Received: from c83-253-10-135.bredband.comhem.se ([83.253.10.135]:64044 helo=falcon.midgard.homeip.net) by ch-smtp01.sth.basefarm.net with smtp (Exim 4.66) (envelope-from ) id 1I4wCM-0008DA-6O for freebsd-hackers@freebsd.org; Sun, 01 Jul 2007 11:58:38 +0200 Received: (qmail 6092 invoked from network); 1 Jul 2007 11:58:27 +0200 Received: from owl.midgard.homeip.net (10.1.5.7) by falcon.midgard.homeip.net with SMTP; 1 Jul 2007 11:58:27 +0200 Received: (qmail 15679 invoked by uid 1001); 1 Jul 2007 11:58:27 +0200 Date: Sun, 1 Jul 2007 11:58:27 +0200 From: Erik Trulsson To: soralx@cydem.org Message-ID: <20070701095827.GA15305@owl.midgard.homeip.net> Mail-Followup-To: soralx@cydem.org, freebsd-hackers@freebsd.org, jgrosch@MooseRiver.com References: <20070701082609.GA1058@mooseriver.com> <20070701023000.5b08947e@soralx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070701023000.5b08947e@soralx> User-Agent: Mutt/1.5.14 (2007-02-12) X-Originating-IP: 83.253.10.135 X-ACL-Warn: Too high rate of unknown addresses received from you X-Scan-Result: No virus found in message 1I4wCM-0008DA-6O. X-Scan-Signature: ch-smtp01.sth.basefarm.net 1I4wCM-0008DA-6O b120645419163b914d4d9578e29e2fd3 Cc: freebsd-hackers@freebsd.org, jgrosch@MooseRiver.com Subject: Re: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 09:58:39 -0000 On Sun, Jul 01, 2007 at 02:30:00AM -0700, soralx@cydem.org wrote: > On Sun, 1 Jul 2007 01:26:09 -0700 > Josef Grosch wrote: > > > There are 3 drivers that I have been using with FreeBSD 6.2-RELEASE > > and 6.2-RELEASE-p5. They are > > > > 1) Intel's new driver for em The em(4) driver in -CURRENT tends to be *very* up-to-date. At the moment it is up to version 6.5.3 (which is actually newer than the 6.4.1 driver that you can download from Intel for FreeBSD 6.x) The driver in 6-STABLE lags a bit behind (as one would expect). It is currently at version 6.2.9 (which is also fairly new.) I don't know if there are any plans for upgrading this before FreeBSD 6.3. > > > > 2) Adaptec's new driver for the 3805 No idea about this one. I don't think there is support for it in either -CURRENT or 6-STABLE at the moment and I don't know if anybody is working on it at the moment. > > > > 3) 3ware's new SATA RAID controller (9500) The latest twa(4) driver from 3ware is already in both -CURRENT and 6-STABLE so it will appear in FreeBSD 6.3 as well as in 7.0 > > 04. myk by Marvell (88E8056 -- gigabit NIC) > The msk(4) driver should support that chip. This driver is also available in both -CURRENT and 6-STABLE and thus will appear in both FreeBSD 6.3 and 7.0. -- Erik Trulsson ertr1013@student.uu.se From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 10:15:09 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 45D1316A469 for ; Sun, 1 Jul 2007 10:15:09 +0000 (UTC) (envelope-from chris@hitnet.RWTH-Aachen.DE) Received: from mta-1.ms.rz.rwth-aachen.de (mta-1.ms.rz.RWTH-Aachen.DE [134.130.7.72]) by mx1.freebsd.org (Postfix) with ESMTP id E361813C455 for ; Sun, 1 Jul 2007 10:15:08 +0000 (UTC) (envelope-from chris@hitnet.RWTH-Aachen.DE) Received: from ironport-out-1.rz.rwth-aachen.de ([134.130.3.58]) by mta-1.ms.rz.RWTH-Aachen.de (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTP id <0JKH00AD7ULEBU00@mta-1.ms.rz.RWTH-Aachen.de> for freebsd-hackers@freebsd.org; Sun, 01 Jul 2007 11:34:26 +0200 (CEST) Received: from talos.rz.rwth-aachen.de (HELO smarthost.rwth-aachen.de) ([134.130.3.22]) by ironport-in-1.rz.rwth-aachen.de with ESMTP; Sun, 01 Jul 2007 11:34:26 +0200 Received: from bigboss.hitnet.rwth-aachen.de (bigspace.hitnet.RWTH-Aachen.DE [137.226.181.2]) by smarthost.rwth-aachen.de (8.13.8/8.13.1/1) with ESMTP id l619YQBg003122; Sun, 01 Jul 2007 11:34:26 +0200 Received: from haakonia.hitnet.rwth-aachen.de ([137.226.181.92]) by bigboss.hitnet.rwth-aachen.de with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1I4vqs-00008v-3f; Sun, 01 Jul 2007 11:36:18 +0200 Received: by haakonia.hitnet.rwth-aachen.de (Postfix, from userid 1001) id BB12A3F41B; Sun, 01 Jul 2007 11:34:25 +0200 (CEST) Date: Sun, 01 Jul 2007 11:34:25 +0200 From: Christian Brueffer In-reply-to: <20070701023000.5b08947e@soralx> To: soralx@cydem.org Message-id: <20070701093425.GA1979@haakonia.hitnet.RWTH-Aachen.DE> MIME-version: 1.0 Content-type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary=ZGiS0Q5IWpPtfppv Content-disposition: inline X-IronPort-AV: E=Sophos;i="4.16,482,1175464800"; d="scan'208";a="5886443" X-Operating-System: FreeBSD 6.2-STABLE X-PGP-Key: http://people.FreeBSD.org/~brueffer/brueffer.key.asc X-PGP-Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D References: <20070701082609.GA1058@mooseriver.com> <20070701023000.5b08947e@soralx> User-Agent: Mutt/1.5.11 Cc: freebsd-hackers@freebsd.org Subject: Re: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 10:15:09 -0000 --ZGiS0Q5IWpPtfppv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 01, 2007 at 02:30:00AM -0700, soralx@cydem.org wrote: > On Sun, 1 Jul 2007 01:26:09 -0700 > Josef Grosch wrote: >=20 > > There are 3 drivers that I have been using with FreeBSD 6.2-RELEASE > > and 6.2-RELEASE-p5. They are=20 > >=20 > > 1) Intel's new driver for em > >=20 > > 2) Adaptec's new driver for the 3805 > >=20 > > 3) 3ware's new SATA RAID controller (9500) >=20 > 04. myk by Marvell (88E8056 -- gigabit NIC) >=20 That one is taken care of, see msk(4) in CURRENT and RELENG_6. - Christian --=20 Christian Brueffer chris@unixpages.org brueffer@FreeBSD.org GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D --ZGiS0Q5IWpPtfppv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGh3UhbHYXjKDtmC0RAjhNAKD+lk4IA0XKYTdwtHbG+QGBMtZkmQCcDYMq HSm4kT3HvQJYpmRAaDEzf/c= =X6H0 -----END PGP SIGNATURE----- --ZGiS0Q5IWpPtfppv-- From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 11:01:35 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AE89916A469 for ; Sun, 1 Jul 2007 11:01:35 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from cydem.org (S01060060977141e2.vc.shawcable.net [24.87.15.201]) by mx1.freebsd.org (Postfix) with ESMTP id 97B3113C43E for ; Sun, 1 Jul 2007 11:01:35 +0000 (UTC) (envelope-from soralx@cydem.org) Received: from soralx (soralx [192.168.0.240]) by cydem.org (Postfix/FreeBSD) with ESMTP id 117257F266; Sun, 1 Jul 2007 04:01:49 -0700 (PDT) Date: Sun, 1 Jul 2007 04:01:30 -0700 From: To: Message-ID: <20070701040130.6954c535@soralx> In-Reply-To: <20070701095827.GA15305@owl.midgard.homeip.net> References: <20070701082609.GA1058@mooseriver.com> <20070701023000.5b08947e@soralx> <20070701095827.GA15305@owl.midgard.homeip.net> X-Mailer: Claws Mail 2.9.1 (GTK+ 2.10.12; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, jgrosch@MooseRiver.com Subject: Re: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 11:01:35 -0000 On Sun, 1 Jul 2007 11:58:27 +0200 Erik Trulsson wrote: > The msk(4) driver should support that chip. This driver is also > available in both -CURRENT and 6-STABLE and thus will appear in both > FreeBSD 6.3 and 7.0. great! is it as good as Marvell's one? [SorAlx] ridin' VN1500-B2 From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 16:11:00 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5A8EF16A400 for ; Sun, 1 Jul 2007 16:11:00 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (pool-71-117-237-217.ptldor.fios.verizon.net [71.117.237.217]) by mx1.freebsd.org (Postfix) with ESMTP id 3A69813C469 for ; Sun, 1 Jul 2007 16:10:58 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (localhost.home.localnet [127.0.0.1]) by schitzo.solgatos.com (8.13.8/8.13.8) with ESMTP id l61GAvnN011244; Sun, 1 Jul 2007 09:10:57 -0700 Received: from sopwith.solgatos.com (uucp@localhost) by schitzo.solgatos.com (8.13.8/8.13.4/Submit) with UUCP id l61GAvbm011241; Sun, 1 Jul 2007 09:10:57 -0700 Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id QAA29911; Sun, 1 Jul 2007 16:01:29 GMT Message-Id: <200707011601.QAA29911@sopwith.solgatos.com> To: Peter B In-reply-to: Your message of "Sun, 01 Jul 2007 05:08:30 +0200." <200707010308.l6138U24009649@brother.ludd.ltu.se> Date: Sun, 01 Jul 2007 09:01:29 +0100 From: Dieter X-Mailman-Approved-At: Sun, 01 Jul 2007 16:22:33 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: SII3512 rev0 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 16:11:00 -0000 > >> Any input on the reliability of SII3512 is also welcomed. > > > >dmesg | grep 3512 > >satalink0: Silicon Image SATALink 3512 (rev. 0x01) > > > >No reliability problems from the 3512. (up 74 days) > > > >This is on an Alpha box running NetBSD. I haven't tried the 3512 > >with FreeBSD. I took a quick look at the drivers, and found a difference. NetBSD: dev/pci/satalink.c /* * Rev. <= 0x01 of the 3112 have a bug that can cause data * corruption if DMA transfers cross an 8K boundary. This is * apparently hard to tickle, but we'll go ahead and play it * safe. */ if (PCI_REVISION(pa->pa_class) <= 0x01) { sc->sc_dma_maxsegsz = 8192; sc->sc_dma_boundary = 8192; FreeBSD: dev/ata/ata-chipset.c if ((ctlr->chip->cfg2 & SIIBUG) && ch->dma) { /* work around errata in early chips */ ch->dma->boundary = 16 * DEV_BSIZE; ch->dma->segsize = 15 * DEV_BSIZE; } And of course there may be other differences I didn't find. From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 18:13:35 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A998616A41F for ; Sun, 1 Jul 2007 18:13:35 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.freebsd.org (Postfix) with ESMTP id 2C79D13C45A for ; Sun, 1 Jul 2007 18:13:34 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by koef.zs64.net (8.14.1/8.14.1) with ESMTP id l61IDYe3078386 for ; Sun, 1 Jul 2007 20:13:34 +0200 (CEST) (envelope-from cracauer@koef.zs64.net) Received: (from cracauer@localhost) by koef.zs64.net (8.14.1/8.14.1/Submit) id l61IDYIX078385 for freebsd-hackers@freebsd.org; Sun, 1 Jul 2007 14:13:34 -0400 (EDT) (envelope-from cracauer) Date: Sun, 1 Jul 2007 14:13:34 -0400 From: Martin Cracauer To: freebsd-hackers@freebsd.org Message-ID: <20070701181334.GA77427@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i Subject: pxeboot and /boot filesystem, share /boot/kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 18:13:35 -0000 I want to tighten up my spaces for diskless machines and I came across this puzzle with pxeboot: I can share /usr and most other filesystems, but my individual roots for the machine each have to have the full kernel. But /boot/kernel is rather large these days and totally identical, so I'd rather share it. %% You can only specify a root filesystem location via the dhcp options. Then, whatever kernel is there in location:/boot/kernel (or rather, loader.rc) will be booted, which will then pick up the / filesystem via location:/etc/fstab. This is not quite what I want, because I want /boot/kernel to be shared, purely for filesize reasons. But I can't specify a separate /boot filesystem. I can't find an easy way to share /boot but not share /. Or in other words, the core of the problem is that I want to share /boot/kernel but not share /etc/fstab. %% So, while writing this mail it occured to me that what I can do is put a /boot/loader.rc on the individual / filesystems that then redirects to a common kernel location. But I don't see how I can make this work as I do not have the option to point to a new NFS mount in /boot/loader.conf, or do I? So what I would want is (on the server): /diskless-usr/ /diskless-kernel/ [has /boot/kernel/ in it] /diskless/root1/ [has /boot/loader.conf in it] /diskless/root2/ DHCP "root-path" is then addr:/diskless/root1 Where /diskless/root1/boot/loader.conf specifies addr:/diskless-kernel/boot/kernel/kernel instead of just a filename. Now, the question is how do I make loader.4th able to mount NFS or use tftp? %% I think I have three paths to go here: 1) make pxeboot understand a separate "boot-path" dhcp option. 2) make loader.4th able to use NFS or tftp. IP is already up by the time it is started. 3) only share /boot/kernel/kernel and share a NFS mount for the modules, but that's very messy. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/ From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 18:03:24 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81E5916A400 for ; Sun, 1 Jul 2007 18:03:24 +0000 (UTC) (envelope-from pb@ludd.ltu.se) Received: from mother.ludd.ltu.se (mother.ludd.ltu.se [130.240.16.3]) by mx1.freebsd.org (Postfix) with ESMTP id 118DE13C447 for ; Sun, 1 Jul 2007 18:03:23 +0000 (UTC) (envelope-from pb@ludd.ltu.se) Received: from brother.ludd.ltu.se (root@brother.ludd.ltu.se [130.240.16.78]) by mother.ludd.ltu.se (8.13.6+Sun/8.12.10) with ESMTP id l61I3LRb014530; Sun, 1 Jul 2007 20:03:21 +0200 (MEST) Received: from brother.ludd.ltu.se (pb@localhost [127.0.0.1]) by brother.ludd.ltu.se (8.13.6+Sun/8.12.2) with ESMTP id l61I3LM9010285; Sun, 1 Jul 2007 20:03:21 +0200 (MEST) Received: (from pb@localhost) by brother.ludd.ltu.se (8.13.6+Sun/8.13.6/Submit) id l61I3KbA010283; Sun, 1 Jul 2007 20:03:20 +0200 (MEST) From: Peter B Message-Id: <200707011803.l61I3KbA010283@brother.ludd.ltu.se> To: freebsd@sopwith.solgatos.com (Dieter) Date: Sun, 1 Jul 2007 20:03:20 +0200 (MEST) In-Reply-To: <200707011601.QAA29911@sopwith.solgatos.com> from "Dieter" at Jul 01, 2007 09:01:29 AM X-Mailer: ELM [version 2.5 PL6] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 01 Jul 2007 18:40:16 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: SII3512 rev0 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 18:03:24 -0000 >NetBSD: > dev/pci/satalink.c > > /* > * Rev. <= 0x01 of the 3112 have a bug that can cause data > * corruption if DMA transfers cross an 8K boundary. This is > * apparently hard to tickle, but we'll go ahead and play it > * safe. > */ > if (PCI_REVISION(pa->pa_class) <= 0x01) { > sc->sc_dma_maxsegsz = 8192; > sc->sc_dma_boundary = 8192; > >FreeBSD: > dev/ata/ata-chipset.c > > if ((ctlr->chip->cfg2 & SIIBUG) && ch->dma) { > /* work around errata in early chips */ > ch->dma->boundary = 16 * DEV_BSIZE; > ch->dma->segsize = 15 * DEV_BSIZE; DEV_BSIZE = (1<<9) = 512 bytes boundary = 8192 segsize = 7680 <- Differs by 512 bytes from NetBSD.. Anyone knows why segsize differs? >And of course there may be other differences I didn't find. The table used to parse pci setup: {{ ATA_SII3114, 0x00, SIIMEMIO, SII4CH, ATA_SA150, "SiI 3114" }, { ATA_SII3512, 0x02, SIIMEMIO, 0, ATA_SA150, "SiI 3512" }, { ATA_SII3112, 0x02, SIIMEMIO, 0, ATA_SA150, "SiI 3112" }, { ATA_SII3112_1, 0x02, SIIMEMIO, 0, ATA_SA150, "SiI 3112" }, { ATA_SII3512, 0x00, SIIMEMIO, SIIBUG, ATA_SA150, "SiI 3512" }, { ATA_SII3112, 0x00, SIIMEMIO, SIIBUG, ATA_SA150, "SiI 3112" }, { ATA_SII3112_1, 0x00, SIIMEMIO, SIIBUG, ATA_SA150, "SiI 3112" }, { ATA_SII3124, 0x00, SIIPRBIO, SII4CH, ATA_SA300, "SiI 3124" }, { ATA_SII3132, 0x00, SIIPRBIO, 0, ATA_SA300, "SiI 3132" }, And it's parsed with this conditional: pci_get_revid(dev) >= index->chiprev /* ata_match_chip() */ So for 3112 rev 0,1 SIIBUG will be enabled on FreeBSD-curr. Ie same behaviour as for when to apply the fix. This hw seems to be like russian roulette ;) Maybe it would be a good idea to have chip revision displayed in the syslog such that an sysadm can quickly spot possible troublesome hw.. In ata_sii_ident() replace: sprintf(buffer, "%s %s controller", idx->text, ata_mode2str(idx->max_dma)); With something along the lines of: int chiprev; chiprev = (int)pci_get_revid(dev); /* assumed unfailable :) */ sprintf(buffer, "%s rev %d %s controller", idx->text, chiprev, ata_mode2str(idx->max_dma) ); /P From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 19:05:42 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 12B3416A421 for ; Sun, 1 Jul 2007 19:05:42 +0000 (UTC) (envelope-from svs@grep.ru) Received: from svs.complife.ru (svs.complife.ru [212.5.80.73]) by mx1.freebsd.org (Postfix) with ESMTP id BBB8113C458 for ; Sun, 1 Jul 2007 19:05:41 +0000 (UTC) (envelope-from svs@grep.ru) Received: from ppp85-141-97-209.pppoe.mtu-net.ru ([85.141.97.209] helo=mononoke.nyo) by svs.complife.ru with esmtpa (Exim 4.54) id 1I54DS-000Atl-MX for freebsd-hackers@freebsd.org; Sun, 01 Jul 2007 22:32:10 +0400 Received: from svs by mononoke.nyo with local (Exim 4.66) (envelope-from ) id 1I54Cw-0003RZ-H1 for freebsd-hackers@freebsd.org; Sun, 01 Jul 2007 22:31:38 +0400 Date: Sun, 1 Jul 2007 22:31:38 +0400 From: Sergey Svishchev To: freebsd-hackers@freebsd.org Message-ID: <20070701183138.GA2232@mononoke.nyo> Mail-Followup-To: freebsd-hackers@freebsd.org References: <466DBF4A.90303@datapipe.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="C7zPtVaVf+AK4Oqc" Content-Disposition: inline In-Reply-To: <466DBF4A.90303@datapipe.com> User-Agent: Mutt/1.5.14cvs (2007-02-28) X-Mailman-Approved-At: Sun, 01 Jul 2007 19:38:27 +0000 Subject: Re: HP SmartArray ( CISS ) and CAMCONTROL X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 19:05:42 -0000 --C7zPtVaVf+AK4Oqc Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 11, 2007 at 05:31:54PM -0400, Mark Saad wrote: >Hello Hackers@ > > I have been searching around and I can not find an answer to a=20 >problem of mine . Does anyone know if its possible >to make camcontrol show the drive health for drives in a HP SMART Array=20 >. Currently it will show the Array heath >by running camcontrol devlist -v . arrayprobe (http://www.strocamp.net/opensource/arrayprobe.php) could be ported, I guess. --=20 Sergey Svishchev --C7zPtVaVf+AK4Oqc Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Comment: '' iD8DBQFGh/MK3P+wmuUns7URAsKTAKChuLuVxwLiB0BC3SIOfQ61HQ2fOACcChC4 plPqteRas79UCtED1hTK4eQ= =B14k -----END PGP SIGNATURE----- --C7zPtVaVf+AK4Oqc-- From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 19:54:29 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2F3CF16A41F for ; Sun, 1 Jul 2007 19:54:29 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id B4DA413C448 for ; Sun, 1 Jul 2007 19:54:28 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de ([10.1.1.7]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id l61Jakjv089274; Sun, 1 Jul 2007 21:36:47 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [10.1.1.14]) by cicely5.cicely.de (8.13.4/8.13.4) with ESMTP id l61JafUl034919 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 1 Jul 2007 21:36:42 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.13.4/8.13.3) with ESMTP id l61Jaf01060608; Sun, 1 Jul 2007 21:36:41 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.13.4/8.13.3/Submit) id l61Jaf1h060607; Sun, 1 Jul 2007 21:36:41 +0200 (CEST) (envelope-from ticso) Date: Sun, 1 Jul 2007 21:36:41 +0200 From: Bernd Walter To: Martin Cracauer Message-ID: <20070701193640.GW50157@cicely12.cicely.de> References: <20070701181334.GA77427@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070701181334.GA77427@cons.org> X-Operating-System: FreeBSD cicely12.cicely.de 5.4-STABLE alpha User-Agent: Mutt/1.5.9i X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.1.7 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on cicely12.cicely.de Cc: freebsd-hackers@freebsd.org Subject: Re: pxeboot and /boot filesystem, share /boot/kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 19:54:29 -0000 On Sun, Jul 01, 2007 at 02:13:34PM -0400, Martin Cracauer wrote: > I want to tighten up my spaces for diskless machines and I came across > this puzzle with pxeboot: > > I can share /usr and most other filesystems, but my individual roots > for the machine each have to have the full kernel. But /boot/kernel > is rather large these days and totally identical, so I'd rather share > it. > > %% > > You can only specify a root filesystem location via the dhcp options. > Then, whatever kernel is there in location:/boot/kernel (or rather, > loader.rc) will be booted, which will then pick up the / filesystem > via location:/etc/fstab. > > This is not quite what I want, because I want /boot/kernel to be > shared, purely for filesize reasons. But I can't specify a separate > /boot filesystem. I can't find an easy way to share /boot but not > share /. > > Or in other words, the core of the problem is that I want to share > /boot/kernel but not share /etc/fstab. > > %% > > So, while writing this mail it occured to me that what I can do is put > a /boot/loader.rc on the individual / filesystems that then redirects > to a common kernel location. But I don't see how I can make this work > as I do not have the option to point to a new NFS mount in > /boot/loader.conf, or do I? > > So what I would want is (on the server): > /diskless-usr/ > /diskless-kernel/ [has /boot/kernel/ in it] > /diskless/root1/ [has /boot/loader.conf in it] > /diskless/root2/ > > DHCP "root-path" is then addr:/diskless/root1 > > Where > /diskless/root1/boot/loader.conf > specifies > addr:/diskless-kernel/boot/kernel/kernel > instead of just a filename. > > Now, the question is how do I make loader.4th able to mount NFS or use > tftp? > > %% > > I think I have three paths to go here: > > 1) make pxeboot understand a separate "boot-path" dhcp option. > > 2) make loader.4th able to use NFS or tftp. IP is already up by the > time it is started. > > 3) only share /boot/kernel/kernel and share a NFS mount for the > modules, but that's very messy. 4) Use different / on the same server filesystem with hardlinked /boot. 5) Use two / - one common for booting and a client specific. loader.rc in common will overwrite rootfs. But I'm unshure if you can get host specific data, e.g. MAC from net interface. 6) Don't install *.symbols and live with multiple file space usage for the rest. -- B.Walter http://www.bwct.de http://www.fizon.de bernd@bwct.de info@bwct.de support@fizon.de From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 21:35:43 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 46FE916A468 for ; Sun, 1 Jul 2007 21:35:43 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by mx1.freebsd.org (Postfix) with ESMTP id CB5B113C45E for ; Sun, 1 Jul 2007 21:35:42 +0000 (UTC) (envelope-from cracauer@koef.zs64.net) Received: from koef.zs64.net (koef.zs64.net [212.12.50.230]) by koef.zs64.net (8.14.1/8.14.1) with ESMTP id l61LZeBt084582; Sun, 1 Jul 2007 23:35:40 +0200 (CEST) (envelope-from cracauer@koef.zs64.net) Received: (from cracauer@localhost) by koef.zs64.net (8.14.1/8.14.1/Submit) id l61LZeb2084581; Sun, 1 Jul 2007 17:35:40 -0400 (EDT) (envelope-from cracauer) Date: Sun, 1 Jul 2007 17:35:40 -0400 From: Martin Cracauer To: ticso@cicely.de Message-ID: <20070701213540.GA83788@cons.org> References: <20070701181334.GA77427@cons.org> <20070701193640.GW50157@cicely12.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070701193640.GW50157@cicely12.cicely.de> User-Agent: Mutt/1.4.2.2i Cc: freebsd-hackers@freebsd.org, Martin Cracauer Subject: Re: pxeboot and /boot filesystem, share /boot/kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 21:35:43 -0000 Bernd Walter wrote on Sun, Jul 01, 2007 at 09:36:41PM +0200: > On Sun, Jul 01, 2007 at 02:13:34PM -0400, Martin Cracauer wrote: > > I want to tighten up my spaces for diskless machines and I came across > > this puzzle with pxeboot: > > > > I can share /usr and most other filesystems, but my individual roots > > for the machine each have to have the full kernel. But /boot/kernel > > is rather large these days and totally identical, so I'd rather share > > it. [...] > > I think I have three paths to go here: > > > > 1) make pxeboot understand a separate "boot-path" dhcp option. > > > > 2) make loader.4th able to use NFS or tftp. IP is already up by the > > time it is started. > > > > 3) only share /boot/kernel/kernel and share a NFS mount for the > > modules, but that's very messy. > > 4) Use different / on the same server filesystem with hardlinked /boot. Hmm, directory hardlinks through NFS server? But might do. I also considered hacking up the NFS server code to resolve individual symlinks of my choice. I like this feature a lot when serving samba shares to Windows clients to make stupid applications actually place stuff where I want it. But this solution is bad news if you rebase your NFS server. %% It has been pointed out to me that BSD used to have /etc/fstab. back in the day. That seems to be easy enough to re-introduce given that pxeboot already set the IP address. However, it is somewhat hackey since in effect your first root filesystem (common for all clients) gets used only once for /etc/fstab and is only used to redirect to a new (indivual) root filesystem. > 5) Use two / - one common for booting and a client specific. > loader.rc in common will overwrite rootfs. > But I'm unshure if you can get host specific data, e.g. MAC from net > interface. That is somewhat similar to /etc/fstab. but would require the 4th to know about networking. > 6) Don't install *.symbols and live with multiple file space usage for > the rest. Right but not sportish :-) %% Overall, I also need to keep in mind that some but but all of my diskless clients will have individual kernels and that I want the most elegant way to express this. The most elegant way to do this would either be a per-host /boot filesystem mount (which I would want to be used both at kernel load time and during runtime). Or a per-host kernel setting in loader.rc which normally points to the common kernel but can point to an individual one. But as mentioned earlier, this is difficult to do if you / is already unshared, unless the loader can deal with NFS or tftp to load from a address:/boot/... location. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/ From owner-freebsd-hackers@FreeBSD.ORG Sun Jul 1 22:55:32 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47AA916A400 for ; Sun, 1 Jul 2007 22:55:32 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 2449213C45A for ; Sun, 1 Jul 2007 22:55:30 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de ([10.1.1.7]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id l61MtS2q094501; Mon, 2 Jul 2007 00:55:28 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [10.1.1.14]) by cicely5.cicely.de (8.13.4/8.13.4) with ESMTP id l61MtNUU036285 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Jul 2007 00:55:23 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.13.4/8.13.3) with ESMTP id l61MtMrE061074; Mon, 2 Jul 2007 00:55:22 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.13.4/8.13.3/Submit) id l61MtM4Z061073; Mon, 2 Jul 2007 00:55:22 +0200 (CEST) (envelope-from ticso) Date: Mon, 2 Jul 2007 00:55:22 +0200 From: Bernd Walter To: Martin Cracauer Message-ID: <20070701225522.GB50157@cicely12.cicely.de> References: <20070701181334.GA77427@cons.org> <20070701193640.GW50157@cicely12.cicely.de> <20070701213540.GA83788@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070701213540.GA83788@cons.org> X-Operating-System: FreeBSD cicely12.cicely.de 5.4-STABLE alpha User-Agent: Mutt/1.5.9i X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.1.7 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on cicely12.cicely.de Cc: freebsd-hackers@freebsd.org, ticso@cicely.de Subject: Re: pxeboot and /boot filesystem, share /boot/kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2007 22:55:32 -0000 On Sun, Jul 01, 2007 at 05:35:40PM -0400, Martin Cracauer wrote: > Bernd Walter wrote on Sun, Jul 01, 2007 at 09:36:41PM +0200: > > On Sun, Jul 01, 2007 at 02:13:34PM -0400, Martin Cracauer wrote: > > 4) Use different / on the same server filesystem with hardlinked /boot. > > Hmm, directory hardlinks through NFS server? But might do. I ment the files in it hardlinked. > I also considered hacking up the NFS server code to resolve individual > symlinks of my choice. I like this feature a lot when serving samba > shares to Windows clients to make stupid applications actually place > stuff where I want it. But this solution is bad news if you rebase > your NFS server. Yes - too easy to forget. > %% > > It has been pointed out to me that BSD used to have > /etc/fstab. back in the day. That seems to be easy enough > to re-introduce given that pxeboot already set the IP address. > > However, it is somewhat hackey since in effect your first root > filesystem (common for all clients) gets used only once for /etc/fstab > and is only used to redirect to a new (indivual) root filesystem. I have something quite similar with a bunch of NetBSD hosts. Common / with different var partitions mounted by custom scripts and not by fstab. Since hostname was already set by dhcp at this point it worked. > > 5) Use two / - one common for booting and a client specific. > > loader.rc in common will overwrite rootfs. > > But I'm unshure if you can get host specific data, e.g. MAC from net > > interface. > > That is somewhat similar to /etc/fstab. but would require the > 4th to know about networking. It does know the bootdev and I just checked the soruce and it even sets variables: common/dev_net.c: setenv("boot.netif.ip", inet_ntoa(myip), 1); common/dev_net.c: setenv("boot.netif.netmask", intoa(netmask), 1); common/dev_net.c: setenv("boot.netif.gateway", inet_ntoa(gateip), 1); common/dev_net.c: setenv("boot.netif.hwaddr", temp, 1); common/dev_net.c: setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); common/dev_net.c: setenv("boot.nfsroot.path", rootpath, 1); No hostname, but IP and MAC. You just have to use the variables in your own loader.rc to overwrite the path. You can fetch them with getenv: s" boot.netif.ip" getenv Well - my forth is too rusty, so I can't help on the if/setenv part. > > 6) Don't install *.symbols and live with multiple file space usage for > > the rest. > > Right but not sportish :-) :-) > Overall, I also need to keep in mind that some but but all of my > diskless clients will have individual kernels and that I want the most > elegant way to express this. If you want you can select the kernel in your loader.rc as well... > Or a per-host kernel setting in loader.rc which normally points to the > common kernel but can point to an individual one. But as mentioned > earlier, this is difficult to do if you / is already unshared, unless > the loader can deal with NFS or tftp to load from a > address:/boot/... location. It can at least read any file from the already mounted NFS path. No problem to read /boot/kernel.hostx/ instead of default. Don't know if you can switch to another NFS path. -- B.Walter http://www.bwct.de http://www.fizon.de bernd@bwct.de info@bwct.de support@fizon.de From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 2 06:04:32 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 04D3716A46C for ; Mon, 2 Jul 2007 06:04:32 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id AB37A13C4B7 for ; Mon, 2 Jul 2007 06:04:31 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1I5F1S-000Mee-2s; Mon, 02 Jul 2007 09:04:30 +0300 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Martin Cracauer In-reply-to: Your message of Sun, 1 Jul 2007 14:13:34 -0400 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Jul 2007 09:04:30 +0300 From: Danny Braniss Message-ID: Cc: freebsd-hackers@freebsd.org Subject: Re: pxeboot and /boot filesystem, share /boot/kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2007 06:04:32 -0000 > I want to tighten up my spaces for diskless machines and I came across > this puzzle with pxeboot: > > I can share /usr and most other filesystems, but my individual roots > for the machine each have to have the full kernel. But /boot/kernel > is rather large these days and totally identical, so I'd rather share > it. > > %% > > You can only specify a root filesystem location via the dhcp options. > Then, whatever kernel is there in location:/boot/kernel (or rather, > loader.rc) will be booted, which will then pick up the / filesystem > via location:/etc/fstab. > > This is not quite what I want, because I want /boot/kernel to be > shared, purely for filesize reasons. But I can't specify a separate > /boot filesystem. I can't find an easy way to share /boot but not > share /. > > Or in other words, the core of the problem is that I want to share > /boot/kernel but not share /etc/fstab. > > %% > > So, while writing this mail it occured to me that what I can do is put > a /boot/loader.rc on the individual / filesystems that then redirects > to a common kernel location. But I don't see how I can make this work > as I do not have the option to point to a new NFS mount in > /boot/loader.conf, or do I? > > So what I would want is (on the server): > /diskless-usr/ > /diskless-kernel/ [has /boot/kernel/ in it] > /diskless/root1/ [has /boot/loader.conf in it] > /diskless/root2/ > > DHCP "root-path" is then addr:/diskless/root1 > > Where > /diskless/root1/boot/loader.conf > specifies > addr:/diskless-kernel/boot/kernel/kernel > instead of just a filename. > > Now, the question is how do I make loader.4th able to mount NFS or use > tftp? > > %% > > I think I have three paths to go here: > > 1) make pxeboot understand a separate "boot-path" dhcp option. > > 2) make loader.4th able to use NFS or tftp. IP is already up by the > time it is started. > > 3) only share /boot/kernel/kernel and share a NFS mount for the > modules, but that's very messy. > look in rc.initdiskless, you'll see there the solution. if you mount unionfs /etc, you can have a /(root) mounted read-only. unless there is something I don't see, there is no real reason to separate / from /usr. danny From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 2 16:13:24 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6C52616A400 for ; Mon, 2 Jul 2007 16:13:24 +0000 (UTC) (envelope-from craig@tobuj.gank.org) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.freebsd.org (Postfix) with ESMTP id 573F213C4CC for ; Mon, 2 Jul 2007 16:13:22 +0000 (UTC) (envelope-from craig@tobuj.gank.org) Received: by ion.gank.org (Postfix, from userid 1001) id 25A151121C; Mon, 2 Jul 2007 11:13:22 -0500 (CDT) Date: Mon, 2 Jul 2007 11:13:20 -0500 From: Craig Boston To: Josef Grosch Message-ID: <20070702161304.GA95593@nowhere> Mail-Followup-To: Craig Boston , Josef Grosch , freebsd-hackers@freebsd.org References: <20070701082609.GA1058@mooseriver.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070701082609.GA1058@mooseriver.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: Device drivers to be commited. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2007 16:13:24 -0000 On Sun, Jul 01, 2007 at 01:26:09AM -0700, Josef Grosch wrote: > 3) 3ware's new SATA RAID controller (9500) Latest -stable seems to have this already; at least one that's recent enough for a shiny new 9650SE to work (which doesn't work in 6.2 release). Craig From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 2 16:55:03 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 39F2016A400 for ; Mon, 2 Jul 2007 16:55:03 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.freebsd.org (Postfix) with ESMTP id 88FFF13C43E for ; Mon, 2 Jul 2007 16:55:02 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so3160957pyb for ; Mon, 02 Jul 2007 09:55:01 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=FaZqDjJ7/EILdd1aYBaMzd3CPa52bTE4BNBLuTUNb3ek53lGx69OXvQfVSeRz0WpxdCvImXiwKVnnukpuYrP8J7szNSW2KdH8qOU6Rhk/S2zUo5HipD5aXU8HwOupkeeZtc9WpkpJmPa7uETSV2CoGaaNFefFaXto9Gn6bP54U0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=r80cyIAqVIMGA4KaBc42C11Vs3wDCyexQLb3cvSuyIiwrSUCWAGoHKNJ5eYql3QveFYamnjoSThF8Ab9NGrENHSjkGA7I/VdFMe6xEiYgk8y6IQGyNs8n9YMkq5Op3TPSZzy7mGHxa13QxoT9VQUeHaWwPbZquywV3b8VUmMzvE= Received: by 10.35.41.8 with SMTP id t8mr6216156pyj.1183395301876; Mon, 02 Jul 2007 09:55:01 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Mon, 2 Jul 2007 09:54:56 -0700 (PDT) Message-ID: Date: Mon, 2 Jul 2007 18:54:56 +0200 From: "Nicolas Cormier" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2007 16:55:03 -0000 Hi, I am trying to map some data allocated in kernel to a user process (via a syscall). I need the proc's vmspace, but the value of p_vmspace of the input proc argument is NULL ... How can I get a valid vmspace ? Thanks ! -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 3 04:18:08 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DE2DD16A41F for ; Tue, 3 Jul 2007 04:18:08 +0000 (UTC) (envelope-from SRS0=D4AtGz=MB=vvelox.net=v.velox@yourhostingaccount.com) Received: from mailout11.yourhostingaccount.com (mailout11.yourhostingaccount.com [65.254.253.88]) by mx1.freebsd.org (Postfix) with ESMTP id B04CB13C447 for ; Tue, 3 Jul 2007 04:18:08 +0000 (UTC) (envelope-from SRS0=D4AtGz=MB=vvelox.net=v.velox@yourhostingaccount.com) Received: from mailscan14.yourhostingaccount.com ([10.1.15.14] helo=mailscan14.yourhostingaccount.com) by mailout11.yourhostingaccount.com with esmtp (Exim) id 1I5ZM1-0008Tb-II for freebsd-hackers@freebsd.org; Mon, 02 Jul 2007 23:47:05 -0400 Received: from impout03.yourhostingaccount.com ([10.1.55.3] ident=exim) by mailscan14.yourhostingaccount.com with spamscanlookuphost (Exim) id 1I5ZM1-0007bs-Ao for freebsd-hackers@freebsd.org; Mon, 02 Jul 2007 23:47:05 -0400 Received: from impout03.yourhostingaccount.com ([10.1.55.3] helo=impout03.yourhostingaccount.com) by mailscan14.yourhostingaccount.com with esmtp (Exim) id 1I5ZM0-0002ix-Te for freebsd-hackers@freebsd.org; Mon, 02 Jul 2007 23:47:05 -0400 Received: from authsmtp10.yourhostingaccount.com ([10.1.18.10]) by impout03.yourhostingaccount.com with NO UCE id Jrn41X0020D2B7u0000000; Mon, 02 Jul 2007 23:47:04 -0400 X-EN-OrigOutIP: 10.1.18.10 X-EN-IMPSID: Jrn41X0020D2B7u0000000 Received: from cpe-24-93-100-44.columbus.res.rr.com ([24.93.100.44] helo=vixen42) by authsmtp10.yourhostingaccount.com with esmtpa (Exim) id 1I5ZLz-0005fM-Bz for freebsd-hackers@freebsd.org; Mon, 02 Jul 2007 23:47:04 -0400 Date: Mon, 2 Jul 2007 23:48:07 -0400 From: "Z.C.B." To: freebsd-hackers@freebsd.org Message-ID: <20070702234807.38be164a@vixen42> X-Mailer: Claws Mail 2.9.2 (GTK+ 2.10.13; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-EN-UserInfo: 0d1ca1697cdb7a831d4877828571b7ab:1570f0de6936c69fef9e164fffc541bc X-EN-AuthUser: vvelox2 Sender: "Z.C.B." X-EN-OrigIP: 24.93.100.44 X-EN-OrigHost: cpe-24-93-100-44.columbus.res.rr.com Subject: where to start on kernel modules and NSS? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2007 04:18:08 -0000 Have just been looking at sitting down and looking at writing a few things, but I am a bit lost on where to start. Any suggestions on things to read in regards to NSS and writing kernel modules? From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 3 09:04:30 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 582C716A400 for ; Tue, 3 Jul 2007 09:04:30 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0ED9713C480 for ; Tue, 3 Jul 2007 09:04:29 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1I5eJ9-0004NV-OJ for freebsd-hackers@freebsd.org; Tue, 03 Jul 2007 11:04:27 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Jul 2007 11:04:27 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 03 Jul 2007 11:04:27 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Tue, 03 Jul 2007 11:04:17 +0200 Lines: 36 Message-ID: References: <20070702234807.38be164a@vixen42> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig042D228142AAB40253793680" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 1.5.0.12 (X11/20060911) In-Reply-To: <20070702234807.38be164a@vixen42> X-Enigmail-Version: 0.94.2.0 Sender: news Subject: Re: where to start on kernel modules and NSS? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2007 09:04:30 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig042D228142AAB40253793680 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Z.C.B. wrote: > Have just been looking at sitting down and looking at writing a few > things, but I am a bit lost on where to start. Any suggestions on > things to read in regards to NSS and writing kernel modules? These are not similar things and you don't need kernel modules for NSS.=20 For kernel modules, you might try reading: http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/inde= x.html http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/index.html= http://www.freebsd.org/doc/en_US.ISO8859-1/articles/geom-class/kernelprog= =2Ehtml --------------enig042D228142AAB40253793680 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFGihERldnAQVacBcgRAsc8AJ9X3WcAGbSSLYr2IsWiEc6Z00bajQCcCFCj XoKdwYp8l5VpCuJ6QOTgY34= =DQAw -----END PGP SIGNATURE----- --------------enig042D228142AAB40253793680-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 3 13:40:29 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5C72916A41F for ; Tue, 3 Jul 2007 13:40:29 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 1C94F13C468 for ; Tue, 3 Jul 2007 13:40:28 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so3687486pyb for ; Tue, 03 Jul 2007 06:40:28 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Oq+6VKw8rsiWo1Ge5eRL6UnCdibZpCxlOT1f0Z+VvaIri7BbnRQU1Oh7punnrNVJk2apQhkoxnhiqKGGYnfpDYaW+8g2RkBhha7VjP6v0+vGD3UN3bzWf4Rea8tjecfRXuozzPOpQyLTFKSM8Sdkpqalt1IvNs6roNi73ZxtjEA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=d9YC+xDJxtubyrn0IFFQ22gn3bNRYRMC7pH3kEJz4UD9LSY70vMxTLXunwHJX80mD+wFBV3pNp8c3enVdIFF6gCnxex4EufykyDxlc7k6dnCNZGhfO0bdlJV+E9WMJkCxAU2YqRJ+cZmwwly0jkUiMQ6Rq9SYU9Iz3JlT9TabsQ= Received: by 10.35.63.2 with SMTP id q2mr7870639pyk.1183470028335; Tue, 03 Jul 2007 06:40:28 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Tue, 3 Jul 2007 06:40:28 -0700 (PDT) Message-ID: Date: Tue, 3 Jul 2007 15:40:28 +0200 From: "Nicolas Cormier" To: freebsd-hackers@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2007 13:40:29 -0000 On 7/2/07, Nicolas Cormier wrote: > Hi, > I am trying to map some data allocated in kernel to a user process > (via a syscall). > I need the proc's vmspace, but the value of p_vmspace of the input > proc argument is NULL ... > How can I get a valid vmspace ? > > Thanks ! Ok, syscall function passed a proc* as arguments, I don't know where this proc* come from but it works with: struct thread *td = curthread; p = td->td_proc; -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 3 17:34:04 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0CAB916A400 for ; Tue, 3 Jul 2007 17:34:04 +0000 (UTC) (envelope-from rdivacky@vlk.vlakno.cz) Received: from vlakno.cz (vlk.vlakno.cz [62.168.28.247]) by mx1.freebsd.org (Postfix) with ESMTP id B7FD113C483 for ; Tue, 3 Jul 2007 17:34:00 +0000 (UTC) (envelope-from rdivacky@vlk.vlakno.cz) Received: from localhost (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 950308BF4AC; Tue, 3 Jul 2007 19:33:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at vlakno.cz Received: from vlakno.cz ([127.0.0.1]) by localhost (vlk.vlakno.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fSSsAAyMMtYX; Tue, 3 Jul 2007 19:33:58 +0200 (CEST) Received: from vlk.vlakno.cz (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 975B68BDAD6; Tue, 3 Jul 2007 19:33:58 +0200 (CEST) Received: (from rdivacky@localhost) by vlk.vlakno.cz (8.13.8/8.13.8/Submit) id l63HXwBK035167; Tue, 3 Jul 2007 19:33:58 +0200 (CEST) (envelope-from rdivacky) Date: Tue, 3 Jul 2007 19:33:58 +0200 From: Roman Divacky To: Nicolas Cormier Message-ID: <20070703173358.GA35125@freebsd.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2007 17:34:04 -0000 > Ok, syscall function passed a proc* as arguments, I don't know where this does not make any sense... userland processes have no way to determine where a proc is stored... what exactly are you trying to achieve? From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 3 23:10:53 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6715B16A41F for ; Tue, 3 Jul 2007 23:10:53 +0000 (UTC) (envelope-from stas@FreeBSD.org) Received: from com1.ht-systems.ru (com1.ht-systems.ru [83.97.104.204]) by mx1.freebsd.org (Postfix) with ESMTP id 1D7E413C44B for ; Tue, 3 Jul 2007 23:10:53 +0000 (UTC) (envelope-from stas@FreeBSD.org) Received: from [85.21.245.235] (helo=phonon.SpringDaemons.com) by com1.ht-systems.ru with esmtpa (Exim 4.62) (envelope-from ) id 1I5rWE-000257-PK; Wed, 04 Jul 2007 03:10:51 +0400 Received: from localhost (localhost [127.0.0.1]) by phonon.SpringDaemons.com (Postfix) with SMTP id 6B90711404; Wed, 4 Jul 2007 03:07:07 +0400 (MSD) Date: Wed, 4 Jul 2007 03:07:01 +0400 From: Stanislav Sedov To: Ivan Voras Message-Id: <20070704030701.24c100dd.stas@FreeBSD.org> In-Reply-To: References: <20070702234807.38be164a@vixen42> Organization: The FreeBSD Project X-Mailer: carrier-pigeon X-Voice: +7 916 849 20 23 X-XMPP: ssedov@jabber.ru X-ICQ: 208105021 X-Yahoo: stanislav_sedov X-PGP-Fingerprint: F21E D6CC 5626 9609 6CE2 A385 2BF5 5993 EB26 9581 X-University: MEPhI Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA1"; boundary="Signature=_Wed__4_Jul_2007_03_07_01_+0400_0lYcjfLbzf=j.vX1" X-Spam-Flag: SKIP X-Spam-Yversion: Spamooborona 1.6.0 Cc: freebsd-hackers@freebsd.org Subject: Re: where to start on kernel modules and NSS? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2007 23:10:53 -0000 --Signature=_Wed__4_Jul_2007_03_07_01_+0400_0lYcjfLbzf=j.vX1 Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, 03 Jul 2007 11:04:17 +0200 Ivan Voras mentioned: > Z.C.B. wrote: > > Have just been looking at sitting down and looking at writing a few > > things, but I am a bit lost on where to start. Any suggestions on > > things to read in regards to NSS and writing kernel modules? >=20 > These are not similar things and you don't need kernel modules for NSS.=20 > For kernel modules, you might try reading: >=20 > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/inde= x.html > http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/index.html > http://www.freebsd.org/doc/en_US.ISO8859-1/articles/geom-class/kernelprog= .html >=20 >=20 Also, this document can serve a very good introduction: http://caia.swin.edu.au/reports/070622A/CAIA-TR-070622A.pdf --=20 Stanislav Sedov ST4096-RIPE --Signature=_Wed__4_Jul_2007_03_07_01_+0400_0lYcjfLbzf=j.vX1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQFGitabK/VZk+smlYERAjA2AJ9gfJ2WU8Jjcqfe4fS118ShC6kg8gCdHDm8 WSRahFKzsW2zwi85J0xiBZs= =n2LL -----END PGP SIGNATURE----- --Signature=_Wed__4_Jul_2007_03_07_01_+0400_0lYcjfLbzf=j.vX1-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 07:01:34 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7FCD616A400; Wed, 4 Jul 2007 07:01:34 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe03.swip.net [212.247.154.65]) by mx1.freebsd.org (Postfix) with ESMTP id E87BD13C4DB; Wed, 4 Jul 2007 07:01:33 +0000 (UTC) (envelope-from hselasky@c2i.net) X-Cloudmark-Score: 0.000000 [] Received: from [194.248.135.20] (account mc467741@c2i.net HELO laptop.lan) by mailfe03.swip.net (CommuniGate Pro SMTP 5.1.10) with ESMTPA id 543776670; Wed, 04 Jul 2007 09:01:31 +0200 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org, freebsd-usb@freebsd.org Date: Wed, 4 Jul 2007 09:01:32 +0200 User-Agent: KMail/1.9.5 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707040901.33019.hselasky@c2i.net> Cc: Subject: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 07:01:34 -0000 Hi, I want to get rid of the copying between DMA'able memory and non-DMA'able memory. Currently I allocate N memory-pages for each USB transfer like separate pages. The bus-dma system then assigns all of these pages each their virtual address. What I see is that when I allocate more than PAGE_SIZE bytes using bus-dma, I get physically contiguous memory. I don't need that for the USB stack. The question is: Should we change bus-dma to support so called scatter and gather allocations, where the physical allocation is non-contiguous, and the virtual allocation is contiguous accross all the scattered pages ? Also: How is the easiest way to load memory pages into DMA ? And I want that the loadig works like this, that when the page must be bounced it should not allocate a bounce buffer, hence I already have a bounce buffer. I only need to know which pages I can forward directly to the USB hardware, and the rest I will bounce somewhere else. --HPS From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 08:48:04 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 38C5916A400 for ; Wed, 4 Jul 2007 08:48:04 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD6B13C457 for ; Wed, 4 Jul 2007 08:48:03 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 17B6D46B14; Wed, 4 Jul 2007 04:16:21 -0400 (EDT) Date: Wed, 4 Jul 2007 09:16:21 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Nicolas Cormier In-Reply-To: Message-ID: <20070704091349.T42421@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 08:48:04 -0000 On Mon, 2 Jul 2007, Nicolas Cormier wrote: > I am trying to map some data allocated in kernel to a user process (via a > syscall). I need the proc's vmspace, but the value of p_vmspace of the input > proc argument is NULL ... How can I get a valid vmspace ? When operating in a system call, the 'td' argument to the system call function is the current thread pointer. You can follow td->td_proc to get to the current process (and therefore, its address space). In general, I prefer mapping user pages into kernel instead of kernel pages into user space, as it reduces the chances of leakage of kernel data to user space, and there are some useful primitives for making this easier. For example, take a look at the sf_buf infrastructure used for things like socket zero-copy send, which manages a temporary kernel mapping for a page. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 09:00:56 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2A65F16A468 for ; Wed, 4 Jul 2007 09:00:56 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.freebsd.org (Postfix) with ESMTP id D499613C45B for ; Wed, 4 Jul 2007 09:00:55 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so4204460pyb for ; Wed, 04 Jul 2007 02:00:55 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=iyp3K0rGFrVcsVP+sgZqDG6D8vX+ztnZU5n94dYAV3/Jp8H8ViW3cTghQej/dKGLfB1BsYX41wkhwSr0mLs+AOxZUzR8kZ8NzzlmcWv2QpoMsAjwndnz/6QYU3gn7uY9ShbsZE2daaaSNdTIavxHkuIjQww5UmWjcKUmy/SbrWA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=L+Fq8GctS5b5yf0axr3I/zYtqXUf3VDZ09X1wSExSs4Jq8+b/G4YqJJMOUCORrJj9AgbXmMQHcnd0VwAiRdZy1ZDXGDuFbwj2ZNS6z2O8ANVRv/C/+9mNNHnwgIuP0oRarNZv8mJayTgo7qpk56Y6hT6Bu2tHpHghPbILN0Zb3c= Received: by 10.35.39.2 with SMTP id r2mr9387257pyj.1183539654590; Wed, 04 Jul 2007 02:00:54 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Wed, 4 Jul 2007 02:00:54 -0700 (PDT) Message-ID: Date: Wed, 4 Jul 2007 11:00:54 +0200 From: "Nicolas Cormier" To: "Robert Watson" In-Reply-To: <20070704091349.T42421@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070704091349.T42421@fledge.watson.org> Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 09:00:56 -0000 On 7/4/07, Robert Watson wrote: > > On Mon, 2 Jul 2007, Nicolas Cormier wrote: > > > I am trying to map some data allocated in kernel to a user process (via a > > syscall). I need the proc's vmspace, but the value of p_vmspace of the input > > proc argument is NULL ... How can I get a valid vmspace ? > > When operating in a system call, the 'td' argument to the system call function > is the current thread pointer. You can follow td->td_proc to get to the > current process (and therefore, its address space). In general, I prefer > mapping user pages into kernel instead of kernel pages into user space, as it > reduces the chances of leakage of kernel data to user space, and there are > some useful primitives for making this easier. For example, take a look at > the sf_buf infrastructure used for things like socket zero-copy send, which > manages a temporary kernel mapping for a page. > Yes Roman told me in private that I'm wrong with the first argument, I thought that it was a proc*... For my module I try to create a simple interface of a network allocator: User code should look like this: unsigned id; void* data = netmalloc(host, size, &id); memcpy(data, "toto", sizeof("toto"); netdetach(data); and later in another process: void* data = netattach(host, id); ... netfree(data); netmalloc syscall does something like that: - query distant host to allocate size - receive an id from distant host - malloc in kernel size - map the buffer to user process (*) netdetach syscall: - send data to distant host netattach syscall: - get data from host - malloc in kernel size - map the buffer to user process (*) * I already watch the function vm_pgmoveco (http://fxr.watson.org/fxr/source/kern/kern_subr.c?v=RELENG62#L78) I used pgmoveco as follow: vm_map_t mapa = &proc->p_vmspace->vm_map, size = round_page(size); void* data = malloc(size, M_NETMALLOC, M_WAITOK); vm_offset_t addr = vm_map_min(mapa); vm_map_find(mapa, NULL, 0, &addr, size, TRUE, VM_PROT_ALL, VM_PROT_ALL, MAP_NOFAULT); vm_pgmoveco(mapa, (vm_offset_t)data, addr); With this I have a panic with vm_page_insert, I am not sure to understand the reason of this panic. I can't have multiple virtual pages on the same physical page ? Thanks! -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 11:17:29 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0AD9016A421 for ; Wed, 4 Jul 2007 11:17:29 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2910D13C483 for ; Wed, 4 Jul 2007 11:17:28 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id F3FE94862E; Wed, 4 Jul 2007 07:17:26 -0400 (EDT) Date: Wed, 4 Jul 2007 12:17:26 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Nicolas Cormier In-Reply-To: Message-ID: <20070704120624.W37059@fledge.watson.org> References: <20070704091349.T42421@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 11:17:29 -0000 On Wed, 4 Jul 2007, Nicolas Cormier wrote: > On 7/4/07, Robert Watson wrote: >> >> On Mon, 2 Jul 2007, Nicolas Cormier wrote: >> >>> I am trying to map some data allocated in kernel to a user process (via a >>> syscall). I need the proc's vmspace, but the value of p_vmspace of the >>> input proc argument is NULL ... How can I get a valid vmspace ? >> >> When operating in a system call, the 'td' argument to the system call >> function is the current thread pointer. You can follow td->td_proc to get >> to the current process (and therefore, its address space). In general, I >> prefer mapping user pages into kernel instead of kernel pages into user >> space, as it reduces the chances of leakage of kernel data to user space, >> and there are some useful primitives for making this easier. For example, >> take a look at the sf_buf infrastructure used for things like socket >> zero-copy send, which manages a temporary kernel mapping for a page. > > Yes Roman told me in private that I'm wrong with the first argument, I > thought that it was a proc*... > > For my module I try to create a simple interface of a network allocator: > User code should look like this: > > unsigned id; > void* data = netmalloc(host, size, &id); > memcpy(data, "toto", sizeof("toto"); > netdetach(data); > > and later in another process: > void* data = netattach(host, id); > ... > netfree(data); > > netmalloc syscall does something like that: > - query distant host to allocate size > - receive an id from distant host > - malloc in kernel size > - map the buffer to user process (*) > > netdetach syscall: > - send data to distant host > > netattach syscall: > - get data from host > - malloc in kernel size > - map the buffer to user process (*) > > * I already watch the function vm_pgmoveco > (http://fxr.watson.org/fxr/source/kern/kern_subr.c?v=RELENG62#L78) > > I used pgmoveco as follow: > > vm_map_t mapa = &proc->p_vmspace->vm_map, > size = round_page(size); > void* data = malloc(size, M_NETMALLOC, M_WAITOK); > vm_offset_t addr = vm_map_min(mapa); > vm_map_find(mapa, NULL, 0, &addr, size, TRUE, VM_PROT_ALL, > VM_PROT_ALL, MAP_NOFAULT); > vm_pgmoveco(mapa, (vm_offset_t)data, addr); > > > With this I have a panic with vm_page_insert, I am not sure to understand > the reason of this panic. I can't have multiple virtual pages on the same > physical page ? I think part of what you're running into here is a conceptual issue. The pages allocated by malloc(9) belong to the kernel memory allocator, and are generally managed by the slab allocator. While in principle you can map them into user space, you're going to have to set up a lot of book-keeping to properly free them again later, etc. There are really two approaches you could be looking at: (1) The user app allocates memory pages, perhaps using mmap() to map anonymous memory or a file. You then borrow those pages to use in-kernel, mapping as required. (2) Your kernel code allocates pages directly from the VM system, possibly anonymous swap-backed pages from the page allocator, and maps them into the kernel as required. In either case, you'll need to think about address space limits, especially if the buffer is large -- the kernel address space on 32-bit systems is limited in size, since it shares the address space with a user application. On 64-bit systems, this is not an issue. You'll also need to make sure that the pages are both paged in and pinned in memory. So before we talk about the details of the calls, we should think about how you plan to use the memory. How much memory are we talking about -- enough to potentially run into kernel address space problems on 32-bit systems? How long will the mappings persist -- do you map them into kernel for a brief period to fill them, and then leave them mapped into user space, or is this going to be a persistent shared mapping over a very long period of time? Is the memory going to be pageable? How will it interact with things like mprotect(), msync(), etc? What should happen if a the pages are released by the process using munmap() or by mapping over the region with mmap()? What should happen in a child process if a process forks after netattach() and the parent calls netdatach()? What happens if the process calls send() using a source address in the memory region, and zero-copy sockets are enabled, which would normally lead the page to be "borrowed" from the user process? The underlying point here is that there is a model by which VM is managed -- pages, pagers, memory objects, mappings, address spaces, etc. We can't just talk about pages being shared or mapped, we need to think about what is to be accomplished, and how to map that into the abstractions that already exist. Memory comes in different flavours, and generally speaking, you don't want to use pages that come from malloc(9) for sharing with userspace, so we need to think about what kind of memory you do need. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 13:04:21 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0553616A421 for ; Wed, 4 Jul 2007 13:04:21 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9076013C46C for ; Wed, 4 Jul 2007 13:04:20 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so4306790pyb for ; Wed, 04 Jul 2007 06:04:19 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=ebYOh5XTfyTvfSnemj5h+ypTXSh7+xYSRLtGWOW5b3KbikcfuTPvs/S9FZAWNQg3oe0UAlJSOhrZHvKmRrRB3ImrHlWoaYzZaOuUKDpr5cRDXArQXjsawn7Q9hgtGTiskE94DRH7+fI33cDDgD2SDFJSjid4nX5AhNv+kcWZSVc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=uQXlnvyyHaH/EfXnth3DWtqEuFdbjP8KL3n6nbVxnd0fB1b/PHWM03fuduPw5BYI/trK9FgdmfASliUvHDVAWSZcyDm/3E0uJgWRV46OSMEVzRnaZh8fpzL2ftYWKT5O4D8weRrgqbElWrMdwsa8zjpLILJ6BQ3bzX7kDwD0PiU= Received: by 10.35.110.13 with SMTP id n13mr9710842pym.1183554259677; Wed, 04 Jul 2007 06:04:19 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Wed, 4 Jul 2007 06:04:19 -0700 (PDT) Message-ID: Date: Wed, 4 Jul 2007 15:04:19 +0200 From: "Nicolas Cormier" To: "Robert Watson" In-Reply-To: <20070704120624.W37059@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070704091349.T42421@fledge.watson.org> <20070704120624.W37059@fledge.watson.org> Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 13:04:21 -0000 On 7/4/07, Robert Watson wrote: > How much memory are we talking about -- enough to potentially run into kernel > address space problems on 32-bit systems? How long will the mappings persist > -- do you map them into kernel for a brief period to fill them, and then leave > them mapped into user space, or is this going to be a persistent shared > mapping over a very long period of time? Is the memory going to be pageable? > How will it interact with things like mprotect(), msync(), etc? What should > happen if a the pages are released by the process using munmap() or by mapping > over the region with mmap()? What should happen in a child process if a > process forks after netattach() and the parent calls netdatach()? What > happens if the process calls send() using a source address in the memory > region, and zero-copy sockets are enabled, which would normally lead the page > to be "borrowed" from the user process? Currently I'm just trying to play with kernel/modules/vm ... I'm a newbie in kernel development and I just want to make a little prototype of an in-kernel network allocator. To start I only need to map a page (1024 bytes) from kernel to user process. This memory will never be used by the kernel between the call of net(malloc/attach) and the call of net(detach/free). So user and kernel will never use this page at the same time. > The underlying point here is that there is a model by which VM is managed -- > pages, pagers, memory objects, mappings, address spaces, etc. We can't just > talk about pages being shared or mapped, we need to think about what is to be > accomplished, and how to map that into the abstractions that already exist. > Memory comes in different flavours, and generally speaking, you don't want to > use pages that come from malloc(9) for sharing with userspace, so we need to > think about what kind of memory you do need. Thank you for your answer. Right now, I just want to do it as easily as possible, I don't know if this kind of project could interest other persons ? It is ok for me to work more on it later on, if there is any further interest in doing it. -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 13:19:48 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 938A416A421 for ; Wed, 4 Jul 2007 13:19:48 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0856413C469 for ; Wed, 4 Jul 2007 13:19:46 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 150BB4771B; Wed, 4 Jul 2007 09:19:45 -0400 (EDT) Date: Wed, 4 Jul 2007 14:19:45 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Nicolas Cormier In-Reply-To: Message-ID: <20070704141257.M37059@fledge.watson.org> References: <20070704091349.T42421@fledge.watson.org> <20070704120624.W37059@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 13:19:48 -0000 On Wed, 4 Jul 2007, Nicolas Cormier wrote: > Currently I'm just trying to play with kernel/modules/vm ... I'm a newbie in > kernel development and I just want to make a little prototype of an > in-kernel network allocator. To start I only need to map a page (1024 bytes) > from kernel to user process. This memory will never be used by the kernel > between the call of net(malloc/attach) and the call of net(detach/free). So > user and kernel will never use this page at the same time. > >> The underlying point here is that there is a model by which VM is managed >> -- pages, pagers, memory objects, mappings, address spaces, etc. We can't >> just talk about pages being shared or mapped, we need to think about what >> is to be accomplished, and how to map that into the abstractions that >> already exist. Memory comes in different flavours, and generally speaking, >> you don't want to use pages that come from malloc(9) for sharing with >> userspace, so we need to think about what kind of memory you do need. > > Thank you for your answer. Right now, I just want to do it as easily as > possible, I don't know if this kind of project could interest other persons > ? It is ok for me to work more on it later on, if there is any further > interest in doing it. What do you mean by a network allocator? How do you plan to use these pages? If you haven't already, you should look at the zero-copy socket code in uipc_cow.c. The main criticism of this approach has been that it uses copy-on-write, leading to potential IPIs for VM shootdowns, etc. An alternative, more along the lines of IO-Lite, would be to allow user space to explicitly abandon the page on send, then map a new page to replace it. In which case you might consider a variation on the send system call that accepts only page-aligned arguments and has the effect of unmapping the pages that are sent. In neither case, on the transmit side, does this require an modification to the kernel memory allocator. The receive side has always been more tricky to deal with... Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 13:48:40 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CB1CF16A46F for ; Wed, 4 Jul 2007 13:48:40 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.179]) by mx1.freebsd.org (Postfix) with ESMTP id 8682813C4BF for ; Wed, 4 Jul 2007 13:48:40 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so4328962pyb for ; Wed, 04 Jul 2007 06:48:39 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=mqT6jjIOfnhCkG3g60FnOoNdYDmRvb2OXYti8nYfXhdSlolefHqPLOKECJNF/kbjS8glnbeyFVhQsN5eJQORe5f1yYiYg8ntLp0jICxu2MngIfh/6pU0Ds5cJI60Ge+p1A5x6NjM76Gn9BRDuYisxJDVAMGDAjRe7WfJM+sFNOg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=kezuUouqvqjuFiP/nVju5OEo84Qe9XRX0075IhOtZepz4laVa1n9TL0+MkfY99vKqXZbTHCZOGB1DTxuhCJbJpnruxOo9u4UrqKdNm46J0MDp/AgE/SjYHWsZzidkdeobrWOENLtk4jBwsXCt2i2Oa7IRszMZugYJpdnC/LjM88= Received: by 10.35.101.1 with SMTP id d1mr9781390pym.1183556919450; Wed, 04 Jul 2007 06:48:39 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Wed, 4 Jul 2007 06:48:39 -0700 (PDT) Message-ID: Date: Wed, 4 Jul 2007 15:48:39 +0200 From: "Nicolas Cormier" To: "Robert Watson" In-Reply-To: <20070704141257.M37059@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070704091349.T42421@fledge.watson.org> <20070704120624.W37059@fledge.watson.org> <20070704141257.M37059@fledge.watson.org> Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 13:48:40 -0000 On 7/4/07, Robert Watson wrote: > What do you mean by a network allocator? How do you plan to use these pages? First I just want to access a local copy of a distant buffer. After the goal is to share memory between hosts (no concurrent access). > If you haven't already, you should look at the zero-copy socket code in > uipc_cow.c. The main criticism of this approach has been that it uses > copy-on-write, leading to potential IPIs for VM shootdowns, etc. An > alternative, more along the lines of IO-Lite, would be to allow user space to > explicitly abandon the page on send, then map a new page to replace it. In > which case you might consider a variation on the send system call that accepts > only page-aligned arguments and has the effect of unmapping the pages that are > sent. In neither case, on the transmit side, does this require an > modification to the kernel memory allocator. > > The receive side has always been more tricky to deal with... > Ok I will take a look at uipc_cow.c, Thank you -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 15:14:35 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0873A16A46C for ; Wed, 4 Jul 2007 15:14:35 +0000 (UTC) (envelope-from david.chosrova@libertysurf.fr) Received: from mail.libertysurf.net (webmail-out.libertysurf.net [213.36.80.105]) by mx1.freebsd.org (Postfix) with ESMTP id C207513C468 for ; Wed, 4 Jul 2007 15:14:34 +0000 (UTC) (envelope-from david.chosrova@libertysurf.fr) Received: from aliceadsl.fr (192.168.10.37) by mail.libertysurf.net (7.1.026) id 465DAC98000A4C11 for freebsd-hackers@freebsd.org; Wed, 4 Jul 2007 17:04:26 +0200 Date: Wed, 4 Jul 2007 17:04:26 +0200 Message-Id: MIME-Version: 1.0 X-Sensitivity: 3 Content-Type: multipart/mixed; boundary="_=__=_XaM3_.1183561466.2A.827234.42.26958.52.42.007.3152" From: "=?iso-8859-1?Q?david.chosrova@libertysurf.fr?=" To: "=?iso-8859-1?Q?freebsd-hackers?=" X-XaM3-API-Version: 3.2 R18 (B34 pl1) X-type: 0 X-SenderIP: 217.128.232.195 Subject: kernel dynamic references X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 15:14:35 -0000 --_=__=_XaM3_.1183561466.2A.827234.42.26958.52.42.007.3152 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable =0D=0AHi,=0D=0A=0D=0A I'm new to Freebsd and interested in system program= ming. So I'have picked up a task from the project ideas list to start wit= h.=0D=0A (part of) the subject is : "This task is to define and implement= a general mechanism for tracking these references and use them in handli= ng module unload requests."=0D=0A=0D=0ASo, to do that, I'have added an "i= nt dynrefs" in struct module (kern_module.c), and functions to increase o= r decrease this count (module_add/remove_dynrefs(const char * modname) an= d module_updatedynrefs(const char * modname, int action) ) in kern_module= .c.=0D=0A=0D=0A To avoid unload of a module which has a dynrefs count !=3D= 0 , I have modified module_unload(), so that unload is process only if = dynrefs=3D0 or flag=3DLINKER_UNLOAD_FORCE.=0D=0A=0D=0A module_unload(modu= le_t mod, int flags)=0D=0A {=0D=0A int error;=0D=0A- error =3D MOD_EVENT= (mod, MOD_QUIESCE);=0D=0A+ MOD_SLOCK;=0D=0A+ (mod->dynrefs =3D=3D 0) ? (e= rror =3D MOD_EVENT(mod, MOD_QUIESCE)) : (error =3D EPERM);=0D=0A+ MOD_SUN= LOCK;=0D=0A=0D=0A=0D=0A=0D=0A I have compiled and tested. with a 6-2 RELE= ASE. For the test I'have used two dummy module, one adding a dynrefs on t= he other.=0D=0A=0D=0A Any comment are welcome=0D=0A=0D=0A David chosrova = =0D=0A=0D=0A=0D=0A=0D=0A=0A=0A------------------------ ALICE C'EST ENCORE= MIEUX AVEC CANAL+ LE BOUQUET ! ---------------=0AD=E9couvrez vite l'offr= e exclusive ALICEBOX et CANAL+ LE BOUQUET, en cliquant ici http://alicebo= x.fr=0ASoumis =E0 conditions.=0A --_=__=_XaM3_.1183561466.2A.827234.42.26958.52.42.007.3152 Content-Type: text/plain;name="module.diff"; name="module.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="module.diff" LS0tIG1vZHVsZS5oLm9yaWcJV2VkIEp1bCAgNCAxMDoyOTo1MyAyMDA3CisrKyBtb2R1bGUu aAlXZWQgSnVsICA0IDEwOjIxOjQwIDIwMDcKQEAgLTE0Nyw3ICsxNDcsOCBAQAogaW50CW1v ZHVsZV9nZXRpZChtb2R1bGVfdCk7CiBtb2R1bGVfdAltb2R1bGVfZ2V0Zm5leHQobW9kdWxl X3QpOwogdm9pZAltb2R1bGVfc2V0c3BlY2lmaWMobW9kdWxlX3QsIG1vZHNwZWNpZmljX3Qg Kik7Ci0KK2ludAltb2R1bGVfYWRkX2R5bnJlZnMoY29uc3QgY2hhciAqKTsKK2ludAltb2R1 bGVfcmVtb3ZlX2R5bnJlZnMoY29uc3QgY2hhciAqKTsKIAogI2lmZGVmCU1PRF9ERUJVRwog ZXh0ZXJuIGludCBtb2RfZGVidWc7Cg== --_=__=_XaM3_.1183561466.2A.827234.42.26958.52.42.007.3152 Content-Type: text/plain;name="kern_module.diff"; name="kern_module.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="kern_module.diff" LS0tIGtlcm5fbW9kdWxlLmMub3JpZwlXZWQgSnVsICA0IDEwOjMzOjA2IDIwMDcKKysrIGtl cm5fbW9kdWxlLmMJV2VkIEp1bCAgNCAxMDoyMToyOCAyMDA3CkBAIC01Miw2ICs1Miw3IEBA CiAJVEFJTFFfRU5UUlkobW9kdWxlKQlmbGluazsJLyogYWxsIG1vZHVsZXMgaW4gYSBmaWxl ICovCiAJc3RydWN0IGxpbmtlcl9maWxlCSpmaWxlOwkvKiBmaWxlIHdoaWNoIGNvbnRhaW5z IHRoaXMgbW9kdWxlICovCiAJaW50CQkJcmVmczsJLyogcmVmZXJlbmNlIGNvdW50ICovCisJ aW50CQkJZHlucmVmczsgICAgICAgIC8qIGR5bmFtaWMgcmVmZXJlbmNlIGNvdW50ICovCiAJ aW50IAkJCWlkOwkvKiB1bmlxdWUgaWQgbnVtYmVyICovCiAJY2hhciAJCQkqbmFtZTsJLyog bW9kdWxlIG5hbWUgKi8KIAltb2RldmVudGhhbmRfdCAJCWhhbmRsZXI7CS8qIGV2ZW50IGhh bmRsZXIgKi8KQEAgLTY1LDYgKzY2LDcgQEAKIHN0cnVjdCBzeCBtb2R1bGVzX3N4Owogc3Rh dGljIGludCBuZXh0aWQgPSAxOwogc3RhdGljIHZvaWQgbW9kdWxlX3NodXRkb3duKHZvaWQg KiwgaW50KTsKK3N0YXRpYyBpbnQgbW9kdWxlX3VwZGF0ZWR5bnJlZnMoY29uc3QgY2hhciog LCBpbnQgKTsKIAogc3RhdGljIGludAogbW9kZXZlbnRfbm9wKG1vZHVsZV90IG1vZCwgaW50 IHdoYXQsIHZvaWQgKmFyZykKQEAgLTE1Miw2ICsxNTQsNyBAQAogCX0KIAluZXdtb2QtPnJl ZnMgPSAxOwogCW5ld21vZC0+aWQgPSBuZXh0aWQrKzsKKwluZXdtb2QtPmR5bnJlZnMgPSAw OwogCW5ld21vZC0+bmFtZSA9IChjaGFyICopKG5ld21vZCArIDEpOwogCXN0cmNweShuZXdt b2QtPm5hbWUsIGRhdGEtPm5hbWUpOwogCW5ld21vZC0+aGFuZGxlciA9IGRhdGEtPmV2aGFu ZCA/IGRhdGEtPmV2aGFuZCA6IG1vZGV2ZW50X25vcDsKQEAgLTIzMSw3ICsyMzQsOSBAQAog bW9kdWxlX3VubG9hZChtb2R1bGVfdCBtb2QsIGludCBmbGFncykKIHsKIAlpbnQgZXJyb3I7 Ci0JZXJyb3IgPSBNT0RfRVZFTlQobW9kLCBNT0RfUVVJRVNDRSk7CisJTU9EX1NMT0NLOwor CShtb2QtPmR5bnJlZnMgPT0gMCkgPyAoZXJyb3IgPSBNT0RfRVZFTlQobW9kLCBNT0RfUVVJ RVNDRSkpIDogKGVycm9yID0gRVBFUk0pOworCU1PRF9TVU5MT0NLOwogCWlmIChlcnJvciA9 PSBFT1BOT1RTVVBQIHx8IGVycm9yID09IEVJTlZBTCkKIAkJZXJyb3IgPSAwOwogCWlmIChm bGFncyA9PSBMSU5LRVJfVU5MT0FEX05PUk1BTCAmJiBlcnJvciAhPSAwKQpAQCAtMjYxLDYg KzI2NiwzNyBAQAogCiAJTU9EX1hMT0NLX0FTU0VSVDsKIAltb2QtPmRhdGEgPSAqZGF0YXA7 Cit9CisKK3N0YXRpYyBpbnQKK21vZHVsZV91cGRhdGVkeW5yZWZzKGNvbnN0IGNoYXIqIG1v ZG5hbWUsIGludCBhY3Rpb24pCit7CisJbW9kdWxlX3QgbW9kOworCQorCU1PRF9YTE9DSzsK Kwltb2QgPSBtb2R1bGVfbG9va3VwYnluYW1lKG1vZG5hbWUpOworCisJaWYobW9kID09IDAp IHsKKwkJTU9EX1hVTkxPQ0s7CisJCXJldHVybigtMSk7CisJfQorCisJKGFjdGlvbiA9PSAx KSA/IG1vZC0+ZHlucmVmcysrIDogbW9kLT5keW5yZWZzLS07CisJTU9EX1hVTkxPQ0s7CisJ cmV0dXJuICgwKTsKK30KKworaW50Cittb2R1bGVfYWRkX2R5bnJlZnMoY29uc3QgY2hhciAq bW9kbmFtZSkKK3sKKworCXJldHVybiAobW9kdWxlX3VwZGF0ZWR5bnJlZnMobW9kbmFtZSwx KSk7Cit9CisKK2ludAorbW9kdWxlX3JlbW92ZV9keW5yZWZzKGNvbnN0IGNoYXIgKiBtb2Ru YW1lKQoreworCXJldHVybiAobW9kdWxlX3VwZGF0ZWR5bnJlZnMobW9kbmFtZSwwKSk7CiB9 CiAKIC8qCQo= --_=__=_XaM3_.1183561466.2A.827234.42.26958.52.42.007.3152-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 15:41:35 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BEDF116A469 for ; Wed, 4 Jul 2007 15:41:35 +0000 (UTC) (envelope-from david.chosrova@libertysurf.fr) Received: from mail.libertysurf.net (webmail-out.libertysurf.net [213.36.80.105]) by mx1.freebsd.org (Postfix) with ESMTP id 805A713C46E for ; Wed, 4 Jul 2007 15:41:35 +0000 (UTC) (envelope-from david.chosrova@libertysurf.fr) Received: from aliceadsl.fr (192.168.10.37) by mail.libertysurf.net (7.1.026) id 465DAC98000A4DE6 for freebsd-hackers@freebsd.org; Wed, 4 Jul 2007 17:41:34 +0200 Date: Wed, 4 Jul 2007 17:41:34 +0200 Message-Id: MIME-Version: 1.0 X-Sensitivity: 3 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable From: "=?iso-8859-1?Q?david.chosrova@libertysurf.fr?=" To: "=?iso-8859-1?Q?freebsd-hackers?=" X-XaM3-API-Version: 3.2 R18 (B34 pl1) X-type: 0 X-SenderIP: 217.128.232.195 Subject: kernel dynamic references X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 15:41:35 -0000 Hi,=0D=0A=0D=0A I'm new to Freebsd and interested in system programming. = So I'have picked up a task from the project ideas list to start with.=0D=0A= (part of) the subject is : "This task is to define and implement a gener= al mechanism for tracking these references and use them in handling modul= e unload requests."=0D=0A=0D=0ASo, to do that, I'have added an "int dynre= fs" in struct module (kern_module.c), and functions to increase or decrea= se this count (module_add/remove_dynrefs(const char * modname) and module= _updatedynrefs(const char * modname, int action) ) in kern_module.c.=0D=0A= =0D=0A=0D=0A To avoid unload of a module which has a dynrefs count !=3D 0= , I have modified module_unload(), so that unload is process only if dy= nrefs=3D0 or flag=3DLINKER_UNLOAD_FORCE.=0D=0A=0D=0A=0D=0A=0D=0A--- modul= e.h.orig Wed Jul 4 10:29:53 2007=0D=0A+++ module.h Wed Jul 4 10:21:40 2= 007=0D=0A@@ -147,7 +147,8 @@=0D=0A int module_getid(module_t);=0D=0A modu= le_t module_getfnext(module_t);=0D=0A void module_setspecific(module_t, m= odspecific_t *);=0D=0A-=0D=0A+int module_add_dynrefs(const char *);=0D=0A= +int module_remove_dynrefs(const char *);=0D=0A =0D=0A #ifdef MOD_DEBUG=0D= =0A extern int mod_debug;=0D=0A=0D=0A--- kern_module.c.orig Wed Jul 4 10= :33:06 2007=0D=0A+++ kern_module.c Wed Jul 4 10:21:28 2007=0D=0A@@ -52,6= +52,7 @@=0D=0A TAILQ_ENTRY(module) flink; /* all modules in a file */=0D= =0A struct linker_file *file; /* file which contains this module */=0D=0A= int refs; /* reference count */=0D=0A+ int dynrefs; /* dynam= ic reference count */=0D=0A int id; /* unique id number */=0D=0A cha= r *name; /* module name */=0D=0A modeventhand_t handler; /* event h= andler */=0D=0A@@ -65,6 +66,7 @@=0D=0A struct sx modules_sx;=0D=0A static= int nextid =3D 1;=0D=0A static void module_shutdown(void *, int);=0D=0A+= static int module_updatedynrefs(const char* , int );=0D=0A =0D=0A static = int=0D=0A modevent_nop(module_t mod, int what, void *arg)=0D=0A@@ -152,6 = +154,7 @@=0D=0A }=0D=0A newmod->refs =3D 1;=0D=0A newmod->id =3D nexti= d++;=0D=0A+ newmod->dynrefs =3D 0;=0D=0A newmod->name =3D (char *)(newmo= d + 1);=0D=0A strcpy(newmod->name, data->name);=0D=0A newmod->handler =3D= data->evhand ? data->evhand : modevent_nop;=0D=0A@@ -231,7 +234,9 @@=0D=0A= module_unload(module_t mod, int flags)=0D=0A {=0D=0A int error;=0D=0A- = error =3D MOD_EVENT(mod, MOD_QUIESCE);=0D=0A+ MOD_SLOCK;=0D=0A+ (mod->dyn= refs =3D=3D 0) ? (error =3D MOD_EVENT(mod, MOD_QUIESCE)) : (error =3D EPE= RM);=0D=0A+ MOD_SUNLOCK;=0D=0A if (error =3D=3D EOPNOTSUPP || error =3D=3D= EINVAL)=0D=0A error =3D 0;=0D=0A if (flags =3D=3D LINKER_UNLOAD_NORMA= L && error !=3D 0)=0D=0A@@ -261,6 +266,37 @@=0D=0A =0D=0A MOD_XLOCK_ASSE= RT;=0D=0A mod->data =3D *datap;=0D=0A+}=0D=0A+=0D=0A+static int=0D=0A+mo= dule_updatedynrefs(const char* modname, int action)=0D=0A+{=0D=0A+ module= _t mod;=0D=0A+ =0D=0A+ MOD_XLOCK;=0D=0A+ mod =3D module_lookupbyname(modn= ame);=0D=0A+=0D=0A+ if(mod =3D=3D 0) {=0D=0A+ MOD_XUNLOCK;=0D=0A+ retur= n(-1);=0D=0A+ }=0D=0A+=0D=0A+ (action =3D=3D 1) ? mod->dynrefs++ : mod->d= ynrefs--;=0D=0A+ MOD_XUNLOCK;=0D=0A+ return (0);=0D=0A+}=0D=0A+=0D=0A+int= =0D=0A+module_add_dynrefs(const char *modname)=0D=0A+{=0D=0A+=0D=0A+ retu= rn (module_updatedynrefs(modname,1));=0D=0A+}=0D=0A+=0D=0A+int=0D=0A+modu= le_remove_dynrefs(const char * modname)=0D=0A+{=0D=0A+ return (module_upd= atedynrefs(modname,0));=0D=0A }=0D=0A =0D=0A /* =0D=0A=0D=0A=0D=0A=0D=0A=0D= =0A I have compiled and tested. with a 6-2 RELEASE. For the test I'have u= sed two dummy module, one adding a dynrefs on the other.=0D=0A=0D=0A Any = comments are welcome.=0D=0A=0D=0A David chosrova =0D=0A=0D=0A=0D=0A=0D=0A= =0A=0A------------------------ ALICE C'EST ENCORE MIEUX AVEC CANAL+ LE BO= UQUET ! ---------------=0AD=E9couvrez vite l'offre exclusive ALICEBOX et = CANAL+ LE BOUQUET, en cliquant ici http://alicebox.fr=0ASoumis =E0 condit= ions.=0A From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 17:35:45 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6391716A468 for ; Wed, 4 Jul 2007 17:35:45 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id E7F5913C44B for ; Wed, 4 Jul 2007 17:35:44 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (anaa50glgw5srse9@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id l64HZWu9077430; Wed, 4 Jul 2007 10:35:32 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id l64HZWaG077429; Wed, 4 Jul 2007 10:35:32 -0700 (PDT) (envelope-from jmg) Date: Wed, 4 Jul 2007 10:35:31 -0700 From: John-Mark Gurney To: Hans Petter Selasky Message-ID: <20070704173531.GO1221@funkthat.com> Mail-Followup-To: Hans Petter Selasky , freebsd-hackers@freebsd.org, freebsd-usb@freebsd.org References: <200707040901.33019.hselasky@c2i.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200707040901.33019.hselasky@c2i.net> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Cc: freebsd-hackers@freebsd.org, freebsd-usb@freebsd.org Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 17:35:45 -0000 Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > I want to get rid of the copying between DMA'able memory and non-DMA'able > memory. > > Currently I allocate N memory-pages for each USB transfer like separate pages. How do you allocate these pages? With malloc(9) or w/ bus_dmamem_alloc(9)? > The bus-dma system then assigns all of these pages each their virtual > address. > > What I see is that when I allocate more than PAGE_SIZE bytes using bus-dma, I > get physically contiguous memory. I don't need that for the USB stack. That is a limitation of how bus_dma currently allocates memory... It calls contigmalloc, which doesn't handle multi-segment memory allocations yet.. > The question is: > > Should we change bus-dma to support so called scatter and gather allocations, > where the physical allocation is non-contiguous, and the virtual allocation > is contiguous accross all the scattered pages ? You can already support this by using malloc, and loading that buffer into your bus_dma map... or using the passing in userland buffer, and loading that into the map... > Also: How is the easiest way to load memory pages into DMA ? And I want that > the loadig works like this, that when the page must be bounced it should not > allocate a bounce buffer, hence I already have a bounce buffer. I only need > to know which pages I can forward directly to the USB hardware, and the rest > I will bounce somewhere else. Why do you not want to let bus_dma do the bouncing for you? If it's to save a copy to another buffer, why don't you load the final buffer into bus_dma? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 20:00:50 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0B31716A400 for ; Wed, 4 Jul 2007 20:00:50 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.freebsd.org (Postfix) with ESMTP id BD91F13C44B for ; Wed, 4 Jul 2007 20:00:49 +0000 (UTC) (envelope-from n.cormier@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so4499641pyb for ; Wed, 04 Jul 2007 13:00:49 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=lEpqp9mc/4oATJyBx8tGuhAhoPNM28dzblI1vkNubm2h7CcQEyScps/FSEb3PkzHlturRBE8qQvV+/9TU0oNsMVNfKWrSQSfzvbLBuTHHeTv9VyNRFcQMwdsbtVJrjassBB94yjJTwed2UPQ5AsoPn3BrX7KiMrG8ajhyVjOkcQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=BJ6imrBmVa0m8mewGxQLCi/9sfzhbHebRAKKfTdWt68F+RU7U7kd4Bsw4iE5ShsJDHModAf4omzRyTRNNyvz8TOVIVtjLQaXmQ5Sq2o0LFtuBLvIOpq/0FDnbfdpf7nxkcXFLTkHlDxq8j3J09IVCPM/5CPP4N4xvK6lDbMagGo= Received: by 10.35.101.1 with SMTP id d1mr10300085pym.1183579249049; Wed, 04 Jul 2007 13:00:49 -0700 (PDT) Received: by 10.35.40.11 with HTTP; Wed, 4 Jul 2007 13:00:48 -0700 (PDT) Message-ID: Date: Wed, 4 Jul 2007 22:00:48 +0200 From: "Nicolas Cormier" To: "Steve Watt" In-Reply-To: <200707041949.l64JntMv013476@wattres.watt.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070704091349.T42421@fledge.watson.org> <200707041949.l64JntMv013476@wattres.watt.com> Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 20:00:50 -0000 On 7/4/07, Steve Watt wrote: > In , > Nicolas Cormier wrote: > >On 7/4/07, Robert Watson wrote: > >> > >> When operating in a system call, the 'td' argument to the system call function > >> is the current thread pointer. You can follow td->td_proc to get to the > >> current process (and therefore, its address space). In general, I prefer > >> mapping user pages into kernel instead of kernel pages into user space, as it > >> reduces the chances of leakage of kernel data to user space, and there are > >> some useful primitives for making this easier. For example, take a look at > >> the sf_buf infrastructure used for things like socket zero-copy send, which > >> manages a temporary kernel mapping for a page. > >> > > > >netmalloc syscall does something like that: > >- query distant host to allocate size > >- receive an id from distant host > >- malloc in kernel size > >- map the buffer to user process (*) > > > >netdetach syscall: > >- send data to distant host > > > >netattach syscall: > >- get data from host > >- malloc in kernel size > >- map the buffer to user process (*) > > What this really sounds like is network shared memory or remote DMA. > I would architect this to remove as much of the management code as > possible from the kernel (i.e. query the distant host, get ID, etc.) > into a userland daemon. Depending on the exact semantics you want, > you'll probably need to write a new kind of pager. > > Basically, at the netmalloc call, you would simply pass the reqest > back to the userland daemon, which would format it in whatever way is > needed to cross the net, send the request off, receive the ID, and > give association information back to the kernel (number of pages, > protections, whatever). Then the call would map the new pages into > the userland process just like it was a shared memory segment. > > At detach time, the message would again go to the userland daemon, > which would map the pages locally and probably use a zero-copy send > to ship the data to the remote host. > > There are some fun potential interactions in there in code I haven't > looked at in a long time. I'll resist the urge to dive in and hack > something together, since VM systems have a way of being tricky in > unexpected places. Thank you for this post ! Your design should be a good start. -- Nicolas Cormier From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 20:00:59 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F37716A421 for ; Wed, 4 Jul 2007 20:00:59 +0000 (UTC) (envelope-from steve@Watt.COM) Received: from wattres.watt.com (wattres.watt.com [66.93.133.130]) by mx1.freebsd.org (Postfix) with ESMTP id 036A113C455 for ; Wed, 4 Jul 2007 20:00:58 +0000 (UTC) (envelope-from steve@Watt.COM) Received: from wattres.watt.com (localhost.watt.com [127.0.0.1]) by wattres.watt.com (8.13.8/8.13.8) with ESMTP id l64JntwF013477; Wed, 4 Jul 2007 12:50:00 -0700 (PDT) (envelope-from steve@wattres.watt.com) Received: (from steve@localhost) by wattres.watt.com (8.13.8/8.13.8/Submit) id l64JntMv013476; Wed, 4 Jul 2007 12:49:55 -0700 (PDT) (envelope-from steve) Message-Id: <200707041949.l64JntMv013476@wattres.watt.com> X-Newsgroups: local.freebsd-hackers In-Reply-To: From: steve@Watt.COM (Steve Watt) References: <20070704091349.T42421@fledge.watson.org> Organization: Watt Consultants, San Jose, CA, USA Date: Wed, 4 Jul 2007 12:49:54 -0700 X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: n.cormier@gmail.com X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (wattres.watt.com [127.0.0.1]); Wed, 04 Jul 2007 12:50:00 -0700 (PDT) X-Archived: 1183578600.225914010@wattres.Watt.COM Cc: freebsd-hackers@freebsd.org Subject: Re: p_vmspace in syscall X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 20:00:59 -0000 In , Nicolas Cormier wrote: >On 7/4/07, Robert Watson wrote: >> >> When operating in a system call, the 'td' argument to the system call function >> is the current thread pointer. You can follow td->td_proc to get to the >> current process (and therefore, its address space). In general, I prefer >> mapping user pages into kernel instead of kernel pages into user space, as it >> reduces the chances of leakage of kernel data to user space, and there are >> some useful primitives for making this easier. For example, take a look at >> the sf_buf infrastructure used for things like socket zero-copy send, which >> manages a temporary kernel mapping for a page. >> > >netmalloc syscall does something like that: >- query distant host to allocate size >- receive an id from distant host >- malloc in kernel size >- map the buffer to user process (*) > >netdetach syscall: >- send data to distant host > >netattach syscall: >- get data from host >- malloc in kernel size >- map the buffer to user process (*) What this really sounds like is network shared memory or remote DMA. I would architect this to remove as much of the management code as possible from the kernel (i.e. query the distant host, get ID, etc.) into a userland daemon. Depending on the exact semantics you want, you'll probably need to write a new kind of pager. Basically, at the netmalloc call, you would simply pass the reqest back to the userland daemon, which would format it in whatever way is needed to cross the net, send the request off, receive the ID, and give association information back to the kernel (number of pages, protections, whatever). Then the call would map the new pages into the userland process just like it was a shared memory segment. At detach time, the message would again go to the userland daemon, which would map the pages locally and probably use a zero-copy send to ship the data to the remote host. There are some fun potential interactions in there in code I haven't looked at in a long time. I'll resist the urge to dive in and hack something together, since VM systems have a way of being tricky in unexpected places. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.5" / 37N 20' 15.3" Internet: steve @ Watt.COM Whois: SW32-ARIN Free time? There's no such thing. It just comes in varying prices... From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 4 20:11:40 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 83DC216A41F for ; Wed, 4 Jul 2007 20:11:40 +0000 (UTC) (envelope-from almarrie@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.249]) by mx1.freebsd.org (Postfix) with ESMTP id 3DDE213C483 for ; Wed, 4 Jul 2007 20:11:39 +0000 (UTC) (envelope-from almarrie@gmail.com) Received: by an-out-0708.google.com with SMTP id c14so540838anc for ; Wed, 04 Jul 2007 13:11:39 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=FJOGal+netgtkOEhsXfZzgYspojvDcpDQVeUCUVol2w/2/zKP0bZsmaPp6BUMeiZgA0WM+tnF1gQDAdZSuvp+UhlRdCmzPSJuAvGTfRSVq8nIyumr96yzn0+JO1jwFM1tvhC78jtnvSORCkEoOtzOH0bspoF2oKldUWjHJaY3JE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=mxylAAXH2YNONWL6p9qmi1dqmAc+HWV8YwGFnz0xWKTFbAstxKqsxW8SJL9YTRoRmR3fZstFX3hMaqheDIbMDxyDLqqcxJg4vGI4EnftniLARB+bRm+yP/G/hB2sHD/KDa53djJvCbM4i4W+2VUCmyk3Ni14iiPi9xYJu/EkVuw= Received: by 10.100.173.19 with SMTP id v19mr4999942ane.1183579898865; Wed, 04 Jul 2007 13:11:38 -0700 (PDT) Received: by 10.100.9.14 with HTTP; Wed, 4 Jul 2007 13:11:38 -0700 (PDT) Message-ID: <499c70c0707041311m77742388r95d06501582be296@mail.gmail.com> Date: Wed, 4 Jul 2007 23:11:38 +0300 From: "Abdullah Ibn Hamad Al-Marri" To: "david.chosrova@libertysurf.fr" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: freebsd-hackers Subject: Re: kernel dynamic references X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2007 20:11:40 -0000 On 7/4/07, david.chosrova@libertysurf.fr wrote: > Hi, > > I'm new to Freebsd and interested in system programming. So I'have picked up a task from the project ideas list to start with. > (part of) the subject is : "This task is to define and implement a general mechanism for tracking these references and use them in handling module unload requests." > David, this is really great thank you! I hope it will be reviewed soon and committed :) -- Regards, -Abdullah Ibn Hamad Al-Marri Arab Portal http://www.WeArab.Net/ From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 00:56:48 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9249E16A41F for ; Thu, 5 Jul 2007 00:56:48 +0000 (UTC) (envelope-from ighighi@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.168]) by mx1.freebsd.org (Postfix) with ESMTP id 2DA4013C4AD for ; Thu, 5 Jul 2007 00:56:47 +0000 (UTC) (envelope-from ighighi@gmail.com) Received: by ug-out-1314.google.com with SMTP id o4so439079uge for ; Wed, 04 Jul 2007 17:56:47 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=MmqpdP/AIpboY5hlEO6nFH8AvblJJ/IU9LVuUVydh2Hv5hV599usMH8HqAYYA9HUW5H6x2lxfqALugX8b6cPwnwTjjAeoy6XHcgoUSSkIj84eAYDm2G1unmUsIPpAmX9ozig8QONeImm5q+Rq8VdqC7sDHBrxrYReZhtk4QY1rY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=lipHlhUo0gYym/wPhzpWvCBwLnVpWSXu/0Talu1d+Sk6DtQ/dwCXxF+n7yQa3iK73DSb1mTFQpfBwAspY7sil6STQ0rasuYaPaQ/OVYH6c3VeOKVxV7hIYjMp8V7rmdrZmpwvNepq84rQPeg1bOEuf0LS0/TG9o7CEuCoA6QA40= Received: by 10.78.140.17 with SMTP id n17mr4436704hud.1183595269263; Wed, 04 Jul 2007 17:27:49 -0700 (PDT) Received: by 10.78.51.18 with HTTP; Wed, 4 Jul 2007 17:27:49 -0700 (PDT) Message-ID: Date: Wed, 4 Jul 2007 20:27:49 -0400 From: "Ighighi Ighighi" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Mailman-Approved-At: Thu, 05 Jul 2007 01:56:39 +0000 Subject: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 00:56:48 -0000 The closefrom() call, available in Solaris, is present in NetBSD since version 3.0. It is implemented with the F_CLOSEM fcntl() available since version 2.0. These 2 fcntl would be cool to have in FreeBSD: F_CLOSEM Close all file descriptors greater than or equal to fd. F_MAXFD Return the maximum file descriptor number currently open by the process. I know that by using kern.file sysctl I could implement both, but it'd be nice if the whole closefrom() is done in kernel space. Another problem with using kern.file is that "struct xfile" may change... By checking http://fxr.watson.org/fxr/ident?v=NETBSD4;i=F_MAXFD I see that it's fairly straightforward to add the F_MAXFD bit. For the F_CLOSEM fcntl, which call is the best to replace the fdrelease() used by NetBSD in kern/kern_descrip.c ? Can someone add these to FreeBSD ? NetBSD's fcntl(2): http://www.freebsd.org/cgi/man.cgi?query=fcntl&apropos=0&sektion=0&manpath=NetBSD+3.0&format=html NetBSD's & Solaris' closefrom(3): http://www.freebsd.org/cgi/man.cgi?query=closefrom&apropos=0&sektion=0&manpath=NetBSD+3.0&format=html http://www.freebsd.org/cgi/man.cgi?query=closefrom&apropos=0&sektion=0&manpath=SunOS+5.9&format=html Cheers, Igh. From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 07:00:35 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D5E6416A400; Thu, 5 Jul 2007 07:00:35 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout5.cac.washington.edu (mxout5.cac.washington.edu [140.142.32.135]) by mx1.freebsd.org (Postfix) with ESMTP id B524D13C459; Thu, 5 Jul 2007 07:00:35 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from smtp.washington.edu (smtp.washington.edu [140.142.33.9] (may be forged)) by mxout5.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.06) with ESMTP id l6570ZHI016035 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 5 Jul 2007 00:00:35 -0700 X-Auth-Received: from [192.168.10.45] (c-24-10-12-194.hsd1.ca.comcast.net [24.10.12.194]) (authenticated authid=youshi10) by smtp.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l6570Ywh020732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 5 Jul 2007 00:00:34 -0700 Message-ID: <468C9718.1050108@u.washington.edu> Date: Thu, 05 Jul 2007 00:00:40 -0700 From: Garrett Cooper User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: ports@FreeBSD.org, hackers@freebsd.org References: <468C96C0.1040603@u.washington.edu> In-Reply-To: <468C96C0.1040603@u.washington.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.3.2.304607, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.7.4.234532 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='__CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __USER_AGENT 0' Cc: Subject: Re: Finding slowdowns in pkg_install (continuations of previous threads) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 07:00:35 -0000 Garrett Cooper wrote: > I'm currently running a gamut of tests (500 tests, per package -- > 128 total on my server), and outputting all data to CSV files to > interpret later, using another Perl script to interpret calculated > averages and standard deviations. > > Using basic printf(2)'s with clock_gettime(2) I have determined > that the majority of the issues are disk-bound (as Tom Kientzle put > it). The scope of my problem is not to analyze tar, but I've > discovered that a lot of time is spent in reading and interpreting the > +CONTENTS and related files (most notably in parsing commands to be > honest). > > Will post more conclusive results tomorrow once all of my results > are available. > > Cheers, > -Garrett Forgot to include hackers@. -Garrett From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 07:32:00 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 923CA16A41F; Thu, 5 Jul 2007 07:32:00 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe07.swip.net [212.247.154.193]) by mx1.freebsd.org (Postfix) with ESMTP id F17FB13C458; Thu, 5 Jul 2007 07:31:59 +0000 (UTC) (envelope-from hselasky@c2i.net) X-Cloudmark-Score: 0.000000 [] Received: from [194.248.135.20] (account mc467741@c2i.net HELO laptop.lan) by mailfe07.swip.net (CommuniGate Pro SMTP 5.1.10) with ESMTPA id 538817661; Thu, 05 Jul 2007 09:31:58 +0200 From: Hans Petter Selasky To: John-Mark Gurney Date: Thu, 5 Jul 2007 09:31:59 +0200 User-Agent: KMail/1.9.5 References: <200707040901.33019.hselasky@c2i.net> <20070704173531.GO1221@funkthat.com> In-Reply-To: <20070704173531.GO1221@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707050931.59256.hselasky@c2i.net> Cc: freebsd-hackers@freebsd.org, freebsd-usb@freebsd.org Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 07:32:00 -0000 On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > > I want to get rid of the copying between DMA'able memory and non-DMA'able > > memory. > > > > Currently I allocate N memory-pages for each USB transfer like separate > > pages. > > How do you allocate these pages? With malloc(9) or w/ bus_dmamem_alloc(9)? bus_dmamem_alloc(). > > > The bus-dma system then assigns all of these pages each their virtual > > address. > > > > What I see is that when I allocate more than PAGE_SIZE bytes using > > bus-dma, I get physically contiguous memory. I don't need that for the > > USB stack. > > That is a limitation of how bus_dma currently allocates memory... It > calls contigmalloc, which doesn't handle multi-segment memory allocations > yet.. > > > The question is: > > > > Should we change bus-dma to support so called scatter and gather > > allocations, where the physical allocation is non-contiguous, and the > > virtual allocation is contiguous accross all the scattered pages ? > > You can already support this by using malloc, and loading that buffer > into your bus_dma map... or using the passing in userland buffer, and > loading that into the map... > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > that the loadig works like this, that when the page must be bounced it > > should not allocate a bounce buffer, hence I already have a bounce > > buffer. I only need to know which pages I can forward directly to the USB > > hardware, and the rest I will bounce somewhere else. > > Why do you not want to let bus_dma do the bouncing for you? If it's > to save a copy to another buffer, why don't you load the final buffer > into bus_dma? Because if I let bus_dma do the bounching, I cannot do this while holding a mutex, hence allocating DMA'able memory on the fly is not so good. --HPS From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 11:43:06 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9C6A716A421 for ; Thu, 5 Jul 2007 11:43:06 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.freebsd.org (Postfix) with ESMTP id 4D74813C448 for ; Thu, 5 Jul 2007 11:43:06 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from localhost (tarsier.geekcn.org [210.51.165.229]) by tarsier.geekcn.org (Postfix) with ESMTP id B890BEB4AE6; Thu, 5 Jul 2007 19:28:54 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([210.51.165.229]) by localhost (mail.geekcn.org [210.51.165.229]) (amavisd-new, port 10024) with ESMTP id 4gYWh1uRcVSj; Thu, 5 Jul 2007 19:28:51 +0800 (CST) Received: from LI-Xins-MacBook.local (sina152-194.staff.sina.com.cn [61.135.152.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 649FDEB4AF0; Thu, 5 Jul 2007 19:28:47 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type; b=TWtMSXXNsixdAn8xEYwlxCr371CnjnbMYlpCvHOrESNv3X6NwnCHTC+3QKmgMMGlU kzYHEFQgWh6haEwSpH1mg== Message-ID: <468CD5E9.7060000@delphij.net> Date: Thu, 05 Jul 2007 19:28:41 +0800 From: LI Xin Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: Ighighi Ighighi References: In-Reply-To: X-Enigmail-Version: 0.95.2 OpenPGP: url=http://www.delphij.net/delphij.asc Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enigE044A3765E07497168DC6A2F" Cc: freebsd-hackers@freebsd.org Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 11:43:06 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigE044A3765E07497168DC6A2F Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Ighighi Ighighi wrote: > The closefrom() call, available in Solaris, is present in NetBSD since > version 3.0. > It is implemented with the F_CLOSEM fcntl() available since version 2.0= =2E I think it might worth an effort (sendmail and perhaps some part of JDK uses it IIRC), but I do not want to rush it into -CURRENT unless we have test cases to make sure that we have done things correctly, because I do not have a NetBSD nor Solaris box at hand to experiment its behavior... := -( Do you have some test cases for this? Cheers, --=20 Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! --------------enigE044A3765E07497168DC6A2F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGjNXpOfuToMruuMARCvdKAJ9L+J6fGnA5dvKur3B88GnQ3FEKIQCeN20x ar+HhgHErqpbjrNM+dhxli8= =WMsm -----END PGP SIGNATURE----- --------------enigE044A3765E07497168DC6A2F-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 12:27:41 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BC66716A421 for ; Thu, 5 Jul 2007 12:27:41 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 8C2DE13C448 for ; Thu, 5 Jul 2007 12:27:40 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id 64E8EE7A3FA for ; Thu, 5 Jul 2007 12:27:35 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id 882937D0F; Thu, 5 Jul 2007 14:26:50 +0200 (CEST) Date: Thu, 5 Jul 2007 14:26:50 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20070705122650.GE1302@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 12:27:41 -0000 On Wed, Jul 04, 2007 at 08:27:49PM -0400, Ighighi Ighighi wrote: > The closefrom() call, available in Solaris, is present in NetBSD since > version 3.0. > It is implemented with the F_CLOSEM fcntl() available since version 2.0. You could also add a system call like it was done in DragonFly. That might be even simpler to implement. Joerg From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 12:55:26 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6BF3216A400 for ; Thu, 5 Jul 2007 12:55:26 +0000 (UTC) (envelope-from zhangweiwu@realss.com) Received: from bossdog.realss.com (bossdog.realss.com [211.157.108.128]) by mx1.freebsd.org (Postfix) with ESMTP id 1D28913C48A for ; Thu, 5 Jul 2007 12:55:26 +0000 (UTC) (envelope-from zhangweiwu@realss.com) Received: from localhost (unknown [127.0.0.1]) by bossdog.realss.com (Postfix) with ESMTP id 247F31C8009 for ; Thu, 5 Jul 2007 20:27:04 +0800 (CST) Received: from bossdog.realss.com ([127.0.0.1]) by localhost (bossdog.realss.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27971-20 for ; Thu, 5 Jul 2007 20:27:03 +0800 (CST) Received: from [218.193.55.202] (unknown [125.77.224.188]) (using SSLv3 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by bossdog.realss.com (Postfix) with ESMTP id 25AA71C8008 for ; Thu, 5 Jul 2007 20:27:00 +0800 (CST) From: Zhang Weiwu To: freebsd-hackers@freebsd.org Content-Type: text/plain Organization: Real Softservice Date: Thu, 05 Jul 2007 20:26:30 +0800 Message-Id: <1183638390.9913.34.camel@esmeralda> Mime-Version: 1.0 X-Mailer: Evolution 2.10.1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at bossdog.realss.com X-Mailman-Approved-At: Thu, 05 Jul 2007 13:18:34 +0000 Subject: should manuals of pcic/card be removed? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 12:55:26 -0000 Dear list sorry I am not sure if I should post to hackers list but I think I should post to a list where people can change FreeBSD distirubtion. It seems old drivers for PCMCIA device are now being replaced, putting either device pcic device card In FreeBSD 6.2 kernel config file would result error in compile. If they are no longer supported, the man page for both pcic and card (they are two different man pages) should be removed. Also pccardc seems to be looking for a device /dev/card0 which doesn't exist, I also occassionally read someone said both pccard_enable="YES" option in rc.conf and pccardd are no longer useful. It's good to remove unusable applications to reduce user confussion. Best Regards Zhang Weiwu From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 21:02:58 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8D90B16A475 for ; Thu, 5 Jul 2007 21:02:58 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 384EE13C468 for ; Thu, 5 Jul 2007 21:02:57 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l65L2pdX027348; Thu, 5 Jul 2007 17:02:53 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 5 Jul 2007 16:25:17 -0400 User-Agent: KMail/1.9.6 References: <200707040901.33019.hselasky@c2i.net> <20070704173531.GO1221@funkthat.com> <200707050931.59256.hselasky@c2i.net> In-Reply-To: <200707050931.59256.hselasky@c2i.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707051625.17954.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 05 Jul 2007 17:02:53 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/3606/Thu Jul 5 13:50:01 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: John-Mark Gurney , freebsd-usb@freebsd.org, Hans Petter Selasky Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 21:02:58 -0000 On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 +0200: > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > > that the loadig works like this, that when the page must be bounced it > > > should not allocate a bounce buffer, hence I already have a bounce > > > buffer. I only need to know which pages I can forward directly to the USB > > > hardware, and the rest I will bounce somewhere else. > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > to save a copy to another buffer, why don't you load the final buffer > > into bus_dma? > > Because if I let bus_dma do the bounching, I cannot do this while holding a > mutex, hence allocating DMA'able memory on the fly is not so good. This is not a hard problem to solve, every other driver using bus_dma solves it. Just make sure your driver is in a sane state and drop the lock while you let bus_dmamap_load() map/copy things for you. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 22:37:04 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 87C4616A41F; Thu, 5 Jul 2007 22:37:04 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 15F1613C45E; Thu, 5 Jul 2007 22:37:04 +0000 (UTC) (envelope-from max@love2party.net) Received: from [88.66.40.129] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu0) with ESMTP (Nemesis), id 0MKwh2-1I6Zwc3jW5-0004Jc; Fri, 06 Jul 2007 00:37:03 +0200 From: Max Laier Organization: FreeBSD To: freebsd-current@freebsd.org Date: Fri, 6 Jul 2007 00:39:08 +0200 User-Agent: KMail/1.9.7 References: <200706242054.54138.max@love2party.net> In-Reply-To: <200706242054.54138.max@love2party.net> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3108306.4ohfARG7Bv"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200707060039.15374.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18Ljz+wju1Xd4Sh2qljgxPq6n1wAGeA2bif+Mh XTXmluSuk0yI7R+k5x8dCJ6JxLNjtKZYql0OEELmeoPOio/vjZ SQ1z8nyT9zcbYyOfFSzLj/wKP+8QKHJaJXwZ2GfsEk= Cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD status reports due: July 7, 2007 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: monthly@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 22:37:04 -0000 --nextPart3108306.4ohfARG7Bv Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello again, kind reminder that the repdigit date is on this Saturday! So far we have=20 received only 18 reports - I know there is more going on out there. =20 Please let the world know what kind of cool stuff is going on in FreeBSD! On Sunday 24 June 2007, Max Laier wrote: > it's that time again. We would like to remind everybody who has > exciting news to share to write a report about their project. This is > a good way to improve exposure of your work, receive feedback and help. > As we are closing in on 7.0 everybody is interested to know what's new > - so please, do tell! > > In addition we would like to gather public relations status reports as > well - did you run a FreeBSD conference or are you going to? Let the > world know how it went or why they should attend. On that note, > everybody remember the early bird deadline for EuroBSDCon? It's July 1 > already! > > Looking forward to your reports. As always you can either use the > template or the generator CGI and mail the result to monthly@ by > 07/07/07 > > http://www.freebsd.org/news/status/ > http://www.freebsd.org/cgi/monthly.cgi > http://www.freebsd.org/news/status/report-sample.xml =2D-=20 =46reeBSD Status reports due: 07/07/07 :-) /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart3108306.4ohfARG7Bv Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBGjXMTXyyEoT62BG0RAmCCAJ0aSWKvAEp7Vf5fNt7iXnOM4ETJyQCeNfQv OfWGHwvSgAxsfUQhDYywXXQ= =ETDZ -----END PGP SIGNATURE----- --nextPart3108306.4ohfARG7Bv-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 5 23:35:53 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0AE7C16A47A; Thu, 5 Jul 2007 23:35:53 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id 8A26113C4C7; Thu, 5 Jul 2007 23:35:52 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l65NZm2W028200; Thu, 5 Jul 2007 19:35:48 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 5 Jul 2007 19:35:32 -0400 User-Agent: KMail/1.9.6 References: <200707040901.33019.hselasky@c2i.net> <200707050931.59256.hselasky@c2i.net> <200707051625.17954.jhb@freebsd.org> In-Reply-To: <200707051625.17954.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707051935.32880.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 05 Jul 2007 19:35:48 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/3606/Thu Jul 5 13:50:01 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: John-Mark Gurney , freebsd-usb@freebsd.org, Hans Petter Selasky Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2007 23:35:53 -0000 On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > +0200: > > > > Also: How is the easiest way to load memory pages into DMA ? And I want > > > > that the loadig works like this, that when the page must be bounced it > > > > should not allocate a bounce buffer, hence I already have a bounce > > > > buffer. I only need to know which pages I can forward directly to the > USB > > > > hardware, and the rest I will bounce somewhere else. > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > to save a copy to another buffer, why don't you load the final buffer > > > into bus_dma? > > > > Because if I let bus_dma do the bounching, I cannot do this while holding a > > mutex, hence allocating DMA'able memory on the fly is not so good. > > This is not a hard problem to solve, every other driver using bus_dma solves > it. Just make sure your driver is in a sane state and drop the lock while > you let bus_dmamap_load() map/copy things for you. Bah, backwards (was thinking of the fact that if you get EINPROGRESS you will have to drop the lock and just wait until the callback is called to make further progress on the request). bus_dmamap_load() already _requires_ you to hold your mutex when you call it, so I don't really see what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and can't properly handle deferred callbacks. You do have to drop the lock around bus_dmamem_alloc(), but not bus_dmamap_load(). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 06:59:40 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 94AB416A41F; Fri, 6 Jul 2007 06:59:40 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe08.swip.net [212.247.154.225]) by mx1.freebsd.org (Postfix) with ESMTP id CE8AC13C457; Fri, 6 Jul 2007 06:59:39 +0000 (UTC) (envelope-from hselasky@c2i.net) X-Cloudmark-Score: 0.000000 [] Received: from [194.248.135.20] (account mc467741@c2i.net HELO laptop.lan) by mailfe08.swip.net (CommuniGate Pro SMTP 5.1.10) with ESMTPA id 536829381; Fri, 06 Jul 2007 08:59:37 +0200 From: Hans Petter Selasky To: John Baldwin Date: Fri, 6 Jul 2007 08:59:39 +0200 User-Agent: KMail/1.9.5 References: <200707040901.33019.hselasky@c2i.net> <200707051625.17954.jhb@freebsd.org> <200707051935.32880.jhb@freebsd.org> In-Reply-To: <200707051935.32880.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707060859.39816.hselasky@c2i.net> Cc: freebsd-hackers@freebsd.org, John-Mark Gurney , freebsd-usb@freebsd.org Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 06:59:40 -0000 On Friday 06 July 2007 01:35, John Baldwin wrote: > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > > > > +0200: > > > > > Also: How is the easiest way to load memory pages into DMA ? And I > > want > > > > > > that the loadig works like this, that when the page must be bounced > > > > > it should not allocate a bounce buffer, hence I already have a > > > > > bounce buffer. I only need to know which pages I can forward > > > > > directly to the > > > > USB > > > > > > > hardware, and the rest I will bounce somewhere else. > > > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > > to save a copy to another buffer, why don't you load the final buffer > > > > into bus_dma? > > > > > > Because if I let bus_dma do the bounching, I cannot do this while > > > holding > > a > > > > mutex, hence allocating DMA'able memory on the fly is not so good. > > > > This is not a hard problem to solve, every other driver using bus_dma > > solves it. Just make sure your driver is in a sane state and drop the > > lock while you let bus_dmamap_load() map/copy things for you. > > Bah, backwards (was thinking of the fact that if you get EINPROGRESS you > will have to drop the lock and just wait until the callback is called to > make further progress on the request). bus_dmamap_load() already > _requires_ you to hold your mutex when you call it, so I don't really see > what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and > can't properly handle deferred callbacks. You do have to drop the lock > around bus_dmamem_alloc(), but not bus_dmamap_load(). The thing is that allocating memory on the fly will be slow, and especially when the xxxx_load() functions will allocate contiguous memory. This only works fine when you load mbufs and things with size less than PAGE_SIZE bytes ?? --HPS From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 10:18:32 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 434BB16A41F; Fri, 6 Jul 2007 10:18:32 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.freebsd.org (Postfix) with ESMTP id 52B0513C44B; Fri, 6 Jul 2007 10:18:31 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from localhost (tarsier.geekcn.org [210.51.165.229]) by tarsier.geekcn.org (Postfix) with ESMTP id 78398EB4E86; Fri, 6 Jul 2007 18:18:30 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([210.51.165.229]) by localhost (mail.geekcn.org [210.51.165.229]) (amavisd-new, port 10024) with ESMTP id ZU1VEEZf5SOb; Fri, 6 Jul 2007 18:18:22 +0800 (CST) Received: from LI-Xins-MacBook.local (sina152-194.staff.sina.com.cn [61.135.152.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id A556DEB4E7B; Fri, 6 Jul 2007 18:18:19 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type; b=jWRTorwXGnBNYtbel2i2+xR9v72St4MMfKxzPbpj1seuq+gXJkaoRIBSCm7ThYuPZ EshQalXbR7NphHZoJPW3A== Message-ID: <468E16E6.6030608@delphij.net> Date: Fri, 06 Jul 2007 18:18:14 +0800 From: LI Xin Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20070705122650.GE1302@britannica.bec.de> In-Reply-To: <20070705122650.GE1302@britannica.bec.de> X-Enigmail-Version: 0.95.2 OpenPGP: url=http://www.delphij.net/delphij.asc Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="------------enig23E1E1748C2AF3C3A2172CF4" Cc: Robert Watson Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 10:18:32 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig23E1E1748C2AF3C3A2172CF4 Content-Type: multipart/mixed; boundary="------------080805090703090505090406" This is a multi-part message in MIME format. --------------080805090703090505090406 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, Joerg Sonnenberger wrote: > On Wed, Jul 04, 2007 at 08:27:49PM -0400, Ighighi Ighighi wrote: >> The closefrom() call, available in Solaris, is present in NetBSD since= >> version 3.0. >> It is implemented with the F_CLOSEM fcntl() available since version 2.= 0. >=20 > You could also add a system call like it was done in DragonFly. That > might be even simpler to implement. Here is my implementation for FreeBSD. Some difference between my and DragonFly's implementation: - closefrom(-1) would be no-op on DragonFly, my version would close all open files (From my understanding of OpenSolaris's userland implementation, this is Solaris's behavior). - my version closefrom(very_big_fd) would result in EBADF. I am not very sure whether this is correct, but it does not hurt for applications that thinks closefrom() would return void. To RW: I have not found a suitable audit event for this, should I create a new event? Cheers, --=20 Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! --------------080805090703090505090406 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="patch-closefrom.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline; filename="patch-closefrom.diff" Index: lib/libc/sys/Symbol.map =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/lib/libc/sys/Symbol.map,v retrieving revision 1.8 diff -u -p -u -r1.8 Symbol.map --- lib/libc/sys/Symbol.map 5 Jun 2007 08:24:34 -0000 1.8 +++ lib/libc/sys/Symbol.map 6 Jul 2007 08:38:55 -0000 @@ -65,6 +65,7 @@ FBSD_1.0 { clock_gettime; clock_settime; close; + closefrom; connect; dup; dup2; Index: sys/kern/init_sysent.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/kern/init_sysent.c,v retrieving revision 1.229 diff -u -p -u -r1.229 init_sysent.c --- sys/kern/init_sysent.c 4 Jul 2007 22:49:54 -0000 1.229 +++ sys/kern/init_sysent.c 6 Jul 2007 08:12:02 -0000 @@ -2,7 +2,7 @@ * System call switch table. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/kern/init_sysent.c,v 1.229 2007/07/04 22:49:54 pete= r Exp $ + * $FreeBSD$ * created from FreeBSD: src/sys/kern/syscalls.master,v 1.232 2007/07/04= 22:47:37 peter Exp=20 */ =20 @@ -510,4 +510,5 @@ struct sysent sysent[] =3D { { AS(lseek_args), (sy_call_t *)lseek, AUE_LSEEK, NULL, 0, 0 }, /* 478 =3D= lseek */ { AS(truncate_args), (sy_call_t *)truncate, AUE_TRUNCATE, NULL, 0, 0 },= /* 479 =3D truncate */ { AS(ftruncate_args), (sy_call_t *)ftruncate, AUE_FTRUNCATE, NULL, 0, 0= }, /* 480 =3D ftruncate */ + { AS(closefrom_args), (sy_call_t *)closefrom, AUE_NULL, NULL, 0, 0 }, /= * 481 =3D closefrom */ }; Index: sys/kern/kern_descrip.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/kern/kern_descrip.c,v retrieving revision 1.312 diff -u -p -u -r1.312 kern_descrip.c --- sys/kern/kern_descrip.c 3 Jul 2007 21:26:06 -0000 1.312 +++ sys/kern/kern_descrip.c 6 Jul 2007 08:04:00 -0000 @@ -989,6 +989,44 @@ fgetown(sigiop) } =20 /* + * Close many file descriptors. + */ +#ifndef _SYS_SYSPROTO_H_ +struct closefrom_args { + int fd; +}; +#endif +/* ARGSUSED */ +int +closefrom(struct thread *td, struct closefrom_args *uap) +{ + + return(kern_closefrom(td, uap->fd)); +} + +int +kern_closefrom(struct thread *td, int fd) +{ + struct filedesc *fdp; + int currfd; + + fdp =3D td->td_proc->p_fd; + + if (fd > fdp->fd_lastfile) + return (EBADF); + else if (fd < 0) + fd =3D 0; + + MPASS(fd >=3D 0); + + while ((currfd =3D fdp->fd_lastfile) >=3D fd) + if (kern_close(td, currfd) =3D=3D EINTR) + return (EINTR); + + return (0); +} + +/* * Close a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ Index: sys/kern/syscalls.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/kern/syscalls.c,v retrieving revision 1.213 diff -u -p -u -r1.213 syscalls.c --- sys/kern/syscalls.c 4 Jul 2007 22:49:55 -0000 1.213 +++ sys/kern/syscalls.c 6 Jul 2007 08:12:02 -0000 @@ -2,7 +2,7 @@ * System call names. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/kern/syscalls.c,v 1.213 2007/07/04 22:49:55 peter E= xp $ + * $FreeBSD$ * created from FreeBSD: src/sys/kern/syscalls.master,v 1.232 2007/07/04= 22:47:37 peter Exp=20 */ =20 @@ -488,4 +488,5 @@ const char *syscallnames[] =3D { "lseek", /* 478 =3D lseek */ "truncate", /* 479 =3D truncate */ "ftruncate", /* 480 =3D ftruncate */ + "closefrom", /* 481 =3D closefrom */ }; Index: sys/kern/syscalls.master =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/kern/syscalls.master,v retrieving revision 1.232 diff -u -p -u -r1.232 syscalls.master --- sys/kern/syscalls.master 4 Jul 2007 22:47:37 -0000 1.232 +++ sys/kern/syscalls.master 6 Jul 2007 07:41:28 -0000 @@ -846,5 +846,6 @@ int whence); } 479 AUE_TRUNCATE STD { int truncate(char *path, off_t length); } 480 AUE_FTRUNCATE STD { int ftruncate(int fd, off_t length); } +481 AUE_NULL STD { int closefrom(int fd); } ; Please copy any additions and changes to the following compatability t= ables: ; sys/compat/freebsd32/syscalls.master Index: sys/kern/systrace_args.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/kern/systrace_args.c,v retrieving revision 1.13 diff -u -p -u -r1.13 systrace_args.c --- sys/kern/systrace_args.c 4 Jul 2007 22:49:55 -0000 1.13 +++ sys/kern/systrace_args.c 6 Jul 2007 08:12:02 -0000 @@ -2,7 +2,7 @@ * System call argument to DTrace register array converstion. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/kern/systrace_args.c,v 1.13 2007/07/04 22:49:55 pet= er Exp $ + * $FreeBSD$ * This file is part of the DTrace syscall provider. */ =20 @@ -2862,6 +2862,13 @@ systrace_args(int sysnum, void *params,=20 *n_args =3D 2; break; } + /* closefrom */ + case 481: { + struct closefrom_args *p =3D params; + iarg[0] =3D p->fd; /* int */ + *n_args =3D 1; + break; + } default: *n_args =3D 0; break; Index: sys/compat/freebsd32/freebsd32_proto.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/compat/freebsd32/freebsd32_proto.h,v retrieving revision 1.77 diff -u -p -u -r1.77 freebsd32_proto.h --- sys/compat/freebsd32/freebsd32_proto.h 4 Jul 2007 23:03:50 -0000 1.77= +++ sys/compat/freebsd32/freebsd32_proto.h 6 Jul 2007 09:57:00 -0000 @@ -2,7 +2,7 @@ * System call prototypes. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/compat/freebsd32/freebsd32_proto.h,v 1.77 2007/07/0= 4 23:03:50 peter Exp $ + * $FreeBSD$ * created from FreeBSD: src/sys/compat/freebsd32/syscalls.master,v 1.90= 2007/07/04 23:02:40 peter Exp=20 */ =20 Index: sys/compat/freebsd32/freebsd32_syscall.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/compat/freebsd32/freebsd32_syscall.h,v retrieving revision 1.75 diff -u -p -u -r1.75 freebsd32_syscall.h --- sys/compat/freebsd32/freebsd32_syscall.h 4 Jul 2007 23:03:50 -0000 1.= 75 +++ sys/compat/freebsd32/freebsd32_syscall.h 6 Jul 2007 09:57:00 -0000 @@ -2,7 +2,7 @@ * System call numbers. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/compat/freebsd32/freebsd32_syscall.h,v 1.75 2007/07= /04 23:03:50 peter Exp $ + * $FreeBSD$ * created from FreeBSD: src/sys/compat/freebsd32/syscalls.master,v 1.90= 2007/07/04 23:02:40 peter Exp=20 */ =20 @@ -337,4 +337,5 @@ #define FREEBSD32_SYS_freebsd32_lseek 478 #define FREEBSD32_SYS_freebsd32_truncate 479 #define FREEBSD32_SYS_freebsd32_ftruncate 480 -#define FREEBSD32_SYS_MAXSYSCALL 481 +#define FREEBSD32_SYS_closefrom 481 +#define FREEBSD32_SYS_MAXSYSCALL 482 Index: sys/compat/freebsd32/freebsd32_syscalls.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/compat/freebsd32/freebsd32_syscalls.c,v retrieving revision 1.66 diff -u -p -u -r1.66 freebsd32_syscalls.c --- sys/compat/freebsd32/freebsd32_syscalls.c 4 Jul 2007 23:03:50 -0000 1= =2E66 +++ sys/compat/freebsd32/freebsd32_syscalls.c 6 Jul 2007 09:57:00 -0000 @@ -2,7 +2,7 @@ * System call names. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/compat/freebsd32/freebsd32_syscalls.c,v 1.66 2007/0= 7/04 23:03:50 peter Exp $ + * $FreeBSD$ * created from FreeBSD: src/sys/compat/freebsd32/syscalls.master,v 1.90= 2007/07/04 23:02:40 peter Exp=20 */ =20 @@ -488,4 +488,5 @@ const char *freebsd32_syscallnames[] =3D { "freebsd32_lseek", /* 478 =3D freebsd32_lseek */ "freebsd32_truncate", /* 479 =3D freebsd32_truncate */ "freebsd32_ftruncate", /* 480 =3D freebsd32_ftruncate */ + "closefrom", /* 481 =3D closefrom */ }; Index: sys/compat/freebsd32/freebsd32_sysent.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/compat/freebsd32/freebsd32_sysent.c,v retrieving revision 1.76 diff -u -p -u -r1.76 freebsd32_sysent.c --- sys/compat/freebsd32/freebsd32_sysent.c 4 Jul 2007 23:03:50 -0000 1.7= 6 +++ sys/compat/freebsd32/freebsd32_sysent.c 6 Jul 2007 09:57:00 -0000 @@ -2,7 +2,7 @@ * System call switch table. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/compat/freebsd32/freebsd32_sysent.c,v 1.76 2007/07/= 04 23:03:50 peter Exp $ + * $FreeBSD$ * created from FreeBSD: src/sys/compat/freebsd32/syscalls.master,v 1.90= 2007/07/04 23:02:40 peter Exp=20 */ =20 @@ -519,4 +519,5 @@ struct sysent freebsd32_sysent[] =3D { { AS(freebsd32_lseek_args), (sy_call_t *)freebsd32_lseek, AUE_LSEEK, NU= LL, 0, 0 }, /* 478 =3D freebsd32_lseek */ { AS(freebsd32_truncate_args), (sy_call_t *)freebsd32_truncate, AUE_TRU= NCATE, NULL, 0, 0 }, /* 479 =3D freebsd32_truncate */ { AS(freebsd32_ftruncate_args), (sy_call_t *)freebsd32_ftruncate, AUE_F= TRUNCATE, NULL, 0, 0 }, /* 480 =3D freebsd32_ftruncate */ + { AS(closefrom_args), (sy_call_t *)closefrom, AUE_NULL, NULL, 0, 0 }, /= * 481 =3D closefrom */ }; Index: sys/compat/freebsd32/syscalls.master =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/compat/freebsd32/syscalls.master,v retrieving revision 1.90 diff -u -p -u -r1.90 syscalls.master --- sys/compat/freebsd32/syscalls.master 4 Jul 2007 23:02:40 -0000 1.90 +++ sys/compat/freebsd32/syscalls.master 6 Jul 2007 09:56:37 -0000 @@ -794,3 +794,4 @@ u_int32_t lengthlo, u_int32_t lengthhi); } 480 AUE_FTRUNCATE STD { int freebsd32_ftruncate(int fd, \ u_int32_t lengthlo, u_int32_t lengthhi); } +481 AUE_NULL NOPROTO { int closefrom(int fd); } Index: sys/sys/syscall.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/sys/syscall.h,v retrieving revision 1.210 diff -u -p -u -r1.210 syscall.h --- sys/sys/syscall.h 4 Jul 2007 22:49:55 -0000 1.210 +++ sys/sys/syscall.h 6 Jul 2007 08:12:02 -0000 @@ -2,7 +2,7 @@ * System call numbers. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/sys/syscall.h,v 1.210 2007/07/04 22:49:55 peter Exp= $ + * $FreeBSD$ * created from FreeBSD: src/sys/kern/syscalls.master,v 1.232 2007/07/04= 22:47:37 peter Exp=20 */ =20 @@ -400,4 +400,5 @@ #define SYS_lseek 478 #define SYS_truncate 479 #define SYS_ftruncate 480 -#define SYS_MAXSYSCALL 481 +#define SYS_closefrom 481 +#define SYS_MAXSYSCALL 482 Index: sys/sys/syscall.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/sys/syscall.mk,v retrieving revision 1.165 diff -u -p -u -r1.165 syscall.mk --- sys/sys/syscall.mk 4 Jul 2007 22:49:55 -0000 1.165 +++ sys/sys/syscall.mk 6 Jul 2007 08:12:02 -0000 @@ -1,6 +1,6 @@ # FreeBSD system call names. # DO NOT EDIT-- this file is automatically generated. -# $FreeBSD: src/sys/sys/syscall.mk,v 1.165 2007/07/04 22:49:55 peter Exp= $ +# $FreeBSD$ # created from FreeBSD: src/sys/kern/syscalls.master,v 1.232 2007/07/04 = 22:47:37 peter Exp=20 MIASM =3D \ syscall.o \ @@ -348,4 +348,5 @@ MIASM =3D \ mmap.o \ lseek.o \ truncate.o \ - ftruncate.o + ftruncate.o \ + closefrom.o Index: sys/sys/syscallsubr.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/sys/syscallsubr.h,v retrieving revision 1.46 diff -u -p -u -r1.46 syscallsubr.h --- sys/sys/syscallsubr.h 7 Jun 2007 19:45:19 -0000 1.46 +++ sys/sys/syscallsubr.h 6 Jul 2007 08:02:13 -0000 @@ -73,6 +73,7 @@ int kern_clock_gettime(struct thread *td int kern_clock_settime(struct thread *td, clockid_t clock_id, struct timespec *ats); int kern_close(struct thread *td, int fd); +int kern_closefrom(struct thread *td, int fd); int kern_connect(struct thread *td, int fd, struct sockaddr *sa); int kern_eaccess(struct thread *td, char *path, enum uio_seg pathseg, int flags); Index: sys/sys/sysproto.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/sys/sys/sysproto.h,v retrieving revision 1.214 diff -u -p -u -r1.214 sysproto.h --- sys/sys/sysproto.h 4 Jul 2007 22:49:55 -0000 1.214 +++ sys/sys/sysproto.h 6 Jul 2007 08:12:02 -0000 @@ -2,7 +2,7 @@ * System call prototypes. * * DO NOT EDIT-- this file is automatically generated. - * $FreeBSD: src/sys/sys/sysproto.h,v 1.214 2007/07/04 22:49:55 peter Ex= p $ + * $FreeBSD$ * created from FreeBSD: src/sys/kern/syscalls.master,v 1.232 2007/07/04= 22:47:37 peter Exp=20 */ =20 @@ -1515,6 +1515,9 @@ struct ftruncate_args { char fd_l_[PADL_(int)]; int fd; char fd_r_[PADR_(int)]; char length_l_[PADL_(off_t)]; off_t length; char length_r_[PADR_(off_t)= ]; }; +struct closefrom_args { + char fd_l_[PADL_(int)]; int fd; char fd_r_[PADR_(int)]; +}; int nosys(struct thread *, struct nosys_args *); void sys_exit(struct thread *, struct sys_exit_args *); int fork(struct thread *, struct fork_args *); @@ -1853,6 +1856,7 @@ int mmap(struct thread *, struct mmap_ar int lseek(struct thread *, struct lseek_args *); int truncate(struct thread *, struct truncate_args *); int ftruncate(struct thread *, struct ftruncate_args *); +int closefrom(struct thread *, struct closefrom_args *); =20 #ifdef COMPAT_43 =20 @@ -2416,6 +2420,7 @@ int freebsd4_sigreturn(struct thread *,=20 #define SYS_AUE_lseek AUE_LSEEK #define SYS_AUE_truncate AUE_TRUNCATE #define SYS_AUE_ftruncate AUE_FTRUNCATE +#define SYS_AUE_closefrom AUE_NULL =20 #undef PAD_ #undef PADL_ Index: crypto/openssh/config.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/crypto/openssh/config.h,v retrieving revision 1.19 diff -u -p -u -r1.19 config.h --- crypto/openssh/config.h 6 Oct 2006 14:27:26 -0000 1.19 +++ crypto/openssh/config.h 6 Jul 2007 08:34:23 -0000 @@ -187,7 +187,7 @@ #define HAVE_CLOCK_T 1 =20 /* Define to 1 if you have the `closefrom' function. */ -/* #undef HAVE_CLOSEFROM */ +#define HAVE_CLOSEFROM 0 =20 /* Define if gai_strerror() returns const char * */ #define HAVE_CONST_GAI_STRERROR_PROTO 1 Index: crypto/openssh/ssh_namespace.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/src/crypto/openssh/ssh_namespace.h,v retrieving revision 1.2 diff -u -p -u -r1.2 ssh_namespace.h --- crypto/openssh/ssh_namespace.h 30 Sep 2006 13:38:05 -0000 1.2 +++ crypto/openssh/ssh_namespace.h 6 Jul 2007 09:11:24 -0000 @@ -145,7 +145,6 @@ #define ciphers_valid ssh_ciphers_valid #define cleanhostname ssh_cleanhostname #define cleanup_exit ssh_cleanup_exit -#define closefrom ssh_closefrom #define colon ssh_colon #define compat_cipher_proposal ssh_compat_cipher_proposal #define compat_datafellows ssh_compat_datafellows --------------080805090703090505090406-- --------------enig23E1E1748C2AF3C3A2172CF4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGjhbmOfuToMruuMARCuP0AJsH6jN/TLKwW8Znfpd2WRkc1C6gqQCfaWQG 2+rX9OYa9lF3YaqBaYB332M= =I82c -----END PGP SIGNATURE----- --------------enig23E1E1748C2AF3C3A2172CF4-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 11:23:47 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2161A16A421 for ; Fri, 6 Jul 2007 11:23:47 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (mx0.hoeg.nl [83.98.131.211]) by mx1.freebsd.org (Postfix) with ESMTP id D3F9113C45D for ; Fri, 6 Jul 2007 11:23:46 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: by palm.hoeg.nl (Postfix, from userid 1000) id AA6DE1CC8A; Fri, 6 Jul 2007 13:25:24 +0200 (CEST) Date: Fri, 6 Jul 2007 13:25:24 +0200 From: Ed Schouten To: d@delphij.net Message-ID: <20070706112453.GA3217@hoeg.nl> References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MW5yreqqjyrRcusr" Content-Disposition: inline In-Reply-To: <468E16E6.6030608@delphij.net> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: FreeBSD Hackers , Robert Watson Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 11:23:47 -0000 --MW5yreqqjyrRcusr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * LI Xin wrote: > Here is my implementation for FreeBSD. Some difference between my and > DragonFly's implementation: >=20 > - closefrom(-1) would be no-op on DragonFly, my version would close all > open files (From my understanding of OpenSolaris's userland > implementation, this is Solaris's behavior). > - my version closefrom(very_big_fd) would result in EBADF. I am not > very sure whether this is correct, but it does not hurt for applications > that thinks closefrom() would return void. Wouldn't it be better to just implement it through fcntl() and implement closefrom() in libc? --=20 Ed Schouten WWW: http://g-rave.nl/ --MW5yreqqjyrRcusr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGjiak52SDGA2eCwURAo3bAJ47S3erS6vgzHwl9RYhdOcqsgdwYQCeKsVK +EFboD/35CVD/ayTO145qVE= =336X -----END PGP SIGNATURE----- --MW5yreqqjyrRcusr-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 11:50:18 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4988116A41F for ; Fri, 6 Jul 2007 11:50:18 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2412F13C457 for ; Fri, 6 Jul 2007 11:50:18 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 89D394830D; Fri, 6 Jul 2007 07:50:17 -0400 (EDT) Date: Fri, 6 Jul 2007 12:50:17 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: d@delphij.net In-Reply-To: <468E16E6.6030608@delphij.net> Message-ID: <20070706124407.T9997@fledge.watson.org> References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 11:50:18 -0000 On Fri, 6 Jul 2007, LI Xin wrote: > Joerg Sonnenberger wrote: > >> On Wed, Jul 04, 2007 at 08:27:49PM -0400, Ighighi Ighighi wrote: >> >>> The closefrom() call, available in Solaris, is present in NetBSD since >>> version 3.0. It is implemented with the F_CLOSEM fcntl() available since >>> version 2.0. >> >> You could also add a system call like it was done in DragonFly. That might >> be even simpler to implement. > > Here is my implementation for FreeBSD. Some difference between my and > DragonFly's implementation: > > - closefrom(-1) would be no-op on DragonFly, my version would close all > open files (From my understanding of OpenSolaris's userland > implementation, this is Solaris's behavior). > - my version closefrom(very_big_fd) would result in EBADF. I am not > very sure whether this is correct, but it does not hurt for applications > that thinks closefrom() would return void. > > To RW: I have not found a suitable audit event for this, should I create a > new event? Solaris side-steps this issue by simply auditing the individual close() system calls. My preference would be that we implement this in user space also, which would likewise generate a series of audit events, one for each system call. The procfs optimization they use (I wonder -- is it really an optimization?) won't work for us, however. Do you think that there's a strong motivation to provide a closefrom(2) system call, rather than a closefrom(3) library call? This would let us neatly avoid the question you've posed :-). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 12:23:04 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3A42D16A468 for ; Fri, 6 Jul 2007 12:23:04 +0000 (UTC) (envelope-from ighighi@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.176]) by mx1.freebsd.org (Postfix) with ESMTP id C66B513C45D for ; Fri, 6 Jul 2007 12:23:03 +0000 (UTC) (envelope-from ighighi@gmail.com) Received: by ik-out-1112.google.com with SMTP id c21so189849ika for ; Fri, 06 Jul 2007 05:23:02 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Lu94sOoem+hY2d9v1UjWOuelwz7e92Yy+hiwmFZjeUMeXrCRzxATn/zLHYSsmuBhexfDXTIfEi9nbVZe6wqTPQ34ME34mhSVBKuZQGPOOcTeOTXOdXawtkMuinVOdPp1J2Ca341PTkkW0sY0AwaZCX5zeVahOn/T1GzLXhDMEw8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=tHbq+/ZwQD3myQ+3VTFKtIHmVaKo1VreWMK/9EVzCHuJRE7pYvnCD7D6DdkInR0bGXQnlZGH00NAUVxgZL3WDaTrdssIaBTYxSUddJXfjjNDPXVEysp8nB3cW8MARrNrXf5Q5QOlxCqd8WaKLyuMt8WGIU5ymYFRoPY0LN/3QfU= Received: by 10.78.160.2 with SMTP id i2mr263214hue.1183724582368; Fri, 06 Jul 2007 05:23:02 -0700 (PDT) Received: by 10.78.51.18 with HTTP; Fri, 6 Jul 2007 05:23:02 -0700 (PDT) Message-ID: Date: Fri, 6 Jul 2007 08:23:02 -0400 From: "Ighighi Ighighi" To: d@delphij.net In-Reply-To: <468CD5E9.7060000@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <468CD5E9.7060000@delphij.net> X-Mailman-Approved-At: Fri, 06 Jul 2007 12:35:23 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 12:23:04 -0000 > LI Xin wrote: > Here is my implementation for FreeBSD. Some difference between my and > DragonFly's implementation: > > - closefrom(-1) would be no-op on DragonFly, my version would close all > open files (From my understanding of OpenSolaris's userland > implementation, this is Solaris's behavior). > - my version closefrom(very_big_fd) would result in EBADF. I am not > very sure whether this is correct, but it does not hurt for applications > that thinks closefrom() would return void. Why not follow current practice and return EBADF for -1 ? It'd be dangerous to close all open files with -1 (a closefrom(0) would suffice), and weird to ignore very_big_fd. I also agree that using fcntl() would be better. Here's the code I'm using to emulate this call on FreeBSD >= 5.0 anyway. int closefrom(int lowfd) { int mib[2] = { CTL_KERN, KERN_FILE }; struct xfile *files = NULL; pid_t pid = getpid(); int i, nfiles; size_t fsize; for (;;) { if (sysctl(mib, 2, files, &fsize, NULL, 0) == -1) { if (errno != ENOMEM) goto bad; else if (files != NULL) { free(files); files = NULL; } } else if (files == NULL) { files = (struct xfile *) malloc(fsize); if (files == NULL) return -1; } else break; } /* XXX This structure may change */ if (files->xf_size != sizeof(struct xfile) || fsize % sizeof(struct xfile)) { errno = ENOSYS; goto bad; } nfiles = fsize / sizeof(struct xfile); for (i = 0; i < nfiles; i++) if (files[i].xf_pid == pid && files[i].xf_fd >= lowfd) close(files[i].xf_fd); free(files); return 0; bad: if (files != NULL) { int save_errno = errno; free(files); errno = save_errno; } return -1; } From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 12:38:18 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B3EF416A41F for ; Fri, 6 Jul 2007 12:38:18 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 7FA3F13C46E for ; Fri, 6 Jul 2007 12:38:15 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id A2C89E7A3FA for ; Fri, 6 Jul 2007 12:38:07 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id C3F157D6C; Fri, 6 Jul 2007 14:37:20 +0200 (CEST) Date: Fri, 6 Jul 2007 14:37:20 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20070706123720.GC427@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> <20070706124407.T9997@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070706124407.T9997@fledge.watson.org> User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 12:38:18 -0000 On Fri, Jul 06, 2007 at 12:50:17PM +0100, Robert Watson wrote: > Solaris side-steps this issue by simply auditing the individual close() > system calls. My preference would be that we implement this in user space > also, which would likewise generate a series of audit events, one for each > system call. The procfs optimization they use (I wonder -- is it really an > optimization?) won't work for us, however. Do you think that there's a > strong motivation to provide a closefrom(2) system call, rather than a > closefrom(3) library call? This would let us neatly avoid the question > you've posed :-). I can think of at least one possible scenario where it makes a difference: multi-threaded applications with concurrent open/closefrom calls. I would expect the kernel version to ensure that all open files start from the given file descriptor. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 12:42:13 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B41C316A46C for ; Fri, 6 Jul 2007 12:42:13 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 8BE2713C448 for ; Fri, 6 Jul 2007 12:42:12 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id B8B4FE7A3FA for ; Fri, 6 Jul 2007 12:42:05 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id 6DA787D6C; Fri, 6 Jul 2007 14:41:19 +0200 (CEST) Date: Fri, 6 Jul 2007 14:41:19 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20070706124119.GD427@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <468E16E6.6030608@delphij.net> User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 12:42:13 -0000 On Fri, Jul 06, 2007 at 06:18:14PM +0800, LI Xin wrote: > - closefrom(-1) would be no-op on DragonFly, my version would close all > open files (From my understanding of OpenSolaris's userland > implementation, this is Solaris's behavior). I think this is a bad idea as -1 is generally an invalid file descriptor. > - my version closefrom(very_big_fd) would result in EBADF. I am not > very sure whether this is correct, but it does not hurt for applications > that thinks closefrom() would return void. I don't think this is a good idea either. One typical use is closefrom(3) and returning an error because no such descriptors are open sounds very wrong. It also just adds another special case as the loop handles this already.... Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 12:50:27 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9633E16A41F for ; Fri, 6 Jul 2007 12:50:27 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.pkgsrc-box.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 502A413C4B7 for ; Fri, 6 Jul 2007 12:50:26 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (www.pkgsrc-box.org [127.0.0.1]) by www.pkgsrc-box.org (Postfix) with ESMTP id 7AED3E7A3FA for ; Fri, 6 Jul 2007 12:50:19 +0000 (UTC) Received: by britannica.bec.de (Postfix, from userid 1000) id C88627D6C; Fri, 6 Jul 2007 14:49:30 +0200 (CEST) Date: Fri, 6 Jul 2007 14:49:30 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20070706124930.GA1174@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 12:50:27 -0000 On Wed, Jul 04, 2007 at 08:27:49PM -0400, Ighighi Ighighi wrote: > It is implemented with the F_CLOSEM fcntl() available since version 2.0. I don't like the fcntl(2) approach as it applies actions to more than the passed in fd. That feels like an interface abuse. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 15:34:02 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 66E9F16A478 for ; Fri, 6 Jul 2007 15:34:02 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from kientzle.com (h-66-166-149-50.snvacaid.covad.net [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 1A28313C4C9 for ; Fri, 6 Jul 2007 15:34:01 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from [10.0.0.222] (p54.kientzle.com [66.166.149.54]) by kientzle.com (8.12.9/8.12.9) with ESMTP id l66FY1H7038895; Fri, 6 Jul 2007 08:34:01 -0700 (PDT) (envelope-from kientzle@freebsd.org) Message-ID: <468E60E9.80507@freebsd.org> Date: Fri, 06 Jul 2007 08:34:01 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060422 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Garrett Cooper References: <468C96C0.1040603@u.washington.edu> <468C9718.1050108@u.washington.edu> In-Reply-To: <468C9718.1050108@u.washington.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Finding slowdowns in pkg_install (continuations of previous threads) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 15:34:02 -0000 >> I'm currently running a gamut of tests (500 tests, per package -- >> 128 total on my server), and outputting all data to CSV files to >> interpret later, using another Perl script to interpret calculated >> averages and standard deviations. Excellent! Much-needed work. >> Using basic printf(2)'s with clock_gettime(2) I have determined >> that the majority of the issues are disk-bound (as Tom Kientzle put >> it). Next question: What are those disk operations and are any of them redundant? >> The scope of my problem is not to analyze tar,... I've spent the last three years+ doing exactly that. Make sure you're using the newest bsdtar/libarchive, which has some very noticable performance improvements. >> but I've >> discovered that a lot of time is spent in reading and interpreting the >> +CONTENTS and related files (most notably in parsing commands to be >> honest). Oh? That's interesting. Is data being re-parsed (in which case some structural changes to parse it once and store the results may help)? Or is the parser just slow? >> Will post more conclusive results tomorrow once all of my results >> are available. I don't follow ports@ so didn't see these "conclusive results" of yours. I'm very interested, though. Tim Kientzle From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 16:23:26 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 063B316A46B; Fri, 6 Jul 2007 16:23:26 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout4.cac.washington.edu (mxout4.cac.washington.edu [140.142.33.19]) by mx1.freebsd.org (Postfix) with ESMTP id D75F013C447; Fri, 6 Jul 2007 16:23:25 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from smtp.washington.edu (smtp.washington.edu [140.142.33.7] (may be forged)) by mxout4.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.06) with ESMTP id l66GNPHu016050 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 6 Jul 2007 09:23:25 -0700 X-Auth-Received: from [192.168.10.45] (c-24-10-12-194.hsd1.ca.comcast.net [24.10.12.194]) (authenticated authid=youshi10) by smtp.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l66GNOGS013145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 6 Jul 2007 09:23:25 -0700 Message-ID: <468E6C81.4060908@u.washington.edu> Date: Fri, 06 Jul 2007 09:23:29 -0700 From: Garrett Cooper User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: Tim Kientzle References: <468C96C0.1040603@u.washington.edu> <468C9718.1050108@u.washington.edu> <468E60E9.80507@freebsd.org> In-Reply-To: <468E60E9.80507@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.3.2.304607, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.7.6.90257 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='__CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __USER_AGENT 0' Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Finding slowdowns in pkg_install (continuations of previous threads) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 16:23:26 -0000 Tim Kientzle wrote: >>> I'm currently running a gamut of tests (500 tests, per package -- >>> 128 total on my server), and outputting all data to CSV files to >>> interpret later, using another Perl script to interpret calculated >>> averages and standard deviations. > > Excellent! Much-needed work. > >>> Using basic printf(2)'s with clock_gettime(2) I have determined >>> that the majority of the issues are disk-bound (as Tom Kientzle put >>> it). > > Next question: What are those disk operations and are any > of them redundant? > >>> The scope of my problem is not to analyze tar,... > > I've spent the last three years+ doing exactly that. > Make sure you're using the newest bsdtar/libarchive, > which has some very noticable performance improvements. > >>> but I've discovered that a lot of time is spent in reading and >>> interpreting the +CONTENTS and related files (most notably in >>> parsing commands to be honest). > > Oh? That's interesting. Is data being re-parsed (in which case > some structural changes to parse it once and store the results > may help)? Or is the parser just slow? > >>> Will post more conclusive results tomorrow once all of my results >>> are available. > > I don't follow ports@ so didn't see these "conclusive results" > of yours. I'm very interested, though. > > Tim Kientzle Some extra notes: -My tests are still running, but almost done (unfortunately I won't be able to post any results before tonight since I'm going to work now). It's taking a lot longer than I originally thought it would (I've produced several gigabytes of logfiles and csv files... eep). -I placed them around what I considered pkg_install specific sensitive areas, i.e. locations where tar was run, or the meta files were processed. -I tried implementing a small buffering technique (read in 10 lines at once, parse the 10 lines, and repeat, instead of read 1 line and parse, then repeat), around the +CONTENTS file parsing function, and the majority of the time it yielded good results (9/10 times the buffering technique won over the non-buffering technique). Given that success I'm going to try implementing the file reading in terms of fgetc(2) to properly read in a number of lines all at once, and see what happens instead (my hunch is those results may be more favorable). Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 16:38:17 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 78FEB16A468 for ; Fri, 6 Jul 2007 16:38:17 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outK.internet-mail-service.net (outK.internet-mail-service.net [216.240.47.234]) by mx1.freebsd.org (Postfix) with ESMTP id 5FE0B13C46A for ; Fri, 6 Jul 2007 16:38:17 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Fri, 06 Jul 2007 09:38:17 -0700 Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 4384F125AE4; Fri, 6 Jul 2007 09:38:16 -0700 (PDT) Message-ID: <468E7007.5050607@elischer.org> Date: Fri, 06 Jul 2007 09:38:31 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: Ed Schouten References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> <20070706112453.GA3217@hoeg.nl> In-Reply-To: <20070706112453.GA3217@hoeg.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , Robert Watson , d@delphij.net Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 16:38:17 -0000 Ed Schouten wrote: > * LI Xin wrote: >> Here is my implementation for FreeBSD. Some difference between my and >> DragonFly's implementation: >> >> - closefrom(-1) would be no-op on DragonFly, my version would close all >> open files (From my understanding of OpenSolaris's userland >> implementation, this is Solaris's behavior). >> - my version closefrom(very_big_fd) would result in EBADF. I am not >> very sure whether this is correct, but it does not hurt for applications >> that thinks closefrom() would return void. > > Wouldn't it be better to just implement it through fcntl() and implement > closefrom() in libc? > that's a possibility but I personally thing the huge difference in efficiency makes it worth putting it in the kernel. Quite a few programs I know of could really help their startup time with this as the first thing they do is "close the first 2000 file descriptors. From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 16:44:50 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D294416A400 for ; Fri, 6 Jul 2007 16:44:50 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id A3E7913C4CB for ; Fri, 6 Jul 2007 16:44:50 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id DEDE8491D0; Fri, 6 Jul 2007 12:44:49 -0400 (EDT) Date: Fri, 6 Jul 2007 17:44:49 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Julian Elischer In-Reply-To: <468E7007.5050607@elischer.org> Message-ID: <20070706174321.M17956@fledge.watson.org> References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> <20070706112453.GA3217@hoeg.nl> <468E7007.5050607@elischer.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Hackers , d@delphij.net, Ed Schouten Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 16:44:50 -0000 On Fri, 6 Jul 2007, Julian Elischer wrote: > Ed Schouten wrote: >> * LI Xin wrote: >>> Here is my implementation for FreeBSD. Some difference between my and >>> DragonFly's implementation: >>> >>> - closefrom(-1) would be no-op on DragonFly, my version would close all >>> open files (From my understanding of OpenSolaris's userland >>> implementation, this is Solaris's behavior). >>> - my version closefrom(very_big_fd) would result in EBADF. I am not very >>> sure whether this is correct, but it does not hurt for applications that >>> thinks closefrom() would return void. >> >> Wouldn't it be better to just implement it through fcntl() and implement >> closefrom() in libc? > > that's a possibility but I personally thing the huge difference in > efficiency makes it worth putting it in the kernel. Quite a few programs I > know of could really help their startup time with this as the first thing > they do is "close the first 2000 file descriptors. The Solaris implementation appears to implement two strategies: (1) If procfs is mounted, list the fd directory to get a list of open fds, then close those by number. (2) If procfs is not mounted, query the number of open fds using the resource limit interface, then sequentially close until the right number close. Hence my question as to whether there's actually a big benefit or not -- do we think closefrom() is a performance-critical function? Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 16:45:07 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6C96916A421; Fri, 6 Jul 2007 16:45:07 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from kientzle.com (h-66-166-149-50.snvacaid.covad.net [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 35B8C13C480; Fri, 6 Jul 2007 16:45:07 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from [10.0.0.222] (p54.kientzle.com [66.166.149.54]) by kientzle.com (8.12.9/8.12.9) with ESMTP id l66Gj6H7039341; Fri, 6 Jul 2007 09:45:06 -0700 (PDT) (envelope-from kientzle@freebsd.org) Message-ID: <468E7192.8030105@freebsd.org> Date: Fri, 06 Jul 2007 09:45:06 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060422 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Garrett Cooper References: <468C96C0.1040603@u.washington.edu> <468C9718.1050108@u.washington.edu> <468E60E9.80507@freebsd.org> <468E6C81.4060908@u.washington.edu> In-Reply-To: <468E6C81.4060908@u.washington.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Finding slowdowns in pkg_install (continuations of previous threads) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 16:45:07 -0000 > -I tried ... buffering ... the +CONTENTS file parsing function, and the > majority of the time it yielded good results .... One approach I prototyped sometime back was to use libarchive in pkg_add as follows: * Open the archive * Read +CONTENTS directly into memory (it's guaranteed to always be first in the archive) * Parse all of +CONTENTS at once * Continue scanning the archive, disposing of each file as it appears in the archive. Based on my experience with this, I would suggest you just read all of +CONTENTS directly into memory at once and parse the whole thing in a single shot. fopen(), then fstat() to get the size, then allocate a buffer and read the whole thing, then fclose(). You can then parse it all at once. As a bonus, your parser then becomes a nice little bit of reusable code that reads a block of memory and returns a structure describing the package metadata. Tim Kientzle From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 16:56:53 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 25FDD16A400 for ; Fri, 6 Jul 2007 16:56:53 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outQ.internet-mail-service.net (outQ.internet-mail-service.net [216.240.47.240]) by mx1.freebsd.org (Postfix) with ESMTP id 0F9EC13C48A for ; Fri, 6 Jul 2007 16:56:53 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Fri, 06 Jul 2007 09:56:52 -0700 Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 63FBB125B6C; Fri, 6 Jul 2007 09:56:52 -0700 (PDT) Message-ID: <468E7463.8020803@elischer.org> Date: Fri, 06 Jul 2007 09:57:07 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: Robert Watson References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> <20070706112453.GA3217@hoeg.nl> <468E7007.5050607@elischer.org> <20070706174321.M17956@fledge.watson.org> In-Reply-To: <20070706174321.M17956@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , d@delphij.net, Ed Schouten Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 16:56:53 -0000 Robert Watson wrote: > > On Fri, 6 Jul 2007, Julian Elischer wrote: > >> Ed Schouten wrote: >>> * LI Xin wrote: >>>> Here is my implementation for FreeBSD. Some difference between my >>>> and DragonFly's implementation: >>>> >>>> - closefrom(-1) would be no-op on DragonFly, my version would close >>>> all open files (From my understanding of OpenSolaris's userland >>>> implementation, this is Solaris's behavior). >>>> - my version closefrom(very_big_fd) would result in EBADF. I am >>>> not very sure whether this is correct, but it does not hurt for >>>> applications that thinks closefrom() would return void. >>> >>> Wouldn't it be better to just implement it through fcntl() and >>> implement closefrom() in libc? >> >> that's a possibility but I personally thing the huge difference in >> efficiency makes it worth putting it in the kernel. Quite a few >> programs I know of could really help their startup time with this as >> the first thing they do is "close the first 2000 file descriptors. > > The Solaris implementation appears to implement two strategies: > > (1) If procfs is mounted, list the fd directory to get a list of open fds, > then close those by number. > > (2) If procfs is not mounted, query the number of open fds using the > resource > limit interface, then sequentially close until the right number close. > > Hence my question as to whether there's actually a big benefit or not -- > do we think closefrom() is a performance-critical function? It's one of those things where it's so simple to do it that it hardly seems worth arguing about the colour, or even whether colour is spelled color or colour. > > Robert N M Watson > Computer Laboratory > University of Cambridge From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 17:56:31 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 94D6316A41F for ; Fri, 6 Jul 2007 17:56:31 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.freebsd.org (Postfix) with ESMTP id 2D47813C455 for ; Fri, 6 Jul 2007 17:56:30 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.13.8/jtpda-5.4) with ESMTP id l66H7qkR044485 ; Fri, 6 Jul 2007 19:07:52 +0200 (CEST) X-Ids: 165 Received: by parthe.lpthe.jussieu.fr (Postfix, from userid 10096) id C8ECDBF6D3; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.9 (2007-02-13) on parthe.lpthe.jussieu.fr X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.9 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 1C9CABF60E; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id 1393180; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) Date: Fri, 6 Jul 2007 19:07:51 +0200 From: Michel Talon To: ports@freebsd.org, hackers@freebsd.org Message-ID: <20070706170750.GA99504@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (shiva.jussieu.fr [134.157.0.165]); Fri, 06 Jul 2007 19:07:52 +0200 (CEST) X-Virus-Scanned: ClamAV 0.88.7/3607/Fri Jul 6 01:51:19 2007 on shiva.jussieu.fr X-Virus-Status: Clean X-j-chkmail-Score: MSGID : 468E76E8.002 on shiva.jussieu.fr : j-chkmail score : X : 0/50 0 0.507 -> 1 X-Miltered: at shiva.jussieu.fr with ID 468E76E8.002 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Mailman-Approved-At: Fri, 06 Jul 2007 18:49:17 +0000 Cc: Subject: Re: Finding slowdowns in pkg_install X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 17:56:31 -0000 Tim Kientzle said: > One approach I prototyped sometime back was to use > libarchive in pkg_add as follows: > * Open the archive > * Read +CONTENTS directly into memory (it's > guaranteed to always be first in the archive) I can only concur with that. In my program http://www.lpthe.jussieu.fr/~talon/check_pkg.py i discovered that memory mapping +CONTENTS and then working in memory before rewriting it was around 5 times faster than reading line by line and parsing each line. See function fix_pkg_database(). By the way i am writing a new +CONTENTS fileand then renaming it, which avoids leaving a mess if something goes astray like portupgrade does. -- Michel TALON From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 20:41:43 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7012A16A41F; Fri, 6 Jul 2007 20:41:43 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.freebsd.org (Postfix) with ESMTP id EB69A13C45B; Fri, 6 Jul 2007 20:41:42 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l66KfcTm036933; Fri, 6 Jul 2007 16:41:38 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Hans Petter Selasky Date: Fri, 6 Jul 2007 16:41:35 -0400 User-Agent: KMail/1.9.6 References: <200707040901.33019.hselasky@c2i.net> <200707051935.32880.jhb@freebsd.org> <200707060859.39816.hselasky@c2i.net> In-Reply-To: <200707060859.39816.hselasky@c2i.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707061641.36396.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Fri, 06 Jul 2007 16:41:38 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/3607/Thu Jul 5 19:51:19 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-hackers@freebsd.org, John-Mark Gurney , freebsd-usb@freebsd.org Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 20:41:43 -0000 On Friday 06 July 2007 02:59:39 am Hans Petter Selasky wrote: > On Friday 06 July 2007 01:35, John Baldwin wrote: > > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at 09:01 > > > > > > +0200: > > > > > > Also: How is the easiest way to load memory pages into DMA ? And I > > > > want > > > > > > > > that the loadig works like this, that when the page must be bounced > > > > > > it should not allocate a bounce buffer, hence I already have a > > > > > > bounce buffer. I only need to know which pages I can forward > > > > > > directly to the > > > > > > USB > > > > > > > > > hardware, and the rest I will bounce somewhere else. > > > > > > > > > > Why do you not want to let bus_dma do the bouncing for you? If it's > > > > > to save a copy to another buffer, why don't you load the final buffer > > > > > into bus_dma? > > > > > > > > Because if I let bus_dma do the bounching, I cannot do this while > > > > holding > > > > a > > > > > > mutex, hence allocating DMA'able memory on the fly is not so good. > > > > > > This is not a hard problem to solve, every other driver using bus_dma > > > solves it. Just make sure your driver is in a sane state and drop the > > > lock while you let bus_dmamap_load() map/copy things for you. > > > > Bah, backwards (was thinking of the fact that if you get EINPROGRESS you > > will have to drop the lock and just wait until the callback is called to > > make further progress on the request). bus_dmamap_load() already > > _requires_ you to hold your mutex when you call it, so I don't really see > > what the issue is, unless you are assuming BUS_DMA_NOWAIT behavior and > > can't properly handle deferred callbacks. You do have to drop the lock > > around bus_dmamem_alloc(), but not bus_dmamap_load(). > > The thing is that allocating memory on the fly will be slow, and especially > when the xxxx_load() functions will allocate contiguous memory. This only > works fine when you load mbufs and things with size less than PAGE_SIZE > bytes ?? Huh? The bounce pages are preallocated, so bus_dmamap_load() isn't going to be allocating things. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Jul 6 21:17:17 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0908E16A421; Fri, 6 Jul 2007 21:17:17 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout3.cac.washington.edu (mxout3.cac.washington.edu [140.142.32.166]) by mx1.freebsd.org (Postfix) with ESMTP id DE86B13C45E; Fri, 6 Jul 2007 21:17:16 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from hymn09.u.washington.edu (hymn09.u.washington.edu [140.142.12.183]) by mxout3.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.06) with ESMTP id l66LHG8S000473 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 6 Jul 2007 14:17:16 -0700 Received: from localhost (localhost [127.0.0.1]) by hymn09.u.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l66LHGvY000562; Fri, 6 Jul 2007 14:17:16 -0700 X-Auth-Received: from [192.55.52.3] by hymn09.u.washington.edu via HTTP; Fri, 06 Jul 2007 14:17:16 PDT Date: Fri, 6 Jul 2007 14:17:16 -0700 (PDT) From: youshi10@u.washington.edu To: ports@freebsd.org In-Reply-To: <20070706170750.GA99504@lpthe.jussieu.fr> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-PMX-Version: 5.3.2.304607, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.7.6.135432 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='SUPERLONG_LINE 0.05, NO_REAL_NAME 0, __CP_URI_IN_BODY 0, __CT 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' Cc: hackers@freebsd.org Subject: Re: Finding slowdowns in pkg_install X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 21:17:17 -0000 On Fri, 6 Jul 2007, Michel Talon wrote: > Tim Kientzle said: > >> One approach I prototyped sometime back was to use >> libarchive in pkg_add as follows: >> * Open the archive >> * Read +CONTENTS directly into memory (it's >> guaranteed to always be first in the archive) > > I can only concur with that. In my program > http://www.lpthe.jussieu.fr/~talon/check_pkg.py > i discovered that memory mapping +CONTENTS and then working > in memory before rewriting it was around 5 times faster > than reading line by line and parsing each line. See function > fix_pkg_database(). By the way i am writing a new +CONTENTS > fileand then renaming it, which avoids leaving a mess if > something goes astray like portupgrade does. > > > -- > > Michel TALON Ok, excellent I'll try that then. I'll work on an improved parser this weekend and probably will have more conclusive results for next week, but for the immediate point in time I'll post results on how slow / fast the critical sections were once I return home and post process my data again for averages and standard deviations. I'll use this as my basis for further conclusions this summer. -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sat Jul 7 07:58:23 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E727616A421 for ; Sat, 7 Jul 2007 07:58:22 +0000 (UTC) (envelope-from xistence@0x58.com) Received: from mailexchange.osnn.net (1e.66.5646.static.theplanet.com [70.86.102.30]) by mx1.freebsd.org (Postfix) with SMTP id 9FE5413C469 for ; Sat, 7 Jul 2007 07:58:22 +0000 (UTC) (envelope-from xistence@0x58.com) Received: (qmail 11279 invoked by uid 0); 7 Jul 2007 07:27:56 -0000 Received: from unknown (HELO ?10.10.10.22?) (xistence@0x58.com@72.208.132.56) by mailexchange.osnn.net with SMTP; 7 Jul 2007 07:27:56 -0000 Mime-Version: 1.0 (Apple Message framework v752.3) In-Reply-To: <468E7463.8020803@elischer.org> References: <20070705122650.GE1302@britannica.bec.de> <468E16E6.6030608@delphij.net> <20070706112453.GA3217@hoeg.nl> <468E7007.5050607@elischer.org> <20070706174321.M17956@fledge.watson.org> <468E7463.8020803@elischer.org> Content-Type: multipart/signed; micalg=sha1; boundary=Apple-Mail-1--672140753; protocol="application/pkcs7-signature" Message-Id: From: Bert JW Regeer Date: Sat, 7 Jul 2007 00:31:29 -0700 To: FreeBSD Hackers X-Mailer: Apple Mail (2.752.3) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: add closefrom() call X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2007 07:58:23 -0000 --Apple-Mail-1--672140753 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed On Jul 6, 2007, at 9:57 AM, Julian Elischer wrote: > Robert Watson wrote: >> On Fri, 6 Jul 2007, Julian Elischer wrote: >>> Ed Schouten wrote: >>>> * LI Xin wrote: >>>>> Here is my implementation for FreeBSD. Some difference between >>>>> my and DragonFly's implementation: >>>>> >>>>> - closefrom(-1) would be no-op on DragonFly, my version would >>>>> close all open files (From my understanding of OpenSolaris's >>>>> userland implementation, this is Solaris's behavior). >>>>> - my version closefrom(very_big_fd) would result in EBADF. I >>>>> am not very sure whether this is correct, but it does not hurt >>>>> for applications that thinks closefrom() would return void. >>>> >>>> Wouldn't it be better to just implement it through fcntl() and >>>> implement closefrom() in libc? >>> >>> that's a possibility but I personally thing the huge difference >>> in efficiency makes it worth putting it in the kernel. Quite a >>> few programs I know of could really help their startup time with >>> this as the first thing they do is "close the first 2000 file >>> descriptors. >> The Solaris implementation appears to implement two strategies: >> (1) If procfs is mounted, list the fd directory to get a list of >> open fds, >> then close those by number. >> (2) If procfs is not mounted, query the number of open fds using >> the resource >> limit interface, then sequentially close until the right >> number close. >> Hence my question as to whether there's actually a big benefit or >> not -- do we think closefrom() is a performance-critical function? > > > It's one of those things where it's so simple to do it that it > hardly seems worth arguing about the colour, or even whether colour > is spelled color or colour. > >> Robert N M Watson >> Computer Laboratory >> University of Cambridge > So guys, where can I pick up my bike-shed? Bert JW Regeer --Apple-Mail-1--672140753-- From owner-freebsd-hackers@FreeBSD.ORG Sat Jul 7 16:14:34 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ECFFB16A41F; Sat, 7 Jul 2007 16:14:34 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.swip.net [212.247.154.161]) by mx1.freebsd.org (Postfix) with ESMTP id 2B91F13C459; Sat, 7 Jul 2007 16:14:33 +0000 (UTC) (envelope-from hselasky@c2i.net) X-Cloudmark-Score: 0.000000 [] Received: from [88.90.156.247] (account mc467741@c2i.net HELO [192.168.0.105]) by mailfe06.swip.net (CommuniGate Pro SMTP 5.1.10) with ESMTPA id 540270416; Sat, 07 Jul 2007 18:14:22 +0200 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org Date: Sat, 7 Jul 2007 18:14:24 +0200 User-Agent: KMail/1.9.5 References: <200707040901.33019.hselasky@c2i.net> <200707060859.39816.hselasky@c2i.net> <200707061641.36396.jhb@freebsd.org> In-Reply-To: <200707061641.36396.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200707071814.24578.hselasky@c2i.net> Cc: John-Mark Gurney , freebsd-usb@freebsd.org Subject: Re: New USB stack and Zero copy. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2007 16:14:35 -0000 On Friday 06 July 2007 22:41, John Baldwin wrote: > On Friday 06 July 2007 02:59:39 am Hans Petter Selasky wrote: > > On Friday 06 July 2007 01:35, John Baldwin wrote: > > > On Thursday 05 July 2007 04:25:17 pm John Baldwin wrote: > > > > On Thursday 05 July 2007 03:31:59 am Hans Petter Selasky wrote: > > > > > On Wednesday 04 July 2007 19:35, John-Mark Gurney wrote: > > > > > > Hans Petter Selasky wrote this message on Wed, Jul 04, 2007 at > > Huh? The bounce pages are preallocated, so bus_dmamap_load() isn't going > to be allocating things. When are they allocated ? --HPS