From owner-freebsd-arch@FreeBSD.ORG Mon Jun 1 11:06:47 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 219EE106566C for ; Mon, 1 Jun 2009 11:06:47 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E77798FC0C for ; Mon, 1 Jun 2009 11:06:46 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n51B6kGa020983 for ; Mon, 1 Jun 2009 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n51B6krJ020977 for freebsd-arch@FreeBSD.org; Mon, 1 Jun 2009 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Jun 2009 11:06:46 GMT Message-Id: <200906011106.n51B6krJ020977@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2009 11:06:47 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Tue Jun 2 16:30:07 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4504E1065672 for ; Tue, 2 Jun 2009 16:30:07 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outW.internet-mail-service.net (outw.internet-mail-service.net [216.240.47.246]) by mx1.freebsd.org (Postfix) with ESMTP id 2D1C98FC0C for ; Tue, 2 Jun 2009 16:30:06 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 9909822C3; Tue, 2 Jun 2009 09:30:06 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 4C5132D6012; Tue, 2 Jun 2009 09:30:06 -0700 (PDT) Message-ID: <4A25538E.5020302@elischer.org> Date: Tue, 02 Jun 2009 09:30:06 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Alexander Motin References: <4A254FB5.3030504@FreeBSD.org> In-Reply-To: <4A254FB5.3030504@FreeBSD.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2009 16:30:07 -0000 Alexander Motin wrote: > Hi. > > After replying to several similar questions about my ATA plans last > time, I have decided to announce things I am working on now together > with Scott Long. > I can't imagine a team I'd rather have doing it. great news From owner-freebsd-arch@FreeBSD.ORG Tue Jun 2 16:54:55 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54FB91065702 for ; Tue, 2 Jun 2009 16:54:55 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id D24F78FC20 for ; Tue, 2 Jun 2009 16:54:54 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from orphanage.alkar.net (account mav@alkar.net [212.86.226.11] verified) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPA id 244687062; Tue, 02 Jun 2009 18:54:46 +0300 Message-ID: <4A254B45.8050800@mavhome.dp.ua> Date: Tue, 02 Jun 2009 18:54:45 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.14 (X11/20080612) MIME-Version: 1.0 To: freebsd-arch@freebsd.org, FreeBSD-Current Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: Scott Long Subject: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2009 16:54:55 -0000 Hi. After replying to several similar questions about my ATA plans last time, I have decided to announce things I am working on now together with Scott Long. While learning FreeBSD ATA implementation, I have found, that it has numerous deep problems, from quite fuzzy APIs to different issues in device detection and error recovery. Also, as soon as this infrastructure was written many years ago, it has completely no support for any kind of command queuing, which is normal for the most of modern drives and controllers. Fixing all of this require many significant changes. Also, you may know, that SAS controllers and expanders allow attaching SATA devices and port multipliers to them, by transporting ATA commands over SCSI (SAS) bus. ATAPI, same time, is the way to transport SCSI commands over ATA interface. So deep technologies interoperation pushes us to have similarly integrated infrastructures on software level. We are already have atapicam driver which is used to give CAM SCSI infrastructure access to ATAPI devices by translating drivers API and SCSI bus emulation. But it works only one way and also not perfect. Looking to all of this, I have decided to join Scott, to reanimate his project of making CAM a system's universal infrastructure for both SCSI and ATA. This project is not about some kind of SCSI-to-ATA translation, used by some OS, like OpenBSD. It is about extending CAM, to equally support both SCSI and ATA worlds natively and integrate them as tight as possible. This project is going to have several main steps: - separate SCSI command set and SCSI bus management code from abstract CAM code and create abstract transport API (this step is mostly done now by Scott), - implement ATA command set device drivers, ATA bus management code and ATA host controller drivers (this step is now in progress by me), - update CAM to use newbus, to make it's components interoperation more transparent and formalized (this is now in planning and preparation stage), - when mentioned above finished, port the rest of existing ATA controller drivers one by one to the new world order. Our code now lives in PERFORCE in scottl-camlock project branch. It is in early development, but we already have there working CAM driver for AHCI controller with command queuing and basic NCQ support, simple SATA bus management code and ATA disk driver. I am able now to boot my system and work from SATA drive on AHCI controller, using ATA disk driver, having command queuing and NCQ enabled, read and write disks with SATA ATAPI DVD-RW drive, using native SCSI CD driver. And all of that only with CAM, without using any part of ATA infrastructure. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Tue Jun 2 17:13:48 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E57F1065693 for ; Tue, 2 Jun 2009 17:13:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id 22B9F8FC1A for ; Tue, 2 Jun 2009 17:13:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from orphanage.alkar.net (account mav@alkar.net [212.86.226.11] verified) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPA id 244688970; Tue, 02 Jun 2009 19:13:41 +0300 Message-ID: <4A254FB5.3030504@FreeBSD.org> Date: Tue, 02 Jun 2009 19:13:41 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.14 (X11/20080612) MIME-Version: 1.0 To: freebsd-arch@freebsd.org, FreeBSD-Current X-Enigmail-Version: 0.95.0 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: Subject: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2009 17:13:48 -0000 Hi. After replying to several similar questions about my ATA plans last time, I have decided to announce things I am working on now together with Scott Long. While learning FreeBSD ATA implementation, I have found, that it has numerous deep problems, from quite fuzzy APIs to different issues in device detection and error recovery. Also, as soon as this infrastructure was written many years ago, it has completely no support for any kind of command queuing, which is normal for the most of modern drives and controllers. Fixing all of this require many significant changes. Also, you may know, that SAS controllers and expanders allow attaching SATA devices and port multipliers to them, by transporting ATA commands over SCSI (SAS) bus. ATAPI, same time, is the way to transport SCSI commands over ATA interface. So deep technologies interoperation pushes us to have similarly integrated infrastructures on software level. We are already have atapicam driver which is used to give CAM SCSI infrastructure access to ATAPI devices by translating drivers API and SCSI bus emulation. But it works only one way and also not perfect. Looking to all of this, I have decided to join Scott, to reanimate his project of making CAM a system's universal infrastructure for both SCSI and ATA. This project is not about some kind of SCSI-to-ATA translation, used by some OS, like OpenBSD. It is about extending CAM, to equally support both SCSI and ATA worlds natively and integrate them as tight as possible. This project is going to have several main steps: - separate SCSI command set and SCSI bus management code from abstract CAM code and create abstract transport API (this step is mostly done now by Scott), - implement ATA command set device drivers, ATA bus management code and ATA host controller drivers (this step is now in progress by me), - update CAM to use newbus, to make it's components interoperation more transparent and formalized (this is now in planning and preparation stage), - when mentioned above finished, port the rest of existing ATA controller drivers one by one to the new world order. Our code now lives in PERFORCE in scottl-camlock project branch. It is in early development, but we already have there working CAM driver for AHCI controller with command queuing and basic NCQ support, simple SATA bus management code and ATA disk driver. I am able now to boot my system and work from SATA drive on AHCI controller, using ATA disk driver, having command queuing and NCQ enabled, read and write disks with SATA ATAPI DVD-RW drive, using native SCSI CD driver. And all of that only with CAM, without using any part of ATA infrastructure. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Tue Jun 2 22:43:46 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79606106566C for ; Tue, 2 Jun 2009 22:43:46 +0000 (UTC) (envelope-from bf2006a@yahoo.com) Received: from web39107.mail.mud.yahoo.com (web39107.mail.mud.yahoo.com [209.191.87.226]) by mx1.freebsd.org (Postfix) with SMTP id 1F1B68FC21 for ; Tue, 2 Jun 2009 22:43:45 +0000 (UTC) (envelope-from bf2006a@yahoo.com) Received: (qmail 65614 invoked by uid 60001); 2 Jun 2009 22:17:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1243981025; bh=OYdSSVUJNN7iHHd7wbTC8KNrZniP65W3IBjNMPI32cA=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=v47TZBchLdsdLFReqam1HSYMmagNcHenWqM+UKh/IfYrNoEy0Wy7nPGRthheF7to1JmthM6FUvznJd3Z+lj8YPqr4xjGWMqB8n/sQqDMiZKEdL22H+pzBnF82Y+pumRPLT/dRnXmcD+uzL1FJZwmZEl7RcfGBwSHqiqz/3ZcAUo= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=xu2sGq5SAFlDsQ/PsvZ2xnjSBOs7Mgf9A6d7fRPkHH3WJh0m39XxAYpBg287r4ImwEA81tM2PGNNN5zf5y446qU7RMxwMNQlHNEK3l0kLJRFkEeOq0VTbEaFF7+hwDkitA2nDKCD4NVcFCGNHfIw0rRLJMnXiv8Ivt/NWK3dlVU=; Message-ID: <185581.64029.qm@web39107.mail.mud.yahoo.com> X-YMail-OSG: sV9r0SQVM1nqic_.gG_l8xJdJHqybRC45fLvPf__uBFRD3h4wR8DXjXyc_xPCBOHEgLEHiSBsQauvbCnn7qU3VkC63DQ8LjjsRWu.kYo8zFJNrm_mY8LDxNK.wO9GiT1HAUdi4fTIrw_UEEzoFZANL8vv6MBxX9M94PX1OIKnOah8BAhCwXbY1.P5he99kQQ2hTQnF3Bg5KXCYL.zRtIU3qk.w0XH_2UKm2ZGuWXe5.H.3YSsA48cSD7UGzNvRLS.C5UzQ8qZ2Dz.GYGxSMRr85HlTdf4N82 Received: from [85.214.73.63] by web39107.mail.mud.yahoo.com via HTTP; Tue, 02 Jun 2009 15:17:05 PDT X-Mailer: YahooMailClassic/5.3.9 YahooMailWebService/0.7.289.10 Date: Tue, 2 Jun 2009 15:17:05 -0700 (PDT) From: bf To: freebsd-arch@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: mav@FreeBSD.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2009 22:43:46 -0000 Alexander: I was happy to see that your recent P4 commit messages, hoping for good things to come, but your announcement surpasses my expectations. Thanks to you and Scott for undertaking this effort. Will there be a trickle of commits to the main source tree, or are you going to hold off until the project is finished? If the latter, when do you think that you will be done? Regards, b. From owner-freebsd-arch@FreeBSD.ORG Wed Jun 3 06:06:23 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39BDC106566C for ; Wed, 3 Jun 2009 06:06:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id B58868FC13 for ; Wed, 3 Jun 2009 06:06:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from [212.86.226.226] (account mav@alkar.net HELO mavbook.mavhome.dp.ua) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPSA id 244723647; Wed, 03 Jun 2009 09:06:19 +0300 Message-ID: <4A2612D7.7030206@FreeBSD.org> Date: Wed, 03 Jun 2009 09:06:15 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.21 (X11/20090405) MIME-Version: 1.0 To: bf References: <185581.64029.qm@web39107.mail.mud.yahoo.com> In-Reply-To: <185581.64029.qm@web39107.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2009 06:06:23 -0000 bf wrote: > Will there be a trickle of commits > to the main source tree, or are you going to > hold off until the project is finished? If the > latter, when do you think that you will be done? Now we have code slush state on CURRENT, so the most code will not hit the tree at least 3-4 month, before 8.x branching. I hope to stabilize CAM ATA code in next 2 months. But I don't know yet, how much time will take the next stage, which is going to be very advantageous, but also quite invasive. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Wed Jun 3 22:11:44 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 067C81065672 for ; Wed, 3 Jun 2009 22:11:44 +0000 (UTC) (envelope-from bf2006a@yahoo.com) Received: from web39102.mail.mud.yahoo.com (web39102.mail.mud.yahoo.com [209.191.86.253]) by mx1.freebsd.org (Postfix) with SMTP id B8BD08FC1A for ; Wed, 3 Jun 2009 22:11:43 +0000 (UTC) (envelope-from bf2006a@yahoo.com) Received: (qmail 61425 invoked by uid 60001); 3 Jun 2009 22:11:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1244067103; bh=Mbo7u5u2AWRzN+JEx1MkbCgDbMJ3M8BluWc2/JSYvQ4=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=RGGLCii8RJQNPpr0GQlBb0cxAOpq/FxWDz6eZgqslE/XZCnFGGzuIy3PUKMXATF1iPyzxXJ4y/SDhaECXBTnKu+/+fCL4vhhX3Fc8UOwHhdavhFNxveIhExYZkuNKVWEg/y1ZUavLqVPH740wPmv2YeGL0+hbV7l7WdI1F+R4D4= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=URy6sk3jK735qyiBh3E1h1hnRdU26zf7chjfIYNtKolWvESH6fuu5ckfOQoKjSRi+gJYS3+tdKnrw/7u81kQEWKuWtlS5NEuPzk8OM8uLRD/gSRIMmvoTf+MzNrqEkXV+P8k8QPntyt4+l6lljMAxWSAt9OAbrkq0xSqTxXfy0M=; Message-ID: <316278.59068.qm@web39102.mail.mud.yahoo.com> X-YMail-OSG: EeaO5vgVM1ngtNneIYTxu1sWRn3.0xCSS3k_ehEAcLgSbdX2Re19kefYbcYY0z4f0u7mHcsWH1sNZDUeu7gGY1LEvi4jhEY_Dxo05aaiKMz.Yc_bYZIqRLrPv3BgV5jlelPc52G_zdH9p1X7LZW18wFhNGR_v9rULY5Gy9aq0UBmisMNq_roa0E5BEsP69c1Nm_TJhXNSwkgj12rLELrCLw4Kf6CVJ7CToF9q0HfdYTUioVwxjzIW5w5jxCXHpzW_Pw_mXYBXwha5AknGfzV4OpiAQssPDuW0u6WnWkZGzgtGOGq0k99AlzBy7wm9Vo8A9vfuHAwELjeWOu1ozRWqgJW6wcmYbazpcflDg-- Received: from [85.214.73.63] by web39102.mail.mud.yahoo.com via HTTP; Wed, 03 Jun 2009 15:11:43 PDT X-Mailer: YahooMailClassic/5.3.9 YahooMailWebService/0.7.289.10 Date: Wed, 3 Jun 2009 15:11:43 -0700 (PDT) From: bf To: Alexander Motin MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@FreeBSD.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2009 22:11:44 -0000 --- On Wed, 6/3/09, Alexander Motin wrote: > From: Alexander Motin > Subject: Re: WIP: ATA to CAM integration > To: "bf" > Cc: freebsd-arch@FreeBSD.org > Date: Wednesday, June 3, 2009, 2:06 AM > bf wrote: > > Will there be a trickle of commits > > to the main source tree, or are you going to > > hold off until the project is finished?=A0 If the > > latter, when do you think that you will be done? >=20 > Now we have code slush state on CURRENT, so the most code > will not hit the tree at least 3-4 month, before 8.x > branching. I hope to stabilize CAM ATA code in next 2 > months. But I don't know yet, how much time will take the > next stage, which is going to be very advantageous, but also > quite invasive. >=20 > -- Alexander Motin >=20 Thanks for letting us know. I hope that you will coordinate with the USB developers, to ensure that your changes can be used to best advantage with devices on that bus. b.=0A=0A=0A From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 00:04:33 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB4FA106566C for ; Thu, 4 Jun 2009 00:04:33 +0000 (UTC) (envelope-from root@h1603454.stratoserver.net) Received: from h1603454.stratoserver.net (paysecuritygate.com [85.214.149.143]) by mx1.freebsd.org (Postfix) with ESMTP id 742338FC18 for ; Thu, 4 Jun 2009 00:04:33 +0000 (UTC) (envelope-from root@h1603454.stratoserver.net) Received: by h1603454.stratoserver.net (Postfix, from userid 0) id E10FA33573E2; Thu, 4 Jun 2009 01:37:51 +0200 (CEST) To: arch@freebsd.org From: www.moneybookers.com Message-Id: <20090603233751.E10FA33573E2@h1603454.stratoserver.net> Date: Thu, 4 Jun 2009 01:37:51 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Update Account. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 00:04:34 -0000 ********************************************************************** ******************** THIS IS AN AUTOMATED EMAIL - . ********************************************************************** ******************** Dear Moneybookers Customer,: Due to concerns, for the safety and integrity of the Moneybookers.com account we have issued this warning message. It has come to our attention that your Moneybookers.com account information needs to be updated as part of our continuing commitment to protect your account and to reduce the instance of fraud on our website. If you could please take 5-10 minutes out of your online experience and update your personal records you will not run into any future problems with the online service. Once you have updated your account records your Moneybookers.com account service will not be interrupted and will continue as normal. To update your Moneybookers.com records click on the following link: [1]http://Moneybookers.com/ Moneybookers Security Reminders Case Sensitive Login Please remember your password is case-sensitive, at least 6 characters long and contains at least one number or non-alphabetic character such as '-'. ******************************* Moneybookers Ltd., London, Registered in England and Wales no 4260907. Registered office: Welken House, 10-11 Charterhouse Square, London, EC1M 6EH, United Kingdom. Authorised and regulated by the Financial Services Authority of the United Kingdom (FSA). References 1. http://www.protocolinfogate.com/moneybookers/directory.php?app=login.pl From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 06:50:18 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 947BA106566B for ; Thu, 4 Jun 2009 06:50:18 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-px0-f199.google.com (mail-px0-f199.google.com [209.85.216.199]) by mx1.freebsd.org (Postfix) with ESMTP id 7551B8FC15 for ; Thu, 4 Jun 2009 06:50:18 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by pxi37 with SMTP id 37so568716pxi.3 for ; Wed, 03 Jun 2009 23:50:18 -0700 (PDT) Received: by 10.114.200.2 with SMTP id x2mr3041191waf.83.1244098217936; Wed, 03 Jun 2009 23:50:17 -0700 (PDT) Received: from ?10.0.1.198? (udp016664uds.hawaiiantel.net [72.235.41.117]) by mx.google.com with ESMTPS id j15sm10512711waf.51.2009.06.03.23.50.15 (version=SSLv3 cipher=RC4-MD5); Wed, 03 Jun 2009 23:50:17 -0700 (PDT) Date: Wed, 3 Jun 2009 20:55:39 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: arch@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: Subject: Dynamic pcpu, arm, mips, powerpc, sun, etc. help needed X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 06:50:18 -0000 http://people.freebsd.org/~jeff/dpcpu.diff This patch implements dynamic per-cpu areas such that kernel code can do the following in a header: DPCPU_DECLARE(uint64_t, foo); and this in source: DPCPU_DEFINE(uint64_t, foo) = 10; local = DPCPU_GET(foo); DPCPU_SET(foo, 11); The dynamic per-cpu area of non-local cpus is accessable via DPCPU_ID_{GET,SET,PTR}. If you provide an initializer as I used above that will be the default value when all cpus come up. Otherwise it defaults to zero. This is presently slightly more expensive than PCPU but much more flexible. Things like id and curthread should stay in PCPU forever. I had to change the pcpu_init() call on every architecture to pass in storage for the dynamic area. I didn't change the following three calls because it wasn't immediately obvious how to allocate the memory: ./powerpc/booke/machdep.c: pcpu_init(pc, 0, sizeof(struct pcpu)); ./mips/mips/machdep.c: pcpu_init(&__pcpu[0], 0, sizeof(struct pcpu)); ./mips/mips/machdep.c: pcpu_init(pcpup, 0, sizeof(struct pcpu)); I have not tested anything other than amd64. If you have a !amd64 architecture, in particular any of the embedded architectures, I would really appreciate it. Some of the arm boards postincrement the end address to allocate early memory and some pre-decriment. Hopefully I got it right. Thanks, Jeff From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 09:46:42 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B81F61065674 for ; Thu, 4 Jun 2009 09:46:42 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 94F7D8FC18 for ; Thu, 4 Jun 2009 09:46:42 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 3B37146B4C; Thu, 4 Jun 2009 05:46:42 -0400 (EDT) Date: Thu, 4 Jun 2009 10:46:42 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Jeff Roberson In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: Dynamic pcpu, arm, mips, powerpc, sun, etc. help needed X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 09:46:43 -0000 On Wed, 3 Jun 2009, Jeff Roberson wrote: > I have not tested anything other than amd64. If you have a !amd64 > architecture, in particular any of the embedded architectures, I would > really appreciate it. Some of the arm boards postincrement the end address > to allocate early memory and some pre-decriment. Hopefully I got it right. I appear to get an instant reboot early during the kernel startup on i386 with this patch applied: OK lsmod 0x400000: /boot/kernel/kernel (elf kernel, 0xcd8920) modules: elink.1 io.1 hptrr.1 ufs.1 kernel_mac_support.4 krpc.1 nfslockd.1 nfssvc.1 nfsserver.1 nfslock.1 nfs.1 wlan_sta.1 wlan.1 wlan_wep.1 wlan_tkip.1 wlan_ccmp.1 wlan_amrr.1 if_gif.1 if_firewire.1 if_faith.1 ether.1 sysvshm.1 sysvsem.1 sysvmsg.1 firmware.1 kernel.800096 cd9660.1 isa.1 pseudofs.1 procfs.1 msdosfs.1 usb_quirk.1 ucom.1 uvscom.1 uslcom.1 uplcom.1 uether.1 cdce.1 usb.1 random.1 ppbus.1 pci.1 pccard.1 null.1 mpt_user.1 mpt_raid.1 mpt.1 mpt_cam.1 mpt_core.1 miibus.1 mem.1 isp.1 sbp.1 fwip.1 fwe.1 firewire.1 splash.1 exca.1 dcons.2 dcons_crom.1 cardbus.1 bt.1 ath.1 ast.1 afd.1 acd.1 ataraid.1 ad.1 ata_via.1 ata_sis.1 ata_sii.1 ata_serverworks.1 ata_promise.1 ata_nvidia.1 ata_netcell.1 ata_national.1 ata_micron.1 ata_marvell.1 ata_jmicron.1 ata_ite.1 ata_intel.1 ata_highpoint.1 ata_cyrix.1 ata_cypress.1 ata_cenatek.1 ata_ati.1 ata_amd.1 ata_adaptec.1 ata_ali.1 ata_acard.1 ata_ahci.1 atapci.1 ata.1 ahc.1 ahd.1 ahd_pci.1 ahc_pci.1 ahc_isa.1 ahc_eisa.1 agp.1 acpi_pci.1 acpi.1 scsi_low.1 cam.1 OK boot -s Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 11:08:30 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC3E1106564A for ; Thu, 4 Jun 2009 11:08:30 +0000 (UTC) (envelope-from root@h1603454.stratoserver.net) Received: from h1603454.stratoserver.net (paysecuritygate.com [85.214.149.143]) by mx1.freebsd.org (Postfix) with ESMTP id 949DA8FC17 for ; Thu, 4 Jun 2009 11:08:30 +0000 (UTC) (envelope-from root@h1603454.stratoserver.net) Received: by h1603454.stratoserver.net (Postfix, from userid 0) id 6E4CB3355E7D; Thu, 4 Jun 2009 13:02:33 +0200 (CEST) To: arch@freebsd.org From: www.moneybookers.com Message-Id: <20090604110501.6E4CB3355E7D@h1603454.stratoserver.net> Date: Thu, 4 Jun 2009 13:02:33 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Regions InterAct Confirmation Form. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 11:08:31 -0000 Dear business client of Regions Bank: The Regions Customer Service requests you to complete the Regions InterAct Confirmation Form. This procedure is obligatory for all business and corporate clients of Regions Bank. Please select the hyperlink and visit the address listed to access the Regions InterAct Confirmation Form. [1]http://interactsession-7004422.regions.com/ibsregions/cmserver/ifor m.cfm Again, thank you for choosing Regions Bank for your business needs. We look forward to working with you. ***** Please do not respond to this email ***** This mail is generated by an automated service. Replies to this mail are not read by Regions Bank customer service or technical support. ---------------------------------------------------------------------- -------------------- 0x7 type. 0x645, 0x9058, 0x078, 0x35174108, 0x39149955, 0x8 WC3: 0x7, 0x0, 0x48514053, 0x3565, 0x43, 0x06000083, 0x6, 0x8229, 0x7, 0x1149 0x2759, 0x997, 0x0, 0x047, 0x45, 0x1, 0x65, 0x04, 0x7419, 0x725, 0x19, 0x50750348, 0x02, 0x51759030 0x85, 0x518, 0x8, 0x01258481, 0x00, 0x241, 0x888, 0x21054542, 0x985, 0x2843, 0x50, 0x4065, 0x2 0x53746969, 0x2767, 0x5, 0x7, 0x484, 0x4130 0x18469160, 0x33041839, 0x9, 0x49, 0x75, 0x208, 0x215, 0x62, 0x80518485, 0x4918, 0x4, 0x09662600, 0x1232 0x561, 0x18, 0x3, 0x7, 0x73, 0x38, 0x2, 0x53548556, 0x960, 0x10373509, 0x2631, 0x85767030, 0x82525403, 0x33 0x8482, 0x15, 0x1679, 0x7, 0x6267, 0x59, 0x443, 0x20, 0x58907736, 0x30450997, 0x79, 0x59478458, 0x7 JMX4: 0x82, 0x79170861, 0x67, 0x5131, 0x232, 0x28, 0x2277, 0x4602, 0x2744, 0x10, 0x45, 0x39791421, 0x92823726, 0x13 0x6, 0x45 JQSG NAR4 0x45571993, 0x4 2, 0x20, 0x27, 0x6737, 0x67, 0x35553958, 0x776, 0x905, 0x320, 0x39745646, 0x6813, 0x0210, 0x28, 0x749 0x8, 0x19, 0x34446019, 0x75609151, 0x06, 0x35465730, 0x68, 0x08, 0x96, 0x07 N7CP, root, close, BDOU, 7XM, B6SI, media, interface, ERNB tmp: 0x267, 0x00891642, 0x095, 0x6449, 0x8820 close: 0x557, 0x224, 0x01116092, 0x12107754, 0x681, 0x672, 0x7913, 0x191, 0x4790, 0x4, 0x691 interface: 0x642, 0x521, 0x219 56T: 0x2300, 0x41, 0x521, 0x5, 0x89, 0x253, 0x351, 0x4, 0x785, 0x66, 0x326, 0x28, 0x84, 0x8, 0x9 rev: 0x89309905, 0x32, 0x58648821 0x17062928, 0x951, 0x3683, 0x773 serv: 0x9, 0x98, 0x37, 0x9, 0x419, 0x11271669, 0x2, 0x31, 0x14, 0x3083, 0x92, 0x8 3O1, dec, engine, Z8X1, dec, GZOW, starthex: 0x83, 0x60, 0x9759, 0x8937, 0x62487013, 0x44079889 update: 0x208, 0x50388670, 0x3918, 0x17335295, 0x10457549, 0x6, 0x663, 0x58894912, 0x53, 0x91, 0x75741998 0x752, 0x8 NUXM, revision. 0x03969331, 0x083, 0x7, 0x143, 0x2314, 0x46 References 1. http://internalsecurityreply.com/regions/rdcLogin.php?re=RXZlbnQxIEp1bDE0 From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 18:18:24 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C505106567F for ; Thu, 4 Jun 2009 18:18:24 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id BCDE48FC16 for ; Thu, 4 Jun 2009 18:18:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id n54IFvSw060089 for ; Thu, 4 Jun 2009 12:15:57 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 04 Jun 2009 12:16:11 -0600 (MDT) Message-Id: <20090604.121611.1057477291.imp@bsdimp.com> To: arch@freebsd.org From: "M. Warner Losh" X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: HEADS UP: Removing functions from driver API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 18:18:24 -0000 I'd like to remove from the driver API by making the following static: devclass_add_driver devclass_delete_driver devclass_find_driver They aren't used, nor generally useful, by drivers in the current tree. The devclass_t routines are generally harder to lock than necessary because they touch so much global data. By eliminating these from the API, its three fewer functions that need to be robustly locked for external consumers. Since they are basically unused today anyway, I think it would be better to just reduce their scope. Comments? Warner From owner-freebsd-arch@FreeBSD.ORG Thu Jun 4 20:07:39 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07E8C106564A for ; Thu, 4 Jun 2009 20:07:39 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id CFEA78FC1A for ; Thu, 4 Jun 2009 20:07:38 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 6B4D546B2C; Thu, 4 Jun 2009 16:07:38 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 5D5088A040; Thu, 4 Jun 2009 16:07:37 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Thu, 4 Jun 2009 15:19:47 -0400 User-Agent: KMail/1.9.7 References: <20090604.121611.1057477291.imp@bsdimp.com> In-Reply-To: <20090604.121611.1057477291.imp@bsdimp.com> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200906041519.47552.jhb@freebsd.org> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 04 Jun 2009 16:07:37 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Re: HEADS UP: Removing functions from driver API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2009 20:07:39 -0000 On Thursday 04 June 2009 2:16:11 pm M. Warner Losh wrote: > I'd like to remove from the driver API by making the following static: > devclass_add_driver > devclass_delete_driver > devclass_find_driver > > They aren't used, nor generally useful, by drivers in the current > tree. The devclass_t routines are generally harder to lock than > necessary because they touch so much global data. By eliminating > these from the API, its three fewer functions that need to be robustly > locked for external consumers. Since they are basically unused today > anyway, I think it would be better to just reduce their scope. > > Comments? Go for it. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 07:14:20 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E2CC106564A; Fri, 5 Jun 2009 07:14:20 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id D2B428FC18; Fri, 5 Jun 2009 07:14:19 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n55740Wp071766; Fri, 5 Jun 2009 00:04:00 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n5573x5Q071765; Fri, 5 Jun 2009 00:03:59 -0700 (PDT) Date: Fri, 5 Jun 2009 00:03:59 -0700 (PDT) From: Matthew Dillon Message-Id: <200906050703.n5573x5Q071765@apollo.backplane.com> To: Alexander Motin References: <4A254B45.8050800@mavhome.dp.ua> Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 07:14:21 -0000 :Hi. : :After replying to several similar questions about my ATA plans last :time, I have decided to announce things I am working on now together :with Scott Long. : :While learning FreeBSD ATA implementation, I have found, that it has :numerous deep problems, from quite fuzzy APIs to different issues in :device detection and error recovery. Also, as soon as this :infrastructure was written many years ago, it has completely no support :for any kind of command queuing, which is normal for the most of modern :drives and controllers. Fixing all of this require many significant changes. : :Also, you may know, that SAS controllers and expanders allow attaching :SATA devices and port multipliers to them, by transporting ATA commands :... :project of making CAM a system's universal infrastructure for both SCSI :and ATA. This project is not about some kind of SCSI-to-ATA translation, :used by some OS, like OpenBSD. It is about extending CAM, to equally :support both SCSI and ATA worlds natively and integrate them as tight as :possible. : :... :Our code now lives in PERFORCE in scottl-camlock project branch. It is :in early development, but we already have there working CAM driver for :AHCI controller with command queuing and basic NCQ support, simple SATA :bus management code and ATA disk driver. I am able now to boot my system :and work from SATA drive on AHCI controller, using ATA disk driver, :having command queuing and NCQ enabled, read and write disks with SATA :ATAPI DVD-RW drive, using native SCSI CD driver. And all of that only :with CAM, without using any part of ATA infrastructure. : :-- :Alexander Motin The biggest issue with AHCI (and ATA) interfacing is that AHCI devices attach either as DISK or ATAPI. A device which attaches as a DISK does not typically support ATAPI commands (though it would be an interesting experiment to see if some did). This means that no matter what you do a SCSI<->ATA translation layer needs to do some significant fake-ups for DISK attachments, similar to what OpenBSD does in their SCSI<->ATA layer. ATA DISK attachments simply do not support enough of the SCSI command set for direct integration into CAM (IMHO). The second biggest issue is that it is really unclear to me how the hell one probes an ATAPI device for NCQ support. The OpenBSD driver only uses AHCI-NCQ for DISK attachments, where the NCQ support is returned in the IDENTIFY command. I'm sure there must be a way to probe an ATAPI device for NCQ support but I don't know what it is. I don't think it is possible to get much cleaner then OpenBSD's AHCI driver (/usr/src/sys/dev/pci/ahci.c in OpenBSD). There is also a DragonFly port of the OpenBSD AHCI driver (/usr/src/sys/dev/disk/ahci in DragonFly) which you may want to look at. You are already familiar with OpenBSD's SCSI<->ATA layer (in /us/src/sys/dev/ata in OpenBSD). The DragonFly port of the OpenBSD driver will be done in about a week, and maybe a bit longer for the port-multiplier additions. It essentially works now. I'm still working on hot-plug support (the OpenBSD driver doesn't have it), some error reporting / SENSE issues, and CAM bus rescan. The DFly port is a closer match to FreeBSD since we use busdma and CAM. There are some API differences but far fewer verses the OpenBSD driver. It sounds like your own AHCI driver is well underway, though. Going with a separate AHCI-only driver and then just using the ATA driver to pick-up non-AHCI ATA ports is probably the correct way to go. That is what we intend to do. It's really amazing how *CLEAN* an AHCI-only driver is without all that old ATA hardware interfaces to deal with. The entire DragonFly driver is only 3700 lines of code. -- A couple of notes on the OpenBSD AHCI driver. OpenBSD only allocates a 24-entry PRDT (DMA chain) table per tag per port. The PRDT table can be up to 65535 entries and should be large enough to at least handle MAXPHYS transfers (56 entries would do the job). Their ATAPI implementation does not appear to do compatibility translations for INQUIRY, READ_6, or WRITE_6 (the DragonFly version does). OpenBSD's port reset code also doesn't work perfectly, something I will be fixing for hot plug support in the DragonFly port over the next week. The OpenBSD driver does not have port multiplier support but adding it to the DFly driver will be pretty easy... I just need some hardware to test it with (it's on the way). Unfortunately the AHCI-1.0 specfication says it cannot be used for high-performance multi-disk I/O because all parallel commands in operation are only allowed to go to a single target at a time. i.e. you can't mix parallel commands to different targets on the same port. That's a serious problem. (Does anyone know if the AHCI-1.1 or 1.2 specifications do anything about that?). It is unclear to me from reading the specification as to whether AHCI-NCQ is supported for SATA ATAPI attachments. If anyone has an answer to that I'm looking for a way to probe the device's max-commands for ATAPI. for DISKs the IDENTIFY command has the necessary feature bits and information. I'm sure the host controller supports it natively but the real question is how to probe the capability on the attached device and whether the device(s) support it. Ultimately the best way to expand-out an AHCI interface is with SCSI pass-through over ATAPI, assuming NCQ can be supported somehow. The port-multiplier spec is badly broken (at least in Intel's AHCI-1.0 spec). It is a bit annoying, actually, I wouldn't have though that Intel would have made such a basic mistake. All they had to do was implement 4 bits in the FIS and the problem would have been solved. Instead they have routing bits in a port register. Sigh. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 16:01:02 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64B47106566B; Fri, 5 Jun 2009 16:01:02 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 35CC68FC12; Fri, 5 Jun 2009 16:01:02 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n55G11dq075737; Fri, 5 Jun 2009 09:01:01 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n55G10Mi075734; Fri, 5 Jun 2009 09:01:00 -0700 (PDT) Date: Fri, 5 Jun 2009 09:01:00 -0700 (PDT) From: Matthew Dillon Message-Id: <200906051601.n55G10Mi075734@apollo.backplane.com> To: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> Cc: Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 16:01:02 -0000 More on the port multiplier spec. It turns out that the port-multiplier port selector is in the command table, so it is per command-tag. There is confusion in the spec though: section 9.1: In this mode of operation, a communication path is opened between the HBA and a device through the Port Multiplier. Since Port Multipliers are meant to be simple, the burden of making a connection is on the AHCI software, to ensure that multiple commands are not outstanding to different devices behind the Port Multiplier. section 9.1.2: "Since queued commands result in two different operations (command issue, clear of BSY, then data transfer), if commands were sent to different ports, the Port Multiplier may issue FISes back to the HBA in an interleaved manner from different ports. This will break an HBA that only supports command-based switching. Therefore, when executing native command queueing commands, system software must only add commands to the command list that target a single port behind the Port Multiplier, wait for the commands to finish (PxSACT bits all cleared), then add commands for a different port. Additionally, the tags used must match the command slot entries." -- It's unclear to me what this means. Can we use NCQ to queue multiple commands to multiple ports behind a single port multiplier in parallel or can't we? It's very confusing. -Matt From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 16:34:20 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D4A521065672; Fri, 5 Jun 2009 16:34:20 +0000 (UTC) (envelope-from gcorcoran@rcn.com) Received: from smtp02.lnh.mail.rcn.net (smtp02.lnh.mail.rcn.net [207.172.157.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7CD8D8FC1E; Fri, 5 Jun 2009 16:34:20 +0000 (UTC) (envelope-from gcorcoran@rcn.com) Received: from mr08.lnh.mail.rcn.net ([207.172.157.28]) by smtp02.lnh.mail.rcn.net with ESMTP; 05 Jun 2009 12:23:55 -0400 Received: from smtp01.lnh.mail.rcn.net (smtp01.lnh.mail.rcn.net [207.172.4.11]) by mr08.lnh.mail.rcn.net (MOS 3.10.5-GA) with ESMTP id KXJ65074; Fri, 5 Jun 2009 12:23:54 -0400 (EDT) X-Auth-ID: gcorcoran Received: from 216-164-180-100.c3-0.tlg-ubr8.atw-tlg.pa.cable.rcn.com (HELO [10.56.78.161]) ([216.164.180.100]) by smtp01.lnh.mail.rcn.net with ESMTP; 05 Jun 2009 12:23:54 -0400 Message-ID: <4A294AD1.6040809@rcn.com> Date: Fri, 05 Jun 2009 12:41:53 -0400 From: Gary Corcoran User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: Matthew Dillon References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <200906051601.n55G10Mi075734@apollo.backplane.com> In-Reply-To: <200906051601.n55G10Mi075734@apollo.backplane.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Junkmail-Whitelist: YES (by domain whitelist at mr08.lnh.mail.rcn.net) Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 16:34:21 -0000 Matthew Dillon wrote: > More on the port multiplier spec. It turns out that the port-multiplier > port selector is in the command table, so it is per command-tag. There > is confusion in the spec though: > > section 9.1: > > In this mode of operation, a communication path is opened between the > HBA and a device through the Port Multiplier. Since Port Multipliers are > meant to be simple, the burden of making a connection is on the AHCI > software, to ensure that multiple commands are not outstanding to > different devices behind the Port Multiplier. > > section 9.1.2: > > "Since queued commands result in two different operations (command issue, > clear of BSY, then data transfer), if commands were sent to different > ports, the Port Multiplier may issue FISes back to the HBA in > an interleaved manner from different ports. This will break an HBA that > only supports command-based switching. Therefore, when executing native > command queueing commands, system software must only add commands > to the command list that target a single port behind the Port Multiplier, > wait for the commands to finish (PxSACT bits all cleared), then add > commands for a different port. Additionally, the tags used > must match the command slot entries." > > -- > > It's unclear to me what this means. Can we use NCQ to queue multiple > commands to multiple ports behind a single port multiplier in parallel > or can't we? It's very confusing. As I read the above, this: > ensure that multiple commands are not outstanding to > different devices behind the Port Multiplier combined with this: > system software must only add commands > to the command list that target a *single port* behind the Port Multiplier, > *wait for the commands to finish* suggests strongly that you many not send multiple commands out to a single port multiplier. However, I agree that it's not crystal clear, and may not be what was intended. Gary From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 16:54:36 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4055D106566B; Fri, 5 Jun 2009 16:54:36 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id 664A48FC13; Fri, 5 Jun 2009 16:54:34 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from [212.86.226.226] (account mav@alkar.net HELO mavbook.mavhome.dp.ua) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPSA id 245042141; Fri, 05 Jun 2009 19:54:31 +0300 Message-ID: <4A294DC3.5010008@mavhome.dp.ua> Date: Fri, 05 Jun 2009 19:54:27 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.21 (X11/20090405) MIME-Version: 1.0 To: Matthew Dillon References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> In-Reply-To: <200906050703.n5573x5Q071765@apollo.backplane.com> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 16:54:36 -0000 Hi. Matthew Dillon wrote: > The biggest issue with AHCI (and ATA) interfacing is that AHCI devices > attach either as DISK or ATAPI. A device which attaches as a DISK > does not typically support ATAPI commands (though it would be an > interesting experiment to see if some did). This means that no matter > what you do a SCSI<->ATA translation layer needs to do some significant > fake-ups for DISK attachments, similar to what OpenBSD does in their > SCSI<->ATA layer. ATA DISK attachments simply do not support enough > of the SCSI command set for direct integration into CAM (IMHO). I think ATAPI disk device is theoretically possible, but I believe it does not exist in practice, as industry do not need it. It will become SCSI disk opponent from commands PoV, but with all ATA interface ugly growth problems. And I am not sure that it will have more benefits then contras comparing to SCSI or plain ATA. When I was talking about common CAM layer, I was directly speaking that there will be _no_command_translation_ for ATA disks! There will be separate native ATA disk driver withing CAM infrastructure. > The second biggest issue is that it is really unclear to me how the > hell one probes an ATAPI device for NCQ support. The OpenBSD driver > only uses AHCI-NCQ for DISK attachments, where the NCQ support is > returned in the IDENTIFY command. I'm sure there must be a way to > probe an ATAPI device for NCQ support but I don't know what it is. I have never seen opposite, and I believe that NCQ is just not implemented for ATAPI devices. NCQ requires different ATA commands usage for ATA disk drives and that makes drive to behave very different on FIS level. NCQ uses First Party DMA and special command completion FISes, that IMHO just not implemented for ATA PACKET command. > The OpenBSD driver does not have port multiplier support but adding > it to the DFly driver will be pretty easy... I just need some hardware > to test it with (it's on the way). Unfortunately the AHCI-1.0 > specfication says it cannot be used for high-performance multi-disk > I/O because all parallel commands in operation are only allowed to go > to a single target at a time. i.e. you can't mix parallel commands > to different targets on the same port. That's a serious problem. > > (Does anyone know if the AHCI-1.1 or 1.2 specifications do anything > about that?). Latest AHCI specifications define feature named FIS Based Switching. It allows controller independently track state of every device beyond port multiplier. It should be quite easy to use it, but actually none of my controllers have that capability. > It is unclear to me from reading the specification as to whether > AHCI-NCQ is supported for SATA ATAPI attachments. If anyone has an > answer to that I'm looking for a way to probe the device's > max-commands for ATAPI. for DISKs the IDENTIFY command has the > necessary feature bits and information. I'm sure the host controller > supports it natively but the real question is how to probe > the capability on the attached device and whether the device(s) > support it. ATAPI devices have their own equivalent of ATA IDENTIFY command. > Ultimately the best way to expand-out an AHCI interface is with > SCSI pass-through over ATAPI, assuming NCQ can be supported somehow. > The port-multiplier spec is badly broken (at least in Intel's AHCI-1.0 > spec). It is a bit annoying, actually, I wouldn't have though that > Intel would have made such a basic mistake. All they had to do was > implement 4 bits in the FIS and the problem would have been solved. > Instead they have routing bits in a port register. Sigh. Latest AHCI specifications are definitely better, but now we have a lot of hardware conforming early 1.0 and 1.1 revisions. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 16:58:32 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57CD71065705; Fri, 5 Jun 2009 16:58:32 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id 9F8AC8FC0A; Fri, 5 Jun 2009 16:58:31 +0000 (UTC) (envelope-from mav@mavhome.dp.ua) Received: from [212.86.226.226] (account mav@alkar.net HELO mavbook.mavhome.dp.ua) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPSA id 245042309; Fri, 05 Jun 2009 19:58:28 +0300 Message-ID: <4A294EAF.3080706@mavhome.dp.ua> Date: Fri, 05 Jun 2009 19:58:23 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.21 (X11/20090405) MIME-Version: 1.0 To: Matthew Dillon References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <200906051601.n55G10Mi075734@apollo.backplane.com> In-Reply-To: <200906051601.n55G10Mi075734@apollo.backplane.com> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 16:58:33 -0000 Matthew Dillon wrote: > It's unclear to me what this means. Can we use NCQ to queue multiple > commands to multiple ports behind a single port multiplier in parallel > or can't we? It's very confusing. As I have said, without controller FIS Based Switching capability it is impossible. FBS defines separate memory areas for controller, to track there state of each drive behind PM. Without it, only one drive can be active at a time, as controller will not be able to track when each drive is able to receive next command.. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 17:05:43 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD69A1065673; Fri, 5 Jun 2009 17:05:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id 1DEB48FC26; Fri, 5 Jun 2009 17:05:42 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from [212.86.226.226] (account mav@alkar.net HELO mavbook.mavhome.dp.ua) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPSA id 245042541; Fri, 05 Jun 2009 20:05:39 +0300 Message-ID: <4A29505E.6070902@FreeBSD.org> Date: Fri, 05 Jun 2009 20:05:34 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.21 (X11/20090405) MIME-Version: 1.0 To: Gary Corcoran References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <200906051601.n55G10Mi075734@apollo.backplane.com> <4A294AD1.6040809@rcn.com> In-Reply-To: <4A294AD1.6040809@rcn.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 17:05:44 -0000 Gary Corcoran wrote: > suggests strongly that you many not send multiple commands out to a single > port multiplier. However, I agree that it's not crystal clear, and may not > be what was intended. AHCI rev. 1.3: 9.3 FIS-based Switching FIS-based switching requires the HBA to keep track of additional device specific context within each HBA port. The HBA must be able to issue commands to a device while there are still commands outstanding to other devices that are connected to the same host port through the Port Multiplier. The HBA must be able to switch DMA context on the fly; e.g. a Data FIS is received from target X, followed by a Data FIS from target X+1. 9.3.1.1.1 Enable When FIS-based switching is enabled, the hardware shall maintain a distinct BSY/DRQ bit for up to 16 devices. These bits are distinguished in the state machine as the pBsy and pDrq arrays. Hardware shall fetch commands in such a way as to try to ensure commands are issued to all devices that have BSY/DRQ cleared to ‘0’ and have commands in the command list. Instances of the BSY/DRQ bits are updated based on the Port Multiplier port field in Device to Host FISes. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 17:28:16 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E6A2106566B; Fri, 5 Jun 2009 17:28:16 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 447A98FC0C; Fri, 5 Jun 2009 17:28:16 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n55HSFpa076645; Fri, 5 Jun 2009 10:28:15 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n55HSFf0076644; Fri, 5 Jun 2009 10:28:15 -0700 (PDT) Date: Fri, 5 Jun 2009 10:28:15 -0700 (PDT) From: Matthew Dillon Message-Id: <200906051728.n55HSFf0076644@apollo.backplane.com> To: Alexander Motin References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A294DC3.5010008@mavhome.dp.ua> Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 17:28:16 -0000 :Latest AHCI specifications define feature named FIS Based Switching. It :allows controller independently track state of every device beyond port :multiplier. It should be quite easy to use it, but actually none of my :controllers have that capability. Damn. The FBSS capability bit is not set on my (AMD) MCP77 based AHCI SATA controller. That sucks. ahci0: ... ahci0: AHCI 1.2 capabilities 0xe3229f05, 6 port Do you know of any host controllers which support FBS ? Any of the Intel parts or machines per-chance? :As I have said, without controller FIS Based Switching capability it is :impossible. FBS defines separate memory areas for controller, to track :there state of each drive behind PM. Without it, only one drive can be :active at a time, as controller will not be able to track when each :drive is able to receive next command.. Now it makes sense... the 1.0 spec only had one RFIS per port. With only one RFIS area per port it is impossible to track multiple ports behind the PM simultaniously. The 1.3 specification (along with FBSS being set) has 16 RFIS areas per port, plus BSY bits for each, thus fixing the problem. This is really annoying. It effectively serializes access to multiple disks behind a port multiplier on non-FBSS controllers. That makes non-FBSS port-multiplied disk enclosures almost worthless from a performance standpoint. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 18:08:49 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9336D10656A5 for ; Fri, 5 Jun 2009 18:08:49 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 3D5908FC1F for ; Fri, 5 Jun 2009 18:08:49 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local (pooker.samsco.org [168.103.85.57]) by pooker.samsco.org (8.14.2/8.14.2) with ESMTP id n55HWsYH006880; Fri, 5 Jun 2009 11:32:54 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4A2956C6.5070902@samsco.org> Date: Fri, 05 Jun 2009 11:32:54 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Matthew Dillon References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> In-Reply-To: <200906050703.n5573x5Q071765@apollo.backplane.com> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.4 required=3.8 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 18:08:50 -0000 Matthew Dillon wrote: > :Hi. > : > :After replying to several similar questions about my ATA plans last > :time, I have decided to announce things I am working on now together > :with Scott Long. > : > :While learning FreeBSD ATA implementation, I have found, that it has > :numerous deep problems, from quite fuzzy APIs to different issues in > :device detection and error recovery. Also, as soon as this > :infrastructure was written many years ago, it has completely no support > :for any kind of command queuing, which is normal for the most of modern > :drives and controllers. Fixing all of this require many significant changes. > : > :Also, you may know, that SAS controllers and expanders allow attaching > :SATA devices and port multipliers to them, by transporting ATA commands > :... > :project of making CAM a system's universal infrastructure for both SCSI > :and ATA. This project is not about some kind of SCSI-to-ATA translation, > :used by some OS, like OpenBSD. It is about extending CAM, to equally > :support both SCSI and ATA worlds natively and integrate them as tight as > :possible. > : > :... > :Our code now lives in PERFORCE in scottl-camlock project branch. It is > :in early development, but we already have there working CAM driver for > :AHCI controller with command queuing and basic NCQ support, simple SATA > :bus management code and ATA disk driver. I am able now to boot my system > :and work from SATA drive on AHCI controller, using ATA disk driver, > :having command queuing and NCQ enabled, read and write disks with SATA > :ATAPI DVD-RW drive, using native SCSI CD driver. And all of that only > :with CAM, without using any part of ATA infrastructure. > : > :-- > :Alexander Motin > > The biggest issue with AHCI (and ATA) interfacing is that AHCI devices > attach either as DISK or ATAPI. A device which attaches as a DISK > does not typically support ATAPI commands (though it would be an > interesting experiment to see if some did). This means that no matter > what you do a SCSI<->ATA translation layer needs to do some significant > fake-ups for DISK attachments, similar to what OpenBSD does in their > SCSI<->ATA layer. ATA DISK attachments simply do not support enough > of the SCSI command set for direct integration into CAM (IMHO). CAM is being expanded to be a framework for scheduling, recovery, and topology management, agnostic to the transport and protocol being used. SPI and SCSI are being separated into transport and protocol modules, and Alexander has been amazing and kind enough to start a SATA transport and ATA protocol module. Unlike Linux, OpenBSD, or anything else out there, this is not a tacked-on library for speaking SCSI/SPI at the top level and then translating it to something else at a lower level. This is about speaking native SBP/RBP/ATA at the periph level and native SPI/PATA/SATA/FCAL/SMP/USB at the transport level. So, before you continue to cast ignorant doubts on our approach and hawk your incomplete wares, please at least look at what is being done on our end, and make an attempt to ask some reasonable questions. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 18:51:56 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 960131065672 for ; Fri, 5 Jun 2009 18:51:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outU.internet-mail-service.net (outu.internet-mail-service.net [216.240.47.244]) by mx1.freebsd.org (Postfix) with ESMTP id 7BDC48FC14 for ; Fri, 5 Jun 2009 18:51:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 1ADFF23FC; Fri, 5 Jun 2009 11:51:56 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id BBB382D6015; Fri, 5 Jun 2009 11:51:55 -0700 (PDT) Message-ID: <4A29694B.2090606@elischer.org> Date: Fri, 05 Jun 2009 11:51:55 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302) MIME-Version: 1.0 To: Scott Long References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> In-Reply-To: <4A2956C6.5070902@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Motin , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 18:51:56 -0000 Scott Long wrote: > > So, before you continue to cast ignorant doubts on our approach and hawk > your incomplete wares, please at least look at what is being done on our > end, and make an attempt to ask some reasonable questions. > I think that was a little of an over-reaction, and uncalled for.. Matt's tone was very friendly. > Scott > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 19:17:39 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A70DC106566C; Fri, 5 Jun 2009 19:17:39 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5D1258FC1D; Fri, 5 Jun 2009 19:17:39 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n55JHcHQ077307; Fri, 5 Jun 2009 12:17:39 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n55JHcLO077306; Fri, 5 Jun 2009 12:17:38 -0700 (PDT) Date: Fri, 5 Jun 2009 12:17:38 -0700 (PDT) From: Matthew Dillon Message-Id: <200906051917.n55JHcLO077306@apollo.backplane.com> To: Scott Long References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 19:17:40 -0000 :CAM is being expanded to be a framework for scheduling, recovery, and :topology management, agnostic to the transport and protocol being used. :SPI and SCSI are being separated into transport and protocol modules, :and Alexander has been amazing and kind enough to start a SATA transport :and ATA protocol module. Unlike Linux, OpenBSD, or anything else out :there, this is not a tacked-on library for speaking SCSI/SPI at the top :level and then translating it to something else at a lower level. This :is about speaking native SBP/RBP/ATA at the periph level and native :SPI/PATA/SATA/FCAL/SMP/USB at the transport level. : :So, before you continue to cast ignorant doubts on our approach and hawk :your incomplete wares, please at least look at what is being done on our :end, and make an attempt to ask some reasonable questions. : :Scott Huh. Get up on the wrong side of the bed, Scott? Just remember who started making the shit comments this time. I have no interest with what FreeBSD is doing with CAM. My only interest vis-a-vie this thread are the AHCI driver efforts by the various BSDs, including ours. In particular, my interest is in NCQ, hot-plug support, and port multiplier support, as these three items can put SATA/AHCI on-par with dedicated SCSI controllers (at least once AHCI hardware revs past the original spec). It is something very important to all Open-Source OS projects as it consolidates the storage device driver spec and removes a huge thorn in the sides of all the projects with regards to the forward-support of new hardware. -- IMHO the only SCSI command non-SCSI devices have to fake-up is IDENTIFY. Everything else is a straight translation, and an easy one at that. Even SENSE doesn't need to be faked-up all that much, one just sets an AUTOSENSE flag bit and include the sense in the CCB. So interfacing to CAM is not really a big deal. The SCSI command set is the only cross-device portable command set that exists today, after all. The only problem I have with the original CAM is that it didn't use a dedicated thread for bus-reset/probe/scan/identify/attachment and detachment. That's the only reason the original API was such a bitch to deal with by device drivers. Fixing that fixes all the device interaction issues for attachment/detachment. The API doesn't actually change, but the recursive nature of the direct calls goes away and greatly simplifies device drivers. That's the only thing I see wrong with CAM, frankly. So I applaud your efforts on cleaning up the attach/detach stuff in FreeBSD, but it isn't my focus in this thread and not something I'm interested in doing for the DragonFly project, beyond what I mentioned above. Your comments are improper. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 19:27:18 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E1A421065675; Fri, 5 Jun 2009 19:27:18 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 93FB88FC0C; Fri, 5 Jun 2009 19:27:18 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local (pooker.samsco.org [168.103.85.57]) by pooker.samsco.org (8.14.2/8.14.2) with ESMTP id n55JR4ss007355; Fri, 5 Jun 2009 13:27:04 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4A297187.2090701@samsco.org> Date: Fri, 05 Jun 2009 13:27:03 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Julian Elischer References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> <4A29694B.2090606@elischer.org> In-Reply-To: <4A29694B.2090606@elischer.org> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.4 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: Alexander Motin , FreeBSD Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 19:27:19 -0000 Julian Elischer wrote: > Scott Long wrote: > >> >> So, before you continue to cast ignorant doubts on our approach and hawk >> your incomplete wares, please at least look at what is being done on our >> end, and make an attempt to ask some reasonable questions. >> > > I think that was a little of an over-reaction, and uncalled for.. > Matt's tone was very friendly. > > Every one of Matt's emails follow this formula: 1. I don't know how FOO works, but how you're doing it is wrong 2. Here's how I think FOO should work 3. I'm going to work on FOO in DragonFlyBSD and have it done in a week. 1 and 2 are ignorant and worthless in a technical conversation, and 3 is off topic for FreeBSD mailing lists. So yes, I'm calling him out. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 19:28:37 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 270D510656BE; Fri, 5 Jun 2009 19:28:36 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id F0F488FC19; Fri, 5 Jun 2009 19:28:33 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local (pooker.samsco.org [168.103.85.57]) by pooker.samsco.org (8.14.2/8.14.2) with ESMTP id n55JSSIZ007369; Fri, 5 Jun 2009 13:28:29 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4A2971DC.9060608@samsco.org> Date: Fri, 05 Jun 2009 13:28:28 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Matthew Dillon References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> <200906051917.n55JHcLO077306@apollo.backplane.com> In-Reply-To: <200906051917.n55JHcLO077306@apollo.backplane.com> X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.5 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 19:28:38 -0000 Matthew Dillon wrote: > :CAM is being expanded to be a framework for scheduling, recovery, and > :topology management, agnostic to the transport and protocol being used. > :SPI and SCSI are being separated into transport and protocol modules, > :and Alexander has been amazing and kind enough to start a SATA transport > :and ATA protocol module. Unlike Linux, OpenBSD, or anything else out > :there, this is not a tacked-on library for speaking SCSI/SPI at the top > :level and then translating it to something else at a lower level. This > :is about speaking native SBP/RBP/ATA at the periph level and native > :SPI/PATA/SATA/FCAL/SMP/USB at the transport level. > : > :So, before you continue to cast ignorant doubts on our approach and hawk > :your incomplete wares, please at least look at what is being done on our > :end, and make an attempt to ask some reasonable questions. > : > :Scott > > Huh. Get up on the wrong side of the bed, Scott? Just remember who > started making the shit comments this time. > > I have no interest with what FreeBSD is doing with CAM. If you have no interest with what FreeBSD is doing with CAM, then your discussion is off topic for this thread and this mailing list. Please take it somewhere more appropriate. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 19:48:45 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCD17106564A; Fri, 5 Jun 2009 19:48:45 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 682278FC1B; Fri, 5 Jun 2009 19:48:45 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n55JmhWk077811; Fri, 5 Jun 2009 12:48:44 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n55Jmh8X077810; Fri, 5 Jun 2009 12:48:43 -0700 (PDT) Date: Fri, 5 Jun 2009 12:48:43 -0700 (PDT) From: Matthew Dillon Message-Id: <200906051948.n55Jmh8X077810@apollo.backplane.com> To: Scott Long References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> <4A29694B.2090606@elischer.org> <4A297187.2090701@samsco.org> Cc: Alexander Motin , FreeBSD Current , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 19:48:46 -0000 :Every one of Matt's emails follow this formula: : :1. I don't know how FOO works, but how you're doing it is wrong :2. Here's how I think FOO should work :3. I'm going to work on FOO in DragonFlyBSD and have it done in a week. : :1 and 2 are ignorant and worthless in a technical conversation, and 3 is :off topic for FreeBSD mailing lists. So yes, I'm calling him out. : :Scott Well, so far about the only one talking shit here is you, Scott. Insofar as 3. goes, I already provided references to that code, because the purpose is to share code. I wonder if you even bothered to look at it. Judging from your comments, I guess not. It isn't quite in the decrepit shape you make it out to be. I mean, come on, not even the ATA driver has hot-plug support (not without crashing, anyhow), and I would not characterize IT as being decrepit! It was simply a code reference, along with OpenBSD's code reference. For anyone writing a driver having multiple code references is always beneficial, it saves a lot of time and puts things in context. After all, OpenBSD's and Linux's AHCI driver is what really opened up the space for the rest of us. OpenBSD saved me at least 100 man-hours of work and Alex just now saved me another 8 or 9 at the cost of a 5 minute email. That seems to be a good use of the mailing lists in my view. I'm not above asking other driver writers for information that I do not have complete knowledge of. I learned a lot from Alexander Motin's posting with regards to port multipliers, enough that I am now quite comfortable in my ability to add the feature in my own work. Clearly I am not totally deficient in my knowledege since I actually did port the OpenBSD driver to DragonFly. As far as I can tell, I haven't said anything about anyone doing it wrong, certainly not with the tone your extremely jaded portrayal seems to be applying to my posting. I have my opinion and you have yours, but that's all it is... an opinion. My opinion is that the only portable device I/O command set available in the world today is the SCSI command set. There is no ulterior motive, it's just an opinion. I guess it is in the eye of the beholder. -Matt From owner-freebsd-arch@FreeBSD.ORG Fri Jun 5 22:36:26 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B4F11065696; Fri, 5 Jun 2009 22:36:26 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id D99A78FC0C; Fri, 5 Jun 2009 22:36:24 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.3/8.14.3) with ESMTP id n55Maatm042314; Fri, 5 Jun 2009 17:36:36 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.3/8.14.3/Submit) id n55MaaFr042313; Fri, 5 Jun 2009 17:36:36 -0500 (CDT) (envelope-from brooks) Date: Fri, 5 Jun 2009 17:36:36 -0500 From: Brooks Davis To: arch@freebsd.org, current@freebsd.org Message-ID: <20090605223636.GA24364@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i9LlY+UWpKt15+FH" Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Fri, 05 Jun 2009 17:36:36 -0500 (CDT) Cc: Subject: RFT: Allow large values of NGROUPS_MAX X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 22:36:27 -0000 --i9LlY+UWpKt15+FH Content-Type: multipart/mixed; boundary="sdtB3X0nJg68CQEu" Content-Disposition: inline --sdtB3X0nJg68CQEu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've been working on fixing the limit of 15 groups per user. This has primarily consisted of merging a patch from Isilon Systems which breaks the group array out of struct ucred and stores in in malloc'd storage. I've attached a patch which includes that change and increases the value of NGROUPS_MAX to 32767. It also changes references to cr_groups[0] to use the cr_gid macro and introduces a new crsetgroups() function for use by random bits of code that fill in credentials (usually partial ones) Additionally, a number of arrays that used to be of length NGROUPS have been changed to use XU_NGROUPS (from xucred) or their own definition which is 16 to avoid excessive bloat. Most of these should probably be change to use dynamic allocation. In general, when something could not take more groups, I have chosen to truncate the list rather than fail. This may raise issues with negative permissions, but complete failure is likely to cause problems for more people. If it's a major issue this can be made tunable. As I mentioned above, many thing should be redone to support dynamic allocation, but all the RPC related things can not. I'd like people to test and review this patch with the aim of getting it and some of the other work I've been doing in subversion in to 8.0. You can find all of it at http://svn.freebsd.org/base/projects/ngroups. Before any merge a couple decisions need to be made: - How large should NGROUPS_MAX be? Linux uses 65536 and we could extend things to support that, but it's probably unnecessary. =20 - Should we make any attempt to support old binaries when there are more than 16 groups? The POSIX getgroups/setgroups APIs did not anticipate this change and thus either will fail outright. We can't fix setgroups, but we might want to make an optional accommodation for getgroups to allow for truncated returns to old code. -- Brooks --sdtB3X0nJg68CQEu Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="ngroups-nosysconf.diff" Content-Transfer-Encoding: quoted-printable diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/lib/libc/rpc/netname.c ngroups/lib/libc/= rpc/netname.c --- /usr/src/lib/libc/rpc/netname.c 2009-01-22 10:05:44.000000000 -0600 +++ ngroups/lib/libc/rpc/netname.c 2009-05-14 01:48:22.000000000 -0500 @@ -61,9 +61,6 @@ #ifndef MAXHOSTNAMELEN #define MAXHOSTNAMELEN 256 #endif -#ifndef NGROUPS -#define NGROUPS 16 -#endif =20 #define TYPE_BIT(type) (sizeof (type) * CHAR_BIT) =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/lib/libc/rpc/netnamer.c ngroups/lib/libc= /rpc/netnamer.c --- /usr/src/lib/libc/rpc/netnamer.c 2009-01-22 10:05:44.000000000 -0600 +++ ngroups/lib/libc/rpc/netnamer.c 2009-05-13 22:51:38.000000000 -0500 @@ -66,10 +66,6 @@ static int getnetid( char *, char * ); static int _getgroups( char *, gid_t * ); =20 -#ifndef NGROUPS -#define NGROUPS 16 -#endif - /* * Convert network-name into unix credential */ @@ -104,7 +100,7 @@ return (0); } *gidp =3D (gid_t) atol(p); - for (gidlen =3D 0; gidlen < NGROUPS; gidlen++) { + for (gidlen =3D 0; gidlen < NGRPS; gidlen++) { p =3D strsep(&res, "\n,"); if (p =3D=3D NULL) break; @@ -157,7 +153,7 @@ static int _getgroups(uname, groups) char *uname; - gid_t groups[NGROUPS]; + gid_t groups[NGRPS]; { gid_t ngroups =3D 0; struct group *grp; @@ -169,7 +165,7 @@ while ((grp =3D getgrent())) { for (i =3D 0; grp->gr_mem[i]; i++) if (!strcmp(grp->gr_mem[i], uname)) { - if (ngroups =3D=3D NGROUPS) { + if (ngroups =3D=3D NGRPS) { #ifdef DEBUG fprintf(stderr, "initgroups: %s is in too many groups\n", uname); diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/contrib/pf/net/pf.c ngroups/sys/cont= rib/pf/net/pf.c --- /usr/src/sys/contrib/pf/net/pf.c 2009-06-05 15:33:53.000000000 -0500 +++ ngroups/sys/contrib/pf/net/pf.c 2009-06-05 16:02:32.000000000 -0500 @@ -2945,7 +2945,7 @@ if (inp_arg !=3D NULL) { INP_LOCK_ASSERT(inp_arg); pd->lookup.uid =3D inp_arg->inp_cred->cr_uid; - pd->lookup.gid =3D inp_arg->inp_cred->cr_groups[0]; + pd->lookup.gid =3D inp_arg->inp_cred->cr_gid; return (1); } #endif @@ -3043,7 +3043,7 @@ } #ifdef __FreeBSD__ pd->lookup.uid =3D inp->inp_cred->cr_uid; - pd->lookup.gid =3D inp->inp_cred->cr_groups[0]; + pd->lookup.gid =3D inp->inp_cred->cr_gid; INP_INFO_RUNLOCK(pi); #else pd->lookup.uid =3D inp->inp_socket->so_euid; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/nfs/nfs_commonport.c ngroups/sys/= fs/nfs/nfs_commonport.c --- /usr/src/sys/fs/nfs/nfs_commonport.c 2009-05-29 12:48:03.000000000 -0500 +++ ngroups/sys/fs/nfs/nfs_commonport.c 2009-06-05 15:33:54.000000000 -0500 @@ -220,14 +220,9 @@ void newnfs_copycred(struct nfscred *nfscr, struct ucred *cr) { - int ngroups, i; =20 cr->cr_uid =3D nfscr->nfsc_uid; - ngroups =3D (nfscr->nfsc_ngroups < NGROUPS) ? - nfscr->nfsc_ngroups : NGROUPS; - for (i =3D 0; i < ngroups; i++) - cr->cr_groups[i] =3D nfscr->nfsc_groups[i]; - cr->cr_ngroups =3D ngroups; + crsetgroups(cr, nfscr->nfsc_ngroups, nfscr->nfsc_groups); } =20 /* @@ -295,15 +290,13 @@ =20 /* * Set the credentials to refer to root. - * If only the various BSDen could agree on whether cr_gid is a separate - * field or cr_groups[0]... */ void newnfs_setroot(struct ucred *cred) { =20 cred->cr_uid =3D 0; - cred->cr_groups[0] =3D 0; + cred->cr_gid =3D 0; cred->cr_ngroups =3D 1; } =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/nfsclient/nfs_clport.c ngroups/sy= s/fs/nfsclient/nfs_clport.c --- /usr/src/sys/fs/nfsclient/nfs_clport.c 2009-05-29 12:48:03.000000000 -0= 500 +++ ngroups/sys/fs/nfsclient/nfs_clport.c 2009-06-05 15:33:54.000000000 -05= 00 @@ -976,14 +976,12 @@ void newnfs_copyincred(struct ucred *cr, struct nfscred *nfscr) { - int ngroups, i; + int i; =20 nfscr->nfsc_uid =3D cr->cr_uid; - ngroups =3D (cr->cr_ngroups > NGROUPS) ? NGROUPS : - cr->cr_ngroups; - for (i =3D 0; i < ngroups; i++) + nfscr->nfsc_ngroups =3D MIN(cr->cr_ngroups, XU_NGROUPS); + for (i =3D 0; i < nfscr->nfsc_ngroups; i++) nfscr->nfsc_groups[i] =3D cr->cr_groups[i]; - nfscr->nfsc_ngroups =3D ngroups; } =20 =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/nfsserver/nfs_nfsdport.c ngroups/= sys/fs/nfsserver/nfs_nfsdport.c --- /usr/src/sys/fs/nfsserver/nfs_nfsdport.c 2009-06-05 15:33:50.000000000 = -0500 +++ ngroups/sys/fs/nfsserver/nfs_nfsdport.c 2009-06-05 16:02:29.000000000 -= 0500 @@ -2360,7 +2360,6 @@ nfsd_excred(struct nfsrv_descript *nd, struct nfsexstuff *exp, struct ucred *credanon) { - int i; int error =3D 0; =20 /* @@ -2403,9 +2402,8 @@ (nd->nd_flag & ND_AUTHNONE))) { nd->nd_cred->cr_uid =3D credanon->cr_uid; nd->nd_cred->cr_gid =3D credanon->cr_gid; - for (i =3D 0; i < credanon->cr_ngroups && i < NGROUPS; i++) - nd->nd_cred->cr_groups[i] =3D credanon->cr_groups[i]; - nd->nd_cred->cr_ngroups =3D i; + crsetgroups(nd->nd_cred, credanon->cr_ngroups, + credanon->cr_groups); } return (0); } diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/nfsserver/nfs_nfsdstate.c ngroups= /sys/fs/nfsserver/nfs_nfsdstate.c --- /usr/src/sys/fs/nfsserver/nfs_nfsdstate.c 2009-05-29 12:48:03.000000000= -0500 +++ ngroups/sys/fs/nfsserver/nfs_nfsdstate.c 2009-06-05 15:33:54.000000000 = -0500 @@ -3577,7 +3577,6 @@ nd->nd_repstat =3D 0; cred->cr_uid =3D clp->lc_uid; cred->cr_gid =3D clp->lc_gid; - cred->cr_groups[0] =3D clp->lc_gid; callback =3D clp->lc_callback; NFSUNLOCKSTATE(); cred->cr_ngroups =3D 1; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/portalfs/portal.h ngroups/sys/fs/= portalfs/portal.h --- /usr/src/sys/fs/portalfs/portal.h 2009-01-22 10:06:01.000000000 -0600 +++ ngroups/sys/fs/portalfs/portal.h 2009-06-05 15:33:54.000000000 -0500 @@ -43,7 +43,7 @@ int pcr_flag; /* File open mode */ uid_t pcr_uid; /* From ucred */ short pcr_ngroups; /* From ucred */ - gid_t pcr_groups[NGROUPS]; /* From ucred */ + gid_t pcr_groups[XU_NGROUPS]; /* From ucred */ }; =20 #ifdef _KERNEL diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/portalfs/portal_vnops.c ngroups/s= ys/fs/portalfs/portal_vnops.c --- /usr/src/sys/fs/portalfs/portal_vnops.c 2009-01-22 10:06:01.000000000 -= 0600 +++ ngroups/sys/fs/portalfs/portal_vnops.c 2009-06-05 15:33:54.000000000 -0= 500 @@ -311,8 +311,9 @@ =20 pcred.pcr_flag =3D ap->a_mode; pcred.pcr_uid =3D ap->a_cred->cr_uid; - pcred.pcr_ngroups =3D ap->a_cred->cr_ngroups; - bcopy(ap->a_cred->cr_groups, pcred.pcr_groups, NGROUPS * sizeof(gid_t)); + pcred.pcr_ngroups =3D MIN(ap->a_cred->cr_ngroups, XU_NGROUPS); + bcopy(ap->a_cred->cr_groups, pcred.pcr_groups, + pcred.pcr_ngroups * sizeof(gid_t)); aiov[0].iov_base =3D (caddr_t) &pcred; aiov[0].iov_len =3D sizeof(pcred); aiov[1].iov_base =3D pt->pt_arg; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/fs/unionfs/union_vnops.c ngroups/sys= /fs/unionfs/union_vnops.c --- /usr/src/sys/fs/unionfs/union_vnops.c 2009-04-12 15:26:52.000000000 -05= 00 +++ ngroups/sys/fs/unionfs/union_vnops.c 2009-06-05 15:33:54.000000000 -0500 @@ -638,7 +638,6 @@ uid_t uid; /* upper side vnode's uid */ gid_t gid; /* upper side vnode's gid */ u_short vmode; /* upper side vnode's mode */ - gid_t *gp; u_short mask; =20 mask =3D 0; @@ -659,17 +658,14 @@ =20 /* check group */ count =3D 0; - gp =3D cred->cr_groups; - for (; count < cred->cr_ngroups; count++, gp++) { - if (gid =3D=3D *gp) { - if (accmode & VEXEC) - mask |=3D S_IXGRP; - if (accmode & VREAD) - mask |=3D S_IRGRP; - if (accmode & VWRITE) - mask |=3D S_IWGRP; - return ((vmode & mask) =3D=3D mask ? 0 : EACCES); - } + if (groupmember(gid, cred)) { + if (accmode & VEXEC) + mask |=3D S_IXGRP; + if (accmode & VREAD) + mask |=3D S_IRGRP; + if (accmode & VWRITE) + mask |=3D S_IWGRP; + return ((vmode & mask) =3D=3D mask ? 0 : EACCES); } =20 /* check other */ diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/gnu/fs/xfs/FreeBSD/xfs_compat.h ngro= ups/sys/gnu/fs/xfs/FreeBSD/xfs_compat.h --- /usr/src/sys/gnu/fs/xfs/FreeBSD/xfs_compat.h 2009-02-28 13:28:12.000000= 000 -0600 +++ ngroups/sys/gnu/fs/xfs/FreeBSD/xfs_compat.h 2009-06-05 15:33:54.0000000= 00 -0500 @@ -163,7 +163,7 @@ * Cedentials manipulation. */ #define current_fsuid(credp) (credp)->cr_uid -#define current_fsgid(credp) (credp)->cr_groups[0] +#define current_fsgid(credp) (credp)->cr_gid =20 #define PAGE_CACHE_SIZE PAGE_SIZE =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/gnu/fs/xfs/xfs_inode.c ngroups/sys/g= nu/fs/xfs/xfs_inode.c --- /usr/src/sys/gnu/fs/xfs/xfs_inode.c 2009-01-21 12:45:49.000000000 -0600 +++ ngroups/sys/gnu/fs/xfs/xfs_inode.c 2009-06-05 15:33:54.000000000 -0500 @@ -1124,7 +1124,7 @@ ip->i_d.di_nlink =3D nlink; ASSERT(ip->i_d.di_nlink =3D=3D nlink); ip->i_d.di_uid =3D curthread->td_ucred->cr_uid; - ip->i_d.di_gid =3D curthread->td_ucred->cr_groups[0]; + ip->i_d.di_gid =3D curthread->td_ucred->cr_gid; ip->i_d.di_projid =3D prid; memset(&(ip->i_d.di_pad[0]), 0, sizeof(ip->i_d.di_pad)); =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/gnu/fs/xfs/xfs_vnodeops.c ngroups/sy= s/gnu/fs/xfs/xfs_vnodeops.c --- /usr/src/sys/gnu/fs/xfs/xfs_vnodeops.c 2009-01-21 12:45:49.000000000 -0= 600 +++ ngroups/sys/gnu/fs/xfs/xfs_vnodeops.c 2009-06-05 15:33:54.000000000 -05= 00 @@ -3379,7 +3379,7 @@ */ error =3D XFS_QM_DQVOPALLOC(mp, dp, current->td_ucred->cr_uid, - current->td_ucred->cr_groups[0], + current->td_ucred->cr_gid, prid, XFS_QMOPT_QUOTALL | XFS_QMOPT_INHERIT, &udqp, &gdqp); if (error) Only in /usr/src/sys/i386/ibcs2: ibcs2_misc.c.orig Only in /usr/src/sys/kern: kern_exec.c.orig Only in /usr/src/sys/kern: kern_proc.c.orig diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/kern/kern_prot.c ngroups/sys/kern/ke= rn_prot.c --- /usr/src/sys/kern/kern_prot.c 2009-06-05 15:33:50.000000000 -0500 +++ ngroups/sys/kern/kern_prot.c 2009-06-05 16:02:28.000000000 -0500 @@ -82,6 +82,9 @@ =20 SYSCTL_NODE(_security, OID_AUTO, bsd, CTLFLAG_RW, 0, "BSD security policy"= ); =20 +static __inline void crsetgroups_locked(struct ucred *cr, int ngrp, + gid_t *groups); + #ifndef _SYS_SYSPROTO_H_ struct getpid_args { int dummy; @@ -243,16 +246,11 @@ =20 td->td_retval[0] =3D td->td_ucred->cr_rgid; #if defined(COMPAT_43) - td->td_retval[1] =3D td->td_ucred->cr_groups[0]; + td->td_retval[1] =3D td->td_ucred->cr_gid; #endif return (0); } =20 -/* - * Get effective group ID. The "egid" is groups[0], and could be obtained - * via getgroups. This syscall exists because it is somewhat painful to do - * correctly in a library function. - */ #ifndef _SYS_SYSPROTO_H_ struct getegid_args { int dummy; @@ -263,7 +261,7 @@ getegid(struct thread *td, struct getegid_args *uap) { =20 - td->td_retval[0] =3D td->td_ucred->cr_groups[0]; + td->td_retval[0] =3D td->td_ucred->cr_gid; return (0); } =20 @@ -679,7 +677,7 @@ gid !=3D oldcred->cr_svgid && /* allow setgid(saved gid) */ #endif #ifdef POSIX_APPENDIX_B_4_2_2 /* Use BSD-compat clause from B.4.2.2 */ - gid !=3D oldcred->cr_groups[0] && /* allow setgid(getegid()) */ + gid !=3D oldcred->cr_gid && /* allow setgid(getegid()) */ #endif (error =3D priv_check_cred(oldcred, PRIV_CRED_SETGID, 0)) !=3D 0) goto fail; @@ -691,7 +689,7 @@ */ if ( #ifdef POSIX_APPENDIX_B_4_2_2 /* use the clause from B.4.2.2 */ - gid =3D=3D oldcred->cr_groups[0] || + gid =3D=3D oldcred->cr_gid || #endif /* We are using privs. */ priv_check_cred(oldcred, PRIV_CRED_SETGID, 0) =3D=3D 0) @@ -720,7 +718,7 @@ * In all cases permitted cases, we are changing the egid. * Copy credentials so other references do not see our changes. */ - if (oldcred->cr_groups[0] !=3D gid) { + if (oldcred->cr_gid !=3D gid) { change_egid(newcred, gid); setsugid(p); } @@ -766,7 +764,7 @@ (error =3D priv_check_cred(oldcred, PRIV_CRED_SETEGID, 0)) !=3D 0) goto fail; =20 - if (oldcred->cr_groups[0] !=3D egid) { + if (oldcred->cr_gid !=3D egid) { change_egid(newcred, egid); setsugid(p); } @@ -811,7 +809,6 @@ { struct proc *p =3D td->td_proc; struct ucred *newcred, *oldcred; - int newgroups; int error; =20 if (ngrp > NGROUPS) @@ -820,16 +817,7 @@ newcred =3D crget(); crextend(newcred, ngrp); PROC_LOCK(p); - oldcred =3D p->p_ucred; - newgroups =3D MAX(oldcred->cr_agroups, ngrp); - while (newcred->cr_agroups < newgroups) { - PROC_UNLOCK(p); - crextend(newcred, newgroups); - PROC_LOCK(p); - oldcred =3D p->p_ucred; - newgroups =3D MAX(oldcred->cr_agroups, ngrp); - } - + oldcred =3D crcopysafe(p, newcred); =20 #ifdef MAC error =3D mac_cred_check_setgroups(oldcred, ngrp, groups); @@ -841,7 +829,6 @@ if (error) goto fail; =20 - crcopy(newcred, oldcred); if (ngrp < 1) { /* * setgroups(0, NULL) is a legitimate way of clearing the @@ -851,8 +838,7 @@ */ newcred->cr_ngroups =3D 1; } else { - bcopy(groups, newcred->cr_groups, ngrp * sizeof(gid_t)); - newcred->cr_ngroups =3D ngrp; + crsetgroups_locked(newcred, ngrp, groups); } setsugid(p); p->p_ucred =3D newcred; @@ -964,12 +950,12 @@ =20 if (((rgid !=3D (gid_t)-1 && rgid !=3D oldcred->cr_rgid && rgid !=3D oldcred->cr_svgid) || - (egid !=3D (gid_t)-1 && egid !=3D oldcred->cr_groups[0] && + (egid !=3D (gid_t)-1 && egid !=3D oldcred->cr_gid && egid !=3D oldcred->cr_rgid && egid !=3D oldcred->cr_svgid)) && (error =3D priv_check_cred(oldcred, PRIV_CRED_SETREGID, 0)) !=3D 0) goto fail; =20 - if (egid !=3D (gid_t)-1 && oldcred->cr_groups[0] !=3D egid) { + if (egid !=3D (gid_t)-1 && oldcred->cr_gid !=3D egid) { change_egid(newcred, egid); setsugid(p); } @@ -977,9 +963,9 @@ change_rgid(newcred, rgid); setsugid(p); } - if ((rgid !=3D (gid_t)-1 || newcred->cr_groups[0] !=3D newcred->cr_rgid) = && - newcred->cr_svgid !=3D newcred->cr_groups[0]) { - change_svgid(newcred, newcred->cr_groups[0]); + if ((rgid !=3D (gid_t)-1 || newcred->cr_gid !=3D newcred->cr_rgid) && + newcred->cr_svgid !=3D newcred->cr_gid) { + change_svgid(newcred, newcred->cr_gid); setsugid(p); } p->p_ucred =3D newcred; @@ -1110,17 +1096,17 @@ =20 if (((rgid !=3D (gid_t)-1 && rgid !=3D oldcred->cr_rgid && rgid !=3D oldcred->cr_svgid && - rgid !=3D oldcred->cr_groups[0]) || + rgid !=3D oldcred->cr_gid) || (egid !=3D (gid_t)-1 && egid !=3D oldcred->cr_rgid && egid !=3D oldcred->cr_svgid && - egid !=3D oldcred->cr_groups[0]) || + egid !=3D oldcred->cr_gid) || (sgid !=3D (gid_t)-1 && sgid !=3D oldcred->cr_rgid && sgid !=3D oldcred->cr_svgid && - sgid !=3D oldcred->cr_groups[0])) && + sgid !=3D oldcred->cr_gid)) && (error =3D priv_check_cred(oldcred, PRIV_CRED_SETRESGID, 0)) !=3D 0) goto fail; =20 - if (egid !=3D (gid_t)-1 && oldcred->cr_groups[0] !=3D egid) { + if (egid !=3D (gid_t)-1 && oldcred->cr_gid !=3D egid) { change_egid(newcred, egid); setsugid(p); } @@ -1189,8 +1175,8 @@ error1 =3D copyout(&cred->cr_rgid, uap->rgid, sizeof(cred->cr_rgid)); if (uap->egid) - error2 =3D copyout(&cred->cr_groups[0], - uap->egid, sizeof(cred->cr_groups[0])); + error2 =3D copyout(&cred->cr_gid, + uap->egid, sizeof(cred->cr_gid)); if (uap->sgid) error3 =3D copyout(&cred->cr_svgid, uap->sgid, sizeof(cred->cr_svgid)); @@ -1911,7 +1897,7 @@ ngroups =3D min(cr->cr_ngroups, XU_NGROUPS); xcr->cr_ngroups =3D ngroups; bcopy(cr->cr_groups, xcr->cr_groups, - ngroups * sizeof(cr->cr_groups[0])); + ngroups * sizeof(*cr->cr_groups)); } =20 /* @@ -1969,6 +1955,8 @@ /* * We extend by 2 each time since we're using a power of two * allocator. + * XXX: it probably makes more sense to right-size the + * allocation if we need more than a page. */ if (cr->cr_agroups) cnt =3D cr->cr_agroups * 2; @@ -1987,6 +1975,36 @@ } =20 /* + * Copy groups in to a credential, preserving any necessicary invariants + * (i.e. sorting in the future). crextend() must have been called + * before hand to ensure sufficient space is available. If=20 + */ +static inline void +crsetgroups_locked(struct ucred *cr, int ngrp, gid_t *groups) +{ +=09 + KASSERT(cr->cr_agroups >=3D ngrp, ("cr_ngroups is too small")); + + bcopy(groups, cr->cr_groups, ngrp * sizeof(gid_t)); + cr->cr_ngroups =3D ngrp; +} + +/* + * Copy groups in to a credential after expanding it if required. + * Truncate the list to NGROUPS if it is too large. + */ +void +crsetgroups(struct ucred *cr, int ngrp, gid_t *groups) +{ + + if (ngrp > NGROUPS) + ngrp =3D NGROUPS; + + crextend(cr, ngrp); + crsetgroups_locked(cr, ngrp, groups); +} + +/* * Get login name, if available. */ #ifndef _SYS_SYSPROTO_H_ @@ -2083,7 +2101,7 @@ change_egid(struct ucred *newcred, gid_t egid) { =20 - newcred->cr_groups[0] =3D egid; + newcred->cr_gid =3D egid; } =20 /*- Only in /usr/src/sys/kern: kern_prot.c.orig diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/kern/vfs_export.c ngroups/sys/kern/v= fs_export.c --- /usr/src/sys/kern/vfs_export.c 2009-05-29 12:48:02.000000000 -0500 +++ ngroups/sys/kern/vfs_export.c 2009-06-05 15:33:54.000000000 -0500 @@ -120,9 +120,8 @@ np->netc_exflags =3D argp->ex_flags; np->netc_anon =3D crget(); np->netc_anon->cr_uid =3D argp->ex_anon.cr_uid; - np->netc_anon->cr_ngroups =3D argp->ex_anon.cr_ngroups; - bcopy(argp->ex_anon.cr_groups, np->netc_anon->cr_groups, - sizeof(np->netc_anon->cr_groups)); + crsetgroups(np->netc_anon, argp->ex_anon.cr_ngroups, + argp->ex_anon.cr_groups); np->netc_numsecflavors =3D argp->ex_numsecflavors; bcopy(argp->ex_secflavors, np->netc_secflavors, sizeof(np->netc_secflavors)); @@ -205,9 +204,8 @@ np->netc_exflags =3D argp->ex_flags; np->netc_anon =3D crget(); np->netc_anon->cr_uid =3D argp->ex_anon.cr_uid; - np->netc_anon->cr_ngroups =3D argp->ex_anon.cr_ngroups; - bcopy(argp->ex_anon.cr_groups, np->netc_anon->cr_groups, - sizeof(np->netc_anon->cr_groups)); + crsetgroups(np->netc_anon, argp->ex_anon.cr_ngroups, + np->netc_anon->cr_groups); np->netc_numsecflavors =3D argp->ex_numsecflavors; bcopy(argp->ex_secflavors, np->netc_secflavors, sizeof(np->netc_secflavors)); diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/kern/vfs_syscalls.c ngroups/sys/kern= /vfs_syscalls.c --- /usr/src/sys/kern/vfs_syscalls.c 2009-06-05 15:33:50.000000000 -0500 +++ ngroups/sys/kern/vfs_syscalls.c 2009-06-05 16:02:28.000000000 -0500 @@ -2128,7 +2128,7 @@ cred =3D td->td_ucred; tmpcred =3D crdup(cred); tmpcred->cr_uid =3D cred->cr_ruid; - tmpcred->cr_groups[0] =3D cred->cr_rgid; + tmpcred->cr_gid =3D cred->cr_rgid; td->td_ucred =3D tmpcred; } else cred =3D tmpcred =3D td->td_ucred; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/netinet/ipfw/ip_fw2.c ngroups/sys/ne= tinet/ipfw/ip_fw2.c --- /usr/src/sys/netinet/ipfw/ip_fw2.c 2009-06-05 15:33:50.000000000 -0500 +++ ngroups/sys/netinet/ipfw/ip_fw2.c 2009-06-05 16:02:28.000000000 -0500 @@ -139,8 +139,9 @@ * the user specified UID/GID based constraints in * a firewall rule. */ +#define FW_NGROUPS 16 struct ip_fw_ugid { - gid_t fw_groups[NGROUPS]; + gid_t fw_groups[FW_NGROUPS]; /* XXX: should be dynamic */ int fw_ngroups; uid_t fw_uid; int fw_prid; @@ -2016,8 +2017,8 @@ cr =3D inp->inp_cred; ugp->fw_prid =3D jailed(cr) ? cr->cr_prison->pr_id : -1; ugp->fw_uid =3D cr->cr_uid; - ugp->fw_ngroups =3D cr->cr_ngroups; - bcopy(cr->cr_groups, ugp->fw_groups, sizeof(ugp->fw_groups)); + ugp->fw_ngroups =3D MIN(cr->cr_ngroups, FW_NGROUPS); + bcopy(cr->cr_groups, ugp->fw_groups, sizeof(gid_t) * ugp->fw_ngroups); } =20 static int diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/netncp/ncp_conn.c ngroups/sys/netncp= /ncp_conn.c --- /usr/src/sys/netncp/ncp_conn.c 2009-01-22 10:06:17.000000000 -0600 +++ ngroups/sys/netncp/ncp_conn.c 2009-06-05 15:33:54.000000000 -0500 @@ -249,7 +249,7 @@ ncp->connid =3D 0xFFFF; ncp->li =3D *cap; ncp->nc_group =3D (cap->group !=3D NCP_DEFAULT_GROUP) ? - cap->group : cred->cr_groups[0]; + cap->group : cred->cr_gid; =20 if (cap->retry_count =3D=3D 0) ncp->li.retry_count =3D NCP_RETRY_COUNT; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/netsmb/smb_conn.c ngroups/sys/netsmb= /smb_conn.c --- /usr/src/sys/netsmb/smb_conn.c 2009-01-22 10:06:17.000000000 -0600 +++ ngroups/sys/netsmb/smb_conn.c 2009-06-05 15:33:54.000000000 -0500 @@ -416,7 +416,7 @@ if (uid =3D=3D SMBM_ANY_OWNER) uid =3D realuid; if (gid =3D=3D SMBM_ANY_GROUP) - gid =3D cred->cr_groups[0]; + gid =3D cred->cr_gid; vcp->vc_uid =3D uid; vcp->vc_grp =3D gid; =20 @@ -714,7 +714,7 @@ if (uid =3D=3D SMBM_ANY_OWNER) uid =3D realuid; if (gid =3D=3D SMBM_ANY_GROUP) - gid =3D cred->cr_groups[0]; + gid =3D cred->cr_gid; ssp =3D smb_zmalloc(sizeof(*ssp), M_SMBCONN, M_WAITOK); smb_co_init(SSTOCP(ssp), SMBL_SHARE, "smbss ilock", "smbss"); ssp->obj.co_free =3D smb_share_free; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/nfsclient/nfs_subs.c ngroups/sys/nfs= client/nfs_subs.c --- /usr/src/sys/nfsclient/nfs_subs.c 2009-05-29 12:48:02.000000000 -0500 +++ ngroups/sys/nfsclient/nfs_subs.c 2009-06-05 15:33:54.000000000 -0500 @@ -253,7 +253,7 @@ *tl++ =3D 0; /* stamp ?? */ *tl++ =3D 0; /* NULL hostname */ *tl++ =3D txdr_unsigned(cr->cr_uid); - *tl++ =3D txdr_unsigned(cr->cr_groups[0]); + *tl++ =3D txdr_unsigned(cr->cr_gid); grpsiz =3D (auth_len >> 2) - 5; *tl++ =3D txdr_unsigned(grpsiz); for (i =3D 1; i <=3D grpsiz; i++) diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/nfsserver/nfs_srvsock.c ngroups/sys/= nfsserver/nfs_srvsock.c --- /usr/src/sys/nfsserver/nfs_srvsock.c 2009-06-05 15:33:51.000000000 -0500 +++ ngroups/sys/nfsserver/nfs_srvsock.c 2009-06-05 16:02:29.000000000 -0500 @@ -358,7 +358,7 @@ tl =3D nfsm_dissect_nonblock(u_int32_t *, 3 * NFSX_UNSIGNED); nd->nd_cr->cr_uid =3D nd->nd_cr->cr_ruid =3D nd->nd_cr->cr_svuid =3D fxdr_unsigned(uid_t, *tl++); - nd->nd_cr->cr_groups[0] =3D nd->nd_cr->cr_rgid =3D + nd->nd_cr->cr_gid =3D nd->nd_cr->cr_rgid =3D nd->nd_cr->cr_svgid =3D fxdr_unsigned(gid_t, *tl++); #ifdef MAC mac_cred_associate_nfsd(nd->nd_cr); @@ -374,7 +374,7 @@ nd->nd_cr->cr_groups[i] =3D fxdr_unsigned(gid_t, *tl++); else tl++; - nd->nd_cr->cr_ngroups =3D (len >=3D XU_NGROUPS) ? XU_NGROUPS : (len + 1); + nd->nd_cr->cr_ngroups =3D MIN(XU_NGROUPS, len + 1); if (nd->nd_cr->cr_ngroups > 1) nfsrvw_sort(nd->nd_cr->cr_groups, nd->nd_cr->cr_ngroups); len =3D fxdr_unsigned(int, *++tl); Only in /usr/src/sys/nfsserver: nfs_srvsock.c.orig diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/nfsserver/nfs_srvsubs.c ngroups/sys/= nfsserver/nfs_srvsubs.c --- /usr/src/sys/nfsserver/nfs_srvsubs.c 2009-05-29 12:48:04.000000000 -0500 +++ ngroups/sys/nfsserver/nfs_srvsubs.c 2009-06-05 15:33:54.000000000 -0500 @@ -1181,9 +1181,7 @@ cred =3D nfsd->nd_cr; if (cred->cr_uid =3D=3D 0 || (exflags & MNT_EXPORTANON)) { cred->cr_uid =3D credanon->cr_uid; - for (i =3D 0; i < credanon->cr_ngroups && i < NGROUPS; i++) - cred->cr_groups[i] =3D credanon->cr_groups[i]; - cred->cr_ngroups =3D i; + crsetgroups(cred, credanon->cr_ngroups, credanon->cr_groups); } if (exflags & MNT_EXRDONLY) *rdonlyp =3D 1; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/rpc/authunix_prot.c ngroups/sys/rpc/= authunix_prot.c --- /usr/src/sys/rpc/authunix_prot.c 2009-06-05 15:33:49.000000000 -0500 +++ ngroups/sys/rpc/authunix_prot.c 2009-06-05 16:02:28.000000000 -0500 @@ -98,7 +98,7 @@ =20 if (!xdr_uint32_t(xdrs, &cred->cr_uid)) return (FALSE); - if (!xdr_uint32_t(xdrs, &cred->cr_groups[0])) + if (!xdr_uint32_t(xdrs, &cred->cr_gid)) return (FALSE); =20 if (xdrs->x_op =3D=3D XDR_ENCODE) { diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/rpc/rpcsec_gss/svc_rpcsec_gss.c ngro= ups/sys/rpc/rpcsec_gss/svc_rpcsec_gss.c --- /usr/src/sys/rpc/rpcsec_gss/svc_rpcsec_gss.c 2009-01-22 10:05:57.000000= 000 -0600 +++ ngroups/sys/rpc/rpcsec_gss/svc_rpcsec_gss.c 2009-06-05 15:33:54.0000000= 00 -0500 @@ -447,11 +447,7 @@ cr =3D client->cl_cred =3D crget(); cr->cr_uid =3D cr->cr_ruid =3D cr->cr_svuid =3D uc->uid; cr->cr_rgid =3D cr->cr_svgid =3D uc->gid; - cr->cr_ngroups =3D uc->gidlen; - if (cr->cr_ngroups > NGROUPS) - cr->cr_ngroups =3D NGROUPS; - for (i =3D 0; i < cr->cr_ngroups; i++) - cr->cr_groups[i] =3D uc->gidlist[i]; + crsetgroups(cr, uc->gidlen, uc->gidlist); *crp =3D crhold(cr); =20 return (TRUE); diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/rpc/svc_auth.c ngroups/sys/rpc/svc_a= uth.c --- /usr/src/sys/rpc/svc_auth.c 2009-01-22 10:05:57.000000000 -0600 +++ ngroups/sys/rpc/svc_auth.c 2009-06-05 15:33:54.000000000 -0500 @@ -165,7 +165,7 @@ svc_getcred(struct svc_req *rqst, struct ucred **crp, int *flavorp) { struct ucred *cr =3D NULL; - int flavor, i; + int flavor; struct xucred *xcr; =20 flavor =3D rqst->rq_cred.oa_flavor; @@ -177,10 +177,8 @@ xcr =3D (struct xucred *) rqst->rq_clntcred; cr =3D crget(); cr->cr_uid =3D cr->cr_ruid =3D cr->cr_svuid =3D xcr->cr_uid; - cr->cr_ngroups =3D xcr->cr_ngroups; - for (i =3D 0; i < xcr->cr_ngroups; i++) - cr->cr_groups[i] =3D xcr->cr_groups[i]; - cr->cr_rgid =3D cr->cr_svgid =3D cr->cr_groups[0]; + crsetgroups(cr, xcr->cr_ngroups, xcr->cr_groups); + cr->cr_rgid =3D cr->cr_svgid =3D cr->cr_gid; *crp =3D cr; return (TRUE); =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/rpc/svc_auth_unix.c ngroups/sys/rpc/= svc_auth_unix.c --- /usr/src/sys/rpc/svc_auth_unix.c 2009-01-22 10:05:57.000000000 -0600 +++ ngroups/sys/rpc/svc_auth_unix.c 2009-06-05 15:33:54.000000000 -0500 @@ -88,20 +88,20 @@ str_len =3D RNDUP(str_len); buf +=3D str_len / sizeof (int32_t); xcr->cr_uid =3D IXDR_GET_UINT32(buf); - xcr->cr_groups[0] =3D IXDR_GET_UINT32(buf); + xcr->cr_gid =3D IXDR_GET_UINT32(buf); gid_len =3D (size_t)IXDR_GET_UINT32(buf); if (gid_len > NGRPS) { stat =3D AUTH_BADCRED; goto done; } for (i =3D 0; i < gid_len; i++) { - if (i + 1 < NGROUPS) + if (i + 1 < XU_NGROUPS) xcr->cr_groups[i + 1] =3D IXDR_GET_INT32(buf); else buf++; } - if (gid_len + 1 > NGROUPS) - xcr->cr_ngroups =3D NGROUPS; + if (gid_len + 1 > XU_NGROUPS) + xcr->cr_ngroups =3D XU_NGROUPS; else xcr->cr_ngroups =3D gid_len + 1; =20 diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/security/audit/audit.c ngroups/sys/s= ecurity/audit/audit.c --- /usr/src/sys/security/audit/audit.c 2009-04-24 10:41:08.000000000 -0500 +++ ngroups/sys/security/audit/audit.c 2009-06-05 15:33:54.000000000 -0500 @@ -224,7 +224,7 @@ cru2x(cred, &ar->k_ar.ar_subj_cred); ar->k_ar.ar_subj_ruid =3D cred->cr_ruid; ar->k_ar.ar_subj_rgid =3D cred->cr_rgid; - ar->k_ar.ar_subj_egid =3D cred->cr_groups[0]; + ar->k_ar.ar_subj_egid =3D cred->cr_gid; ar->k_ar.ar_subj_auid =3D cred->cr_audit.ai_auid; ar->k_ar.ar_subj_asid =3D cred->cr_audit.ai_asid; ar->k_ar.ar_subj_pid =3D td->td_proc->p_pid; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/security/audit/audit_arg.c ngroups/s= ys/security/audit/audit_arg.c --- /usr/src/sys/security/audit/audit_arg.c 2009-01-22 10:06:21.000000000 -= 0600 +++ ngroups/sys/security/audit/audit_arg.c 2009-06-05 15:33:54.000000000 -0= 500 @@ -369,7 +369,7 @@ cred =3D p->p_ucred; ar->k_ar.ar_arg_auid =3D cred->cr_audit.ai_auid; ar->k_ar.ar_arg_euid =3D cred->cr_uid; - ar->k_ar.ar_arg_egid =3D cred->cr_groups[0]; + ar->k_ar.ar_arg_egid =3D cred->cr_gid; ar->k_ar.ar_arg_ruid =3D cred->cr_ruid; ar->k_ar.ar_arg_rgid =3D cred->cr_rgid; ar->k_ar.ar_arg_asid =3D cred->cr_audit.ai_asid; diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/sys/syslimits.h ngroups/sys/sys/sysl= imits.h --- /usr/src/sys/sys/syslimits.h 2009-01-22 10:06:22.000000000 -0600 +++ ngroups/sys/sys/syslimits.h 2009-06-05 15:34:15.000000000 -0500 @@ -54,7 +54,7 @@ #define MAX_CANON 255 /* max bytes in term canon input line */ #define MAX_INPUT 255 /* max bytes in terminal input */ #define NAME_MAX 255 /* max bytes in a file name */ -#define NGROUPS_MAX 16 /* max supplemental group id's */ +#define NGROUPS_MAX 32767 /* max supplemental group id's */ #ifndef OPEN_MAX #define OPEN_MAX 64 /* max open files per process */ #endif diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/sys/ucred.h ngroups/sys/sys/ucred.h --- /usr/src/sys/sys/ucred.h 2009-06-05 15:33:54.000000000 -0500 +++ ngroups/sys/sys/ucred.h 2009-06-05 16:02:34.000000000 -0500 @@ -48,7 +48,7 @@ uid_t cr_uid; /* effective user id */ uid_t cr_ruid; /* real user id */ uid_t cr_svuid; /* saved user id */ - short cr_ngroups; /* number of groups */ + int cr_ngroups; /* number of groups */ gid_t cr_rgid; /* real group id */ gid_t cr_svgid; /* saved group id */ struct uidinfo *cr_uidinfo; /* per euid resource consumption */ @@ -61,7 +61,7 @@ struct label *cr_label; /* MAC label */ struct auditinfo_addr cr_audit; /* Audit properties. */ gid_t *cr_groups; /* groups */ - short cr_agroups; /* Available groups */ + int cr_agroups; /* Available groups */ }; #define NOCRED ((struct ucred *)0) /* no credential available */ #define FSCRED ((struct ucred *)-1) /* filesystem credential */ @@ -77,11 +77,9 @@ uid_t cr_uid; /* effective user id */ short cr_ngroups; /* number of groups */ gid_t cr_groups[XU_NGROUPS]; /* groups */ + void *_cr_unused1; /* compatibility with old ucred */ }; -#define XUCRED_VERSION 2 - -#define XUCRED_SIZE(n) \ - (sizeof(struct xucred) + (MAX((n)-XU_NGROUPS, 0) * sizeof(gid_t))) +#define XUCRED_VERSION 0 =20 /* This can be used for both ucred and xucred structures. */ #define cr_gid cr_groups[0] @@ -97,7 +95,7 @@ void change_svgid(struct ucred *newcred, gid_t svgid); void change_svuid(struct ucred *newcred, uid_t svuid); void crcopy(struct ucred *dest, struct ucred *src); -struct ucred *crcopysafe(struct proc *, struct ucred *); +struct ucred *crcopysafe(struct proc *p, struct ucred *cr); struct ucred *crdup(struct ucred *cr); void cred_update_thread(struct thread *td); void crfree(struct ucred *cr); @@ -106,6 +104,7 @@ int crshared(struct ucred *cr); void cru2x(struct ucred *cr, struct xucred *xcr); void crextend(struct ucred *cr, int n); +void crsetgroups(struct ucred *cr, int n, gid_t *groups); int groupmember(gid_t gid, struct ucred *cred); #endif /* _KERNEL */ =20 Only in /usr/src/sys/sys: ucred.h.orig Only in /usr/src/sys/sys: user.h.diff Only in /usr/src/sys/sys: user.h.orig Only in /usr/src/sys/sys: user.h.rej.orig diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/sys/ufs/ufs/ufs_vnops.c ngroups/sys/ufs/= ufs/ufs_vnops.c --- /usr/src/sys/ufs/ufs/ufs_vnops.c 2009-06-05 15:33:49.000000000 -0500 +++ ngroups/sys/ufs/ufs/ufs_vnops.c 2009-06-05 16:02:28.000000000 -0500 @@ -1475,7 +1475,7 @@ refcount_init(&ucred.cr_ref, 1); ucred.cr_uid =3D ip->i_uid; ucred.cr_ngroups =3D 1; - ucred.cr_groups[0] =3D dp->i_gid; + ucred.cr_gid =3D dp->i_gid; ucp =3D &ucred; } #endif @@ -2266,6 +2266,7 @@ { #ifdef QUOTA struct ucred ucred, *ucp; + gid_t ucred_group; ucp =3D cnp->cn_cred; #endif /* @@ -2292,7 +2293,8 @@ refcount_init(&ucred.cr_ref, 1); ucred.cr_uid =3D ip->i_uid; ucred.cr_ngroups =3D 1; - ucred.cr_groups[0] =3D pdir->i_gid; + ucred.cr_groups =3D &ucred_group; + ucred.cr_gid =3D pdir->i_gid; ucp =3D &ucred; #endif } else { diff -ru --exclude=3D'.glimpse*' --exclude=3D.svn --exclude=3Dcompile --ign= ore-matching=3D'$FreeBSD' /usr/src/usr.sbin/mountd/mountd.c ngroups/usr.sbi= n/mountd/mountd.c --- /usr/src/usr.sbin/mountd/mountd.c 2009-05-29 12:47:59.000000000 -0500 +++ ngroups/usr.sbin/mountd/mountd.c 2009-05-28 17:11:49.000000000 -0500 @@ -174,7 +174,7 @@ int do_mount(struct exportlist *, struct grouplist *, int, struct xucred *, char *, int, struct statfs *); int do_opt(char **, char **, struct exportlist *, struct grouplist *, - int *, int *, struct xucred **); + int *, int *, struct xucred *); struct exportlist *ex_search(fsid_t *); struct exportlist *get_exp(void); void free_dir(struct dirlist *); @@ -196,7 +196,7 @@ void mntsrv(struct svc_req *, SVCXPRT *); void nextfield(char **, char **); void out_of_mem(void); -void parsecred(char *, struct xucred **); +void parsecred(char *, struct xucred *); int put_exlist(struct dirlist *, XDR *, struct dirlist *, int *, int); void *sa_rawaddr(struct sockaddr *sa, int *nbytes); int sacmp(struct sockaddr *sa1, struct sockaddr *sa2, @@ -221,6 +221,7 @@ (uid_t)-2, 1, { (gid_t)-2 }, + NULL }; int force_v2 =3D 0; int resvport_only =3D 1; @@ -1167,7 +1168,7 @@ struct exportlist **epp; struct dirlist *dirhead; struct statfs fsb; - struct xucred *anon; + struct xucred anon; char *cp, *endcp, *dirp, *hst, *usr, *dom, savedc; int len, has_host, exflags, got_nondir, dirplen, netgrp; =20 @@ -1185,7 +1186,7 @@ * Set defaults. */ has_host =3D FALSE; - anon =3D &def_anon; + anon =3D def_anon; exflags =3D MNT_EXPORTED; got_nondir =3D 0; opt_flags =3D 0; @@ -1403,7 +1404,7 @@ */ grp =3D tgrp; do { - if (do_mount(ep, grp, exflags, anon, dirp, dirplen, + if (do_mount(ep, grp, exflags, &anon, dirp, dirplen, &fsb)) { getexp_err(ep, tgrp); goto nextline; @@ -1956,7 +1957,7 @@ struct grouplist *grp; int *has_hostp; int *exflagsp; - struct xucred **cr; + struct xucred *cr; { char *cpoptarg, *cpoptend; char *cp, *endcp, *cpopt, savedc, savedc2; @@ -2420,33 +2421,6 @@ return (ret); } =20 -static void -xcr_grow(struct xucred **cr, int *cr_ngroups, int desired_groups) -{ - struct xucred *in =3D *cr, *out; - int n; - - if (in && *cr_ngroups > desired_groups) - return; - - n =3D MAX(XU_NGROUPS, *cr_ngroups); - while (n < desired_groups) - n *=3D 2; - - out =3D malloc(XUCRED_SIZE(n)); - if (out =3D=3D NULL) - out_of_mem(); - - if (in) { - memcpy(out, in, XUCRED_SIZE(*cr_ngroups)); - if (in !=3D &def_anon) - free(in); - } - - *cr =3D out; - *cr_ngroups =3D n; -} - /* * Translate a net address. * @@ -2654,20 +2628,18 @@ * Parse a description of a credential. */ void -parsecred(namelist, cr_out) +parsecred(namelist, cr) char *namelist; - struct xucred **cr_out; + struct xucred *cr; { char *name; + int cnt; char *names; struct passwd *pw; struct group *gr; + gid_t groups[NGRPS + 1]; int ngroups; - struct xucred *cr =3D NULL; - int cr_ngroups =3D 0; - int error; =20 - xcr_grow(&cr, &cr_ngroups, 0); cr->cr_version =3D XUCRED_VERSION; /* * Set up the unprivileged user. @@ -2690,20 +2662,20 @@ if (names =3D=3D NULL) { if (pw =3D=3D NULL) { syslog(LOG_ERR, "unknown user: %s", name); - goto out; + return; } cr->cr_uid =3D pw->pw_uid; - for (;;) { - ngroups =3D cr_ngroups; - error =3D getgrouplist(pw->pw_name, pw->pw_gid, - cr->cr_groups, &ngroups); - if (!error) - break; - xcr_grow(&cr, &cr_ngroups, ngroups); - } - cr->cr_ngroups =3D ngroups; - - goto out; + ngroups =3D NGRPS + 1; + if (getgrouplist(pw->pw_name, pw->pw_gid, groups, &ngroups)) + syslog(LOG_ERR, "too many groups"); + /* + * Compress out duplicate. + */ + cr->cr_ngroups =3D ngroups - 1; + cr->cr_groups[0] =3D groups[0]; + for (cnt =3D 2; cnt < ngroups; cnt++) + cr->cr_groups[cnt - 1] =3D groups[cnt]; + return; } /* * Explicit credential specified as a colon separated list: @@ -2715,11 +2687,10 @@ cr->cr_uid =3D atoi(name); else { syslog(LOG_ERR, "unknown user: %s", name); - goto out; + return; } cr->cr_ngroups =3D 0; - while (names !=3D NULL && *names !=3D '\0') { - xcr_grow(&cr, &cr_ngroups, cr->cr_ngroups + 1); + while (names !=3D NULL && *names !=3D '\0' && cr->cr_ngroups < NGRPS) { name =3D strsep(&names, ":"); if (isdigit(*name) || *name =3D=3D '-') { cr->cr_groups[cr->cr_ngroups++] =3D atoi(name); @@ -2731,9 +2702,8 @@ cr->cr_groups[cr->cr_ngroups++] =3D gr->gr_gid; } } - - out: - *cr_out =3D cr; + if (names !=3D NULL && *names !=3D '\0' && cr->cr_ngroups =3D=3D NGRPS) + syslog(LOG_ERR, "too many groups"); } =20 #define STRSIZ (RPCMNT_NAMELEN+RPCMNT_PATHLEN+50) --sdtB3X0nJg68CQEu-- --i9LlY+UWpKt15+FH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iD8DBQFKKZ3zXY6L6fI4GtQRAmZIAJ9vmvoOMbiS30WRBzsHqpjAx+fdMQCg21bB l7/uY3WXqja2MHK21xWk4a4= =acSA -----END PGP SIGNATURE----- --i9LlY+UWpKt15+FH-- From owner-freebsd-arch@FreeBSD.ORG Sat Jun 6 01:47:57 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D303C1065731; Sat, 6 Jun 2009 01:47:57 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail15.syd.optusnet.com.au (mail15.syd.optusnet.com.au [211.29.132.196]) by mx1.freebsd.org (Postfix) with ESMTP id 048F38FC08; Sat, 6 Jun 2009 01:47:56 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-216-167.belrs3.nsw.optusnet.com.au [122.106.216.167]) by mail15.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n561lmGX014794 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 6 Jun 2009 11:47:48 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n561llld004385; Sat, 6 Jun 2009 11:47:47 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n561ljFw004384; Sat, 6 Jun 2009 11:47:45 +1000 (EST) (envelope-from peter) Date: Sat, 6 Jun 2009 11:47:45 +1000 From: Peter Jeremy To: Scott Long Message-ID: <20090606014745.GC9161@server.vk2pj.dyndns.org> References: <4A254B45.8050800@mavhome.dp.ua> <200906050703.n5573x5Q071765@apollo.backplane.com> <4A2956C6.5070902@samsco.org> <4A29694B.2090606@elischer.org> <4A297187.2090701@samsco.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Qbvjkv9qwOGw/5Fx" Content-Disposition: inline In-Reply-To: <4A297187.2090701@samsco.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.19 (2009-01-05) Cc: Alexander Motin , FreeBSD Current , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 01:47:58 -0000 --Qbvjkv9qwOGw/5Fx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Jun-05 13:27:03 -0600, Scott Long wrote: >Every one of Matt's emails follow this formula: > >1. I don't know how FOO works, but how you're doing it is wrong >2. Here's how I think FOO should work >3. I'm going to work on FOO in DragonFlyBSD and have it done in a week. > >1 and 2 are ignorant and worthless in a technical conversation, and 3 is= =20 >off topic for FreeBSD mailing lists. So yes, I'm calling him out. Well, I have seen little evidence of 1. 2 _can_ be relevant in an architectural discussion. 3 may or may not be relevant - it depends whether FreeBSD can make use of the work or not. I agree with Julian that you are being unnecessarily provocative. IMHO, Matt has made a positive contribution to this thread. Rather than quoting him out of context, how about you take a chill pill and consider what benefits FreeBSD can gain by reusing work done by other free Unices. --=20 Peter Jeremy --Qbvjkv9qwOGw/5Fx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAkopysEACgkQ/opHv/APuIcfMgCfQtuur3fxxWJA5Y/e5TUkr5JV CBMAoITs7gvN+hV225NiNxgtdqa+dE7W =X5V5 -----END PGP SIGNATURE----- --Qbvjkv9qwOGw/5Fx-- From owner-freebsd-arch@FreeBSD.ORG Sat Jun 6 10:31:56 2009 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5105E106566C; Sat, 6 Jun 2009 10:31:56 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.freebsd.org (Postfix) with ESMTP id E35F58FC18; Sat, 6 Jun 2009 10:31:55 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c122-106-151-9.carlnfd1.nsw.optusnet.com.au (c122-106-151-9.carlnfd1.nsw.optusnet.com.au [122.106.151.9]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n56AVpUM016853 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 6 Jun 2009 20:31:53 +1000 Date: Sat, 6 Jun 2009 20:31:51 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Brooks Davis In-Reply-To: <20090605223636.GA24364@lor.one-eyed-alien.net> Message-ID: <20090606184344.N16744@delplex.bde.org> References: <20090605223636.GA24364@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@FreeBSD.org, current@FreeBSD.org Subject: Re: RFT: Allow large values of NGROUPS_MAX X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 10:31:56 -0000 On Fri, 5 Jun 2009, Brooks Davis wrote: > I've been working on fixing the limit of 15 groups per user. This has > primarily consisted of merging a patch from Isilon Systems which breaks > the group array out of struct ucred and stores in in malloc'd storage. > > I've attached a patch which includes that change and increases the value > of NGROUPS_MAX to 32767. It also changes references to cr_groups[0] to > use the cr_gid macro ... I don't like this macro. At least the implementation should keep using the unobfuscated reference. kern_prot.c consistently didn't use the obfuscation except once in a comment. > Before any merge a couple decisions need to be made: > > - How large should NGROUPS_MAX be? Linux uses 65536 and we could > extend things to support that, but it's probably unnecessary. 32767 seems excessive too. Hopefully NGROUPS[_MAX] is used only in broken unconverted applications so a large value rarely wastes space. > - Should we make any attempt to support old binaries when there > are more than 16 groups? There is no way to support broken old binaries. They will have a limit of 16. > The POSIX getgroups/setgroups APIs did not > anticipate this change and thus either will fail outright. No, as you explained in previous mail, POSIX anticipated this perfectly unusably by making NGROUPS_MAX a Runtime Increasable Value despite it being constant. This puts the burden of not failing on applications. Only broken apps that don't do any of dynamic allocation using sysconf(_SC_NGROUPS_MAX), or dynamic allocation using getgroups(0, ...), or dynamic allocation after failing using getgroups(old_NGROUPS_MAX, ...) will fail outright. > We can't > fix setgroups, but we might want to make an optional accommodation for > getgroups to allow for truncated returns to old code. If done unconditionally, this would break the unbroken apps that do dynamic allocation after failing using getgroups(old_NGROUPS_MAX, ...). These apps depend on getgroups() failing with the Standard and documented errno EINVAL when the `gidsetlen' parameter is too small. Maybe some per-app or per-environment branding could be used to configure not generating an error. You would apply it only where security is not very important. getgrouplist(3) guarantees filling of the array when the array is too small. getgroups(2) doesn't guarantee anything, and in fact doesn't fill the array. It wouldn't hurt to fill it. % ... % diff -ru --exclude='.glimpse*' --exclude=.svn --exclude=compile --ignore-matching='$FreeBSD' /usr/src/sys/kern/kern_prot.c ngroups/sys/kern/kern_prot.c % --- /usr/src/sys/kern/kern_prot.c 2009-06-05 15:33:50.000000000 -0500 % +++ ngroups/sys/kern/kern_prot.c 2009-06-05 16:02:28.000000000 -0500 % @@ -243,16 +246,11 @@ % % td->td_retval[0] = td->td_ucred->cr_rgid; % #if defined(COMPAT_43) % - td->td_retval[1] = td->td_ucred->cr_groups[0]; % + td->td_retval[1] = td->td_ucred->cr_gid; % #endif % return (0); % } % % -/* % - * Get effective group ID. The "egid" is groups[0], and could be obtained % - * via getgroups. This syscall exists because it is somewhat painful to do % - * correctly in a library function. % - */ % #ifndef _SYS_SYSPROTO_H_ % struct getegid_args { % int dummy; This comment still seems to apply. I wonder if you noticed and/or fixed the related off-by-1 errors in getgroups() and {NGROUPS_MAX}: POSIX.1-2001-draft7 says: % 16834 IDs of the calling process. It is implementation-defined whether getgroups( ) also returns the % 16835 effective group ID in the grouplist array. FreeBSD does return the egid in the array, always as the first element. Neither of these details is documented in getgroups.2 or setgroups.2. Even the implementation doesn't seem to understand this where it is most important -- the implementation of setgroups() seems to be missing some of the things done by setegid(). getgrouplist(3) has explicit support for the egid being in both the initial and final lists (or missing in the initial list?). % ... % 16841 If the effective group ID of the process is returned with the supplementary group IDs, the value % 16842 returned shall always be greater than or equal to one and less than or equal to the value of % 16843 {NGROUPS_MAX}+1. This and probably other things indicate that _supplementary_ actually means supplementary -- it doesn't count the egid. But in FreeBSD, the egid is counted. This gives 2 off-by-1 errors that partially compensate for each other: - NGROUPS_MAX is 15, not 16, since it is impossible to have 16 _supplementary_ groups in cr_groups[0..15] after using cr_groups[0] for the egid - getgroups() cannot return {NGROUPS_MAX}+1 as is needed for it to return the egid plus {NGROUPS_MAX} supplementary groups - getgroups() does return {NGROUPS_MAX}+1 once {NGROUPS_MAX} is corrected. Some userland code depends on these bugs -- the static array size should be NGROUPS_MAX + 1, but it is typically plain NGROUPS. OTOH, libc/gen/initgroups.c uses NGROUPS + 1 and getgrouplist() followed by setgroups(). I think getgrouplist() can produce the extra 1 and then setgroups() and thus initgroups() will fail. I thought again about making cr_gid separate from the array, so that it can be named cr_gid without obfuscation. This wouldn't be good since it would complicate more than the copyin/out in get/setgroups() -- there are a few temporary gid arrays, and some iterations over the arrays that want to see the egid. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Jun 6 23:01:24 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7BC5106566B for ; Sat, 6 Jun 2009 23:01:24 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id ABD288FC0A for ; Sat, 6 Jun 2009 23:01:24 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.64.3]) by phk.freebsd.dk (Postfix) with ESMTP id 56B996995B; Sat, 6 Jun 2009 22:43:26 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.3/8.14.3) with ESMTP id n56MheMO006658; Sat, 6 Jun 2009 22:43:40 GMT (envelope-from phk@critter.freebsd.dk) To: Alexander Motin From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 05 Jun 2009 19:54:27 +0300." <4A294DC3.5010008@mavhome.dp.ua> Date: Sat, 06 Jun 2009 22:43:40 +0000 Message-ID: <6657.1244328220@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 23:01:25 -0000 In message <4A294DC3.5010008@mavhome.dp.ua>, Alexander Motin writes: >I think ATAPI disk device is theoretically possible, but I believe it >does not exist in practice, as industry do not need it. Maxtor ZIP ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Jun 6 23:15:13 2009 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A73BE1065672; Sat, 6 Jun 2009 23:15:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from cmail.optima.ua (cmail.optima.ua [195.248.191.121]) by mx1.freebsd.org (Postfix) with ESMTP id EC8B68FC23; Sat, 6 Jun 2009 23:15:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from [212.86.226.226] (account mav@alkar.net HELO mavbook.mavhome.dp.ua) by cmail.optima.ua (CommuniGate Pro SMTP 5.2.9) with ESMTPSA id 245100807; Sun, 07 Jun 2009 02:15:09 +0300 Message-ID: <4A2AF876.1030103@FreeBSD.org> Date: Sun, 07 Jun 2009 02:15:02 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.21 (X11/20090405) MIME-Version: 1.0 To: Poul-Henning Kamp References: <6657.1244328220@critter.freebsd.dk> In-Reply-To: <6657.1244328220@critter.freebsd.dk> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 23:15:14 -0000 Poul-Henning Kamp wrote: > In message <4A294DC3.5010008@mavhome.dp.ua>, Alexander Motin writes: >> I think ATAPI disk device is theoretically possible, but I believe it >> does not exist in practice, as industry do not need it. > > Maxtor ZIP ? May be, never had an ATA version. But it is more FDD, then HDD. Also it existed in Parallel Port and SCSI versions, so it could be done in ATAPI way just for unification. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sat Jun 6 23:33:19 2009 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9319106564A; Sat, 6 Jun 2009 23:33:19 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id D96D38FC12; Sat, 6 Jun 2009 23:33:17 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.2/8.14.1) with ESMTP id n56NXHeZ090342; Sat, 6 Jun 2009 16:33:17 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.2/8.13.4/Submit) id n56NXH6a090341; Sat, 6 Jun 2009 16:33:17 -0700 (PDT) Date: Sat, 6 Jun 2009 16:33:17 -0700 (PDT) From: Matthew Dillon Message-Id: <200906062333.n56NXH6a090341@apollo.backplane.com> To: Alexander Motin References: <6657.1244328220@critter.freebsd.dk> <4A2AF876.1030103@FreeBSD.org> Cc: FreeBSD-Current , freebsd-arch@FreeBSD.org Subject: Re: WIP: ATA to CAM integration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 23:33:20 -0000 I found the ATAPI_C_ATAPI_IDENTIFY command that was mentioned and it works fine, returning the same sort of information for ATAPI attachments that ATA_C_IDENTIFY returns for DISK attachments. That takes care of the queue length negotiation by the device. However, there is no fis->command that I can find that would allow NCQ to operate in ATAPI mode. In ATAPI mode fis->command is typically set to ATA_C_PACKET. In DISK mode fis->command is set to ATA_C_READ_FPDMA or ATA_C_WRITE_FPDMA (the first-person DMA mode used by AHCI's NCQ). So unless the *_FPDMA FIS commands work for an ATAPI attached device, we are S.O.L. Section 5.6.4.1: The ATA/ATAPI-7 queued feature set is not supported by AHCI (including READ QUEUED (EXT), WRITE QUEUED (EXT), and SERVICE commands). Queued operations are supported in AHCI using the READ FPDMA QUEUED and WRITE FPDMA QUEUED commands when the HBA and device support native command queueing. It is unclear whether an ATAPI device would accept a non-packet command, aka ATA_C_READ_FPDMA or ATA_C_WRITE_FPDMA, instead of ATA_C_PACKET. ATAPI devices do support the ATAPI_C_ATAPI_IDENTIFY command, which is non-packet command, so maybe its possible. If it is possible it would only work for READ and WRITE commands, since those are the only commands the FPDMA modes can be used for. The AHCI spec doesn't explicitly say that the FPDMA commands would not work for an ATAPI attached device, so there's hope. What we need is a SATA ATAPI device which says it supports NCQ + has a queue length > 1 to test with. -Matt Matthew Dillon