From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 07:13:27 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA77E1065672; Sun, 29 Jan 2012 07:13:27 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 575BA8FC12; Sun, 29 Jan 2012 07:13:27 +0000 (UTC) Received: from 63.imp.bsdimp.com (63.imp.bsdimp.com [10.0.0.63]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id q0T75otE054565 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Sun, 29 Jan 2012 00:05:50 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sun, 29 Jan 2012 00:05:50 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> To: Juli Mallett X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Sun, 29 Jan 2012 00:05:53 -0700 (MST) Cc: Aleksandr Rybalko , Stefan Bethke , freebsd-net@FreeBSD.org, Adrian Chadd , Aleksandr Rybalko , freebsd-arch@FreeBSD.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 07:13:27 -0000 On Jan 28, 2012, at 4:00 PM, Juli Mallett wrote: > It makes me wonder if the understanding of the relationship in FreeBSD > isn't backwards. Yes, the MAC sits on a bus and is memory-mapped, but > you can conceptualize of it as a child of the PHY, rather than the > parent of it, especially in systems with switch chipsets. Especially > in systems where there is a switch chipset attached to multiple MACs. >=20 > In that model, it makes sense to semi-generically attach a > CPU-to-switch port's pseudo-PHY (or actual PHY, depending on hardware) > to a MAC generically, but that doesn't meant that the switch itself is > attached generically to the MAC. I think that the main issue here is that we have an assumption that we = have a tree structure. However, it is more of a DAG across different = domains. The hierarchy works well when each device owns all the devices = below it and only interacted with them. Most devices are that way. = However, in the embedded world, there's lots of reach-accrosses that are = expected that break the model. Plus, MDIO is more than what we call mii/phy, so there's an imperfect = match there. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 11:14:55 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7389E1065672; Sun, 29 Jan 2012 11:14:55 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 189FE8FC08; Sun, 29 Jan 2012 11:14:53 +0000 (UTC) Received: by eekb47 with SMTP id b47so1236538eek.13 for ; Sun, 29 Jan 2012 03:14:53 -0800 (PST) Received: by 10.14.8.196 with SMTP id 44mr4216886eer.21.1327835692938; Sun, 29 Jan 2012 03:14:52 -0800 (PST) Received: from rnote.ddteam.net (238-239-92-178.pool.ukrtel.net. [178.92.239.238]) by mx.google.com with ESMTPS id e12sm56906377eea.5.2012.01.29.03.14.50 (version=SSLv3 cipher=OTHER); Sun, 29 Jan 2012 03:14:51 -0800 (PST) Date: Sun, 29 Jan 2012 13:14:37 +0200 From: Aleksandr Rybalko To: Juli Mallett Message-Id: <20120129131437.27981686.ray@ddteam.net> In-Reply-To: References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Adrian Chadd , Stefan Bethke , freebsd-net@freebsd.org, Aleksandr, Rybalko , freebsd-arch@freebsd.org Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 11:14:55 -0000 On Sat, 28 Jan 2012 15:00:01 -0800 Juli Mallett wrote: > On Sat, Jan 28, 2012 at 14:12, Aleksandr Rybalko > wrote: > > As I see from your patch, mdio/miiproxy require special bits in MAC > > driver. When I design switch framework, I keeping in mind that > > MAC drivers should be standard as possible > > I don't understand why this is desirable in practice. It's a nice > theory, but it falls down when one thinks in depth about how Ethernet > interfaces are used and administered vs. how switches are used and > administered. What should media report? What should media changes > do? What is link status? Do you show the CPU-to-switch port, or all > switch ports? IMO, if we about why to make MAC driver more "standard", it is desired because most MAC, used in SoC's, used also in NIC cards or other SoC's which have no switch, but MII and MDIO pins to attach external PHY. Real example bfe(4), it is supported long ago as PCI card, but also found in BCM4701 family SoC's. In BCM5354 it attached to embedded switch. In BCM5836 it is two independent MAC. Last one used in D-Link DIR-330, there is one MAC attached to PHY, second to BCM5325 switch). Since BCM4701 family have internal SSB bus, a.k.a siba(4), I add only siba(4) bus glue. So if we able to attach switch driver to regular MAC drivers it will make our life simpler :) > > It makes me wonder if the understanding of the relationship in FreeBSD > isn't backwards. Yes, the MAC sits on a bus and is memory-mapped, but > you can conceptualize of it as a child of the PHY, rather than the > parent of it, especially in systems with switch chipsets. Especially > in systems where there is a switch chipset attached to multiple MACs. > > In that model, it makes sense to semi-generically attach a > CPU-to-switch port's pseudo-PHY (or actual PHY, depending on hardware) > to a MAC generically, but that doesn't meant that the switch itself is > attached generically to the MAC. > > There are a lot of switches out there that don't look or act much like > MII-driven PHYs, but which are connected over MDIO, as I've tried to > stress before. Yeah, for that case Marius commit phymask support for our miibus, to not touch reserved PHY IDs. >I hope there will be as much separation between the > MII work that is being done and the switch work that is being done as > possible. I think anything else will prove rapidly-obsolete and > perhaps even obstructive as soon as anyone seeks to add support for > more switch chipsets to FreeBSD. > > I suppose the most likely reality, though, is that people simply won't > add switch support to FreeBSD, and will administer them out-of-band > from userland, using gross kernel shims. That is probably true > whether any of the currently-outstanding work is committed or not, > unfortunately :( I just hope we'll end up with something flexible > enough, broad enough in applicability, narrow enough in requirements, > etc., that we'll have feature-rich support for a few chipsets in tree > that work in sufficiently-different manners that they can be models > for other drivers in the future. > > Thanks, > Juli. Thank you for comments Juli! WBW -- Aleksandr Rybalko From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 11:32:00 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF8FA106566B; Sun, 29 Jan 2012 11:31:59 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail13.syd.optusnet.com.au (mail13.syd.optusnet.com.au [211.29.132.194]) by mx1.freebsd.org (Postfix) with ESMTP id 5F30F8FC18; Sun, 29 Jan 2012 11:31:58 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail13.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q0TBVpHs003567 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 29 Jan 2012 22:31:55 +1100 Date: Sun, 29 Jan 2012 22:31:51 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Jilles Tjoelker In-Reply-To: <20120128233712.GA92694@stack.nl> Message-ID: <20120129215808.V844@besplex.bde.org> References: <20120110005155.S2378@besplex.bde.org> <20120110153807.H943@besplex.bde.org> <20120128233712.GA92694@stack.nl> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: flo@FreeBSD.org, Giovanni Trematerra , Attilio Rao , Konstantin Belousov , freebsd-arch@FreeBSD.org Subject: Re: pipe/fifo code merged. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 11:32:00 -0000 On Sun, 29 Jan 2012, Jilles Tjoelker wrote: > On Tue, Jan 17, 2012 at 12:43:19PM +0100, Giovanni Trematerra wrote: >> Did you agree that this patch >> http://www.trematerra.net/patches/pipefifo_merge2.4.diff Sorry I haven;t got around to reviewing this. >> doesn't introduce any further regressions while it fixes > >> - style bugs you pointed out. >> - pipe_stat now use underlying filesystem information for pipes. The last still can't work. It is necessary to use the underlying file system for all read and write metadata operations. It is too hard to emulate what the underlying file system would do and write to it when the pipe state is released. >> - comment about pipeinfo was intended for. >> - race into pipe_poll (look at fifo_iseof line). > > I tested this version of the patch and found that it breaks opening a > fifo with O_TRUNC: it fails with [EINVAL]. This appears to be > pipe_truncate()'s doing. Previously, truncate requests went to the > vnode. This probably affects truncate() and ftruncate() too. Most file systems accept bogus truncation requests and do nothing, but pipe_truncate() always returns EINVAL. POSIX requires that O_TRUNC have no effect for fifos and terminals, and of course to work if the file is regular. Its effect for other file types is implementation-defined. The FreeBSD implementation doesn't actually define (document) the behaviour anywhere AFAIK. Its documentation in open(2) is just broken, since says that O_TRUNC causes truncation to size 0 without mentioning any restrictions on the file type. POSIX requires ftruncate() to fail if the fd refers to a directory, and of course to work if the fd refers to a regular file, and to have certain behaviour if the fd refers to a shared memory object. Its effect for other file types is unspecifed. ftruncate is normally implemented in the file system's VOP_SETATTR() (fo_truncate for vnodes is vn_truncate() which calls VOP_SETATTR()). If you fix the attributes, then O_TRUNC will be fixed automatically. devfs implements fo_truncate bogusly, using a lot of code to do almost nothing. It starts with a perfectly suitable fileop for this, namely devfs_ops_f = devfs_truncate_f(). This just calls vop.fo_truncate = vn_truncate(), which executes a lot of code that reduces to calling VOP_SETATTR() = devfs_vnodeops.vop_setattr = devfs_settatr(). devfs_setattr() then silenly ignores the request to set the size. But this is wrong for directories. However, an error is returned for directories because vn_truncate() doesn't quite reduce to VOP_SETATTR(). It also checks for directories and returns EISDIR for them. Perhaps always going through the vn layer is a good idea after all. ffs has the following complications in ufs_setattr() for truncation: - truncation works on symlinks too. This seems to be undocumented. - but a read-only mount gives error EROFS - a snapshot for the inode gives EPERM. This seems to be undocumented. - immutability or append-only for the inode gives EPERM. BTW, most or all setattr() vops have a broken test for va_bytes == VNOVAL. They break va_bytes in the test by casting it to int. Thus 2**32-1 values for it with the low 32 bits set are misclassified as VNOVAL (1 of such 2**32 values is correctly classified). But this has no effect, since va_bytes is always VNOVAL. I think most or all the other unsettable attributes that are laboriously checked for by setattr vnops are also impossible. It would take a kernel bug for a generic unsettable attributes to be requested to be set. A really large bug, like requesting to change the file flags instead of the file mode. Checking for the generic unsettable attributes just obscures the checks for unsupported attributes that some file systems should be doing. Bruce From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 12:26:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8BF8106566C; Sun, 29 Jan 2012 12:26:02 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 915038FC12; Sun, 29 Jan 2012 12:26:02 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id C7D632772C; Sun, 29 Jan 2012 12:26:00 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: Date: Sun, 29 Jan 2012 13:26:00 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <045BA867-B83B-4C2E-AB4F-4D09E49B7F3D@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> To: Juli Mallett X-Mailer: Apple Mail (2.1251.1) Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , Aleksandr Rybalko , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 12:26:03 -0000 Am 29.01.2012 um 00:00 schrieb Juli Mallett: > On Sat, Jan 28, 2012 at 14:12, Aleksandr Rybalko = wrote: >> As I see from your patch, mdio/miiproxy require special bits in MAC >> driver. When I design switch framework, I keeping in mind that >> MAC drivers should be standard as possible >=20 > I don't understand why this is desirable in practice. It's a nice > theory, but it falls down when one thinks in depth about how Ethernet > interfaces are used and administered vs. how switches are used and > administered. What should media report? What should media changes > do? What is link status? Do you show the CPU-to-switch port, or all > switch ports? The main thrust here is to reuse the existing PHY code to be able to = configure the PHYs that are embedded in the switch chips. To confuse = things, one of these PHYs might be connected to the SoCs ethernet = interface via MII, GMII, etc. To confuse things further, these PHYs are = controlled by an MDIO bus that has it's master in the switch chip, while = the switch chip is a slave to the MDIO master in the ethernet = controller. The goal is to be able to configure the switch ports and set media on = one of them, for example. That code path could be entirely independent = from the ethernet infrastructure, if dev/miibus didn't require if_media = and hence a struct ifnet. This discussion is also about how to deal = with this entanglement. The MII connection between the ethernet controller and the switch chip = (usually referred to as the "CPU" port) is hard-coded and has no media = settings, so there's no question what if_media settings should be = presented on the interface. > are a lot of switches out there that don't look or act much like > MII-driven PHYs, but which are connected over MDIO, as I've tried to > stress before. I hope there will be as much separation between the > MII work that is being done and the switch work that is being done as > possible. I think anything else will prove rapidly-obsolete and > perhaps even obstructive as soon as anyone seeks to add support for > more switch chipsets to FreeBSD. >=20 > I suppose the most likely reality, though, is that people simply won't > add switch support to FreeBSD, and will administer them out-of-band > from userland, using gross kernel shims. That is probably true > whether any of the currently-outstanding work is committed or not, > unfortunately :( I just hope we'll end up with something flexible > enough, broad enough in applicability, narrow enough in requirements, > etc., that we'll have feature-rich support for a few chipsets in tree > that work in sufficiently-different manners that they can be models > for other drivers in the future. Which is a valid approach from a vendor's viewpoint. The reason we're = talking is to try and make it easier to write a switch driver for this = framework than roll your own hack. :-) Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 12:44:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 128CF1065672; Sun, 29 Jan 2012 12:44:31 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id D3B488FC14; Sun, 29 Jan 2012 12:44:28 +0000 (UTC) Received: by eaaa14 with SMTP id a14so1254080eaa.13 for ; Sun, 29 Jan 2012 04:44:28 -0800 (PST) Received: by 10.213.102.12 with SMTP id e12mr2259927ebo.128.1327841067864; Sun, 29 Jan 2012 04:44:27 -0800 (PST) Received: from rnote.ddteam.net (238-239-92-178.pool.ukrtel.net. [178.92.239.238]) by mx.google.com with ESMTPS id t11sm30969724eea.10.2012.01.29.04.44.25 (version=SSLv3 cipher=OTHER); Sun, 29 Jan 2012 04:44:27 -0800 (PST) Date: Sun, 29 Jan 2012 14:44:13 +0200 From: Aleksandr Rybalko To: Stefan Bethke Message-Id: <20120129144413.a8045873.ray@ddteam.net> In-Reply-To: <045BA867-B83B-4C2E-AB4F-4D09E49B7F3D@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <045BA867-B83B-4C2E-AB4F-4D09E49B7F3D@lassitu.de> X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Juli Mallett , FreeBSD Net , Adrian Chadd , Aleksandr Rybalko , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 12:44:31 -0000 On Sun, 29 Jan 2012 13:26:00 +0100 Stefan Bethke wrote: > Am 29.01.2012 um 00:00 schrieb Juli Mallett: > > > On Sat, Jan 28, 2012 at 14:12, Aleksandr Rybalko > > wrote: > >> As I see from your patch, mdio/miiproxy require special bits in MAC > >> driver. When I design switch framework, I keeping in mind that > >> MAC drivers should be standard as possible > > > > I don't understand why this is desirable in practice. It's a nice > > theory, but it falls down when one thinks in depth about how > > Ethernet interfaces are used and administered vs. how switches are > > used and administered. What should media report? What should > > media changes do? What is link status? Do you show the > > CPU-to-switch port, or all switch ports? > > The main thrust here is to reuse the existing PHY code to be able to > configure the PHYs that are embedded in the switch chips. To confuse > things, one of these PHYs might be connected to the SoCs ethernet > interface via MII, GMII, etc. To confuse things further, these PHYs > are controlled by an MDIO bus that has it's master in the switch > chip, while the switch chip is a slave to the MDIO master in the > ethernet controller. > > The goal is to be able to configure the switch ports and set media on > one of them, for example. That code path could be entirely > independent from the ethernet infrastructure, if dev/miibus didn't > require if_media and hence a struct ifnet. This discussion is also > about how to deal with this entanglement. > > The MII connection between the ethernet controller and the switch > chip (usually referred to as the "CPU" port) is hard-coded and has no > media settings, so there's no question what if_media settings should > be presented on the interface. Most switches (AR8x16 for example) have configurable MII port, so you can choose to use MII/RMII/GMII/RGMII. If you decide to use MII media can not be higher than 100baseTX. So it is possible to use some auto-negotiation here. > > > are a lot of switches out there that don't look or act much like > > MII-driven PHYs, but which are connected over MDIO, as I've tried to > > stress before. I hope there will be as much separation between the > > MII work that is being done and the switch work that is being done > > as possible. I think anything else will prove rapidly-obsolete and > > perhaps even obstructive as soon as anyone seeks to add support for > > more switch chipsets to FreeBSD. > > > > I suppose the most likely reality, though, is that people simply > > won't add switch support to FreeBSD, and will administer them > > out-of-band from userland, using gross kernel shims. That is > > probably true whether any of the currently-outstanding work is > > committed or not, unfortunately :( I just hope we'll end up with > > something flexible enough, broad enough in applicability, narrow > > enough in requirements, etc., that we'll have feature-rich support > > for a few chipsets in tree that work in sufficiently-different > > manners that they can be models for other drivers in the future. > > Which is a valid approach from a vendor's viewpoint. The reason > we're talking is to try and make it easier to write a switch driver > for this framework than roll your own hack. :-) > > > Stefan > > -- > Stefan Bethke Fon +49 151 14070811 > > > -- Aleksandr Rybalko From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 12:49:26 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D800A106566B; Sun, 29 Jan 2012 12:49:26 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 8FDE38FC0C; Sun, 29 Jan 2012 12:49:26 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 8E6AC27835; Sun, 29 Jan 2012 12:49:25 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20120129144413.a8045873.ray@ddteam.net> Date: Sun, 29 Jan 2012 13:49:24 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5DD20168-B908-4737-9983-3570E2BF32D3@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <045BA867-B83B-4C2E-AB4F-4D09E49B7F3D@lassitu.de> <20120129144413.a8045873.ray@ddteam.net> To: Aleksandr Rybalko X-Mailer: Apple Mail (2.1251.1) Cc: Juli Mallett , FreeBSD Net , Adrian Chadd , Aleksandr Rybalko , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 12:49:26 -0000 Am 29.01.2012 um 13:44 schrieb Aleksandr Rybalko: >> The MII connection between the ethernet controller and the switch >> chip (usually referred to as the "CPU" port) is hard-coded and has no >> media settings, so there's no question what if_media settings should >> be presented on the interface. >=20 > Most switches (AR8x16 for example) have configurable MII port, so you > can choose to use MII/RMII/GMII/RGMII. If you decide to use MII > media can not be higher than 100baseTX. So it is possible to use some > auto-negotiation here. The point is that the admin has no need to change it, it's hard coded in = the driver or statically configured via hints. And for all of this = discussion, I'm using MII as a synonym for all xMII busses. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 13:14:44 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 804E51065672; Sun, 29 Jan 2012 13:14:44 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 3930A8FC0A; Sun, 29 Jan 2012 13:14:44 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id EE9D827A42; Sun, 29 Jan 2012 13:14:42 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: Date: Sun, 29 Jan 2012 14:14:42 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> To: Warner Losh X-Mailer: Apple Mail (2.1251.1) Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 13:14:44 -0000 Am 29.01.2012 um 08:05 schrieb Warner Losh: > I think that the main issue here is that we have an assumption that we = have a tree structure. However, it is more of a DAG across different = domains. The hierarchy works well when each device owns all the devices = below it and only interacted with them. Most devices are that way. = However, in the embedded world, there's lots of reach-accrosses that are = expected that break the model. What do people think would be a good approach to solve the concrete = issue without too much hackery? Aleksandr and I have presented three = possible approaches: 1. have a PHY driver that acts as a proxy and talks to a separate MDIO = master 2. have a proxy between the ethernet driver and the miibus driver 3. modify miibus to have a separate device for mediachg() etc. All three suffer from a dependency problem: miibus attachment can only = complete successfully when both the ethernet driver and the mdio driver = have been attached. In the current model, the ethernet driver provides = both the MDIO access as well as the target for the mediachg, etc. = messages, so there's no attachment ordering issue. In my miiproxy patch (2.), it is necessary for the ethernet driver to = cooperate in this delayed attachment, by splitting out the miibus = attachment from the device_attach method implementation. That's a = fairly intrusive change, and that would need to be made to any ethernet = driver that is to support such a split MDIO/MII setup. I tried delaying just the call to mii_attach(), but it seems that = interacts badly with an already attached interface. Specifically, I got = a non-working interface when I tried to call mii_attach() long after = if_etherattach(). Additionally, if_arge (and probably most other = drivers) assumes that the miibus is attached after they've successfully = attached themselves, and subsequently runs into null pointer issues when = it isn't. Would it be possible to attach miibus without having a working PHY, and = only attach the PHYs once the MDIO device has been attached? For the mips/atheros platforms, we could introduce ordering hints for = nexus to make sure the mdio driver gets attached before the arge's. = That's a fairly small changes (two lines), and does work, but it's more = a hack than a general solution. With my proposed change to miibus = (split devices), that would bring down the necessary changes to just a = handful of lines. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 15:32:08 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FE75106564A; Sun, 29 Jan 2012 15:32:08 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id ED6358FC0C; Sun, 29 Jan 2012 15:32:07 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0TFW0AG044415; Sun, 29 Jan 2012 16:32:01 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0TFVxEk044414; Sun, 29 Jan 2012 16:31:59 +0100 (CET) (envelope-from marius) Date: Sun, 29 Jan 2012 16:31:59 +0100 From: Marius Strobl To: Stefan Bethke Message-ID: <20120129153159.GA44362@alchemy.franken.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 15:32:08 -0000 On Sun, Jan 29, 2012 at 02:14:42PM +0100, Stefan Bethke wrote: > Am 29.01.2012 um 08:05 schrieb Warner Losh: > > > I think that the main issue here is that we have an assumption that we have a tree structure. However, it is more of a DAG across different domains. The hierarchy works well when each device owns all the devices below it and only interacted with them. Most devices are that way. However, in the embedded world, there's lots of reach-accrosses that are expected that break the model. > > What do people think would be a good approach to solve the concrete issue without too much hackery? Aleksandr and I have presented three possible approaches: > 1. have a PHY driver that acts as a proxy and talks to a separate MDIO master > 2. have a proxy between the ethernet driver and the miibus driver > 3. modify miibus to have a separate device for mediachg() etc. > > All three suffer from a dependency problem: miibus attachment can only complete successfully when both the ethernet driver and the mdio driver have been attached. In the current model, the ethernet driver provides both the MDIO access as well as the target for the mediachg, etc. messages, so there's no attachment ordering issue. > > In my miiproxy patch (2.), it is necessary for the ethernet driver to cooperate in this delayed attachment, by splitting out the miibus attachment from the device_attach method implementation. That's a fairly intrusive change, and that would need to be made to any ethernet driver that is to support such a split MDIO/MII setup. > > I tried delaying just the call to mii_attach(), but it seems that interacts badly with an already attached interface. Specifically, I got a non-working interface when I tried to call mii_attach() long after if_etherattach(). Additionally, if_arge (and probably most other drivers) assumes that the miibus is attached after they've successfully attached themselves, and subsequently runs into null pointer issues when it isn't. > > Would it be possible to attach miibus without having a working PHY, and only attach the PHYs once the MDIO device has been attached? This would break the concept of knowing that when mii_attach() returns success you can talk to all the hardware necessary to present a working interface. So you'll just need another way instead to ensure that there's also at least a single PHY before attaching the whole thing which I don't see getting us further with the attach ordering problem of the MDIO provider. > > For the mips/atheros platforms, we could introduce ordering hints for nexus to make sure the mdio driver gets attached before the arge's. That's a fairly small changes (two lines), and does work, but it's more a hack than a general solution. With my proposed change to miibus (split devices), that would bring down the necessary changes to just a handful of lines. > How about adding the MDIO provider via multi-pass probing? That idea of that model was to attach things like drivers for interrupt controllers before any other driver that requires that resource. This seems like a perfect match here and requires neither a specific hack to the nexus (the platform generally needs to be multi-pass aware though and the thing enabled in subr_bus.c) nor a hack to miibus(4). We really need to find a proper way of dealing with the constraints of the embedded- world rather than to sprinkle hacks all over the place. Marius From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 16:00:41 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BEF81065678; Sun, 29 Jan 2012 16:00:41 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 537CC8FC18; Sun, 29 Jan 2012 16:00:41 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id B69E927CAD; Sun, 29 Jan 2012 16:00:39 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20120129153159.GA44362@alchemy.franken.de> Date: Sun, 29 Jan 2012 17:00:38 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <3B5A749B-9553-4152-9441-3F343BA219FB@lassitu.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> To: Marius Strobl X-Mailer: Apple Mail (2.1251.1) Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 16:00:41 -0000 Am 29.01.2012 um 16:31 schrieb Marius Strobl: > How about adding the MDIO provider via multi-pass probing? That idea > of that model was to attach things like drivers for interrupt = controllers > before any other driver that requires that resource. This seems like a > perfect match here and requires neither a specific hack to the nexus > (the platform generally needs to be multi-pass aware though and the > thing enabled in subr_bus.c) nor a hack to miibus(4). Please recall the devinfo graph that I posted earlier. The PHY arge0 = needs to talk to is attached to the MDIO master on the switch = controller, which in turn is attached to GE1's MDIO master. This would = require early attachment of the switch, in turn requiring early = attachment of arge_mdio, in turn requiring early attachment of = mips/mips/nexus. > We really need > to find a proper way of dealing with the constraints of the embedded- > world rather than to sprinkle hacks all over the place. Why is the above is less of a hack than making the ordering in nexus = configurable through a hint? Stefan --=20 Stefan Bethke Fon +49 151 14070811 diff --git a/sys/mips/mips/nexus.c b/sys/mips/mips/nexus.c index b51357d..c4c207a 100644 --- a/sys/mips/mips/nexus.c +++ b/sys/mips/mips/nexus.c @@ -240,8 +240,11 @@ nexus_hinted_child(device_t bus, const char *dname, = int dunit) int result; int irq; int mem_hints_count; + int order; =20 - child =3D BUS_ADD_CHILD(bus, 0, dname, dunit); + order =3D 1000; /* there should be a define for the default = order */ + resource_int_value(dname, dunit, "order", &order); + child =3D BUS_ADD_CHILD(bus, order, dname, dunit); if (child =3D=3D NULL) return; From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 16:19:44 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1B98106566C; Sun, 29 Jan 2012 16:19:44 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 374F08FC18; Sun, 29 Jan 2012 16:19:42 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0TGJcXQ044634; Sun, 29 Jan 2012 17:19:38 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0TGJcnA044633; Sun, 29 Jan 2012 17:19:38 +0100 (CET) (envelope-from marius) Date: Sun, 29 Jan 2012 17:19:38 +0100 From: Marius Strobl To: Stefan Bethke Message-ID: <20120129161938.GD39861@alchemy.franken.de> References: <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> <3B5A749B-9553-4152-9441-3F343BA219FB@lassitu.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3B5A749B-9553-4152-9441-3F343BA219FB@lassitu.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 16:19:44 -0000 On Sun, Jan 29, 2012 at 05:00:38PM +0100, Stefan Bethke wrote: > Am 29.01.2012 um 16:31 schrieb Marius Strobl: > > > How about adding the MDIO provider via multi-pass probing? That idea > > of that model was to attach things like drivers for interrupt controllers > > before any other driver that requires that resource. This seems like a > > perfect match here and requires neither a specific hack to the nexus > > (the platform generally needs to be multi-pass aware though and the > > thing enabled in subr_bus.c) nor a hack to miibus(4). > > Please recall the devinfo graph that I posted earlier. The PHY arge0 needs to talk to is attached to the MDIO master on the switch controller, which in turn is attached to GE1's MDIO master. This would require early attachment of the switch, in turn requiring early attachment of arge_mdio, in turn requiring early attachment of mips/mips/nexus. > Yes > > We really need > > to find a proper way of dealing with the constraints of the embedded- > > world rather than to sprinkle hacks all over the place. > > Why is the above is less of a hack than making the ordering in nexus configurable through a hint? > If it's generally true that driver A must be attached before driver B as B has a dependency on A than this should be expressed and configured in the drivers themselves and not need an extra hint to configure it at the runtime of the kernel. Marius From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 16:21:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3635106564A; Sun, 29 Jan 2012 16:21:54 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [IPv6:2001:470:1f0b:105e::1ea]) by mx1.freebsd.org (Postfix) with ESMTP id 585B68FC1E; Sun, 29 Jan 2012 16:21:54 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 7352827D58; Sun, 29 Jan 2012 16:21:53 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20120129161938.GD39861@alchemy.franken.de> Date: Sun, 29 Jan 2012 17:21:52 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5785C7EF-B886-459B-911C-870A4594AC4C@lassitu.de> References: <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> <3B5A749B-9553-4152-9441-3F343BA219FB@lassitu.de> <20120129161938.GD39861@alchemy.franken.de> To: Marius Strobl X-Mailer: Apple Mail (2.1251.1) Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 16:21:54 -0000 Am 29.01.2012 um 17:19 schrieb Marius Strobl: >>> We really need >>> to find a proper way of dealing with the constraints of the = embedded- >>> world rather than to sprinkle hacks all over the place. >>=20 >> Why is the above is less of a hack than making the ordering in nexus = configurable through a hint? >>=20 >=20 > If it's generally true that driver A must be attached before driver > B as B has a dependency on A than this should be expressed and > configured in the drivers themselves and not need an extra hint to > configure it at the runtime of the kernel. Except that it is not generally true, but only in specific = configurations. Other boards have other combinations of devices. The = Atheros family of switches is available both embedded into certain SoC = as well as separate silicon. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 17:32:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A25BF106564A; Sun, 29 Jan 2012 17:32:23 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 17F1B8FC13; Sun, 29 Jan 2012 17:32:22 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0THWHLE044852; Sun, 29 Jan 2012 18:32:17 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0THWHrX044851; Sun, 29 Jan 2012 18:32:17 +0100 (CET) (envelope-from marius) Date: Sun, 29 Jan 2012 18:32:17 +0100 From: Marius Strobl To: Stefan Bethke Message-ID: <20120129173217.GF39861@alchemy.franken.de> References: <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> <3B5A749B-9553-4152-9441-3F343BA219FB@lassitu.de> <20120129161938.GD39861@alchemy.franken.de> <5785C7EF-B886-459B-911C-870A4594AC4C@lassitu.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5785C7EF-B886-459B-911C-870A4594AC4C@lassitu.de> User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , Aleksandr Rybalko , Adrian Chadd , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 17:32:23 -0000 On Sun, Jan 29, 2012 at 05:21:52PM +0100, Stefan Bethke wrote: > > Am 29.01.2012 um 17:19 schrieb Marius Strobl: > > >>> We really need > >>> to find a proper way of dealing with the constraints of the embedded- > >>> world rather than to sprinkle hacks all over the place. > >> > >> Why is the above is less of a hack than making the ordering in nexus configurable through a hint? > >> > > > > If it's generally true that driver A must be attached before driver > > B as B has a dependency on A than this should be expressed and > > configured in the drivers themselves and not need an extra hint to > > configure it at the runtime of the kernel. > > Except that it is not generally true, but only in specific configurations. Other boards have other combinations of devices. The Atheros family of switches is available both embedded into certain SoC as well as separate silicon. > It still seems to be true to me that as soon as you have a separate MDIO driver that this one needs to be attached before any Ethernet, switch or whatever driver that needs to talk via the MDIO lines of the former. Similarly, if the switch attaches to an MDIO instance directly, that switch would need to be attached before the Ethernet driver (if there is one) that the switch is the MDIO master for (this isn't exactly the problematic arge-case). Multi-pass is per driver module, so if the switch attaches to a "regular" Ethernet driver providing both the MAC and the MDIO master instead, you can leave the default probe order and attach the switch after the Ethernet driver. So for the problematic arge-case you would attach the MDIO driver first (corresponding to the MDIO of arge1), then the switch to the MDIO driver and both arge0 and arge1 in whatever order last. Allowing that one special case to work properly shouldn't interfere with other device combinations as it basically boils down to the presence of a separate MDIO instance. Actually that should also work just fine when the MDIO master sits on the IIC bus as that again would mean that it needs to be attached before a Ethernet or switch driver can use it. Marius From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 19:00:27 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6EBF1065675; Sun, 29 Jan 2012 19:00:27 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 66D188FC0A; Sun, 29 Jan 2012 19:00:23 +0000 (UTC) Received: by vcmm1 with SMTP id m1so3624036vcm.13 for ; Sun, 29 Jan 2012 11:00:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=S+16ggzHz2I9dV8lc5XXjT/wxLTJd8WnLPIbAgQx6NM=; b=nQwbq6s1P0vfu2yKG9gxhK4U6G5xnzDS/rRt91+IqRA7FuH0ArJXxTJf7eqOy64gpH Q8DnF/y8gRJ2IG6AOGKM9YOFiu3bicIqG+FMQ8sOTQlZKJcBWysRT9+I+rnPG2kdhg9p zzfjB12QUTk1RPlio628QMmGlh/xSsjBjJvAE= MIME-Version: 1.0 Received: by 10.220.156.199 with SMTP id y7mr4940381vcw.2.1327863622709; Sun, 29 Jan 2012 11:00:22 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.52.172.37 with HTTP; Sun, 29 Jan 2012 11:00:22 -0800 (PST) In-Reply-To: <20120129153159.GA44362@alchemy.franken.de> References: <20120120221319.ca8b631f.ray@freebsd.org> <30A45A1E-CA13-4AC8-86FB-F8E06301D1F6@lassitu.de> <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> Date: Sun, 29 Jan 2012 11:00:22 -0800 X-Google-Sender-Auth: -AkCx4Pg7npYgNDM18SU2ksap2M Message-ID: From: Adrian Chadd To: Marius Strobl Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Net , Aleksandr Rybalko , Stefan Bethke , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 19:00:28 -0000 I think for switches, that doesn't necessarily hold. ie, mii_attach() for single-PHY devices may work that way, but the weird and wonderful world of embedded switch SoC's doesn't. You're lucky in most instances since the bootloader does give you a mostly-working switch config. But I doubt that's a guarantee on all platforms. So for those (ethernet) devices, splitting out the mii/mdio stuff and _not_ necessarily presenting a working PHY may be acceptable. For now it's going to be a subset, to having it export an MDIO bus and doing late MII attachment may not be as architecturally "no-no" as I interpret your post. Adrian From owner-freebsd-arch@FreeBSD.ORG Sun Jan 29 22:13:19 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C01AF1065787; Sun, 29 Jan 2012 22:13:19 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 321A58FC17; Sun, 29 Jan 2012 22:13:18 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.4/8.14.4/ALCHEMY.FRANKEN.DE) with ESMTP id q0TMDGq3046343; Sun, 29 Jan 2012 23:13:17 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.4/8.14.4/Submit) id q0TMDGhq046342; Sun, 29 Jan 2012 23:13:16 +0100 (CET) (envelope-from marius) Date: Sun, 29 Jan 2012 23:13:16 +0100 From: Marius Strobl To: Adrian Chadd Message-ID: <20120129221316.GG39861@alchemy.franken.de> References: <20120122195130.360261ce.ray@freebsd.org> <0E31FEC4-963D-4AC8-9AB7-EE6D6D7F86EE@lassitu.de> <20120129001251.7e4cfe83.ray@ddteam.net> <81B88904-6894-4AC3-80F3-2866216E494B@lassitu.de> <20120129153159.GA44362@alchemy.franken.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: FreeBSD Net , Aleksandr Rybalko , Stefan Bethke , FreeBSD-arch Subject: Re: Ethernet Switch Framework X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 22:13:19 -0000 On Sun, Jan 29, 2012 at 11:00:22AM -0800, Adrian Chadd wrote: > I think for switches, that doesn't necessarily hold. Err, what exactly doesn't hold? The suggestion about using multi-pass probing was just for the case when there's a separate MDIO master other drivers depend on. The latter would just use the default ordering (unless there are again ordering constraints whithin them). So if there's no separate MDIO master driver invovled that is attached first, the other drivers would still just be attached in the default ordering. > > ie, mii_attach() for single-PHY devices may work that way, but the weird What way? > and wonderful world of embedded switch SoC's doesn't. You're lucky in most > instances since the bootloader does give you a mostly-working switch > config. But I doubt that's a guarantee on all platforms. > > So for those (ethernet) devices, splitting out the mii/mdio stuff and _not_ > necessarily presenting a working PHY may be acceptable. For now it's going If there isn't a single working PHY, why would one want to attach miibus(4) in the first place? What is about the opposite case, when all you have is a MDIO master and a switch connected to it, but no MAC on the MII lines of the switch? > to be a subset, to having it export an MDIO bus and doing late MII > attachment may not be as architecturally "no-no" as I interpret your post. > Originally, Stefan said that he wants to find a way to support all the odd combinations found in the embedded-world and I try to come up with model that is able to do that without needing hacks and hints left and right. As imp@ said, interrupt controllers and GPIO basically suffer the same dependency constraints across otherwise unrelated devices there, so we really should find a way to properly deal with that situation without again needing another set of hacks and hints in other types of drivers. As for MDIO you actually can have another layer of dependencies between MDIO master, Ethernet switch and PHY, f.e. at work we use a single MDIO master and switch it to different slave busses using a multiplexer managed via GPIO ... not that I'd ever wanted to support this via generic means in FreeBSD. Marius From owner-freebsd-arch@FreeBSD.ORG Mon Jan 30 10:11:52 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C0B91106566B; Mon, 30 Jan 2012 10:11:52 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7B6788FC0A; Mon, 30 Jan 2012 10:11:51 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 584FC6AA3; Mon, 30 Jan 2012 10:11:50 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 345668570; Mon, 30 Jan 2012 11:11:50 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Attilio Rao References: <201111081018.pA8AI7ha027020@svn.freebsd.org> Date: Mon, 30 Jan 2012 11:11:50 +0100 In-Reply-To: (Attilio Rao's message of "Sat, 28 Jan 2012 17:03:11 +0100") Message-ID: <86wr89jy8p.fsf_-_@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD FS , freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Retiring non-mpsafe filesystems (was: Re: svn commit: r227333 - in head: . sys/amd64/conf sys/arm/conf sys/conf sys/i386/conf sys/ia64/conf sys/kern sys/mips/conf sys/pc98/conf sys/powerpc/conf sys/sparc64/conf) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 10:11:52 -0000 Attilio Rao writes: > In one month I'm going to disable VFS_ALLOW_NONMPSAFE by defaults in > order to see how well the users do with this option down. At the > present times this means that from 1st March you won't be able to > mount smbfs or ntfs volumes, for example. Hmm, wasn't there a GSoC project to reimplement NTFS? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Jan 30 22:08:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9C16106566C; Mon, 30 Jan 2012 22:08:11 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id CC6358FC0A; Mon, 30 Jan 2012 22:08:11 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 62BFAC3F0; Mon, 30 Jan 2012 14:08:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1327961291; bh=aC2nNiL04hPXIstov4tsWxVoiSWdc++TUND5/S4HwVo=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=vBHEHGdb+tUbeCcsaUsw/8IPRqIOt7Ey6cXH/w16v95sjh9k0ht3LB8O1S8pKeHry LEQgEtetLeV5X5S0g6z33aUYg7PuKg4UFO5tAmFp1EzJNQKu4v0N5N1Ld0ClQ6yIag OMs5lbRgKD3LF+dIoqEhAV7MZOCdYt1VRnSq4SCQ= Message-ID: <4F2714CA.2030401@delphij.net> Date: Mon, 30 Jan 2012 14:08:10 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Robert Millan References: <20120122201814.GA32081@thorin> <4F1DBB94.900@delphij.net> <4F204FDD.60807@delphij.net> <20120128150126.GA14522@thorin> In-Reply-To: <20120128150126.GA14522@thorin> X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , Adrian Chadd , d@delphij.net, freebsd-arch@freebsd.org Subject: Re: RFC: MK_BLOBS build option X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 22:08:12 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, On 01/28/12 07:01, Robert Millan wrote: > On Wed, Jan 25, 2012 at 10:54:21AM -0800, Xin Li wrote: >> >> I meant this, basically we have: >> >> GENERIC -- default kernel in FreeBSD SOURCELESS_EXCLUDES -- a >> meta-'overlay' that contains 'include SOURCELESS_HOST_EXCLUDES' >> and 'include SOURCELESS_UCODE_EXCLUDES' SOURCELESS_HOST_EXCLUDES >> -- a 'overlay' that contains a few 'nodevice 's >> SOURCELESS_UCODE_EXCLUDES -- ditto >> >> This way, e.g. GENERIC-DEBIAN would be simply: >> >> include GENERIC include SOURCELESS_EXCLUDES >> >> This way would minimize the maintenance of the GENERIC-DEBIAN I >> think while not compromising your policy of not including these >> stuff in binary? > > Sounds fine to me. Please let me know what you think about this. > I took your idea but used "WITHOUT" rather than "EXCLUDES" for > consistency with the MK_SOURCELESS_* options (which would be > disabled using e.g. -DWITHOUT_SOURCELESS_UCODE in command-line). > > I've also rewritten descriptions of the MK_* options so they use > the word "sourceless" instead of referring to "binary-only blobs". Sorry for the late response. The patch looks good to me. Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJPJxTKAAoJEG80Jeu8UPuzl78IAIWpMlg9Hfty5Ep2xBZI5tPW fCzQA1dtEaPpeSw/duh/CRLVUKbQQtKtIrmyFBHMFJsgTxruHCUxEeuw6B6N7vZL 8NZxUVbWvchQYBEcz26vbro2ff2nBrVajPvJDRb+sj7OHI2qZT0sg1iLfn30Xydv Mb/alPsWaRcKIQKYsL+mRpRpYJ+ksaTEH8H5NBn86f5dhFe4scwbW+d+nm/mXLZb kC+MvF+/LdjnQOIRiW6/e8seQDZTilFpUyLS/dzg9VM0kZw91j6KakFlW13FKLwy bVEWNqYkxsdyLNf+c5UCLELzE9N3alsdqpKaNlxF/J/ZKMLzzqOVEtI6tMIvPoY= =kXJ1 -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 19:39:14 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD27E1065670 for ; Tue, 31 Jan 2012 19:39:14 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 65FF18FC19 for ; Tue, 31 Jan 2012 19:39:14 +0000 (UTC) Received: by eaan10 with SMTP id n10so134682eaa.13 for ; Tue, 31 Jan 2012 11:39:13 -0800 (PST) Received: by 10.213.8.199 with SMTP id i7mr602607ebi.129.1328038753210; Tue, 31 Jan 2012 11:39:13 -0800 (PST) Received: from rnote.ddteam.net (119-219-133-95.pool.ukrtel.net. [95.133.219.119]) by mx.google.com with ESMTPS id b3sm38528954een.2.2012.01.31.11.39.11 (version=SSLv3 cipher=OTHER); Tue, 31 Jan 2012 11:39:12 -0800 (PST) Date: Tue, 31 Jan 2012 21:38:57 +0200 From: Aleksandr Rybalko To: freebsd-arch@freebsd.org Message-Id: <20120131213857.86c81626.ray@ddteam.net> X-Mailer: Sylpheed 3.1.2 (GTK+ 2.24.5; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: dynamic attach of hinted devices X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 19:39:14 -0000 Hi FreeBSD hackers, at first I want to say this: :) WARNING: FOLLOWING DEVCTL PATCH MAY EASILY PANIC YOUR SYSTEM, PLEASE DO NOT TRY IT ON PRODUCTION SERVERS AND TRY IT WITH FILESYSTEMS MOUNTED AS READONLY :))))) So I introduce two patches first one [1] used to migrate from static_hints or hints in the static_kenv to dynamic hints. sysctl kern.hintmode=2 will copy hints from static hints or from static kenv and put it into dynamic kenv. Those will allow to manipulate hints values and attach hinted devices with devctl tool. Second [2] allow attach/detach devices with userland tool devctl. devctl tool allow add and initialize new devices which is not able to be autoenumerating, such a hinted devices. Both designed to have ability update EEPROM items in runtime, since some device can't work in mode when it accessible like a EEPROM chip. Example: # sysctl kern.hintmode=2 # kenv hint.mx25l.0.at="spibus0" # kenv hint.mx25l.0.cs=0 # kenv hint.mx25l.0.chipname="at25128" # devctl hinted spibus 0 mx25l 0 mx25l0: at cs 0 mode 0 on spibus0 mx25l0: at25128, sector 64 bytes, 256 sectors GEOM: new disk flash/spi0 Someone may found it also useful for testing device attach/detach code (memory leaks, resource allocation, etc). So, say me please your opinion. 1. http://my.ddteam.net/files/2012-01-31.to_dynamic_hints.patch 2. http://my.ddteam.net/files/2012-01-31.devctl2.patch WBW -- Aleksandr Rybalko From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 20:02:16 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22B17106566C for ; Tue, 31 Jan 2012 20:02:16 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD948FC0A for ; Tue, 31 Jan 2012 20:02:16 +0000 (UTC) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id B2498C1FB; Tue, 31 Jan 2012 12:02:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1328040135; bh=yUmfoa1HXbmL5Ooi/rfGA3IcSL4cUo2kE36GzfAooB0=; h=Message-ID:Date:From:Reply-To:MIME-Version:To:CC:Subject: Content-Type:Content-Transfer-Encoding; b=H8SqQmTx+V/AeANr7bGmx/7MzR/i1oAiKY7qX8jBIUREBCRRaY0vfpx0EWuswDN4B +DPb/kky9F0o+u5eG9iZJcOcJnnASvtV/m6vf4vbqFqNMRllrDlZ5oO1EpPKvtFDlj mfARplCovBbxyzPSB6hYdVBSU4RjqBvPfQx73fRw= Message-ID: <4F2848C3.2090606@delphij.net> Date: Tue, 31 Jan 2012 12:02:11 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net Subject: Testing to make sure there is no ABI change X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 20:02:16 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, Is there a better way to test if there is a breaking ABI change than eye inspection for a C library? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBCAAGBQJPKEjDAAoJEG80Jeu8UPuzY2oH/0GPyvGPKYSuG2bxfSsaVBmt SbyXtYbHHK8qRVy0AoD5Uq0v3UElrSSQKwYyPieq3WoxRsh/Clekpg1bdvnthMcU OahMUqFmySgmXflNxHaMmmDPEKTEVALqABB8nzPqqTEOxx91p2aDw8X+rPrWxT18 AajVwvM+c9LOgsu+KVrVcG/e1lMWizuo7PtCSxsuyBGNAq0b5Gi9GUai0AY6Xm7e awCVjeA1mI04v8J6130ePrOV8z7eEIl9QK6r2JHPRma6CDr687uXAzZsAJ5YPBoT n1pRmOqTTvIaezO/DjORC74rMOkaohrdIj8wpJnQQmHUuM/dYUwNzg/ePmFX/A0= =6215 -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 20:31:00 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52DF61065675 for ; Tue, 31 Jan 2012 20:31:00 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 2C01F8FC18 for ; Tue, 31 Jan 2012 20:30:59 +0000 (UTC) Received: by daec6 with SMTP id c6so324636dae.13 for ; Tue, 31 Jan 2012 12:30:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=tTNyV+D/wxDXaYQ0rj1YVhmMfwYDWSds8WgKL147OE8=; b=kV4EgNx1vt/fNHtdAY1kHEHiMcy2PcbtiB8n4bwTi/F8MUGzOysfUFn7wJOJlq57Q7 tiLGR+GceV+kwjC4YT85D5liM9xwoB1aYbpRoPtJ/HBDLoEO36nsVKNli5cWFVHCerYc 8FUp3/IeUVQECDFB5C3sqmEq4ZrzIb6rzx3+Y= MIME-Version: 1.0 Received: by 10.68.74.134 with SMTP id t6mr53090067pbv.26.1328040329240; Tue, 31 Jan 2012 12:05:29 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.0.71 with HTTP; Tue, 31 Jan 2012 12:05:29 -0800 (PST) In-Reply-To: <4F2848C3.2090606@delphij.net> References: <4F2848C3.2090606@delphij.net> Date: Tue, 31 Jan 2012 12:05:29 -0800 X-Google-Sender-Auth: Zsr-z5VMQo6bWptrhLl7toBEYOw Message-ID: From: mdf@FreeBSD.org To: d@delphij.net Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Gleb Kurtsou , freebsd-arch@freebsd.org Subject: Re: Testing to make sure there is no ABI change X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 20:31:00 -0000 Gleb has a tool that examines DWARF debug code, and some wrappers to add the libc syscall shims as well. I got stalled by $WORK on the ino64 branch so I haven't used it yet. For KBI, though, I think it's still by hand. Cheers, matthew On Tue, Jan 31, 2012 at 12:02 PM, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Hi, > > Is there a better way to test if there is a breaking ABI change than > eye inspection for a C library? > > Cheers, > - -- > Xin LI =A0 =A0https://www.delphij.net/ > FreeBSD - The Power to Serve! =A0 =A0 =A0 =A0 =A0 Live free or die > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.18 (FreeBSD) > > iQEcBAEBCAAGBQJPKEjDAAoJEG80Jeu8UPuzY2oH/0GPyvGPKYSuG2bxfSsaVBmt > SbyXtYbHHK8qRVy0AoD5Uq0v3UElrSSQKwYyPieq3WoxRsh/Clekpg1bdvnthMcU > OahMUqFmySgmXflNxHaMmmDPEKTEVALqABB8nzPqqTEOxx91p2aDw8X+rPrWxT18 > AajVwvM+c9LOgsu+KVrVcG/e1lMWizuo7PtCSxsuyBGNAq0b5Gi9GUai0AY6Xm7e > awCVjeA1mI04v8J6130ePrOV8z7eEIl9QK6r2JHPRma6CDr687uXAzZsAJ5YPBoT > n1pRmOqTTvIaezO/DjORC74rMOkaohrdIj8wpJnQQmHUuM/dYUwNzg/ePmFX/A0=3D > =3D6215 > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 21:12:56 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59360106564A for ; Tue, 31 Jan 2012 21:12:56 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 1E4DC8FC15 for ; Tue, 31 Jan 2012 21:12:55 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id q0VKvXhl089209 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 31 Jan 2012 14:57:34 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.5/8.14.5) with ESMTP id q0VKvXNh023377 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 31 Jan 2012 14:57:33 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.5/8.14.5/Submit) id q0VKvXZN023376; Tue, 31 Jan 2012 14:57:33 -0600 (CST) (envelope-from dan) Date: Tue, 31 Jan 2012 14:57:33 -0600 From: Dan Nelson To: d@delphij.net Message-ID: <20120131205732.GA5775@dan.emsphone.com> References: <4F2848C3.2090606@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F2848C3.2090606@delphij.net> X-OS: FreeBSD 8.2-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Tue, 31 Jan 2012 14:57:35 -0600 (CST) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: freebsd-arch@freebsd.org Subject: Re: Testing to make sure there is no ABI change X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 21:12:56 -0000 In the last episode (Jan 31), Xin Li said: > Is there a better way to test if there is a breaking ABI change than > eye inspection for a C library? The devel/abi-compliance-checker port might do what you want. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 21:49:03 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C54C106566C; Tue, 31 Jan 2012 21:49:03 +0000 (UTC) (envelope-from giovanni.trematerra@gmail.com) Received: from mail-qw0-f47.google.com (mail-qw0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 99C258FC0A; Tue, 31 Jan 2012 21:49:02 +0000 (UTC) Received: by qadz30 with SMTP id z30so3519592qad.13 for ; Tue, 31 Jan 2012 13:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Qv7W0pKN0Qf+bbryD8C9uoei1PbZ/Vx9NKUeiCzcdpA=; b=s9vTKfAX/HQP7bJFmIQt/eJuca9pIQH0hv/Gj8wS8+sruuWFNC/Ckz99Px4MS2Zusi rqfWuvyiQcJ64KtVuc6CIPX42u/nJiCQBBziOhrTxBmHElzghFEMSnftk14zGtNCKa2B 4JF1GSSPD+jdLXoLwvqbPJrhKaxIZve5Xs374= MIME-Version: 1.0 Received: by 10.224.182.147 with SMTP id cc19mr17398896qab.76.1328046540966; Tue, 31 Jan 2012 13:49:00 -0800 (PST) Sender: giovanni.trematerra@gmail.com Received: by 10.229.81.200 with HTTP; Tue, 31 Jan 2012 13:49:00 -0800 (PST) In-Reply-To: <20120128233712.GA92694@stack.nl> References: <20120110005155.S2378@besplex.bde.org> <20120110153807.H943@besplex.bde.org> <20120128233712.GA92694@stack.nl> Date: Tue, 31 Jan 2012 22:49:00 +0100 X-Google-Sender-Auth: z1P4qkCvEliqSLxENK7NxSkPt9E Message-ID: From: Giovanni Trematerra To: Jilles Tjoelker Content-Type: text/plain; charset=ISO-8859-1 Cc: Attilio Rao , flo@freebsd.org, Konstantin Belousov , freebsd-arch@freebsd.org Subject: Re: pipe/fifo code merged. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 21:49:03 -0000 On Sun, Jan 29, 2012 at 12:37 AM, Jilles Tjoelker wrote: > On Tue, Jan 17, 2012 at 12:43:19PM +0100, Giovanni Trematerra wrote: >> Did you agree that this patch >> http://www.trematerra.net/patches/pipefifo_merge2.4.diff > >> doesn't introduce any further regressions while it fixes > >> - style bugs you pointed out. >> - pipe_stat now use underlying filesystem information for pipes. >> - comment about pipeinfo was intended for. >> - race into pipe_poll (look at fifo_iseof line). > > I tested this version of the patch and found that it breaks opening a > fifo with O_TRUNC: it fails with [EINVAL]. This appears to be > pipe_truncate()'s doing. Previously, truncate requests went to the > vnode. > > In particular, this happens when opening a fifo for writing using > sh(1)'s > (unless 'set -C' is in effect) or >| redirection operators. > The open properly blocks until a reader arrives but then fails with > [EINVAL]. Several tests in tools/regression/bin/sh do this. > Hi Jilles, thanks a lot for looking at the patch. I'll fix the issue in a new version of the patch. -- Gianni From owner-freebsd-arch@FreeBSD.ORG Tue Jan 31 22:16:41 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E5A9106566B; Tue, 31 Jan 2012 22:16:41 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6AF628FC0C; Tue, 31 Jan 2012 22:16:39 +0000 (UTC) Received: by lagz14 with SMTP id z14so394757lag.13 for ; Tue, 31 Jan 2012 14:16:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=WFNyU7sRUpwk7/fq3NTDnvGUsVHWHdFPfpsM9Y4mL1Y=; b=Yd/zzYlO7VO77uR/tLc58nD/OA9uEN9IhILnhkQyA0lJFtAOTZEbwacRxZ1LDRNr3J DPK5FWg3pTcwJpj8mS5SMzhA0kq8NNPQZ8bU+tgYmJLaf8Am9OoKkxn2qmRXNQ4CniL1 2nhXSrbVZ9XsGO+z84RcbS6HfKcsNV0zT3nZQ= Received: by 10.152.110.6 with SMTP id hw6mr12344335lab.37.1328046590615; Tue, 31 Jan 2012 13:49:50 -0800 (PST) Received: from localhost ([78.157.92.5]) by mx.google.com with ESMTPS id a8sm6008844lba.15.2012.01.31.13.49.49 (version=SSLv3 cipher=OTHER); Tue, 31 Jan 2012 13:49:49 -0800 (PST) Date: Tue, 31 Jan 2012 23:49:55 +0200 From: Gleb Kurtsou To: d@delphij.net Message-ID: <20120131214955.GA23246@reks> References: <4F2848C3.2090606@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: mdf@FreeBSD.org, freebsd-arch@freebsd.org Subject: Re: Testing to make sure there is no ABI change X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 22:16:41 -0000 On (31/01/2012 12:05), mdf@FreeBSD.org wrote: > Gleb has a tool that examines DWARF debug code, and some wrappers to > add the libc syscall shims as well. I got stalled by $WORK on the > ino64 branch so I haven't used it yet. > > For KBI, though, I think it's still by hand. It's also hosted on github: https://github.com/glk/shlib-compat shlib-compat can be used on kernel code as well to compare *.o files with debugging info. You'll need to specify list of symbols, e.g. with --include-sym/--exclude-sym. But it's scope is limited by function signature checks, you won't be able to compare binary sysctls, ioctls, etc. Thanks, Gleb. > > Cheers, > matthew > > On Tue, Jan 31, 2012 at 12:02 PM, Xin Li wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > > > > Hi, > > > > Is there a better way to test if there is a breaking ABI change than > > eye inspection for a C library? > > > > Cheers, > > - -- > > Xin LI    https://www.delphij.net/ > > FreeBSD - The Power to Serve!           Live free or die > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG v2.0.18 (FreeBSD) > > > > iQEcBAEBCAAGBQJPKEjDAAoJEG80Jeu8UPuzY2oH/0GPyvGPKYSuG2bxfSsaVBmt > > SbyXtYbHHK8qRVy0AoD5Uq0v3UElrSSQKwYyPieq3WoxRsh/Clekpg1bdvnthMcU > > OahMUqFmySgmXflNxHaMmmDPEKTEVALqABB8nzPqqTEOxx91p2aDw8X+rPrWxT18 > > AajVwvM+c9LOgsu+KVrVcG/e1lMWizuo7PtCSxsuyBGNAq0b5Gi9GUai0AY6Xm7e > > awCVjeA1mI04v8J6130ePrOV8z7eEIl9QK6r2JHPRma6CDr687uXAzZsAJ5YPBoT > > n1pRmOqTTvIaezO/DjORC74rMOkaohrdIj8wpJnQQmHUuM/dYUwNzg/ePmFX/A0= > > =6215 > > -----END PGP SIGNATURE----- > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Fri Feb 3 19:37:26 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 191E8106566B for ; Fri, 3 Feb 2012 19:37:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 80BF78FC15 for ; Fri, 3 Feb 2012 19:37:24 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q13JbKjL013317 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 3 Feb 2012 21:37:20 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q13JbJQk029323 for ; Fri, 3 Feb 2012 21:37:19 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q13JbJvv029322 for arch@freebsd.org; Fri, 3 Feb 2012 21:37:19 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 3 Feb 2012 21:37:19 +0200 From: Konstantin Belousov To: arch@freebsd.org Message-ID: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sZvnRN25x3w09J/6" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Subject: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2012 19:37:26 -0000 --sZvnRN25x3w09J/6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline FreeBSD I/O infrastructure has well known issue with deadlock caused by vnode lock order reversal when buffers supplied to read(2) or write(2) syscalls are backed by mmaped file. I previously published the patches to convert i/o path to use VMIO, based on the Jeff Roberson proposal, see http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the deadlock. Since that work is very intrusive and did not got any follow-up, it get stalled. Below is very lightweight patch which only goal is to fix deadlock in the least intrusive way. This is possible after FreeBSD got the vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. http://people.freebsd.org/~kib/misc/vm1.3.patch Theory of operation is described in the patched sys/kern/vfs_vnops.c, see preamble comment for vn_io_fault(). The patch borrows the rangelocks implementation from VM6, which was discussed and improved together with Attilio Rao. I was not able to reproduce the deadlock in the targeted test running for several hours, while stock HEAD deadlocks in the first iteration. Below is the benchmark for the worst-case situation for the patched system, reading 1 byte from a file in a loop. The value is the time in seconds to execute read(2) for single byte and lseek back to the start of the file. The loop is executed 100,000,000 times. Machine has 3.4Ghz Core i7 2600K and used HEAD@230866 with debugging options turned off. As you see, the rangelock overhead for the worst (but uncontented) case is less then 10%. x stock-1-byte.txt + vm1-1-byte.txt +--------------------------------------------------------------------------+ |xx ++| |xxx +++| ||A |A|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 1.063206e-06 1.065569e-06 1.064172e-06 1.064109e-06 9.8031959e-10 + 5 1.167145e-06 1.170244e-06 1.168939e-06 1.1690444e-06 1.2477022e-09 Difference at 95.0% confidence 1.04935e-07 +/- 1.63638e-09 9.86134% +/- 0.153779% (Student's t, pooled s = 1.122e-09) --sZvnRN25x3w09J/6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk8sN28ACgkQC3+MBN1Mb4j5YwCgvBRtHeZMSQrKiXG7AZX2sJf8 fbkAoLaL/489HyVNCImU/pq2yNVOJVHS =o8ql -----END PGP SIGNATURE----- --sZvnRN25x3w09J/6-- From owner-freebsd-arch@FreeBSD.ORG Fri Feb 3 19:47:17 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E6EC106566C; Fri, 3 Feb 2012 19:47:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 801428FC13; Fri, 3 Feb 2012 19:47:16 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q13Jl8rd014745 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 3 Feb 2012 21:47:08 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q13Jl8Ss029373; Fri, 3 Feb 2012 21:47:08 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q13Jl8T8029372; Fri, 3 Feb 2012 21:47:08 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 3 Feb 2012 21:47:08 +0200 From: Konstantin Belousov To: Attilio Rao Message-ID: <20120203194708.GC3283@deviant.kiev.zoral.com.ua> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7KvBbaRjXS2SFKKp" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2012 19:47:17 -0000 --7KvBbaRjXS2SFKKp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Feb 03, 2012 at 07:40:37PM +0000, Attilio Rao wrote: > Do you have an ETA for reviews? When do you plan to commit this? Obviously, I cannot have an ETA for reviews, and I am not sure that I get any review at all (as usual). I think it is reasonable to commit in 2-3 weeks after the first post. > it would be valuable to get a grasp on the benchmark and refine the > performance difference as much as possible. The benchmark is trivial and available at=20 http://people.freebsd.org/~kib/misc/bench_read.c There is also a targeted test for uio functionality http://people.freebsd.org/~kib/misc/uio.c which I used together with intensive swapping workload to verify the correctness of the patch. Regarding the performance hit, I consider the < 10% on the worst case a reasonable cost for finally fixing this issue. If anybody can provide a benchmark result for e.g. postgresql tests, I will be very grateful. --7KvBbaRjXS2SFKKp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk8sObwACgkQC3+MBN1Mb4jLCQCeIBu368//Hwq4rZmtqcUQgVRP Yq4AoONvlv43L7ABlzDggAPBNbCDz2vV =FQiX -----END PGP SIGNATURE----- --7KvBbaRjXS2SFKKp-- From owner-freebsd-arch@FreeBSD.ORG Fri Feb 3 20:10:21 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B07891065678 for ; Fri, 3 Feb 2012 20:10:21 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2CA708FC16 for ; Fri, 3 Feb 2012 20:10:20 +0000 (UTC) Received: by wgbdq11 with SMTP id dq11so4372743wgb.31 for ; Fri, 03 Feb 2012 12:10:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=P9CjPqk0COYuEZIj0GF8LV+JyHchr7EoESR9hYxCSQw=; b=FIAFm32vh+0VgI30KTjBl8c+BwLB8HV5rH5U4X0kxgTK+m7QiFTAyOAT3YgyIF0O4f Rs47H/5//eORccvZj5OzxCOQxT9DYZigE7D4xYwF6n2zZZ+evHBvrYKUii5rbT3dgzq8 HA5Th0McJt1BUFvhPul9Te+P6lIA+Zlsy4ScE= MIME-Version: 1.0 Received: by 10.180.92.226 with SMTP id cp2mr13657714wib.10.1328298037851; Fri, 03 Feb 2012 11:40:37 -0800 (PST) Sender: asmrookie@gmail.com Received: by 10.216.177.73 with HTTP; Fri, 3 Feb 2012 11:40:37 -0800 (PST) In-Reply-To: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> Date: Fri, 3 Feb 2012 19:40:37 +0000 X-Google-Sender-Auth: kqo88koH42lAy3SzxAxGoq17LRc Message-ID: From: Attilio Rao To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2012 20:10:21 -0000 2012/2/3 Konstantin Belousov : > FreeBSD I/O infrastructure has well known issue with deadlock caused > by vnode lock order reversal when buffers supplied to read(2) or > write(2) syscalls are backed by mmaped file. > > I previously published the patches to convert i/o path to use VMIO, > based on the Jeff Roberson proposal, see > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > deadlock. Since that work is very intrusive and did not got any > follow-up, it get stalled. > > Below is very lightweight patch which only goal is to fix deadlock in > the least intrusive way. This is possible after FreeBSD got the > vm_fault_quick_hold_pages(9) and vm_fault_disable_pagefaults(9) KPIs. > http://people.freebsd.org/~kib/misc/vm1.3.patch > > Theory of operation is described in the patched sys/kern/vfs_vnops.c, > see preamble comment for vn_io_fault(). The patch borrows the > rangelocks implementation from VM6, which was discussed and improved > together with Attilio Rao. > > I was not able to reproduce the deadlock in the targeted test running > for several hours, while stock HEAD deadlocks in the first iteration. > > Below is the benchmark for the worst-case situation for the patched > system, reading 1 byte from a file in a loop. The value is the time in > seconds to execute read(2) for single byte and lseek back to the start > of the file. The loop is executed 100,000,000 times. Machine has > 3.4Ghz Core i7 2600K and used HEAD@230866 with debugging options > turned off. > > As you see, the rangelock overhead for the worst (but uncontented) > case is less then 10%. > > x stock-1-byte.txt > + vm1-1-byte.txt > +------------------------------------------------------------------------= --+ > |xx =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0++| > |xxx =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0+++| > ||A =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 |A|| > +------------------------------------------------------------------------= --+ > =C2=A0 =C2=A0N =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Min =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Max =C2=A0 =C2=A0 =C2=A0 =C2=A0Median =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 Avg =C2=A0 =C2=A0 =C2=A0 =C2=A0Stddev > x =C2=A0 5 =C2=A01.063206e-06 =C2=A01.065569e-06 =C2=A01.064172e-06 =C2= =A01.064109e-06 9.8031959e-10 > + =C2=A0 5 =C2=A01.167145e-06 =C2=A01.170244e-06 =C2=A01.168939e-06 1.169= 0444e-06 1.2477022e-09 > Difference at 95.0% confidence > =C2=A0 =C2=A0 =C2=A0 =C2=A01.04935e-07 +/- 1.63638e-09 > =C2=A0 =C2=A0 =C2=A0 =C2=A09.86134% +/- 0.153779% > =C2=A0 =C2=A0 =C2=A0 =C2=A0(Student's t, pooled s =3D 1.122e-09) Do you have an ETA for reviews? When do you plan to commit this? it would be valuable to get a grasp on the benchmark and refine the performance difference as much as possible. Thanks, Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Sat Feb 4 15:29:47 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BD93106564A for ; Sat, 4 Feb 2012 15:29:47 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id EDC658FC0C for ; Sat, 4 Feb 2012 15:29:46 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 2E51E6BF1; Sat, 4 Feb 2012 15:29:46 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id E84AA8991; Sat, 4 Feb 2012 16:29:45 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> Date: Sat, 04 Feb 2012 16:29:45 +0100 In-Reply-To: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Fri, 3 Feb 2012 21:37:19 +0200") Message-ID: <8662fmk45y.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Feb 2012 15:29:47 -0000 Konstantin Belousov writes: > I previously published the patches to convert i/o path to use VMIO, > based on the Jeff Roberson proposal, see > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > deadlock. Since that work is very intrusive and did not got any > follow-up, it get stalled. If it works, and you commit it now, we have at least a year to shake it down before the 10.0 release cycle... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Feb 4 16:20:24 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BD721065670 for ; Sat, 4 Feb 2012 16:20:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id CE5FE8FC0C for ; Sat, 4 Feb 2012 16:20:23 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q14GJ21q020850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 4 Feb 2012 18:19:02 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q14GJ2me045426; Sat, 4 Feb 2012 18:19:02 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q14GJ2JR045425; Sat, 4 Feb 2012 18:19:02 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Feb 2012 18:19:02 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Message-ID: <20120204161902.GQ3283@deviant.kiev.zoral.com.ua> References: <20120203193719.GB3283@deviant.kiev.zoral.com.ua> <8662fmk45y.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="apqMg07LpBOyFFAR" Content-Disposition: inline In-Reply-To: <8662fmk45y.fsf@ds4.des.no> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: Prefaulting for i/o buffers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Feb 2012 16:20:24 -0000 --apqMg07LpBOyFFAR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Feb 04, 2012 at 04:29:45PM +0100, Dag-Erling Sm??rgrav wrote: > Konstantin Belousov writes: > > I previously published the patches to convert i/o path to use VMIO, > > based on the Jeff Roberson proposal, see > > http://wiki.freebsd.org/VM6. As a side effect, the VM6 fixed the > > deadlock. Since that work is very intrusive and did not got any > > follow-up, it get stalled. >=20 > If it works, and you commit it now, we have at least a year to shake it > down before the 10.0 release cycle... If it were KibBSD, the suggested route of action might be indeed reasonable. But, since our VM and VFS is the community efforts, I cannot and do not want to force other developers working in the same area to suffer from my mistakes. In other words, this cannot go in without community review and acceptance. --apqMg07LpBOyFFAR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk8tWnUACgkQC3+MBN1Mb4hG0QCg9S1o2htFfKRT/sL8aE55kwPL kSgAoOU8aHDstfX9mM4TRwEvMQEHPeXj =sL5Y -----END PGP SIGNATURE----- --apqMg07LpBOyFFAR--