From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 04:15:38 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 838FE649 for ; Sun, 6 Oct 2013 04:15:38 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 44B9D25A1 for ; Sun, 6 Oct 2013 04:15:37 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id e14so12275435iej.24 for ; Sat, 05 Oct 2013 21:15:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=HThZfqvNLVRoG+TzEW2KeGGGbkJCettBc6uvmn/KHeQ=; b=ge1wZxf+c9KvqbY0psCml2ANUSPFVlKunE7AC5nwy6MCg2sDkDIt786Q0PKjS6JOEL dNr3YuNrnsGBZUJppJW2tR9a6IkWKeJWxUxKNpTjzFEz2DITe+VRkEtvl+Rah6MvQg0X M3EQlrav5tbPxihBHpxYGj3/i2zJPCgOCUQmyHNH4Npml57ugcA6LBZt6hQMYm/gX6D3 pZE+rTx4soG7kOhEsiBDFXOhp79hs6BQa6bq7jXmqmQhN7mtFwuHMC1pwjueT/uX0E4d hQp3vyQtoT+eAKzdFp1F3lOkIScT5fgUrxA9rCtf4PQv4rVScYTZ57/ZMqTHk7RWH4Kf N17A== X-Gm-Message-State: ALoCoQkeXwyLd28bM+GuKu8xWorScMfbxsJ4v1Ovqx35dllOf3sl9XE4d8aZPgF/7ApGYb7fR0aP X-Received: by 10.50.26.36 with SMTP id i4mr12143551igg.33.1381032931222; Sat, 05 Oct 2013 21:15:31 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id p7sm18251725iga.3.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 05 Oct 2013 21:15:30 -0700 (PDT) Sender: Warner Losh Subject: Re: How's bus-space stuff supposed to work with superscalar MIPS? Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sat, 5 Oct 2013 22:15:29 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0ED4F6A1-458A-474F-97F1-488336AB9A4D@bsdimp.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1085) Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 04:15:38 -0000 On Oct 5, 2013, at 11:18 AM, Adrian Chadd wrote: > Hi all, >=20 > I've been bringing up the AR9344 PHY and after a lot of digging, I > discovered that I can fix things by changing ARGE_WRITE() (ie, write = to the > ethernet space registers) to: >=20 > bus_write_4(); > bus_read_4(); >=20 > .. to (what I'm guessing here) flush the write out before the next > instruction is run. read after write ensures that the write is flushed by definition. > So, given this particular hilarity has shown up, what's the story with > doing IO accesses on a superscalar MIPS CPU? If it's going to kseg1, = is it > somehow going to magically enforce ordering? Or am I right in thinking = we > will need explicit barriers here? If you are on a super-scalor architecture, then you'll need memory = barriers. It doesn't look like we've implemented them just yet. > I'd like to sneak this into the initial mips74k bringup support that = I'm > going to commit to -HEAD soon. Sneak what in? Warner > Thanks, >=20 >=20 > -adrian > _______________________________________________ > freebsd-mips@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org" From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 04:22:03 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7939E6F8 for ; Sun, 6 Oct 2013 04:22:03 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 411F725D9 for ; Sun, 6 Oct 2013 04:22:03 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id e14so12281227iej.24 for ; Sat, 05 Oct 2013 21:22:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=mL6YRwo4quUW966cDq3D/gx2CpcrabK/kT/q+yU0E0g=; b=N3nhh55GTaqAAwNpQoRyWQ7EFgcAyL0InBhYp84ctQeEHdnqaWoSgOSKHxhYH0GsjZ G0rBuyM4diUUSyRQBch7A6nTOy7/IAEdQXRXagT18BJGu+dDKiPTzhCRpmHmZBE/9qWU 3GsnVMqqnrDdCjL3t2BX6V7NN4aDfg0RTRdLYNQdzYuWjkAS2zZVNt97IXTq07B0bgTP xt5AwI0/32GIZYOBdWfvt+Q1gjKO5YOmM2PwZT2LE8W4qHoDhrpT8t8uxl7Vj78IYo7+ nxz/lgQeMcHipuJle4R39EsMiri6XZx0FphHqEQXhXl4CFxEr4w2uQeNkAnEbjp5D/bT XnyQ== X-Gm-Message-State: ALoCoQmjRKR0ljG9cUUfORFFL/ftte87ZuRukhpBVKAqY/Sg/Kg79ktL4c9h9hx82e0WFyt7e/+t X-Received: by 10.43.172.4 with SMTP id nw4mr13729615icc.25.1381033322534; Sat, 05 Oct 2013 21:22:02 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id ih14sm18254005igb.7.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 05 Oct 2013 21:22:02 -0700 (PDT) Sender: Warner Losh Subject: Re: How's bus-space stuff supposed to work with superscalar MIPS? Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Sat, 5 Oct 2013 22:22:01 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> References: <5AD9EE93-9D19-4A07-B189-43DA0C4A85E9@FreeBSD.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1085) Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 04:22:03 -0000 On Oct 5, 2013, at 5:51 PM, Adrian Chadd wrote: > On 5 October 2013 16:06, Stanislav Sedov wrote: >=20 >>=20 >> On Oct 5, 2013, at 10:18 AM, Adrian Chadd wrote: >>=20 >>> Hi all, >>>=20 >>> I've been bringing up the AR9344 PHY and after a lot of digging, I >>> discovered that I can fix things by changing ARGE_WRITE() (ie, write = to >> the >>> ethernet space registers) to: >>>=20 >>> bus_write_4(); >>> bus_read_4(); >>>=20 >>> .. to (what I'm guessing here) flush the write out before the next >>> instruction is run. >>>=20 >>> So, given this particular hilarity has shown up, what's the story = with >>> doing IO accesses on a superscalar MIPS CPU? If it's going to kseg1, = is >> it >>> somehow going to magically enforce ordering? Or am I right in = thinking we >>> will need explicit barriers here? >>>=20 >>=20 >> I don't know specifics of mips74k, but usually one indeed needs = memory >> barriers >> when performing read of write operation sequences that require = ordering on >> device I/O (e.g changing the ring and writing the new ring index >> afterwards). I would >> not be surprised if the cpu reorders i/o bus memory access, = especially a >> multi-issue >> one. >>=20 >> It is a good idea to have barriers where needed regardless. We have >> special macros >> for them which are defined to nothing on the in-order platforms. >=20 >=20 > Right. I know this stuff. I really though want to know this kind of = stuff: >=20 > * What the specifics are for superscalar MIPS CPUs; I believe they document that writes can be reordered unless there's an = intervening read or memory barrier. I've not looked it up. > * What the bus space stuff should be be providing by default (and I've = been > down this path once, with ath(4) bugs, PPC, and the bus space macros = not > enforcing flushes after IO operations, even though the API requires = drivers > do it themselves..); It isn't so much flushes as barriers to prevent reordering. By doing the = read after write, you are forcing an expensive memory barrier. Drivers = that depend on a particular write ordering need to have explicit = barriers. > * Whether it should be enough to map space COHERENT - then it's up to = the > underlying bus implementation to implement enforcing ordering. The question here is whether there should be an implied barrier in write = operations. On x86 there is, but as you are discovering on other = architectures there isn't. While it would be convenient to force a = memory barrier between every write (something trivial to do with an = explicit barrier in your driver), it is not very performant to do so, = since most writes don't have an explicit ordering... Warner= From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 07:09:48 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D9859F87; Sun, 6 Oct 2013 07:09:47 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qe0-x236.google.com (mail-qe0-x236.google.com [IPv6:2607:f8b0:400d:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8728D2B2D; Sun, 6 Oct 2013 07:09:47 +0000 (UTC) Received: by mail-qe0-f54.google.com with SMTP id 1so622666qec.27 for ; Sun, 06 Oct 2013 00:09:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EXQPYtC7y/+owI5XMxK51HplEviY0NKR3jXi+Hu5SAw=; b=Gj0Gv+Ms+H79df4PHX0013SLcvleS14GXuL+SonjKNqviXOZuxBHTYjND2oB795T00 bcrd5R0ivFlys5cWtplf5nN5HRmZ/CX1U/w42MTod2qEtksLuMuoMq4q25pJwzRMcq2v PfJmy7mBwVWizaJpgxCrEgoCMzIoI6/MmANGn+C7uAJc3GGABbwR/cAacMvVH6ZlMggf FiyMTduV2loelyB40eP5A2xPTChIOCcqr17MzDMCiQXy+boJu5Uy1AtFi8sdWbz20DEY oLzvXFLPbDirPfyXKhUSQwLaEwt2xfCAHbwcpvsb1yjrmRsjmyK5Ewh1jk5WG67iYpvZ +ZtA== MIME-Version: 1.0 X-Received: by 10.224.95.10 with SMTP id b10mr29950171qan.6.1381043386724; Sun, 06 Oct 2013 00:09:46 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Sun, 6 Oct 2013 00:09:46 -0700 (PDT) In-Reply-To: <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> References: <5AD9EE93-9D19-4A07-B189-43DA0C4A85E9@FreeBSD.org> <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> Date: Sun, 6 Oct 2013 00:09:46 -0700 X-Google-Sender-Auth: fF7OvUAf02tMxKlir5IrQCzCgo8 Message-ID: Subject: Re: How's bus-space stuff supposed to work with superscalar MIPS? From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 07:09:48 -0000 Hi, On 5 October 2013 21:22, Warner Losh wrote: > > On Oct 5, 2013, at 5:51 PM, Adrian Chadd wrote: > > > On 5 October 2013 16:06, Stanislav Sedov wrote: > > > >> > >> On Oct 5, 2013, at 10:18 AM, Adrian Chadd wrote: > >> > >>> Hi all, > >>> > >>> I've been bringing up the AR9344 PHY and after a lot of digging, I > >>> discovered that I can fix things by changing ARGE_WRITE() (ie, write to > >> the > >>> ethernet space registers) to: > >>> > >>> bus_write_4(); > >>> bus_read_4(); > >>> > >>> .. to (what I'm guessing here) flush the write out before the next > >>> instruction is run. > >>> > >>> So, given this particular hilarity has shown up, what's the story with > >>> doing IO accesses on a superscalar MIPS CPU? If it's going to kseg1, is > >> it > >>> somehow going to magically enforce ordering? Or am I right in thinking > we > >>> will need explicit barriers here? > >>> > >> > >> I don't know specifics of mips74k, but usually one indeed needs memory > >> barriers > >> when performing read of write operation sequences that require ordering > on > >> device I/O (e.g changing the ring and writing the new ring index > >> afterwards). I would > >> not be surprised if the cpu reorders i/o bus memory access, especially a > >> multi-issue > >> one. > >> > >> It is a good idea to have barriers where needed regardless. We have > >> special macros > >> for them which are defined to nothing on the in-order platforms. > > > > > > Right. I know this stuff. I really though want to know this kind of > stuff: > > > > * What the specifics are for superscalar MIPS CPUs; > > I believe they document that writes can be reordered unless there's an > intervening read or memory barrier. I've not looked it up. > Would you mind helping me find where this is documented? > * What the bus space stuff should be be providing by default (and I've > been > > down this path once, with ath(4) bugs, PPC, and the bus space macros not > > enforcing flushes after IO operations, even though the API requires > drivers > > do it themselves..); > > It isn't so much flushes as barriers to prevent reordering. By doing the > read after write, you are forcing an expensive memory barrier. Drivers that > depend on a particular write ordering need to have explicit barriers. > > > * Whether it should be enough to map space COHERENT - then it's up to the > > underlying bus implementation to implement enforcing ordering. > > The question here is whether there should be an implied barrier in write > operations. On x86 there is, but as you are discovering on other > architectures there isn't. While it would be convenient to force a memory > barrier between every write (something trivial to do with an explicit > barrier in your driver), it is not very performant to do so, since most > writes don't have an explicit ordering... > I'm happy to treat MIPS as the "platform we try to properly fix this on" versus just doing what we did over in PPC land when this came up. So, what should it look like? Is the barrier() busdma instruction the right method to implement this as and use in drivers? Thanks! -adrian From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 16:31:13 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5BE1325F; Sun, 6 Oct 2013 16:31:13 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EC6BF230B; Sun, 6 Oct 2013 16:31:12 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id ii20so2397470qab.0 for ; Sun, 06 Oct 2013 09:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7QSVFHR7NgHGA6fm0MUKfhlm/z7vXDhbS7y5YdJc0iA=; b=DMCFgdX1spJ9Md/w0ibh+JCBwldgsfXbMHiRRvFH/qtDpzcB5Fj2dpBN1Wtk6kclhi wMyrR+ebnUgXuA/5uS9q8tJDmylwzgeb0tNlavqJU/mzNsD6hX1YAGuveKryITrR58Xc pPNtOfPy2TZk+V42+iXNONCNoIYO0rvt7HfWjgDTVuL7LgIczaaUTS8uZaayrqyqY9tr oWDGqaKAuMhDchZGK1kZx2A0gQV7tH0X+B1xm6LBmNNLYD+BoNZE7dtr5LfK7lOYYON9 H9tsrZAoClQtD1oTIexJU4g4Qy5RHSQzKM5wUFux+VWRXZSebCuCC4dvdol2qfUrY+hy Y0Ng== MIME-Version: 1.0 X-Received: by 10.49.12.14 with SMTP id u14mr3606800qeb.74.1381077072059; Sun, 06 Oct 2013 09:31:12 -0700 (PDT) Received: by 10.224.207.66 with HTTP; Sun, 6 Oct 2013 09:31:11 -0700 (PDT) Received: by 10.224.207.66 with HTTP; Sun, 6 Oct 2013 09:31:11 -0700 (PDT) In-Reply-To: <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> References: <5AD9EE93-9D19-4A07-B189-43DA0C4A85E9@FreeBSD.org> <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> Date: Sun, 6 Oct 2013 09:31:11 -0700 Message-ID: Subject: Re: How's bus-space stuff supposed to work with superscalar MIPS? From: Adrian Chadd To: "M. Warner Losh" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 16:31:13 -0000 On Oct 6, 2013 12:22 AM, "Warner Losh" wrote: > > > On Oct 5, 2013, at 5:51 PM, Adrian Chadd wrote: > > > On 5 October 2013 16:06, Stanislav Sedov wrote: > > > >> > >> On Oct 5, 2013, at 10:18 AM, Adrian Chadd wrote: > >> > >>> Hi all, > >>> > >>> I've been bringing up the AR9344 PHY and after a lot of digging, I > >>> discovered that I can fix things by changing ARGE_WRITE() (ie, write to > >> the > >>> ethernet space registers) to: > >>> > >>> bus_write_4(); > >>> bus_read_4(); > >>> > >>> .. to (what I'm guessing here) flush the write out before the next > >>> instruction is run. > >>> > >>> So, given this particular hilarity has shown up, what's the story with > >>> doing IO accesses on a superscalar MIPS CPU? If it's going to kseg1, is > >> it > >>> somehow going to magically enforce ordering? Or am I right in thinking we > >>> will need explicit barriers here? > >>> > >> > >> I don't know specifics of mips74k, but usually one indeed needs memory > >> barriers > >> when performing read of write operation sequences that require ordering on > >> device I/O (e.g changing the ring and writing the new ring index > >> afterwards). I would > >> not be surprised if the cpu reorders i/o bus memory access, especially a > >> multi-issue > >> one. > >> > >> It is a good idea to have barriers where needed regardless. We have > >> special macros > >> for them which are defined to nothing on the in-order platforms. > > > > > > Right. I know this stuff. I really though want to know this kind of stuff: > > > > * What the specifics are for superscalar MIPS CPUs; > > I believe they document that writes can be reordered unless there's an intervening read or memory barrier. I've not looked it up. > > > * What the bus space stuff should be be providing by default (and I've been > > down this path once, with ath(4) bugs, PPC, and the bus space macros not > > enforcing flushes after IO operations, even though the API requires drivers > > do it themselves..); > > It isn't so much flushes as barriers to prevent reordering. By doing the read after write, you are forcing an expensive memory barrier. Drivers that depend on a particular write ordering need to have explicit barriers. > > > * Whether it should be enough to map space COHERENT - then it's up to the > > underlying bus implementation to implement enforcing ordering. > > The question here is whether there should be an implied barrier in write operations. On x86 there is, but as you are discovering on other architectures there isn't. While it would be convenient to force a memory barrier between every write (something trivial to do with an explicit barrier in your driver), it is not very performant to do so, since most writes don't have an explicit ordering... The other thing is how correct the shared driver code is, like pci, usb, etc. I think that allocing bus space coherent means non cached, not non speculative/in order. So, what should we do? And whats the busdma barrier method do? Is it a cache barrier, or did its definition include ordering? Its a stub in mips, with the cache invalidate call commented out. My idea here is to change the definition of coherent, making it imply in order. Then add another flag saying space is potentially non ordered. That puts the onus on drivers to do the right thing if they want the performance boost, but buys us correctness now. I know that ppc modified their bus space to enforce ordered writes. Thanks, > Warner From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 17:10:02 2013 Return-Path: Delivered-To: freebsd-mips@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3BFFCA05 for ; Sun, 6 Oct 2013 17:10:02 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F36C0249F for ; Sun, 6 Oct 2013 17:10:01 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VSrq6-0000cK-NL; Sun, 06 Oct 2013 17:09:54 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r96H9qMO016489; Sun, 6 Oct 2013 11:09:52 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19/tVTUaykLNlkVMPLp24HE Subject: Re: How's bus-space stuff supposed to work with superscalar MIPS? From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <5AD9EE93-9D19-4A07-B189-43DA0C4A85E9@FreeBSD.org> <21AC10EC-BAA6-4F1A-BC17-F781CF77D224@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Sun, 06 Oct 2013 11:09:52 -0600 Message-ID: <1381079392.1152.45.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 17:10:02 -0000 On Sun, 2013-10-06 at 09:31 -0700, Adrian Chadd wrote: > On Oct 6, 2013 12:22 AM, "Warner Losh" wrote: > > > > > > On Oct 5, 2013, at 5:51 PM, Adrian Chadd wrote: > > > > > On 5 October 2013 16:06, Stanislav Sedov wrote: > > > > > >> > > >> On Oct 5, 2013, at 10:18 AM, Adrian Chadd wrote: > > >> > > >>> Hi all, > > >>> > > >>> I've been bringing up the AR9344 PHY and after a lot of digging, I > > >>> discovered that I can fix things by changing ARGE_WRITE() (ie, write > to > > >> the > > >>> ethernet space registers) to: > > >>> > > >>> bus_write_4(); > > >>> bus_read_4(); > > >>> > > >>> .. to (what I'm guessing here) flush the write out before the next > > >>> instruction is run. > > >>> > > >>> So, given this particular hilarity has shown up, what's the story with > > >>> doing IO accesses on a superscalar MIPS CPU? If it's going to kseg1, > is > > >> it > > >>> somehow going to magically enforce ordering? Or am I right in > thinking we > > >>> will need explicit barriers here? > > >>> > > >> > > >> I don't know specifics of mips74k, but usually one indeed needs memory > > >> barriers > > >> when performing read of write operation sequences that require > ordering on > > >> device I/O (e.g changing the ring and writing the new ring index > > >> afterwards). I would > > >> not be surprised if the cpu reorders i/o bus memory access, especially > a > > >> multi-issue > > >> one. > > >> > > >> It is a good idea to have barriers where needed regardless. We have > > >> special macros > > >> for them which are defined to nothing on the in-order platforms. > > > > > > > > > Right. I know this stuff. I really though want to know this kind of > stuff: > > > > > > * What the specifics are for superscalar MIPS CPUs; > > > > I believe they document that writes can be reordered unless there's an > intervening read or memory barrier. I've not looked it up. > > > > > * What the bus space stuff should be be providing by default (and I've > been > > > down this path once, with ath(4) bugs, PPC, and the bus space macros not > > > enforcing flushes after IO operations, even though the API requires > drivers > > > do it themselves..); > > > > It isn't so much flushes as barriers to prevent reordering. By doing the > read after write, you are forcing an expensive memory barrier. Drivers that > depend on a particular write ordering need to have explicit barriers. > > > > > * Whether it should be enough to map space COHERENT - then it's up to > the > > > underlying bus implementation to implement enforcing ordering. > > > > The question here is whether there should be an implied barrier in write > operations. On x86 there is, but as you are discovering on other > architectures there isn't. While it would be convenient to force a memory > barrier between every write (something trivial to do with an explicit > barrier in your driver), it is not very performant to do so, since most > writes don't have an explicit ordering... > > The other thing is how correct the shared driver code is, like pci, usb, > etc. > > I think that allocing bus space coherent means non cached, not non > speculative/in order. So, what should we do? > > And whats the busdma barrier method do? Is it a cache barrier, or did its > definition include ordering? Its a stub in mips, with the cache invalidate > call commented out. > > My idea here is to change the definition of coherent, making it imply in > order. Then add another flag saying space is potentially non ordered. That > puts the onus on drivers to do the right thing if they want the performance > boost, but buys us correctness now. > > I know that ppc modified their bus space to enforce ordered writes. > > Thanks, There is mixing here between the concepts of bus_space and busdma, and they're not miscible. They're two separate subsystems that live side by side in driverland. The bus_space system is how the cpu accesses peripherals that live on a memory or IO bus, and busdma is how the cpu and peripherals share access to main memory. You speak of "allocating bus space" and of "busdma barrier" -- that's backwards. You can allocate busdma memory, but not bus_space. There are barrier operations available in bus_space, not in busdma (there are sync operations in busdma). Normally I try not to be overly pedantic, but this is an area where you really can't discuss things properly without using the correct terminology, or the discusssion will become hopelessly muddled. So for bus_space, the documentation states that each individual driver must call bus_space_barrier() as needed after other bus_space accesses. Very few drivers currently do so, and it just seems to accidentally work out okay on most platforms. On ARM for example the memory-mapped devices are mapped with MMU attributes that force all access to be strongly ordered (each read or write happens in the order it was issued, without caching or buffering, without speculative access or prefetching, etc). Fixing the lack of bus_space_barrier() calls would be a monumental task. Pretty much every existing bus_space_read() and bus_space_write() call in all the various flavors in the whole system has to be examined in the context of the code that surrounds it with thoughts in mind such as "what would happen if this read/write happened before the prior one?" IMO, the right way to handle this kind of fix would be to change the bus_space API so that every access function had a flag that said what to do about barriers. That's the only way you'll ever be sure that you've fixed every existing driver and that new drivers in the future will always be written correctly. Or you could just implement the ordering at the bus_space implementation layer and rewrite the docs to match the existing practice (and effectively eliminate the bus_space_barrier() call). To me, this makes a lot more sense -- the bus_space implementation is closer to the host hardware, and seems like the right place to know about things such as the ordering of bus accesses. When it comes to busdma and coherent mappings, that's a whole different can of worms, a whole 'nother area full of "works by accident" right now. But since I think bus_space is what you're really concerned with in this thread, we probably shouldn't muddy the discussion with busdma issues. -- Ian From owner-freebsd-mips@FreeBSD.ORG Sun Oct 6 18:15:15 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 36F7CFEB; Sun, 6 Oct 2013 18:15:15 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qe0-x22a.google.com (mail-qe0-x22a.google.com [IPv6:2607:f8b0:400d:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DA2C22824; Sun, 6 Oct 2013 18:15:14 +0000 (UTC) Received: by mail-qe0-f42.google.com with SMTP id nc12so1699179qeb.1 for ; Sun, 06 Oct 2013 11:15:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LZHE05LkSB3bsvVjY1Di7nrZs0/p1GWDMdT9Vwkspe4=; b=aVhaf56fm/wVN2E/OBUY3VAWMAlHfCKZiINkuwHHJBXOHgin9ztyQVUA2+12jKsGrp VarVjhRJCqtxoq/H3ZuEDErZbX2enffMHuInVI58B4O9omiVgCh9dKy0P0w+UKGsDKJV H9By/HwSWYU6Q5OU+cUniHXzTpBkZWPTxxbz3jEMpb/ABPu0ChvHLER9O1FyeDuWhkPC oduBPWbfWgbi7o6FFg/o1NhOmw82bKBx/A5jna4nHtnf6ms9+v5qEz2GkgOjI5iiB/1n UTj3JkBufq3cDvYd8+sF7iCWwUQ4j5tKTDYqRsK4/yJ1sG4YInde1SjVrLUhy9DoFOzJ noDQ== MIME-Version: 1.0 X-Received: by 10.49.127.179 with SMTP id nh19mr31004412qeb.1.1381083314038; Sun, 06 Oct 2013 11:15:14 -0700 (PDT) Received: by 10.224.207.66 with HTTP; Sun, 6 Oct 2013 11:15:13 -0700 (PDT) In-Reply-To: <3CB8B78B-43F3-4891-9E68-E2CE3EDED96D@bsdimp.com> References: <180563C9-541F-47F9-8C90-35686BBC6471@bluezbox.com> <3CB8B78B-43F3-4891-9E68-E2CE3EDED96D@bsdimp.com> Date: Sun, 6 Oct 2013 11:15:13 -0700 Message-ID: Subject: Re: FreeBSD/MIPS emulation From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2013 18:15:15 -0000 On Tuesday, October 1, 2013, Warner Losh wrote: > > On Sep 30, 2013, at 11:09 AM, Adrian Chadd wrote: > > > Hi! > > > > I've verified that MALTA + mipsel works. > > > > I'll check MALTA + mipseb later tonight/tomorrow. I don't know if my qemu > > package has the relevant patch though; but I'll see if I can get it > > committed to the port so it isn't such a big problem for us. > > > > It's about time that we generated some generic initial rootfs for this > > stuff as part of the release process. > > > I'd love to ship big-endian mips64 and little endian mips images as part > of the release process. Maybe 10.0 as a dry/unofficial run with a real run > in 10.1? > > Warner > Yup. We should hammer the hell out of the -HEAD emulated images first (ie, doing buildworlds inside the emulators to ensure that things don't get overly kinked up) and actively backport fixes from -HEAD to -10 though.) Now that this is working, I'm going to make good on my promise to take gonzo's great work and flesh out the missing documentation (ie, rootdev isn't right by default, the specific bits to put in /etc/fstab, adding a swap partition, etc) so that everyone can do it. Then, likely, I'll extend my build scripts to do exactly the documented steps so it will spit out the relevant images. Then, no excuse. -adrian From owner-freebsd-mips@FreeBSD.ORG Mon Oct 7 11:06:48 2013 Return-Path: Delivered-To: freebsd-mips@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B4E3C86C for ; Mon, 7 Oct 2013 11:06:48 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A2574283C for ; Mon, 7 Oct 2013 11:06:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r97B6mi6077773 for ; Mon, 7 Oct 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r97B6mjJ077771 for freebsd-mips@FreeBSD.org; Mon, 7 Oct 2013 11:06:48 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 7 Oct 2013 11:06:48 GMT Message-Id: <201310071106.r97B6mjJ077771@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-mips@FreeBSD.org Subject: Current problem reports assigned to freebsd-mips@FreeBSD.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Oct 2013 11:06:48 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/177876 mips [mips] kernel stack overflow panic on mips64, EdgeRout o kern/165951 mips [ar913x] [ath] DDR flush isn't being done for the WMAC 2 problems total. From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 00:33:03 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8CB02A8A for ; Wed, 9 Oct 2013 00:33:03 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 50A782D79 for ; Wed, 9 Oct 2013 00:33:03 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id m20so58730qcx.15 for ; Tue, 08 Oct 2013 17:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=f660OdaGuIUm+Ns/xURg0fGnyGjiwdWxavPVpwfbW5M=; b=bES7OTfSkR5/uZN6gqgQE1asectavQ/LIg/6I7kcW1VDeRDetisbO2np+VlFM+LoVN 9/5rNhW0v8wO/9Tb1Qk9MZdOvnx1a3cSIp/DAWxFrZGFmvLc96eh1GOAhfnLs+3sdB27 ltpPJBnu584Dv1h+ahP3B8zfGBxtt80EGGTDiKLcwOZni7ovolC3yRPMPnM2HUTd+qme 2Zk3OBNT2zMasBIxQX6oud+nNJ4HPqNXRcRfv1ep3YvxXOJLQM58byPzuHylzSBfsmsd IiO89ofjJdS09lOAPg1yvJIr1UXfbtna27Ks9F6QtqllKoE5ixllxDCWdC7XtqGw5p/F LoXQ== MIME-Version: 1.0 X-Received: by 10.49.59.115 with SMTP id y19mr6034600qeq.8.1381278782457; Tue, 08 Oct 2013 17:33:02 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Tue, 8 Oct 2013 17:33:02 -0700 (PDT) Date: Tue, 8 Oct 2013 17:33:02 -0700 X-Google-Sender-Auth: _WyBqyQ8dM6ULtUbK-nZuwbS-Q4 Message-ID: Subject: [rfc] implement busdma barrier operation From: Adrian Chadd To: "freebsd-mips@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 00:33:03 -0000 Hi, I'd like to implement the busdma barrier operation. This is required for (at least) correct behaviour of devices on the ar9344 (mips74k) core, as now I actually need to care about the order of device operations. This (and some local uncommitted changes) are required for the ar9344 ethernet and switchport devices to correctly function. I've tested this on the other mips24k hardware I have and it works. But nothing else (including ath, grr) explicitly uses read/write barriers. I'll eventually add them. I'd like to bounce this to re@ ASAP to get the approval to commit. Thanks, -adrian Index: sys/mips/mips/bus_space_generic.c =================================================================== --- sys/mips/mips/bus_space_generic.c (revision 256173) +++ sys/mips/mips/bus_space_generic.c (working copy) @@ -749,4 +749,8 @@ if (flags & BUS_SPACE_BARRIER_WRITE) mips_dcache_wbinv_all(); #endif + if (flags & BUS_SPACE_BARRIER_READ) + rmb(); + if (flags & BUS_SPACE_BARRIER_WRITE) + wmb(); } From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 09:35:44 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3AE1A2EE for ; Wed, 9 Oct 2013 09:35:44 +0000 (UTC) (envelope-from kamel.alipoor@gmail.com) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F1E1F283B for ; Wed, 9 Oct 2013 09:35:43 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id c3so363887qcv.32 for ; Wed, 09 Oct 2013 02:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:reply-to:from:to:subject:date:mime-version:content-type :content-transfer-encoding; bh=I2oot/BTSFAxNuzXqLTRB1T2nJfrMwJ4Q9LPvffpVd0=; b=KncaNFGs93harfkTvel/td6Jy78qcEoN+uLOJavwRTIszPdls1+WZhAvuLcRk1eS0E oS3ByldKBPMNXiNg33a61EY376AkmaaZd+b09ZfZfEG2D3ILhvBCHVQjNe8vqTgecvHc K4nhESGPtvZdUq/EoSkenk4o/LYYEnO5e6Di0Wf/yW/d+jKHy686StG8eoRvbzDLTAvA dWDPag5TXFHQXlOHKRN7StI9AGyLkAFMdU62sahBaojkflJkoWnihvjvRyfX6/lfv1uX bIC4ACuqaioFdVPMbdX9SHHCbtm45B4WkhI6btNSRoLdZVAnqYA1K8tgi7VKxVGttS/K 3dYA== X-Received: by 10.224.72.81 with SMTP id l17mr9715138qaj.54.1381311343114; Wed, 09 Oct 2013 02:35:43 -0700 (PDT) Received: from Almani-VAIO ([130.255.231.40]) by mx.google.com with ESMTPSA id 4sm81573873qak.11.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 09 Oct 2013 02:35:42 -0700 (PDT) Message-ID: <02dcfa3c-41556-01ac5455593403@almani-vaio> From: Global Researchers Journals To: freebsd-mips@freebsd.org Subject: Call for Paper October 2013 |Volume 3 : Issue 10| Date: Wed, 9 Oct 2013 13:05:42 +0330 X-Priority: 3 MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Global Researchers Journals List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 09:35:44 -0000 Call for Paper Dear Colleagues You are cordially invited to submit or recommend papers to: [1]http://www.grjournals.com October 2013 (Volume 3 | Issue 10) · Journal of Physiology and Pharmacology Advances (JPPA) [2]http://grjournals.com/Default.aspx?tabid=6605&articleType=CategoryVi ew&categoryId=2302 · Journal of Animal Production Advances (JAPA) [3]http://grjournals.com/Default.aspx?tabid=6606&articleType=CategoryVi ew&categoryId=2301 · Journal of Veterinary Advances (JVA) http://grjournals.com/Default.aspx?tabid=6604&articleType=CategoryView& categoryId=2303 · Journal of Animal Science Advances (JASA) [4]http://grjournals.com/Default.aspx?tabid=6529&articleType=Category View&categoryId=2272 Top 20 Hottest Articles (Last 7 Days) [5]http://www.scopemed.org/ Global Researchers Journals, a fast track peer-reviewed and open access academic journal published by Grjournals Publishing, which is one of the largest open access journal publishers around the world. Grjournals is using online article submission, review and tracking system for quality and quick review processing. Journal provides rapid publication of research article. After 30 days Rapid Review Process by the editorial/review board members or outside experts, an accepted paperwill be placed under In Press within 24 hours and will be published in the next issue. Instructions for authors are available on our website: [6]http://www.grjournals.com Submitted papers must follow the Instructions to authors to be considered for review and publication. Refereeing of manuscripts is conducted anonymously and the identity of the referees is not disclosed. The manuscripts which get an acceptance will publish with DOI number. Your Manuscript(s) can be one of these kinds: Review, Original Article, Case Report, Short Communications, Technical Notes, Mini Review Article and Hypothesis. Some of Abstracted/Index in: CAB reviews, Chemical Abstract Service (CAS), Genamics JournalSeek, Index Directory of Open Access Journals (DOAJ), Index Electronic Journals Library and SCIRUS, ISC and the World most Popular University Electronic Library. [7]http://grjournals.com/Defaul t.aspx?tabid=7329 Now you can clear the clutter by accessing your favorite journals online: · Full text, full archive that's always there when you need it · Easy access anywhere, anytime and anyhow · Impact your practice, not the environment NOTICE: Authors that cite [8]www.grjournals.com manuscripts as reference in their ISI articles, they can send their manuscripts to one of above journals as FREE of charge. After evaluation and get an acceptance it will publish without any Article Processing Fee with DOI. We apologize if you have received this email twice, or our journal is not your field. With Warm Regards Sincerely, Grjournals team Site: [9]www.grjournals.com E_Mail: [10]grjournals@gmail.com References 1. http://www.grjournals.com/ 2. http://grjournals.com/Default.aspx?tabid=6605&articleType=CategoryView&categoryId=2302 3. http://grjournals.com/Default.aspx?tabid=6606&articleType=CategoryView&categoryId=2301 4. http://grjournals.com/Default.aspx?tabid=6529&articleType=CategoryView&categoryId=2272 5. http://www.scopemed.org/ 6. http://www.grjournals.com/ 7. http://grjournals.com/Default.aspx?tabid=7329 8. http://www.grjournals.com/ 9. http://www.grjournals.com/ 10. mailto:grjournals@gmail.com From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 18:52:39 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B1045533; Wed, 9 Oct 2013 18:52:39 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qe0-x235.google.com (mail-qe0-x235.google.com [IPv6:2607:f8b0:400d:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5FF012D09; Wed, 9 Oct 2013 18:52:39 +0000 (UTC) Received: by mail-qe0-f53.google.com with SMTP id cy11so916840qeb.40 for ; Wed, 09 Oct 2013 11:52:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cGQj235prHypXcHT6RW70q//dE5hdh7hVJipeSAhkZU=; b=zv1lhtLhRtnMMKL5oEdAe4MWY3dZLU3rk0gTUO6n7U67oBfxWqwdha1UrLiDccnXcU Yke/4+AE1s29YKYAm9gYLrctxrUrJdije+bAf3c4qwq9yQwcbl/WS0Tbyj2KhnlEkI/T w73BH4zoasxHUuxDG2KvHKHb6TRuBzLFg8llHt432mxiCzn6yCOB5ShzgoiLEHjTodGD tM2unmUDQAE8Iftf7O2cGIdaVuuwZftzSxZH5DfyWJi+/xw0l+fukj1qJpgflYY5/Dba i5M3UEDhybQd+twK41XvsKqp2WpNPCJHidbgGjGTbJYcJRvOeyhKwcuKgWmubtP9iBP2 4nbw== MIME-Version: 1.0 X-Received: by 10.229.75.9 with SMTP id w9mr11500437qcj.0.1381344756341; Wed, 09 Oct 2013 11:52:36 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Wed, 9 Oct 2013 11:52:36 -0700 (PDT) In-Reply-To: References: <180563C9-541F-47F9-8C90-35686BBC6471@bluezbox.com> <3CB8B78B-43F3-4891-9E68-E2CE3EDED96D@bsdimp.com> Date: Wed, 9 Oct 2013 11:52:36 -0700 X-Google-Sender-Auth: 2LJWAXniPOPVnzc-MU6__L-YOIg Message-ID: Subject: Re: FreeBSD/MIPS emulation From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 18:52:39 -0000 Hi, As promised: https://wiki.freebsd.org/FreeBSD/MipsEmulation This requires a bunch of rewriting before I'm happy with it. Next, testing out mipsle and mips64 builds. Thanks gonzo! -adrian From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 20:46:28 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DD1BAADB for ; Wed, 9 Oct 2013 20:46:27 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f170.google.com (mail-ie0-f170.google.com [209.85.223.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ABA4C24A3 for ; Wed, 9 Oct 2013 20:46:27 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id x13so3258687ief.29 for ; Wed, 09 Oct 2013 13:46:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=R7Y8sbn1SCMpxRaGVlkUI0IkZqwROXPM506IPRbPR9Y=; b=TKRkDpC+4i/Jp+AEQ0LsvbOWUMWyOuXvEdkndD6UJ+XmfLXw/ARQFQQkKIzBIyi2fg 2ssh9LW4IELqO9Mne3OCnord09cKrfgZcA88pDBjTbpoXPs8QZAqOnKJdGTnPqjbCv/t q7oO1pQ3OLc4xVgNWDXJYZDIadrUsBDrybQe+lH/0lraUlmiBQmaFawlyJcPqcVDWT42 zOUBJZJXwuy/zoXDWqA8iY6LM4GUGtdW7WbFR8463YtCooJWAMBbH6vGGYV2eYVSxc2U EeDCtFC+uhmVb6vUudbAgmtqRJxzswKlxm3EpSwx2INzIqkO8H5W9Bln6EVzhEUiWQ0r gEoQ== X-Gm-Message-State: ALoCoQlJ0N+Kd0Y7JH2xS99qJouwCZm/RYx+SaHORWcSn6L2jHsDkhbOGufPMCMTAx8QTMGpCFmS X-Received: by 10.50.61.205 with SMTP id s13mr3650993igr.29.1381351580803; Wed, 09 Oct 2013 13:46:20 -0700 (PDT) Received: from [10.0.0.53] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id y10sm9450575igl.4.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Oct 2013 13:46:20 -0700 (PDT) Sender: Warner Losh Subject: Re: [rfc] implement busdma barrier operation Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Wed, 9 Oct 2013 14:46:16 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0ABAE1CE-C4C3-4A3E-843E-E9B450559012@bsdimp.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1085) Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 20:46:28 -0000 On Oct 8, 2013, at 6:33 PM, Adrian Chadd wrote: >=20 > I'd like to implement the busdma barrier operation. You'd like to improve the bus space read and write barrier = implementation on MIPS. > This is required for > (at least) correct behaviour of devices on the ar9344 (mips74k) core, = as > now I actually need to care about the order of device operations. Yup. We should have had something like this all along. You can get rid, I think, of the cache invalidation and write back too, = since that's such a huge hit we'll never do that in a generic = implementation. The rest of the change looks good to me. Warner > This (and some local uncommitted changes) are required for the ar9344 > ethernet and switchport devices to correctly function. >=20 > I've tested this on the other mips24k hardware I have and it works. = But > nothing else (including ath, grr) explicitly uses read/write barriers. = I'll > eventually add them. >=20 > I'd like to bounce this to re@ ASAP to get the approval to commit. >=20 > Thanks, >=20 >=20 > -adrian >=20 >=20 > Index: sys/mips/mips/bus_space_generic.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/mips/mips/bus_space_generic.c (revision 256173) > +++ sys/mips/mips/bus_space_generic.c (working copy) > @@ -749,4 +749,8 @@ > if (flags & BUS_SPACE_BARRIER_WRITE) > mips_dcache_wbinv_all(); > #endif > + if (flags & BUS_SPACE_BARRIER_READ) > + rmb(); > + if (flags & BUS_SPACE_BARRIER_WRITE) > + wmb(); > } > _______________________________________________ > freebsd-mips@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org" From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 21:01:58 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 413E3210; Wed, 9 Oct 2013 21:01:58 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E5A00258D; Wed, 9 Oct 2013 21:01:57 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id k4so5377782qaq.11 for ; Wed, 09 Oct 2013 14:01:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=i3vJ6sxH3ZQFhj1PPqbsDUwkbj7It+nyYWNDDwiHZeI=; b=cui0uRoazdP0l8Kp1/WzfYj4HiB3qxTw/0gCgNiiV3nLjCcqDdA9sg1FJC7HbSpZJ0 7nOCyDYX+uVOCX0hePDmenwfugPkKKh7F6FD+FYM/q77Ejb5BmD3kcqPKkiaveHAZZeA 35ijUmR9CElVAeLORLmYmwOCMPjHKmrRivWjGdCuD6NuFYiXuFjS9SozS/25iiNtFLqI hN7n8MZVFr0DYI5+0dxVIdmKrr2R/CTneUivgqhS9Vr/d2fRpWmv6RKyew2dPaobSWl7 onK+FOA60+FnvGC7lhyi+Di2aSIY4mx6O8GE7ROcKLggdW3kuSKPlXGO+dAb4n77VHzJ 7tAw== MIME-Version: 1.0 X-Received: by 10.229.130.135 with SMTP id t7mr1367314qcs.18.1381352516853; Wed, 09 Oct 2013 14:01:56 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.224.213.136 with HTTP; Wed, 9 Oct 2013 14:01:56 -0700 (PDT) In-Reply-To: References: <180563C9-541F-47F9-8C90-35686BBC6471@bluezbox.com> <3CB8B78B-43F3-4891-9E68-E2CE3EDED96D@bsdimp.com> Date: Wed, 9 Oct 2013 17:01:56 -0400 X-Google-Sender-Auth: XFTcIXzXvbE8PEjBTorpYCdwaK8 Message-ID: Subject: Re: FreeBSD/MIPS emulation From: Ed Maste To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 21:01:58 -0000 On 9 October 2013 14:52, Adrian Chadd wrote: > Hi, > > As promised: > > https://wiki.freebsd.org/FreeBSD/MipsEmulation > > This requires a bunch of rewriting before I'm happy with it. > > Next, testing out mipsle and mips64 builds. Thanks for putting this on the wiki Adrian. Mips64 works -- I have used both qemu-mips64 for emulation of individual binaries and qemu-system-mips64, while working on mips64 support for LLDB. -Ed From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 21:05:33 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 66B3A34B; Wed, 9 Oct 2013 21:05:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qc0-x22e.google.com (mail-qc0-x22e.google.com [IPv6:2607:f8b0:400d:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E021125B2; Wed, 9 Oct 2013 21:05:32 +0000 (UTC) Received: by mail-qc0-f174.google.com with SMTP id n9so1003773qcw.19 for ; Wed, 09 Oct 2013 14:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jhDx6n19P4ZjM+WgaQdWIXRQHEfcB1h5lt2Fp4FY7Tg=; b=I+feG4CNeQ+eo4DRIVDNwTM6Jr1EWD5+A+b+jqvBMB4HTx3qGx8tjR7Smv75ZGBdcZ ZFYNceGT2Lg3t9YobdL/h8ZMB2FDGFoFA6Fnkd93lDt8owr9zzwHsNvRqtX/jp93djAW sjAqMGxOeAdynKbyhIMw7NnKMYLgZafgh0FX5nS6vR+kaDTzEDJUgNItn4Mnhj+VQcJr 2r/iTqneTajEF2hrWGwcr9QuoJ0orLaB/zonCDqFkx8Rd7bmjpW9nKeRqHULfn/AmZn9 J2Et7RYhbAwbo0EslXJuAcOr+vYrKRb6/Ol4Z97wyMtXP+Xohu3vOAMWJGYYDUOEwdV5 dU5Q== MIME-Version: 1.0 X-Received: by 10.229.244.69 with SMTP id lp5mr1423963qcb.14.1381352731950; Wed, 09 Oct 2013 14:05:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Wed, 9 Oct 2013 14:05:31 -0700 (PDT) In-Reply-To: References: <180563C9-541F-47F9-8C90-35686BBC6471@bluezbox.com> <3CB8B78B-43F3-4891-9E68-E2CE3EDED96D@bsdimp.com> Date: Wed, 9 Oct 2013 14:05:31 -0700 X-Google-Sender-Auth: BgrKHhYla4ZrNZqOVjQ85_qmyQ0 Message-ID: Subject: Re: FreeBSD/MIPS emulation From: Adrian Chadd To: Ed Maste Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 21:05:33 -0000 On 9 October 2013 14:01, Ed Maste wrote: > On 9 October 2013 14:52, Adrian Chadd wrote: > > Hi, > > > > As promised: > > > > https://wiki.freebsd.org/FreeBSD/MipsEmulation > > > > This requires a bunch of rewriting before I'm happy with it. > > > > Next, testing out mipsle and mips64 builds. > > Thanks for putting this on the wiki Adrian. Mips64 works -- I have > used both qemu-mips64 for emulation of individual binaries and > qemu-system-mips64, while working on mips64 support for LLDB. Next - can we build live image snapshots of mips le/be for both 32 bit and 64 bit? What would be the right vehicle for this? Crochet, maybe? -adrian From owner-freebsd-mips@FreeBSD.ORG Wed Oct 9 22:53:51 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7FB568B7 for ; Wed, 9 Oct 2013 22:53:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3F2552C95 for ; Wed, 9 Oct 2013 22:53:51 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id ii20so5474339qab.14 for ; Wed, 09 Oct 2013 15:53:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=C7t7FDVDw195KsPPjFnwmpyyMqcxTLWHyPqnERN8j3k=; b=k3k4GtY6zLUmLUbhUEWQ03Otkg5EpZzoJU2DlMH4hBJWAjYod0Klbx+54o05O56U0R 0+3cysBBxFdLew/7PXm2vA97RIp5O7YwPJVxxFiCTst5i+zUd2eAxYMZoCamfLMgtvz3 yY6RwuYJ/VfoO19OY8S26f7YfvpnbfeuwTJwiE18dsPjewSZc3oME7FZZz1KXfiq2Z71 zNdht7ClR08N0jghRtpDWIorqoE5+SicMSg08b5e6Z+zYPogTuwtNmZWKtlF2HoH/y4J c33xbWyECDvLzFPhIJO0gyJXGg9gxuJNAklSNbOLKsCqYPQ+qOtIFmM6leZUCSq5Xtgt N6Bw== MIME-Version: 1.0 X-Received: by 10.49.103.161 with SMTP id fx1mr356244qeb.68.1381359230468; Wed, 09 Oct 2013 15:53:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Wed, 9 Oct 2013 15:53:50 -0700 (PDT) In-Reply-To: <0ABAE1CE-C4C3-4A3E-843E-E9B450559012@bsdimp.com> References: <0ABAE1CE-C4C3-4A3E-843E-E9B450559012@bsdimp.com> Date: Wed, 9 Oct 2013 15:53:50 -0700 X-Google-Sender-Auth: eK436JOu4L9C_L8E3jYR5i5Mkww Message-ID: Subject: Re: [rfc] implement busdma barrier operation From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-mips@freebsd.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 22:53:51 -0000 Committed! -adrian On 9 October 2013 13:46, Warner Losh wrote: > On Oct 8, 2013, at 6:33 PM, Adrian Chadd wrote: > > > > I'd like to implement the busdma barrier operation. > > You'd like to improve the bus space read and write barrier implementation > on MIPS. > > > This is required for > > (at least) correct behaviour of devices on the ar9344 (mips74k) core, as > > now I actually need to care about the order of device operations. > > Yup. We should have had something like this all along. > > You can get rid, I think, of the cache invalidation and write back too, > since that's such a huge hit we'll never do that in a generic > implementation. > > The rest of the change looks good to me. > > Warner > > > This (and some local uncommitted changes) are required for the ar9344 > > ethernet and switchport devices to correctly function. > > > > I've tested this on the other mips24k hardware I have and it works. But > > nothing else (including ath, grr) explicitly uses read/write barriers. > I'll > > eventually add them. > > > > I'd like to bounce this to re@ ASAP to get the approval to commit. > > > > Thanks, > > > > > > -adrian > > > > > > Index: sys/mips/mips/bus_space_generic.c > > =================================================================== > > --- sys/mips/mips/bus_space_generic.c (revision 256173) > > +++ sys/mips/mips/bus_space_generic.c (working copy) > > @@ -749,4 +749,8 @@ > > if (flags & BUS_SPACE_BARRIER_WRITE) > > mips_dcache_wbinv_all(); > > #endif > > + if (flags & BUS_SPACE_BARRIER_READ) > > + rmb(); > > + if (flags & BUS_SPACE_BARRIER_WRITE) > > + wmb(); > > } > > _______________________________________________ > > freebsd-mips@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-mips > > To unsubscribe, send any mail to "freebsd-mips-unsubscribe@freebsd.org" > >