From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 18:49:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47817106566C for ; Mon, 20 Aug 2012 18:49:17 +0000 (UTC) (envelope-from benlaurie@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 04CD28FC12 for ; Mon, 20 Aug 2012 18:49:16 +0000 (UTC) Received: by yenl7 with SMTP id l7so6454520yen.13 for ; Mon, 20 Aug 2012 11:49:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=QYUkDKve7g2c/YDdDI6tJiNDYz6GQKqTYSW8+l+/wm0=; b=ZwjmsPG3cOI7AB2Jmxf0LSY/hC0MBb86lmf3Ig2MDKoLTuYrvR7bY6EQvnXnuf+R3g aRtIOF335zRG8wgaR5zCRF5J7vawTZeR1ibreDDHGu7UoVsGnXXNb0f+/bW15lmu6J/T 17dzK3F5DWIc9OYwDiFDg0pfUhfxE7w7G5aDDrDYVtqQdd6HxcBIetmxqoVFY7KuULCO yyMS7M8XvXxSeJ9H7t0Wtg52QM6FN1v5PmjDZVoAGg3MPNQb7/bdVOwtwvpcKa2ZMaeS 1BNFIKfMcQO6svcK3ib7dX6pyFILMfhoP6Eo7pBz9QzitjgpUsHbwfDXqDKtE0P2aCE4 oz3g== MIME-Version: 1.0 Received: by 10.50.194.165 with SMTP id hx5mr10876949igc.56.1345488556147; Mon, 20 Aug 2012 11:49:16 -0700 (PDT) Sender: benlaurie@gmail.com Received: by 10.50.101.232 with HTTP; Mon, 20 Aug 2012 11:49:16 -0700 (PDT) Date: Mon, 20 Aug 2012 19:49:16 +0100 X-Google-Sender-Auth: e6mKEiTdUco1rTv_Rs4tkHRGc5E Message-ID: From: Ben Laurie To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 18:49:17 -0000 Apparently /dev/random uses h/w PRNGs if available, discarding all other sources of randomness. This seems like a mistake to me - we should fix it. Also, it seems that entropy is available in detailed interrupt timing (see http://www.issihosts.com/haveged/) which might be useful on entropy-starved systems - I suspect we are not exploiting this source, and in light of recent developments, we probably should be, Comments? From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 22:02:50 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 65C4C106564A for ; Mon, 20 Aug 2012 22:02:50 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 430068FC0C for ; Mon, 20 Aug 2012 22:02:50 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7KM2hnr096789; Mon, 20 Aug 2012 15:02:43 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7KM2hC0096788; Mon, 20 Aug 2012 15:02:43 -0700 (PDT) (envelope-from sgk) Date: Mon, 20 Aug 2012 15:02:43 -0700 From: Steve Kargl To: Ben Laurie Message-ID: <20120820220243.GA96700@troutmask.apl.washington.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 22:02:50 -0000 On Mon, Aug 20, 2012 at 07:49:16PM +0100, Ben Laurie wrote: > Apparently /dev/random uses h/w PRNGs if available, discarding all > other sources of randomness. > > This seems like a mistake to me - we should fix it. > > Also, it seems that entropy is available in detailed interrupt timing > (see http://www.issihosts.com/haveged/) which might be useful on > entropy-starved systems - I suspect we are not exploiting this source, > and in light of recent developments, we probably should be, > > Comments? Well, it's hard to comment when you failed to explain *why* you think it is a mistake. In addition, I'm having a hard time parsing your 1st sentence, which appears to be related to this sentence: The device will probe for certain hardware entropy sources, and use these in preference to the fallback, which is a generator implemented in software. from 'man 4 random'. Your 'all other sources of randomness' would then need to be amended to 'all other sources of entropy'. But, then 'all other sources' does not make sense, because only the 'generator implemented in software' is discarded. It is also unclear why a linux-only project for gathering entropy is relevant here given that FreeBSD already has the ability to use both hardware and software interrupts as sources of entropy. -- Steve From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 22:05:40 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 042881065673 for ; Mon, 20 Aug 2012 22:05:40 +0000 (UTC) (envelope-from benlaurie@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id BE8DC8FC08 for ; Mon, 20 Aug 2012 22:05:39 +0000 (UTC) Received: by ialo14 with SMTP id o14so3653151ial.13 for ; Mon, 20 Aug 2012 15:05:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=FiDDRDP2zDEoD1YWqvW5aefXdEdYJMIcfByNO7Iubr4=; b=ADJ19FUz/jFq0LW59T0aGMfpiy5osouN0mYmElKiI7949+qDyWJNh2Dw+QNIZxUMpG KmOs3p2ZBI0jiNFevn3xkiCEL2vCfJl39rejEjtkzlx69rUz/BIVKkD/TSzrOky+mk1J XlRXPeGxZnAXPnWrqErQOKuuQNElza9NoswGkFnXYvpuqj+P8wT173m/qvHzCUiq7H/y CxuFMoxuLhiK4SZh362n9jFN0EVbn/WulyZk6O/DDYGtRFOiqMBK2HVaQHHjSeDA3z3C jjczW22FPFVEr0DQ7KTdGS5A+AbJt3dgAFUu/vY8UN5hpvpeZleIYFnjRQpa21sqTK7Y ayjg== MIME-Version: 1.0 Received: by 10.50.217.137 with SMTP id oy9mr11327521igc.56.1345500339168; Mon, 20 Aug 2012 15:05:39 -0700 (PDT) Sender: benlaurie@gmail.com Received: by 10.50.101.232 with HTTP; Mon, 20 Aug 2012 15:05:39 -0700 (PDT) In-Reply-To: <20120820220243.GA96700@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> Date: Mon, 20 Aug 2012 23:05:39 +0100 X-Google-Sender-Auth: 8GQ0Xc9CzAkIYsylIyh4A4zitoo Message-ID: From: Ben Laurie To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 22:05:40 -0000 On Mon, Aug 20, 2012 at 11:02 PM, Steve Kargl wrote: > On Mon, Aug 20, 2012 at 07:49:16PM +0100, Ben Laurie wrote: >> Apparently /dev/random uses h/w PRNGs if available, discarding all >> other sources of randomness. >> >> This seems like a mistake to me - we should fix it. >> >> Also, it seems that entropy is available in detailed interrupt timing >> (see http://www.issihosts.com/haveged/) which might be useful on >> entropy-starved systems - I suspect we are not exploiting this source, >> and in light of recent developments, we probably should be, >> >> Comments? > > Well, it's hard to comment when you failed to explain > *why* you think it is a mistake. Sorry - because I do not think it is wise to trust the h/w prng so much we discard other entropy. > In addition, I'm having > a hard time parsing your 1st sentence, which appears to > be related to this sentence: > > The device will probe for certain hardware entropy sources, > and use these in preference to the fallback, which is a > generator implemented in software. > > from 'man 4 random'. Your 'all other sources of randomness' > would then need to be amended to 'all other sources of entropy'. > But, then 'all other sources' does not make sense, because only > the 'generator implemented in software' is discarded. That is everything except the hardware, right? So ... all other sources. > It is also unclear why a linux-only project for gathering entropy > is relevant here given that FreeBSD already has the ability to > use both hardware and software interrupts as sources of entropy. It is relevant because it seems there is entropy available in fine-grained timing. From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 22:43:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9DC8D1065670 for ; Mon, 20 Aug 2012 22:43:17 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 5A4288FC15 for ; Mon, 20 Aug 2012 22:43:17 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7KMhHW2097152; Mon, 20 Aug 2012 15:43:17 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7KMhGTC097151; Mon, 20 Aug 2012 15:43:16 -0700 (PDT) (envelope-from sgk) Date: Mon, 20 Aug 2012 15:43:16 -0700 From: Steve Kargl To: Ben Laurie Message-ID: <20120820224316.GC96700@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 22:43:17 -0000 On Mon, Aug 20, 2012 at 11:05:39PM +0100, Ben Laurie wrote: > On Mon, Aug 20, 2012 at 11:02 PM, Steve Kargl > wrote: > > On Mon, Aug 20, 2012 at 07:49:16PM +0100, Ben Laurie wrote: > >> Apparently /dev/random uses h/w PRNGs if available, discarding all > >> other sources of randomness. > >> > >> This seems like a mistake to me - we should fix it. > >> > >> Also, it seems that entropy is available in detailed interrupt timing > >> (see http://www.issihosts.com/haveged/) which might be useful on > >> entropy-starved systems - I suspect we are not exploiting this source, > >> and in light of recent developments, we probably should be, > >> > >> Comments? > > > > Well, it's hard to comment when you failed to explain > > *why* you think it is a mistake. > > Sorry - because I do not think it is wise to trust the h/w prng so > much we discard other entropy. > I guess I simply don't understand your terseness here. FreeBSD collects entropy (on my system) from kern.random.sys.harvest.ethernet: 1 kern.random.sys.harvest.point_to_point: 1 kern.random.sys.harvest.interrupt: 1 kern.random.sys.harvest.swi: 0 What do you mean by h/w prng here? Are you talking about these sysctls and what each collects? Or, are you talking about h/w crypto accelerators such as hifn(4) and padlock(4)? > > In addition, I'm having > > a hard time parsing your 1st sentence, which appears to > > be related to this sentence: > > > > The device will probe for certain hardware entropy sources, > > and use these in preference to the fallback, which is a > > generator implemented in software. > > > > from 'man 4 random'. Your 'all other sources of randomness' > > would then need to be amended to 'all other sources of entropy'. > > But, then 'all other sources' does not make sense, because only > > the 'generator implemented in software' is discarded. > > That is everything except the hardware, right? So ... all other sources. It's a single source. Not sure why you want to write 'all other sources.' Unless you're being overly dramatic to sway opinion to your desire result that it is a bad idea. > > It is also unclear why a linux-only project for gathering entropy > > is relevant here given that FreeBSD already has the ability to > > use both hardware and software interrupts as sources of entropy. > > It is relevant because it seems there is entropy available in > fine-grained timing. kern.random.sys.harvest.swi controls collection of entropy from SWI, implicit would (implicitly?) be controlled by timing of the interrupt. -- Steve From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 22:55:13 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3AFA1065670 for ; Mon, 20 Aug 2012 22:55:13 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id 49AA88FC08 for ; Mon, 20 Aug 2012 22:55:12 +0000 (UTC) Received: from server.rulingia.com (c220-239-249-137.belrs5.nsw.optusnet.com.au [220.239.249.137]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q7KMtAOs079612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 21 Aug 2012 08:55:10 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q7KMt4LF078926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Aug 2012 08:55:04 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q7KMt4nf078925; Tue, 21 Aug 2012 08:55:04 +1000 (EST) (envelope-from peter) Date: Tue, 21 Aug 2012 08:55:04 +1000 From: Peter Jeremy To: Ben Laurie Message-ID: <20120820225504.GA78528@server.rulingia.com> References: <20120820220243.GA96700@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sm4nu43k4a2Rpi4c" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 22:55:13 -0000 --sm4nu43k4a2Rpi4c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: >> Well, it's hard to comment when you failed to explain >> *why* you think it is a mistake. > >Sorry - because I do not think it is wise to trust the h/w prng so >much we discard other entropy. This depends on the relative predictability of Yarrow vs the hardware RNG. FreeBSD random(4) currently only supports one hardware RNG - the one in the VIA Nehemiah. VIA have published an independent evaluation of their RNG which suggests it is a good source of entropy. Additionally, the RNG is not used in a raw form, instead a Davies- Meyer hash is performed using the AES-128 CBC with random key, IV and data to further whiten the output. I am not sure whether anyone has done any comparison of the relative randomness of these approaches. >That is everything except the hardware, right? So ... all other sources. The FreeBSD random(4) device implementation currently allows only one RNG to be active at a time, though it should be possible to create a kernel thread that regularly adds entropy from a hardware RNG to the Yarrow state. >It is relevant because it seems there is entropy available in >fine-grained timing. Part of the entropy harvested at each of the sampling points is the CPU cyclecounter (eg TSC). It's difficult to see what finer grained timing you expect to be used. --=20 Peter Jeremy --sm4nu43k4a2Rpi4c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlAywEgACgkQ/opHv/APuIcFKwCfd10vSexKn3uwiqV+8rsGcN3J /BkAniKFchi+OQNUky8sYPh4GN5ZZ+8q =xPYF -----END PGP SIGNATURE----- --sm4nu43k4a2Rpi4c-- From owner-freebsd-arch@FreeBSD.ORG Mon Aug 20 22:55:22 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9EA0106566B for ; Mon, 20 Aug 2012 22:55:22 +0000 (UTC) (envelope-from si@hibs.net) Received: from hibs-net.nh-serv.co.uk (hibs-net.nh-serv.co.uk [176.56.58.105]) by mx1.freebsd.org (Postfix) with ESMTP id AAD468FC12 for ; Mon, 20 Aug 2012 22:55:22 +0000 (UTC) Received: from hibs-net.nh-serv.co.uk (unknown [127.0.0.1]) by hibs-net.nh-serv.co.uk (Postfix) with ESMTP id 1D4CE2AFA6DA for ; Mon, 20 Aug 2012 14:44:21 +0000 (UTC) Received: from hibs.net (unknown [82.102.13.82]) by hibs-net.nh-serv.co.uk (Postfix) with ESMTP for ; Mon, 20 Aug 2012 14:44:20 +0000 (UTC) Date: Mon, 20 Aug 2012 14:59:57 +0000 From: =?windows-1251?Q?=C2=E5=F0=EA=E0_=D2=E5=EF=EB=EE=E2=E0?= Organization: mafoktowhmkl X-Priority: 3 (Normal) Message-ID: <496677363.20120820145957@hibs.net> To: freebsd-arch@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 8bit Cc: Subject: =?windows-1251?b?5fHr6CDi5PDz4yDs7ubl8iDy+yDx4u7h7uTl7SDoIPXu?= =?windows-1251?b?+CDn4Oft4Oru7Ojy/PH/?= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Aug 2012 22:55:23 -0000 From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 07:10:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id E83AD1065670 for ; Tue, 21 Aug 2012 07:10:36 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from opti.dougb.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id AB4FA14F3CE; Tue, 21 Aug 2012 07:10:36 +0000 (UTC) Message-ID: <5033346C.3080907@FreeBSD.org> Date: Tue, 21 Aug 2012 00:10:36 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: Peter Jeremy References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> In-Reply-To: <20120820225504.GA78528@server.rulingia.com> X-Enigmail-Version: 1.4.3 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Ben Laurie , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 07:10:37 -0000 On 08/20/2012 15:55, Peter Jeremy wrote: > On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: >>> Well, it's hard to comment when you failed to explain >>> *why* you think it is a mistake. >> >> Sorry - because I do not think it is wise to trust the h/w prng so >> much we discard other entropy. > > This depends on the relative predictability of Yarrow vs the hardware > RNG. Throughout this thread people have been mixing up entropy sources, and hardware and software PRNGs. A PRNG has (at least) 2 components, the entropy source(s), and the software that turns the entropy into a stream of pseudo-random output. You can't directly compare "yarrow" vs. Padlock without comparing both elements. > FreeBSD random(4) currently only supports one hardware RNG - the > one in the VIA Nehemiah. VIA have published an independent evaluation > of their RNG which suggests it is a good source of entropy. I'm not sure what paper you're referring to, but according to the padlock programming guide it's a random number generator, not (directly) an entropy source. That said, it certainly *could* be used as an entropy source for yarrow. The way I see it, if padlock is available, there should be 3 options: 1. Use it as the exclusive feed for /dev/random 2. Allow the user to bypass it for the regular yarrow implementation 3. Allow padlock to be utilized as a source of entropy for yarrow. > Additionally, the RNG is not used in a raw form, instead a Davies- > Meyer hash is performed using the AES-128 CBC with random key, IV and > data to further whiten the output. I am not sure whether anyone has > done any comparison of the relative randomness of these approaches. That's the software component of the RNG. >> That is everything except the hardware, right? So ... all other sources. > > The FreeBSD random(4) device implementation currently allows only one > RNG to be active at a time, though it should be possible to create a > kernel thread that regularly adds entropy from a hardware RNG to the > Yarrow state. Right. The mechanism already exists to use devices as feeders to yarrow's entropy pool. It should be trivial to add another one. hth, Doug -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 07:39:33 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0CF1106566B; Tue, 21 Aug 2012 07:39:33 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 305DE8FC0A; Tue, 21 Aug 2012 07:39:33 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7L7dWCV099550; Tue, 21 Aug 2012 00:39:32 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7L7dVMQ099549; Tue, 21 Aug 2012 00:39:31 -0700 (PDT) (envelope-from sgk) Date: Tue, 21 Aug 2012 00:39:31 -0700 From: Steve Kargl To: Doug Barton Message-ID: <20120821073931.GA99502@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5033346C.3080907@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Cc: Ben Laurie , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 07:39:33 -0000 On Tue, Aug 21, 2012 at 12:10:36AM -0700, Doug Barton wrote: > On 08/20/2012 15:55, Peter Jeremy wrote: > > On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: > >>> Well, it's hard to comment when you failed to explain > >>> *why* you think it is a mistake. > >> > >> Sorry - because I do not think it is wise to trust the h/w prng so > >> much we discard other entropy. > > > > This depends on the relative predictability of Yarrow vs the hardware > > RNG. > > Throughout this thread people have been mixing up entropy sources, and > hardware and software PRNGs. A PRNG has (at least) 2 components, the > entropy source(s), and the software that turns the entropy into a stream > of pseudo-random output. > > You can't directly compare "yarrow" vs. Padlock without comparing both > elements. Well, only one person seems confused, but OP seems to remain adament in being terse in his questions. Yes, it seems OP has conflated PRNG and entropy, but again he seems to not want to explain his point of view. > > FreeBSD random(4) currently only supports one hardware RNG - the > > one in the VIA Nehemiah. VIA have published an independent evaluation > > of their RNG which suggests it is a good source of entropy. > > I'm not sure what paper you're referring to, but according to the > padlock programming guide it's a random number generator, not (directly) > an entropy source. That said, it certainly *could* be used as an entropy > source for yarrow. I suspect Jeremy has read the /dev/random code; not some paper. UTSL. -- Steve From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 08:18:03 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3638D106566B for ; Tue, 21 Aug 2012 08:18:03 +0000 (UTC) (envelope-from lev@serebryakov.spb.ru) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id E38178FC14 for ; Tue, 21 Aug 2012 08:18:02 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:8870:d54b:33e8:55b2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 7CC624AC2D; Tue, 21 Aug 2012 12:17:55 +0400 (MSK) Date: Tue, 21 Aug 2012 12:17:52 +0400 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1701421006.20120821121752@serebryakov.spb.ru> To: Peter Jeremy In-Reply-To: <20120820225504.GA78528@server.rulingia.com> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Ben Laurie , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 08:18:03 -0000 Hello, Peter. You wrote 21 =D0=B0=D0=B2=D0=B3=D1=83=D1=81=D1=82=D0=B0 2012 =D0=B3., 2:55:= 04: PJ> RNG. FreeBSD random(4) currently only supports one hardware RNG - the PJ> one in the VIA Nehemiah. VIA have published an independent evaluation But `man glxsb' says, for example, that its hardware RNG is used to harvest entropy... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 08:25:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 400551065670; Tue, 21 Aug 2012 08:25:38 +0000 (UTC) (envelope-from benlaurie@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id E7D0F8FC14; Tue, 21 Aug 2012 08:25:37 +0000 (UTC) Received: by ialo14 with SMTP id o14so4128648ial.13 for ; Tue, 21 Aug 2012 01:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=ahfSDNBDJcK99+gy6udO5cBwI7Gviex+cCiAkjMZoTI=; b=ChSmdQIi7h/uFUsRvKD/P5pxlSOHEo43Dq4bvnH1CBXSPkBPDXGx1j9NIyu2MszuPB l/MFVQrowune3VaDDdZgIhSID1YRKaHJEZ09duz8CkDIbYJ1IxXmTEiXSvjzovVxmPtL evA6XUmAQV5ognu8IeQDXhaHYKESaVH4gG7n1Jv5TucIXELvOAollymL4I10QrNazYx+ X8RJZDYgroaIsqGUtyFMOZ4e9tJKSk1OGJVKJjejkLlgMf9VuT7iA5M15KCLae9yUW6l HERUE90YIvzjxhw/KVIm1BpAQJiE6Dc3Orp8hino8ynqA2iBVooxCGJLN2dK4jtbD08m BzYg== MIME-Version: 1.0 Received: by 10.50.1.204 with SMTP id 12mr12156005igo.56.1345537537137; Tue, 21 Aug 2012 01:25:37 -0700 (PDT) Sender: benlaurie@gmail.com Received: by 10.50.101.232 with HTTP; Tue, 21 Aug 2012 01:25:37 -0700 (PDT) In-Reply-To: <20120821073931.GA99502@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> Date: Tue, 21 Aug 2012 09:25:37 +0100 X-Google-Sender-Auth: O2IRJqwLGfemRFSZsKOwcFZMcc4 Message-ID: From: Ben Laurie To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 Cc: Doug Barton , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 08:25:38 -0000 On Tue, Aug 21, 2012 at 8:39 AM, Steve Kargl wrote: > On Tue, Aug 21, 2012 at 12:10:36AM -0700, Doug Barton wrote: >> On 08/20/2012 15:55, Peter Jeremy wrote: >> > On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: >> >>> Well, it's hard to comment when you failed to explain >> >>> *why* you think it is a mistake. >> >> >> >> Sorry - because I do not think it is wise to trust the h/w prng so >> >> much we discard other entropy. >> > >> > This depends on the relative predictability of Yarrow vs the hardware >> > RNG. >> >> Throughout this thread people have been mixing up entropy sources, and >> hardware and software PRNGs. A PRNG has (at least) 2 components, the >> entropy source(s), and the software that turns the entropy into a stream >> of pseudo-random output. >> >> You can't directly compare "yarrow" vs. Padlock without comparing both >> elements. > > Well, only one person seems confused, but OP seems to > remain adament in being terse in his questions. Yes, > it seems OP has conflated PRNG and entropy, but again > he seems to not want to explain his point of view. Entropy is a poorly defined word, and PRNGs have some (we hope). So, as someone else already explained, there's a hardware entropy source in the VIA Nehemiah. I don't think it really matters whether this is "raw" entropy or a PRNG, what matters is that it seems like a bad idea to trust it so much that we don't use other entropy sources in conjunction with it. As for how it should be fixed, my view is that it should be used as an entropy source for yarrow, but I'm not against Doug's 3 options. >> > FreeBSD random(4) currently only supports one hardware RNG - the >> > one in the VIA Nehemiah. VIA have published an independent evaluation >> > of their RNG which suggests it is a good source of entropy. >> >> I'm not sure what paper you're referring to, but according to the >> padlock programming guide it's a random number generator, not (directly) >> an entropy source. That said, it certainly *could* be used as an entropy >> source for yarrow. > > I suspect Jeremy has read the /dev/random code; not some paper. > > UTSL. > > -- > Steve From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 08:33:57 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 624E61065675; Tue, 21 Aug 2012 08:33:57 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1F6A38FC15; Tue, 21 Aug 2012 08:33:56 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 9DF073B788; Tue, 21 Aug 2012 08:33:49 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q7L8Xm46076711; Tue, 21 Aug 2012 08:33:49 GMT (envelope-from phk@phk.freebsd.dk) To: Ben Laurie From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 21 Aug 2012 09:25:37 +0100." Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 21 Aug 2012 08:33:48 +0000 Message-ID: <76710.1345538028@critter.freebsd.dk> Cc: Doug Barton , Steve Kargl , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 08:33:57 -0000 In message , Ben Laurie writes: >>> > FreeBSD random(4) currently only supports one hardware RNG - [...] I belive this is wrong: hifn7751.c also feeds Yarrow/random(4). That said, purely on principle I'm with Ben here: All sources of entropy should be fed to Yarrow by default. I don't mind there being separate devices so you can get your bits directly from a hardware device if you want to. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 08:43:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D47B5106564A; Tue, 21 Aug 2012 08:43:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 491A18FC16; Tue, 21 Aug 2012 08:43:22 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q7L8hSP0020203; Tue, 21 Aug 2012 11:43:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q7L8hGNR093790; Tue, 21 Aug 2012 11:43:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q7L8hF5o093789; Tue, 21 Aug 2012 11:43:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 21 Aug 2012 11:43:15 +0300 From: Konstantin Belousov To: Poul-Henning Kamp Message-ID: <20120821084315.GL33100@deviant.kiev.zoral.com.ua> References: <76710.1345538028@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mQQrPwpUFXnyQ/Ik" Content-Disposition: inline In-Reply-To: <76710.1345538028@critter.freebsd.dk> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Doug Barton , Ben Laurie , Steve Kargl , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 08:43:23 -0000 --mQQrPwpUFXnyQ/Ik Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 21, 2012 at 08:33:48AM +0000, Poul-Henning Kamp wrote: > In message > , Ben Laurie writes: >=20 > >>> > FreeBSD random(4) currently only supports one hardware RNG - [...] >=20 > I belive this is wrong: hifn7751.c also feeds Yarrow/random(4). >=20 > That said, purely on principle I'm with Ben here: All sources of > entropy should be fed to Yarrow by default. >=20 > I don't mind there being separate devices so you can get your bits > directly from a hardware device if you want to. The question should become much more practical in the short term, since IvyBridge has supposedly high-quality RNG in CPU (uncore). It would be on large percentage of the machines in the near future, as opposed to somewhat exotic add-on cards or VIA CPUs. --mQQrPwpUFXnyQ/Ik Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAzSiMACgkQC3+MBN1Mb4hLyACgo5FurwdHTOqgTFj4BoyBA2he 3QMAn3MCbVPWHItC6OBjQs8Xw5HxsOAM =OXIg -----END PGP SIGNATURE----- --mQQrPwpUFXnyQ/Ik-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 08:55:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 901C9106566B for ; Tue, 21 Aug 2012 08:55:02 +0000 (UTC) (envelope-from benlaurie@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 53ACC8FC14 for ; Tue, 21 Aug 2012 08:55:02 +0000 (UTC) Received: by ialo14 with SMTP id o14so4154230ial.13 for ; Tue, 21 Aug 2012 01:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=gO2h7hxfqIL7+5giinSe4Nb9LfNWF0x0qcw8hyDUKcY=; b=ApKV0P74J18hu/OrM6G8e5JPA0KosZyZB1yDUU4IC3oFBIZa6R1Ks7qBPOVzEVyulL VY8hNU1tASn1d63gpqeX03DEviCX4aY9xUh3qJj12o3n/WxXCSgX+Sjne53LziigaSiW dIGVgMWSmM8Rhkv82+41T2g4SHcvN5DP+ANt6fyMpjkleWMYPqN0O7/HaEl7ggfiXzBq MrEMeDpiQ3+8MjLX5vCqDgCikxYOlgynWZ5jB5zJZIHndEgMXCUckFL+2QKvGkuw83I8 FjtR/dKysdua1slAj3o/zEQhFQzhu025S5aVi1ZlB7hw6Tk9fLpkjzp3Jp5W5t1PK6Z3 mI+Q== MIME-Version: 1.0 Received: by 10.42.61.16 with SMTP id s16mr13418877ich.7.1345539301723; Tue, 21 Aug 2012 01:55:01 -0700 (PDT) Sender: benlaurie@gmail.com Received: by 10.50.101.232 with HTTP; Tue, 21 Aug 2012 01:55:01 -0700 (PDT) In-Reply-To: <20120820225504.GA78528@server.rulingia.com> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> Date: Tue, 21 Aug 2012 09:55:01 +0100 X-Google-Sender-Auth: uAwycw-1NuvCrKaKCuFGQQBaHiM Message-ID: From: Ben Laurie To: Peter Jeremy Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 08:55:02 -0000 On Mon, Aug 20, 2012 at 11:55 PM, Peter Jeremy wrote: > On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: >>It is relevant because it seems there is entropy available in >>fine-grained timing. > > Part of the entropy harvested at each of the sampling points is > the CPU cyclecounter (eg TSC). It's difficult to see what finer > grained timing you expect to be used. In the wake of https://factorable.net/weakkeys12.conference.pdf, I'm wondering how well we do on entropy-starved devices. The thing that worries me about TSC is that multiple identical devices may get similar values during initialisation (I don't know if they do, has anyone studied this?). Skew between TSC and a real-time clock might be useful (because ultimately the RTC relies on a clock that is not synchronised with the CPU clock), but AFAICS we don't use the RTC to provide randomness. I could be missing something, of course, I've only recently started looking at this code. From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 10:03:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4248F106564A; Tue, 21 Aug 2012 10:03:11 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id F14728FC08; Tue, 21 Aug 2012 10:03:10 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id F35A6669C; Tue, 21 Aug 2012 12:03:09 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id BF2CC88CA; Tue, 21 Aug 2012 12:03:09 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Steve Kargl References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> Date: Tue, 21 Aug 2012 12:03:09 +0200 In-Reply-To: <20120821073931.GA99502@troutmask.apl.washington.edu> (Steve Kargl's message of "Tue, 21 Aug 2012 00:39:31 -0700") Message-ID: <86sjbga7fm.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Doug Barton , Ben Laurie , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 10:03:11 -0000 Steve Kargl writes: > Well, only one person seems confused, but OP seems to remain adament > in being terse in his questions. Yes, it seems OP has conflated PRNG > and entropy, but again he seems to not want to explain his point of > view. Maybe your attitude scared him off? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 10:22:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CA9D106564A for ; Tue, 21 Aug 2012 10:22:11 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id BCFFB8FC16 for ; Tue, 21 Aug 2012 10:22:10 +0000 (UTC) Received: from server.rulingia.com (c220-239-249-137.belrs5.nsw.optusnet.com.au [220.239.249.137]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q7LAM8Uc081344 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 21 Aug 2012 20:22:08 +1000 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q7LAM2ZB065827 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 21 Aug 2012 20:22:02 +1000 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q7LAM2Gd065826 for freebsd-arch@freebsd.org; Tue, 21 Aug 2012 20:22:02 +1000 (EST) (envelope-from peter) Date: Tue, 21 Aug 2012 20:22:02 +1000 From: Peter Jeremy To: freebsd-arch@freebsd.org Message-ID: <20120821102202.GA85982@server.rulingia.com> References: <76710.1345538028@critter.freebsd.dk> <20120821084315.GL33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OXfL5xGRrasGEqWY" Content-Disposition: inline In-Reply-To: <20120821084315.GL33100@deviant.kiev.zoral.com.ua> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 10:22:11 -0000 --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Aug-21 00:10:36 -0700, Doug Barton wrote: >On 08/20/2012 15:55, Peter Jeremy wrote: >> one in the VIA Nehemiah. VIA have published an independent evaluation >> of their RNG which suggests it is a good source of entropy. > >I'm not sure what paper you're referring to, but according to the >padlock programming guide it's a random number generator, not (directly) >an entropy source. That said, it certainly *could* be used as an entropy >source for yarrow. I was referring to: http://www.via.com.tw/en/downloads/whitepapers/initiatives/padlock/evaluati= on_padlock_rng.pdf >The way I see it, if padlock is available, there should be 3 options: > >1. Use it as the exclusive feed for /dev/random This is currently the only option. On 2012-Aug-21 12:17:52 +0400, Lev Serebryakov wro= te: >PJ> RNG. FreeBSD random(4) currently only supports one hardware RNG - the >PJ> one in the VIA Nehemiah. VIA have published an independent evaluation > But `man glxsb' says, for example, that its hardware RNG is used to >harvest entropy... On 2012-Aug-21 08:33:48 +0000, Poul-Henning Kamp wrote: >I belive this is wrong: hifn7751.c also feeds Yarrow/random(4). The random(4) man page (and my repetition of it) is somewhat misleading here. The current random(4) code uses the VIA Nehemiah Padlock (with a Davies-Meyer hash) if it's available, otherwise it uses Yarrow. If Yarrow is selected, it uses a variety of entropy sources (as available): "Pure entropy" is regularly harvested from: glxsb(4), hifn(4), safe(4), ubsec(4), MIPS Octeon rnd(4) syscons(4) mouse and keyboard events Various events controlled via kern.random.sys.harvest sysctls: kern.random.sys.harvest.ethernet (default enabled) incoming ethernet packets kern.random.sys.harvest.point_to_point (default enabled) packets written to tun(4), netgraph receive hook kern.random.sys.harvest.interrupt (default enabled): adv(4), adw(4), aha(4), ahb(4), ahci(4), aic(4), amr(4), asr(4), ata(4), bm(4), bt(4), cuda(4), dpt(4), fdc(4), glc(4), ida(4), isp(4), mlx(4), mly(4), mpt(4), mvs(4), ncr(4), ncv(4), nsp(4), pmu(4), ps3cdrom(4), ps3disk(4), pst(4), siis(4), stg(4), sym(4), twe(4), vtblk(4), wds(4) kern.random.sys.harvest.swi (default disabled) Not currently used (or supported) Note that there's also a rndtest(4) device that can monitor the output =66rom hifn(4), safe(4) and ubsec(4). >That said, purely on principle I'm with Ben here: All sources of >entropy should be fed to Yarrow by default. The only reason I can think of for bypassing Yarrow would be to increase the rate at which you can read bits from /dev/random. On 2012-Aug-21 11:43:15 +0300, Konstantin Belousov wr= ote: >The question should become much more practical in the short term, >since IvyBridge has supposedly high-quality RNG in CPU (uncore). Someone(TM) just needs to copy implement the relevant code. --=20 Peter Jeremy --OXfL5xGRrasGEqWY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlAzYUoACgkQ/opHv/APuIc2RACeL3beBc2fe99yGpZNhCJ0aeiD dGAAnRZVcW3h3bJdrWZbDX5lbeWy8lXD =2Vmo -----END PGP SIGNATURE----- --OXfL5xGRrasGEqWY-- From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 12:11:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 25DDD1065675 for ; Tue, 21 Aug 2012 12:11:54 +0000 (UTC) (envelope-from CashmereWorld@emarketing.aplf.com) Received: from mta3.primary.na.epidm.net (mta3.primary.na.epidm.net [146.82.220.238]) by mx1.freebsd.org (Postfix) with ESMTP id 8E54C8FC14 for ; Tue, 21 Aug 2012 12:11:53 +0000 (UTC) DomainKey-Signature: q=dns; a=rsa-sha1; c=nofws; s=dk; d=emarketing.aplf.com; h=Received:Date:Message-Id:X-DMHA-ID-Packet:From:Reply-To:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Dmd9qzul7LdMsyrazoAribN9zmSNZ2efW6HI6zpxtuZeq8GFimLkIAbjTvRJk3M7 6AY2CzKEt0wGWagcinUnKg84tFh1K4gJKZjfKBFUo2UC017RuvdzHRqU6hs29AyM Received: from [10.21.250.231] ([10.21.250.231:2149] helo=CORE18PUMPER1) by pcomtanode10 (envelope-from ) (ecelerity 2.2.2.45 r(34222M)) with ESMTP id 47/05-26318-20B73305; Tue, 21 Aug 2012 08:11:46 -0400 Date: Tue, 21 Aug 2012 21:11:46 +0900 (JST) Message-Id: X-DMHA-ID-Packet: Z4gvK5o622gzTBxXWzQ7-zhVQA0QbBqdYlg-VQ47NHl4DDQg6BEtCVL9FZgEXG9As2khkdY-Sqw= From: "Cashmere World" To: freebsd-arch@freebsd.org Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Cashmere World September 2012 Seminar Registration X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Cashmere World List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 12:11:54 -0000 If you cannot read this email, please press [1]here. Ref. No.: INFASHION_FT01243 | Country: Russia | Company: , Fashion Accessories Purchasing Director Eng | [2]δΈ Cashmere World is the annual business platform for the international cashmere trade. This year, it marks its move from Beijing to Hong Kong with an interesting seminar series that provides a professional forum to discuss the latest developments in the global cashmere industry. Don't miss this opportunity to learn more about the wonderful world of cashmere and make a difference to your business. Our professional speakers will be exploring in-depth the following issues that concern the industry. SEMINAR SCHEDULE 29 September 2012 (Saturday) - Paid Admission Buying Accessories for the European Market Time: 09:30 - 11:00 Venue: Rm S428, Level 4 Speaker: Ms. Sandrine WILLIAMSON Head Accessories Buyer, The Galaries Lafayette Group, France Development & Trends in the Cashmere Trade Time: 10.00 - 13.00 Venue: Rm S427, Level 4 Speakers: Dr. KC HO Director of R&D, Hong Kong Research Institute of Textiles & Apparel Session1: Innovative technologies for cashmere sweater manufacture Dr. Riaz SHAH, Professor cum Chief Scientist, Sher-E University Session 2: The effects of goat cloning on the cashmere industry Ms. Kristina SALMINEN, Arelalizza Session 3: Arelalizza: The Evolution of a brand - A case study of the growth and development of the Finnish cashmere brand Mr. Ricky WONG, M.D., Nameson Group Session 4: To be confirmed Supporter: Sponsors: TEAM LEE FASHION KNITTERS LTD. SEMINAR FEES (USD) Early Bird Offers 3 Admissions or more Standard Confirm on or before 22 Aug Confirm on or before 5 Sep Confirm on or before 5 Sep After 5 Sep 29 Sep (09.30 - 11.00) $120 each $150 each $120 each $180 each 29 Sep (10.00 - 13.00) $120 each $150 each $120 each $180 each Have you already registered as a Buyer for Cashmere World? If not, _________________________________________________________________ Concurrent Event: Fashion Access is Asia's leading international fashion fair for bags, footwear, luggage, small leather goods and lifestyle accessories. Twice a year, we are joined by fashion suppliers and buyers from around the world. The September fair focuses on the Spring-Summer 2013 collections of over 400 exhibitors. [3]www.fashionaccess.aplf.com [4]>>> View more Seminars at Fashion Access _________________________________________________________________ UBM Asia Ltd 17/F China Resources Building 26 Harbour Road Wanchai Hong Kong [5]www.cashmereworldfair.com T: 852 2827 6211 F: 852 2827 7831 Organisers: CFNA / UBM Managers: UBM [6]www.cashmereworldfair.com This newsletter is intended for the cashmere buyers in your company. If you are a proper recipient and wish to contact a Cashmere World representative for more information on the fair, please [7]click here. If this email has been sent to the wrong person, please forward to the right colleague. To unsubscribe from our newsletter, click on [8]unsubscribe. [flosensing?z=5w10YEVWZ0lIV0Bn] References Visible links 1. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGN0Et 2. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGO0Eu 3. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGS0Ey 4. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGT0Ez 5. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGP0Ev 6. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGP0Ev 7. mailto:info@cashmereworldfair.com?subject=Visitor%20Enquiry%20-%20Cashmere%20World 8. mailto:unsubscribe@cashmereworldfair.com?subject=Unsubscribe%20Cashmere%20World%20eNewsletter Hidden links: 9. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGP0Ev 10. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGQ0Ew 11. http://www.ubmonlinereg.com/Registration/default.aspx?fid=91&lang=en 12. http://regadmin.ubmonlinereg.com/Registration/default.aspx?fid=50&lang=en&code=seminar 13. http://emarketing.aplf.com/cgi-bin18/DM/t/h5w10YEVWZ0lIV0LZGR0Ex From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 15:34:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 48BC91065672; Tue, 21 Aug 2012 15:34:17 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 212118FC19; Tue, 21 Aug 2012 15:34:17 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7LFY9kj001849; Tue, 21 Aug 2012 08:34:09 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7LFY89r001848; Tue, 21 Aug 2012 08:34:08 -0700 (PDT) (envelope-from sgk) Date: Tue, 21 Aug 2012 08:34:08 -0700 From: Steve Kargl To: Dag-Erling Sm??rgrav Message-ID: <20120821153408.GA1782@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> <86sjbga7fm.fsf@ds4.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86sjbga7fm.fsf@ds4.des.no> User-Agent: Mutt/1.4.2.3i Cc: Doug Barton , Ben Laurie , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 15:34:17 -0000 On Tue, Aug 21, 2012 at 12:03:09PM +0200, Dag-Erling Sm??rgrav wrote: > Steve Kargl writes: > > Well, only one person seems confused, but OP seems to remain adament > > in being terse in his questions. Yes, it seems OP has conflated PRNG > > and entropy, but again he seems to not want to explain his point of > > view. > > Maybe your attitude scared him off? > What attitude? Look at my first post. I asked Ben to explain *why* he thought it was a mistake. His 3 sentence response can be summarized as "I think it is a mistake, because that's what I think." -- Steve From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 15:44:44 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB0AE1065670; Tue, 21 Aug 2012 15:44:44 +0000 (UTC) (envelope-from benlaurie@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5BC458FC19; Tue, 21 Aug 2012 15:44:44 +0000 (UTC) Received: by ggnk4 with SMTP id k4so7197781ggn.13 for ; Tue, 21 Aug 2012 08:44:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=fXKX5ACsqYgxzYpbGZzbGt10MQsELXvrHZum+X/0iyQ=; b=rVIY5Ba/Ds2Py5XIgJbzmqPXQgFrTpdY5m9tNszBHm47o9+keBI/KCrXUMKDjZrpxz uD5hCpVyB8752No6Wc7uamgiiwF7Npm+VZtqmS0+F2vujiWy4NpACScKdMUKuB6ZsoLc YGVkzJ67WAk+1dM0oVtNciMXvWmHtbTDd3iu1cC0VsRvNqVdzEmrhJJxWjfnZ3ONZGk7 q2S4pin3aclEEz9Thoa+cV3x7i/NONXG9VrMJag0pi9NjMOjAgLPKamOA/YZz4udizkC gpDP9i05j0wWi6pyuqUDnr19QYzii4XZqSiHZTElR4PAW1kuKUxEhXx7juA4JJ9eNmSY 0s/A== MIME-Version: 1.0 Received: by 10.50.213.97 with SMTP id nr1mr14122089igc.40.1345563878086; Tue, 21 Aug 2012 08:44:38 -0700 (PDT) Sender: benlaurie@gmail.com Received: by 10.50.101.232 with HTTP; Tue, 21 Aug 2012 08:44:38 -0700 (PDT) In-Reply-To: <20120821153408.GA1782@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> <86sjbga7fm.fsf@ds4.des.no> <20120821153408.GA1782@troutmask.apl.washington.edu> Date: Tue, 21 Aug 2012 16:44:38 +0100 X-Google-Sender-Auth: __jNwO7O8UWr7VgjvnCun1WLYlU Message-ID: From: Ben Laurie To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 Cc: Dag-Erling Sm??rgrav , Doug Barton , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 15:44:44 -0000 On Tue, Aug 21, 2012 at 4:34 PM, Steve Kargl wrote: > On Tue, Aug 21, 2012 at 12:03:09PM +0200, Dag-Erling Sm??rgrav wrote: >> Steve Kargl writes: >> > Well, only one person seems confused, but OP seems to remain adament >> > in being terse in his questions. Yes, it seems OP has conflated PRNG >> > and entropy, but again he seems to not want to explain his point of >> > view. >> >> Maybe your attitude scared him off? >> > > What attitude? Look at my first post. I asked > Ben to explain *why* he thought it was a mistake. > His 3 sentence response can be summarized as "I > think it is a mistake, because that's what I > think." Whatever. Have my other posts clarified sufficiently for you? From owner-freebsd-arch@FreeBSD.ORG Tue Aug 21 17:18:47 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4C7A9106564A; Tue, 21 Aug 2012 17:18:47 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 220628FC0C; Tue, 21 Aug 2012 17:18:47 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q7LHIiCg002355; Tue, 21 Aug 2012 10:18:44 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q7LHIhSq002354; Tue, 21 Aug 2012 10:18:43 -0700 (PDT) (envelope-from sgk) Date: Tue, 21 Aug 2012 10:18:43 -0700 From: Steve Kargl To: Ben Laurie Message-ID: <20120821171843.GA2142@troutmask.apl.washington.edu> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> <86sjbga7fm.fsf@ds4.des.no> <20120821153408.GA1782@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: Dag-Erling Sm??rgrav , Doug Barton , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Aug 2012 17:18:47 -0000 On Tue, Aug 21, 2012 at 04:44:38PM +0100, Ben Laurie wrote: > On Tue, Aug 21, 2012 at 4:34 PM, Steve Kargl > wrote: > > On Tue, Aug 21, 2012 at 12:03:09PM +0200, Dag-Erling Sm??rgrav wrote: > >> Steve Kargl writes: > >> > Well, only one person seems confused, but OP seems to remain adament > >> > in being terse in his questions. Yes, it seems OP has conflated PRNG > >> > and entropy, but again he seems to not want to explain his point of > >> > view. > >> > >> Maybe your attitude scared him off? > >> > > > > What attitude? Look at my first post. I asked > > Ben to explain *why* he thought it was a mistake. > > His 3 sentence response can be summarized as "I > > think it is a mistake, because that's what I > > think." > > Whatever. > > Have my other posts clarified sufficiently for you? Yes, Peter Jeremy's posts have been helpful as well as reading dev/random/probe.c. It is clear that this issue can effect only i386 and amd64 class platforms and in particular those platforms with a Via processors. -- Steve From owner-freebsd-arch@FreeBSD.ORG Wed Aug 22 03:17:02 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 472A9106566B; Wed, 22 Aug 2012 03:17:02 +0000 (UTC) (envelope-from gnn@freebsd.org) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 138178FC1C; Wed, 22 Aug 2012 03:17:01 +0000 (UTC) Received: from pool-96-250-5-62.nycmny.fios.verizon.net ([96.250.5.62]:57620 helo=minion.home) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1T41RE-0005HT-Qb; Tue, 21 Aug 2012 23:17:01 -0400 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1485\)) From: George Neville-Neil In-Reply-To: Date: Tue, 21 Aug 2012 23:17:00 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <2533EA82-0ACA-48B1-B56D-A3B922A02CF3@freebsd.org> References: <8B16B6D3-A9C1-487E-9C42-6EDD07D689D4@freebsd.org> To: Oleksandr Tymoshenko X-Mailer: Apple Mail (2.1485) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - freebsd.org Cc: arm@freebsd.org, arch@freebsd.org, hackers@freebsd.org Subject: Re: projects/armv6 merged to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2012 03:17:02 -0000 On Aug 17, 2012, at 05:24 , Robert Watson wrote: >=20 > On Thu, 16 Aug 2012, Oleksandr Tymoshenko wrote: >=20 >> projects/armv6 branch was merged to HEAD and should be considered = dead now. This patch is a result of a joint effort by many people. = Including but not limited to: >=20 > Amazing work -- many thanks are due to to everyone who was involved! >=20 And this ought to simplify work on both the Rasberry Pi and BeagleBone, = as well as the rest of the arm systems. Great! Best, George From owner-freebsd-arch@FreeBSD.ORG Wed Aug 22 05:22:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B551B106566B; Wed, 22 Aug 2012 05:22:17 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7A4128FC16; Wed, 22 Aug 2012 05:22:17 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so976403pbb.13 for ; Tue, 21 Aug 2012 22:22:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=fQx4dWsNbKvi2UsL96zsF23jQIDNlWTWxnM936WVYdg=; b=luTJMIwT0gidB2K9gvLOpEGGc/iutUuv/KaydRy/H9a225bv9Ij7pW3CuDvKZNx4Xw 6bzfwyf38e3GVve/NPt342kmjNQgOez1ps7Dyo/xvMQrnDjOaSAxlgfiL11RKjWVxsKr BuV+8fIbyc5gXYrY8Q5rcZmGTeAoqm/2aZ/olLa8+Ih76t9JHvkhcBubwy9UcjWjmiVh 0X5UChIiRXg2Gnyt7GY0B5EHeLNoYjD5qvzFZPTMywI/aUTB0wG+sBI53XZD6n/zcO3e IOeJ/I2JtMYEV2Kwm+TkjJyfPEE8du4DgouohMXJWoBgL7LpKjj2odkuiBHXYQoYskUt 3EqQ== Received: by 10.68.216.136 with SMTP id oq8mr50519917pbc.68.1345612937105; Tue, 21 Aug 2012 22:22:17 -0700 (PDT) Received: from localhost (c-24-130-155-143.hsd1.ca.comcast.net. [24.130.155.143]) by mx.google.com with ESMTPS id qx8sm2933239pbc.63.2012.08.21.22.22.15 (version=SSLv3 cipher=OTHER); Tue, 21 Aug 2012 22:22:16 -0700 (PDT) Date: Tue, 21 Aug 2012 22:22:21 -0700 From: Gleb Kurtsou To: Ben Laurie Message-ID: <20120822052221.GA1692@reks> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Doug Barton , Steve Kargl , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2012 05:22:17 -0000 On (21/08/2012 09:25), Ben Laurie wrote: > On Tue, Aug 21, 2012 at 8:39 AM, Steve Kargl > wrote: > > On Tue, Aug 21, 2012 at 12:10:36AM -0700, Doug Barton wrote: > >> On 08/20/2012 15:55, Peter Jeremy wrote: > >> > On 2012-Aug-20 23:05:39 +0100, Ben Laurie wrote: > >> >>> Well, it's hard to comment when you failed to explain > >> >>> *why* you think it is a mistake. > >> >> > >> >> Sorry - because I do not think it is wise to trust the h/w prng so > >> >> much we discard other entropy. > >> > > >> > This depends on the relative predictability of Yarrow vs the hardware > >> > RNG. > >> > >> Throughout this thread people have been mixing up entropy sources, and > >> hardware and software PRNGs. A PRNG has (at least) 2 components, the > >> entropy source(s), and the software that turns the entropy into a stream > >> of pseudo-random output. > >> > >> You can't directly compare "yarrow" vs. Padlock without comparing both > >> elements. > > > > Well, only one person seems confused, but OP seems to > > remain adament in being terse in his questions. Yes, > > it seems OP has conflated PRNG and entropy, but again > > he seems to not want to explain his point of view. > > Entropy is a poorly defined word, and PRNGs have some (we hope). In my experience using *single* explicit entropy source was often a requirement. In some cases it was even forbidden to use internal PRNG of any sort, you had to wait for external randomness to become available. Anyway mixing several entropy sources was never acceptable. You either have good entropy/randomness source or not. The whole thing was due to conformance to standards, acquiring licenses, etc. Clearly it's not what FreeBSD aims at. And let me stress it - randomness always was a crucial point. People would care more about randomness than algorithm implementation issues. Just my 2 cents. p.s. It doesn't answer your original question, but hopefully can be useful. > > So, as someone else already explained, there's a hardware entropy > source in the VIA Nehemiah. I don't think it really matters whether > this is "raw" entropy or a PRNG, what matters is that it seems like a > bad idea to trust it so much that we don't use other entropy sources > in conjunction with it. > > As for how it should be fixed, my view is that it should be used as an > entropy source for yarrow, but I'm not against Doug's 3 options. > > >> > FreeBSD random(4) currently only supports one hardware RNG - the > >> > one in the VIA Nehemiah. VIA have published an independent evaluation > >> > of their RNG which suggests it is a good source of entropy. > >> > >> I'm not sure what paper you're referring to, but according to the > >> padlock programming guide it's a random number generator, not (directly) > >> an entropy source. That said, it certainly *could* be used as an entropy > >> source for yarrow. > > > > I suspect Jeremy has read the /dev/random code; not some paper. > > > > UTSL. > > > > -- > > Steve From owner-freebsd-arch@FreeBSD.ORG Wed Aug 22 07:02:52 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id EEFFF106564A for ; Wed, 22 Aug 2012 07:02:52 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from [127.0.0.1] (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 0E50614E7D1; Wed, 22 Aug 2012 07:02:49 +0000 (UTC) Message-ID: <50348419.8060604@FreeBSD.org> Date: Wed, 22 Aug 2012 00:02:49 -0700 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: Gleb Kurtsou References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <5033346C.3080907@FreeBSD.org> <20120821073931.GA99502@troutmask.apl.washington.edu> <20120822052221.GA1692@reks> In-Reply-To: <20120822052221.GA1692@reks> X-Enigmail-Version: 1.4.3 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Ben Laurie , Steve Kargl , freebsd-arch@freebsd.org Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2012 07:02:53 -0000 On 8/21/2012 10:22 PM, Gleb Kurtsou wrote: > In my experience using *single* explicit entropy source was often a > requirement. In some cases it was even forbidden to use internal PRNG of > any sort, you had to wait for external randomness to become available. > Anyway mixing several entropy sources was never acceptable. You either > have good entropy/randomness source or not. Been there, done that. :) Hence my suggestion that we make it possible for a dedicated device to be the sole feeder for /dev/random as one of the three possible options. Doug -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) From owner-freebsd-arch@FreeBSD.ORG Wed Aug 22 10:29:32 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 844D41065672 for ; Wed, 22 Aug 2012 10:29:32 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [94.23.254.147]) by mx1.freebsd.org (Postfix) with ESMTP id 4523C8FC0A for ; Wed, 22 Aug 2012 10:29:31 +0000 (UTC) Received: from baby-jane.lamaiziere.net (unknown [192.168.1.10]) by smtp.lamaiziere.net (Postfix) with ESMTP id C55C859E8 for ; Wed, 22 Aug 2012 12:22:22 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by baby-jane.lamaiziere.net (Postfix) with ESMTP id 381312CEC18 for ; Wed, 22 Aug 2012 12:22:22 +0200 (CEST) Date: Wed, 22 Aug 2012 12:22:21 +0200 From: Patrick Lamaiziere To: freebsd-arch@freebsd.org Message-ID: <20120822122221.518d1807@davenulle.org> In-Reply-To: <1701421006.20120821121752@serebryakov.spb.ru> References: <20120820220243.GA96700@troutmask.apl.washington.edu> <20120820225504.GA78528@server.rulingia.com> <1701421006.20120821121752@serebryakov.spb.ru> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.6; i386-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: /dev/random X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2012 10:29:32 -0000 Le Tue, 21 Aug 2012 12:17:52 +0400, Lev Serebryakov a ιcrit : > But `man glxsb' says, for example, that its hardware RNG is used > to harvest entropy... For the record, I've tested (in FreeBSD 7) the output of glxsb with rndtest(4) and that looked good. glxsb collects one uint32 of "pure entropy" each hz/100 (10 ms), I'm not sure if this is enough (?) Regards. From owner-freebsd-arch@FreeBSD.ORG Wed Aug 22 10:39:18 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3AD601065672; Wed, 22 Aug 2012 10:39:18 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by mx1.freebsd.org (Postfix) with ESMTP id BE0E08FC08; Wed, 22 Aug 2012 10:39:17 +0000 (UTC) Received: from [194.32.164.22] (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id q7MAdB08011468; Wed, 22 Aug 2012 11:39:11 +0100 (BST) (envelope-from rb@gid.co.uk) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: Bob Bishop In-Reply-To: <2533EA82-0ACA-48B1-B56D-A3B922A02CF3@freebsd.org> Date: Wed, 22 Aug 2012 11:39:05 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <3440928F-900B-4894-BCFE-5E738B644D59@gid.co.uk> References: <8B16B6D3-A9C1-487E-9C42-6EDD07D689D4@freebsd.org> <2533EA82-0ACA-48B1-B56D-A3B922A02CF3@freebsd.org> To: George Neville-Neil X-Mailer: Apple Mail (2.1278) Cc: arm@freebsd.org, Oleksandr Tymoshenko , hackers@freebsd.org, arch@freebsd.org Subject: Re: projects/armv6 merged to HEAD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2012 10:39:18 -0000 On 22 Aug 2012, at 04:17, George Neville-Neil wrote: >=20 > On Aug 17, 2012, at 05:24 , Robert Watson wrote: >=20 >>=20 >> On Thu, 16 Aug 2012, Oleksandr Tymoshenko wrote: >>=20 >>> projects/armv6 branch was merged to HEAD and should be considered = dead now. This patch is a result of a joint effort by many people. = Including but not limited to: >>=20 >> Amazing work -- many thanks are due to to everyone who was involved! >>=20 >=20 > And this ought to simplify work on both the Rasberry Pi and = BeagleBone, as well as the > rest of the arm systems. Great! What he said. Big thanks to all concerned! -- Bob Bishop rb@gid.co.uk From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 21:28:29 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D9C4106566B for ; Thu, 23 Aug 2012 21:28:29 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id 5D1788FC17 for ; Thu, 23 Aug 2012 21:28:29 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta06.emeryville.ca.mail.comcast.net with comcast id qMCH1j0010cQ2SLA6MUPw1; Thu, 23 Aug 2012 21:28:23 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta10.emeryville.ca.mail.comcast.net with comcast id qMUN1j00P4NgCEG8WMUPgg; Thu, 23 Aug 2012 21:28:23 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7NLSK2M025408; Thu, 23 Aug 2012 15:28:20 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, freebsd-arm@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 15:28:20 -0600 Message-ID: <1345757300.27688.535.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Subject: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 21:28:29 -0000 A recent innocuous change to the USB driver code caused intermittant errors in the umass(4) driver on ARM and MIPS platforms, and this message is some background on partial cacheline flushes, and info on what I found while investigating the cause, which is rooted in the DMA cache coherency code. First I need to say that when I say ARM in this message I mean "ARM and MIPS and any other platform where cache coherency around DMA operations is maintained with help from software (as opposed to being implemented purely in hardware as it is in i386/amd64)." I have no experience on MIPS, but I believe that it's similar to ARM in regards to cache coherency. As far as I know, this is not specific to VIVT caches, but rather is specific to software cache coherency, so it probably applies to armv6/v7 as well as v4 and v5 that I'm working with. Over the years, we've had a series of glitches and patches in the partial cacheline flush logic for arm. Even with all the fixes, I've thought that there are two ways that the scheme could fail, but I've never been able to demonstrate it experimentally, and empirically it seems that the failures are rare. Both ways involve the fact that we have several software entities trying to maintain the caches concurrently, without knowledge of each others' actions or needs. The two ways are basically two variations of a situation where a dirty cacheline is flushed by the software while a DMA operation that overlaps that cacheline is in progress: * A cpu_dcache_wbinv_all() call happens after some DMA data has hit main memory, but before bus_dmamap_sync(POSTREAD) is called. * Two independent DMA operations are happening in different parts of the same cacheline and some DMA data from both devices has hit main memory; whichever operation does the POSTREAD sync first can wipe out data in main memory from the other operation that is still in progress. For problems to happen, the CPU has to also modify the same memory location / cacheline as the DMA, so that the cache holds the newest data for part of the cacheline, and main memory holds the newest data for part of the cacheline. Our logic for handling POSTREAD partial cacheline flushes creates this condition even if it doesn't already exist on entry to the sync routine. The wbinv_all() situation seemed to me the most likely to occur. It gets called from a variety of places for a variety of reasons. It is called from a several places in pmap.c; it appears to me that many of those invocations can happen at completely arbitrary points with respect to any IO that's in progress. Another way wbinv_all() can get invoked is during a call to wbinv_range() or even just inv_range(), when the range to be invalidated is so large that it's too inefficient to loop through the range discarding a line at a time on a given platform. In those cases, for some arm platforms, the inv_range() implementation just calls wbinv_all() internally. The implication of the last paragraph is that wbinv_all() can potentially be invoked as part of the busdma sync operations for any IO, PREREAD, PREWRITE, or POSTREAD, from any device at any time. A recent USB driver change moved some things around in memory, such that a small (13 byte) IO buffer became split across two cachelines, and suddenly we had intermittant (but fairly frequent) failures reported by umass(4). Some logging from the usb driver showed that there was stale data from previous IO operations in part of the IO buffer. I added some code to pre-initialize the buffer to various byte patterns before starting the IO, and after the IO, part of the buffer would contain those patterns, and the rest of the buffer (after the cacheline split point) contained newer data from the IO. It looked pretty conclusively as if the partial cacheline flush logic was failing. First I investigated the logic for handling such splits, but it was working correctly. So I moved on to assuming that the cause was one of the two potential problems I've long suspected. I received a helpful clue from Hans that the buffer in question was allocated once at device creation and remained allocated from that point on. That made it easy to save that buffer pointer when it was created, and write wrappers for all the cache writeback and invalidate routines that checked whether the cacheline containing that buffer was part of the cache operation. What I expected to see was that USB would call the busdma sync ops before starting the IO, and then before it called the post-IO sync ops I would see that something else in the system called wbinv_all() or a [wb]inv_range() that included the umass buffer address. What I actually saw was that that never happened. Not even once. Very rarely I would see some other [wb]inv_range() calls happen, but the ranges never involved the umass buffer, and the unit I'm doing these tests on (a DreamPlug) is not one that ever turns an inv_range into wbinv_all. It eventually occurred to me that I had been overlooking the most obvious way a dirty cacheline can get written back to main memory: the cache hardware needs to evict a line to make room for a new incoming line, and the line it has chosen is dirty and has to be written back before being evicted. Unfortunately, there is no way to instrument the software to detect that happening, so now I'm in the position of proving something based on the complete lack of evidence that anything else is the cause. That's a great way to promote a conspiracy theory, not so great for debugging. In addition to showing that no software-triggered flush/invalidate operations are affecting the cacheline, I was able to show that the problem wasn't just that a partial cacheline flush was involved, but that error condition depended on the specific memory addresses (and thus the specific cacheline) involved. At the point in the usb code where that buffer is allocated I changed the code to add 32 bytes to the buffer offset, so that the buffer is still split across two cachelines in exactly the same way as before, but now it's two different cachelines. When doing this, the error doesn't occur. I think that may lend some weight to the theory that it is hardware-based cacheline eviction which is causing a flush of a dirty cacheline while IO into that memory is in progress, but it's just more circumstantial evidence. I think the intermittant-but-frequent nature of the error may also be circumstantial evidence that hardware eviction is the cause. My DreamPlug unit has a 4-way set-associative cache that selects one of the ways at random when it needs to evict a line for refill. That would seem to imply that there's a one in four chance that the cacheline holding the umass status buffer is the one that gets hit, and that seems to match the symptoms I see of "this usb drive kind of works but there are tons of errors spewing on the console about it". Sometimes you get several failures in a row and the drive fails to attach, but most of the time it limps along with lots of errors followed by succesful retries. I considered trying to lock the cacheline in question into the cache as a way of confirming this situation (that should make the error go away). It turns out that's not especially easy to do on this platform, and you can't lock a single cacheline, you have to lock a whole cache way. That's a pretty big change that will perturb system operations in general, it may be hard to draw conclusions from the results. The ARM Architecture Reference Manual mentions the following guidelines as part of the strategy for handling DMA/cache coherency: * marking the memory areas involved in the DMA operation as uncachable and/or unbufferable * cleaning and/or invalidating the data cache, at least with respect to the address range involved in the DMA operation * draining the write buffer * restrictions on processor accesses to the address range involved in the DMA operation until it is known that the DMA operation is complete. Our partial cacheline flush logic is trying to wish away the last bullet item, but now I think we can never successfully do so. Until last week I thought we had a theoretical problem that could eventually be fixed with a sufficiently-clever cache maintenance implementation that somehow avoided having unrelated parts of the OS interfering with each other's operations. Now it appears that hardware operations we have no direct control over can also lead to memory corruption, and no amount of software cleverness is ever going to allow concurrent CPU and DMA access to the same memory without disabling the cache for that memory range. At this point I was going to launch into some "what we can do about it" rambling, but this is long enough already; I think I'll leave this message as a summary of where we've come from and what I learned in the past few days, and leave "what next" for followups. -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 21:50:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF0E71065672 for ; Thu, 23 Aug 2012 21:50:38 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8B9F48FC12 for ; Thu, 23 Aug 2012 21:50:38 +0000 (UTC) Received: by dadr6 with SMTP id r6so618158dad.13 for ; Thu, 23 Aug 2012 14:50:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=WSYlzD3T5FYG0E17Iqr7sZ9KknY3rYR8v2QRn3PxG8E=; b=X4ardj4Un7wUm8+Z3exMdonqf7Q8fsKmhExRobjDbB0HbvytjaBgsYs+JPgGVxmvPH Wm9KFruzcwJBs/M1aAOz0xXBPArmAICPP2vGa2r9lkSZWvitffK7/fMwKSn9M4DzCYGi vuA8QAnl76ednWOEaA0Y0G5P1eA4QcWT3S2s8G6fx0OOqm1fqNT/NAhbyTFsB6E3mXp4 KpuTig44I8wSJ67E/EixtGpyxOUglwRTZasF5ZCh9EfC3qpzJHZZlemi87Ok6q0TO4ys K3TFav4zd7tSgMRESfi3NcFpYE6OrnZDqrfAJD+uCoutduFYw47ZIUnQeW3wr5xm0Nxo yUTg== Received: by 10.66.80.193 with SMTP id t1mr6152901pax.69.1345758638309; Thu, 23 Aug 2012 14:50:38 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id gf3sm6757324pbc.74.2012.08.23.14.50.36 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 23 Aug 2012 14:50:37 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1345757300.27688.535.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 15:50:35 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQnf+tsffohyLcybQi/VwHvYZUU86SLIF5eA+gxV+u4LFaIsp4AK6bD0WFh3DbyMnSbzH828 Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 21:50:38 -0000 On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: > A recent innocuous change to the USB driver code caused intermittant > errors in the umass(4) driver on ARM and MIPS platforms, and this I think the proper solution is to segregate DMA and non-DMA parts of = structures so that you don't have both sharing a cache line. I also wonder why we don't allocate the DMA memory for these structures = separately from the non-DMA parts. This would eliminate the = USB_CACHE_BYTES kludge (which is CPU dependent, not arch dependent) and = move the knowledge of this junk into busdma layer where it belongs. = =46rom my understanding of the issue, this would completely eliminate = the problem forever! Sharing a cacheline between something that is DMA aware and something = that is just begging for trouble. We're doing more work than we need = to to support this dubious feature and we'd be miles ahead if we could = not share at all. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:09:55 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EB858106566B; Thu, 23 Aug 2012 23:09:55 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id B2EEE8FC16; Thu, 23 Aug 2012 23:09:55 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so2387126pbb.13 for ; Thu, 23 Aug 2012 16:09:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=lPP5YvtAv3fjOFO37wLOSmHkqwKO0tRK39RRHX3KsdA=; b=iJlel0NdvHa0pb0YIuPboM2W+Rhof6sYq13DQpJWAFdf6lK+rJ5UaFiLI+eYNL7Q1h uqaT7qobOE4KQkVME6sDZV2/NaWe2eFrX6VP1YWOUw41MY5mf4sneuJNr4sscZMIiEGm wRkDb3DXJcY+Uj4x3p4A1gRJHMSw6UQNlIU3CvJ2vvecgXvW5nzwmwRe5Hv2gGsIlwWl L7evO1FlXhUm3hEt/RrblcgQpJmFvp/Kcq2GLYMmIKsSIBFkCijwlha/McepSbcOowMD kMSao4x61/eWBKHA36z7yml0i9E9dHQR+dmw3ddp0iu4WSaD+Ugid52ev7CrW2OsBozE 59ww== MIME-Version: 1.0 Received: by 10.66.78.73 with SMTP id z9mr6742852paw.9.1345763395120; Thu, 23 Aug 2012 16:09:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Thu, 23 Aug 2012 16:09:55 -0700 (PDT) In-Reply-To: <1345757300.27688.535.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 16:09:55 -0700 X-Google-Sender-Auth: knDTxNAgW4qmvzZwfhemm3Ea6BQ Message-ID: From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:09:56 -0000 [snip] Whoa, there's USB code that has these small buffers straddle cache lines? Why aren't they just allocated to always be in their own separate buffers, so they don't ever have to worry about overlapping cache lines? Adrian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:11:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8D18B106567C for ; Thu, 23 Aug 2012 23:11:02 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id 42E248FC15 for ; Thu, 23 Aug 2012 23:11:02 +0000 (UTC) Received: from omta06.emeryville.ca.mail.comcast.net ([76.96.30.51]) by qmta14.emeryville.ca.mail.comcast.net with comcast id qNmx1j00116AWCUAEP9wpl; Thu, 23 Aug 2012 23:09:56 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta06.emeryville.ca.mail.comcast.net with comcast id qP9v1j00g4NgCEG8SP9ws5; Thu, 23 Aug 2012 23:09:56 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7NN9spk025527; Thu, 23 Aug 2012 17:09:54 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: freebsd-arch@freebsd.org, freebsd-arm@freebsd.org, freebsd-mips@freebsd.org In-Reply-To: <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 17:09:53 -0600 Message-ID: <1345763393.27688.578.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:11:02 -0000 On Thu, 2012-08-23 at 15:50 -0600, Warner Losh wrote: > On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: > > A recent innocuous change to the USB driver code caused intermittant > > errors in the umass(4) driver on ARM and MIPS platforms, and this > > I think the proper solution is to segregate DMA and non-DMA parts of structures so that you don't have both sharing a cache line. > > I also wonder why we don't allocate the DMA memory for these structures separately from the non-DMA parts. This would eliminate the USB_CACHE_BYTES kludge (which is CPU dependent, not arch dependent) and move the knowledge of this junk into busdma layer where it belongs. From my understanding of the issue, this would completely eliminate the problem forever! > > Sharing a cacheline between something that is DMA aware and something that is just begging for trouble. We're doing more work than we need to to support this dubious feature and we'd be miles ahead if we could not share at all. > > Warner > It seems to me that what we have here is a new type of constraint on DMA operations, and we need a way to communicate that constraint from the part of the platform support code that knows about it to the drivers and driver support code that needs to know. The way we communicate DMA constraints is with a busdma tag, but right now that tag only communicates constraints that were needed for ISA and PCI busses, namely buffer alignment, boundary-crossing restrictions, and exclusion regions. Now we have a new type of constraint, I think of it as "granularity". In effect, we have a DMA system that can only do DMA in cacheline sized chunks. Even when the IO size -- and thus the number of "bits on the wire" -- is less than the cacheline size, at the end of the DMA operation (which includes the software-assisted coherency operations) the number of bytes in memory that may be modified is the size of a cacheline. This is because "the DMA system" is not just the engine that moves bytes around, it's the combination of hardware and software that work together to maintain cache coherency. Ideally we'd find a way to communicate this new constraint using the existing mechanism, the busdma tag, and ideally we'd not have to change every existing call to bus_dma_tag_create() to add a new parm. As I understand it, parent tags are now passed down through the newbus hierarchy consistantly, such that a tag at the nexus level could describe a platform requirement such as granularity, and all devices and the helper code they use will have access to that constraint via inheritance from ancestors' tags. Maybe we could have a new flavor of bus_dma_tag_create() that takes a struct of parms, and existing calls wouldn't have to be changed. Communicating the constraint is only part of the problem; it also has to be easy for drivers to work with that constraint, especially drivers that are not targeted specifically at platforms with granular DMA. I think we can achieve a huge chunk of that purely within the arm/mips implementation of bus_dmamem_alloc(), but even so there would be a new conceptual limitation on using that routine: it is specifically for allocating DMA buffers, and that means that there will be a new a rule that the CPU cannot access any memory within that buffer while an IO operation is in progress. I'd also like to say there's a new rule that you cannot subdivide a buffer obtained from bus_dmamem_alloc() into multiple buffers, or into a combination of DMA and CPU-accessed data. That would be bad news for the USB subsystem, and perhaps other drivers. If this idea is either impossible or particularly contentious, then I guess we'd need some new helper routines so that a driver can subdivide the memory in a way that doesn't violate any constraints implied by the tag used to allocate the buffer. Not all IO occurs using buffers obtained from bus_dmamem_alloc(), and I doubt we can reasonably ever require that it be so. I think the only hope we have of handling that problem is to bounce the requests that don't meet the granularity constraint, just as we'd have to do if the caller-supplied buffer fell into an exclusion zone or violated an alignment or boundary constraint. When I've tossed this idea out in the past there was instant resistance. Yeah, bounce buffers are massively inefficient, but my experience has been that most of the IO that isn't aligned and sized to a multiple of a cacheline is small IO (a few to a few dozen bytes). I've never seen a case of page-sized or larger IO requests that required partial-cacheline handling. I'm sure some examples exist, but they're probably more the exception than the rule. (And the bad performance you'd get from bouncing and copying massive bulk data flow would be a powerful incentive to track down the culprit and improve the driver.) -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:26:26 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8ECE61065673 for ; Thu, 23 Aug 2012 23:26:26 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 129AB8FC12 for ; Thu, 23 Aug 2012 23:26:25 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so2404539pbb.13 for ; Thu, 23 Aug 2012 16:26:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=huWGlQ/K+J9q23N2Dy+i8d3okVdSWb7iIk3+N02V218=; b=odCPfzmLpT2B359L/kELYD5Y1XIOsqw6z2sKsDQ9MC89O/bvlc/TamkPXrDpx4vz9n izn4iBw1lCAs8f0oPVlj5S5G3z3W1PO9Kk4DUo2PIEf5+WYoPk7EKSlH/v2hhD02cUid LQfmtxEW3Iu2f83NMmOtgpmJGZSwv4MKJwlMgu/8EP/cUXMop9LK8yND8sGKaZy/RF+z IlTkDxFFaLM6ZhB6QDLku8rbtRo6KSL9CvpGjeaGekK7gXGvW4MfyiBte715L8jYI5M1 Zx8SvNlXzKGZvRYMcInHepaWg/hqsOoVkGmi5eh939h10AOmsuTfmNgkKivqGBdbYMt3 YEbw== Received: by 10.68.196.193 with SMTP id io1mr8509341pbc.17.1345764385480; Thu, 23 Aug 2012 16:26:25 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id pj10sm6923195pbb.46.2012.08.23.16.26.24 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 23 Aug 2012 16:26:25 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1345763393.27688.578.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 17:26:19 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQltwyQOgjXTXVeLYnXnSEx+Q3Li/O4FlDobGOnLyo9tXlzUKQYMQS/KA/b7CC44P+T+Fps/ Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:26:26 -0000 On Aug 23, 2012, at 5:09 PM, Ian Lepore wrote: > On Thu, 2012-08-23 at 15:50 -0600, Warner Losh wrote:=20 >> On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: >>> A recent innocuous change to the USB driver code caused intermittant >>> errors in the umass(4) driver on ARM and MIPS platforms, and this >>=20 >> I think the proper solution is to segregate DMA and non-DMA parts of = structures so that you don't have both sharing a cache line. >>=20 >> I also wonder why we don't allocate the DMA memory for these = structures separately from the non-DMA parts. This would eliminate the = USB_CACHE_BYTES kludge (which is CPU dependent, not arch dependent) and = move the knowledge of this junk into busdma layer where it belongs. = =46rom my understanding of the issue, this would completely eliminate = the problem forever! >>=20 >> Sharing a cacheline between something that is DMA aware and something = that is just begging for trouble. We're doing more work than we need = to to support this dubious feature and we'd be miles ahead if we could = not share at all. >>=20 >> Warner >>=20 >=20 > It seems to me that what we have here is a new type of constraint on = DMA > operations, and we need a way to communicate that constraint from the > part of the platform support code that knows about it to the drivers = and > driver support code that needs to know. The way we communicate DMA > constraints is with a busdma tag, but right now that tag only > communicates constraints that were needed for ISA and PCI busses, = namely > buffer alignment, boundary-crossing restrictions, and exclusion > regions. =20 >=20 > Now we have a new type of constraint, I think of it as "granularity". > In effect, we have a DMA system that can only do DMA in cacheline = sized > chunks. Even when the IO size -- and thus the number of "bits on the > wire" -- is less than the cacheline size, at the end of the DMA > operation (which includes the software-assisted coherency operations) > the number of bytes in memory that may be modified is the size of a > cacheline. This is because "the DMA system" is not just the engine = that > moves bytes around, it's the combination of hardware and software that > work together to maintain cache coherency. But this isn't new. It is an alignment requirement, which carries with = it an implicit size requirement. If you enforce the alignment, and = force all 'sub buffers' to have this alignment, you don't need the new = thing. > Ideally we'd find a way to communicate this new constraint using the > existing mechanism, the busdma tag, and ideally we'd not have to = change > every existing call to bus_dma_tag_create() to add a new parm. As I > understand it, parent tags are now passed down through the newbus > hierarchy consistantly, such that a tag at the nexus level could > describe a platform requirement such as granularity, and all devices = and > the helper code they use will have access to that constraint via > inheritance from ancestors' tags. Maybe we could have a new flavor of > bus_dma_tag_create() that takes a struct of parms, and existing calls > wouldn't have to be changed. Wouldn't a simpler solution be to just make this alignment requirement = be part of the global parent tag on these platforms and to make sure all = drivers on those platforms use it and don't cop-out and pass NULL? > Communicating the constraint is only part of the problem; it also has = to > be easy for drivers to work with that constraint, especially drivers > that are not targeted specifically at platforms with granular DMA. I > think we can achieve a huge chunk of that purely within the arm/mips > implementation of bus_dmamem_alloc(), but even so there would be a new > conceptual limitation on using that routine: it is specifically for > allocating DMA buffers, and that means that there will be a new a rule > that the CPU cannot access any memory within that buffer while an IO > operation is in progress. I don't think we should pander to drivers that don't know how to do DMA = properly. We get it almost right in bus_dma now. However, going from = almost right to completely right is hard and we keep uncovering edge = cases that bite us. Wouldn't it be better to eliminate all these weird = edge cases? > I'd also like to say there's a new rule that you cannot subdivide a > buffer obtained from bus_dmamem_alloc() into multiple buffers, or into = a > combination of DMA and CPU-accessed data. That would be bad news for > the USB subsystem, and perhaps other drivers. If this idea is either > impossible or particularly contentious, then I guess we'd need some = new > helper routines so that a driver can subdivide the memory in a way = that > doesn't violate any constraints implied by the tag used to allocate = the > buffer. When the USB subsystem went into the tree, this was one of the = criticisms that was ignored. It has come back to bite us time and time = again. Perhaps it is time to fix it once and for all. > Not all IO occurs using buffers obtained from bus_dmamem_alloc(), and = I > doubt we can reasonably ever require that it be so. =20 True, but the I/O that's not in memory from bus_dmamem_alloc is page = aligned. > I think the only > hope we have of handling that problem is to bounce the requests that > don't meet the granularity constraint, just as we'd have to do if the > caller-supplied buffer fell into an exclusion zone or violated an > alignment or boundary constraint. When I've tossed this idea out in = the > past there was instant resistance. Yeah, bounce buffers are massively > inefficient, but my experience has been that most of the IO that isn't > aligned and sized to a multiple of a cacheline is small IO (a few to a > few dozen bytes). I've never seen a case of page-sized or larger IO > requests that required partial-cacheline handling. I'm sure some > examples exist, but they're probably more the exception than the rule. > (And the bad performance you'd get from bouncing and copying massive > bulk data flow would be a powerful incentive to track down the culprit > and improve the driver.) That's also the underlying idea in the bus_dma stuff. You give the = constraints, you get the buffers and if you have a buffer that's outside = the constraints it gets bounced. That's why the sync operations on on = DMA items, not on cache line items. While cache lines are one issue, = memory placement can be another. Floppy drives, for example, couldn't = DMA past the first 16MB and if you have a buffer passed in that's = outside of that, it bounces. If this bouncing produces slower code, = then the drivers should be updated to have better alignment. The USB subsystem is making assumptions about the underlying cache = mechanisms that aren't really true. Ideally, we could get it to stop = doing that. Warner= From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:27:12 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1BBE71065670 for ; Thu, 23 Aug 2012 23:27:12 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id EE4128FC19 for ; Thu, 23 Aug 2012 23:27:11 +0000 (UTC) Received: from omta17.emeryville.ca.mail.comcast.net ([76.96.30.73]) by qmta01.emeryville.ca.mail.comcast.net with comcast id qEbq1j04X1afHeLA1PTBbX; Thu, 23 Aug 2012 23:27:11 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta17.emeryville.ca.mail.comcast.net with comcast id qPTA1j00D4NgCEG8dPTAp2; Thu, 23 Aug 2012 23:27:11 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7NNR8Lw025556; Thu, 23 Aug 2012 17:27:08 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 17:27:08 -0600 Message-ID: <1345764428.27688.591.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:27:12 -0000 On Thu, 2012-08-23 at 16:09 -0700, Adrian Chadd wrote: > [snip] > > Whoa, there's USB code that has these small buffers straddle cache lines? > > Why aren't they just allocated to always be in their own separate > buffers, so they don't ever have to worry about overlapping cache > lines? Yes, but note that with our current code, a small buffer doesn't have to straddle a cacheline boundary to invoke the partial cacheline flush logic. Its very smallness does so -- if a buffer doesn't start on a cacheline boundary, and/or if its size doesn't exactly equal a multiple of the cacheline size, then it's a partial cacheline flush situation. So even a driver striving to work around this trouble can get into trouble -- it uses bus_dmamem_alloc() to grab a buffer for a 13 byte status message. It gets a 16-byte chunk from the kernel allocator, and the other 16 bytes of that cacheline are given to something else in the kernel, and now you have DMA and CPU access going on in the same cacheline despite your efforts to Do The Right Thing with buffer allocation. (At work we avoid this by setting the kernel minimum allocation size to 32 bytes instead of 16. We did so after being burned by exactly this situation -- the "other 16 bytes" in our case belonged to a struct ithread, and trying to track down a lockup that was caused by a partial cacheline flush corrupting the struct ithread kept me entertained for a couple weeks.) -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:36:46 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5376E106564A; Thu, 23 Aug 2012 23:36:46 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 17DD18FC15; Thu, 23 Aug 2012 23:36:45 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so2415467pbb.13 for ; Thu, 23 Aug 2012 16:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=ZqDIjzxMgU3yZwSHpcg2vnW8k3VCuuGcmucl0nHa5xc=; b=qFD3SNi7TjXoHyZyjZc9Cgz9jPt4O0ldrMYgS8HZgp7xzg9tLQju4GahEViI36qO7G 69jHiCDwYe7Unv2VK5NwuH8/LSHXHoH7+cQYP30Oq/Dy7aUEd5k07rptR1hfkbTbXXsp dp5tVjalA9IKe87/jGIPFa4op6MpTriSAw93In2RDvlV5a29SSzxJ6NEXxz+EQB8QQhH CEh2GWHctCICw98iocvHxn0xjQYiRupsd1hVXuKZ9a+V5dEZfUjGGhdinEC+OjmH9Idi 2x3D/BEge6egfP/6T8/ziCRcpdiu7a2IpNOFwMVrzTq6XF7gcz/OQ/p9M2YZvfAbgbSZ qSPA== MIME-Version: 1.0 Received: by 10.68.136.40 with SMTP id px8mr8215347pbb.153.1345765005641; Thu, 23 Aug 2012 16:36:45 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Thu, 23 Aug 2012 16:36:45 -0700 (PDT) In-Reply-To: <1345764428.27688.591.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> <1345764428.27688.591.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 16:36:45 -0700 X-Google-Sender-Auth: 8N9-uFx59Shvyoz92HPvSALf2iE Message-ID: From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:36:46 -0000 Right, that's what Linux does for ARM/MIPS. It just sets the minimum allocation size to be cache line sized. That way they didn't have to fix their USB and network stack code. .. or, we could fix the USB stack code by saying that anything being used as a DMA buffer needs to be minimum cache line size (which can be determined at run time if appropriate) and make the minimum allocation that. Then either it uses a separate allocation for each buffer or it allocates one big set of buffers and chops them up in at least "cache line size" bits. That reminds me, I should do that to the descriptor allocation in the Atheros driver - ie, round up the descriptor allocation size to a multiple of a cache line. That way DMAs don't conflict with the next DMAed buffer.. Adrian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:45:08 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5FBAD106564A for ; Thu, 23 Aug 2012 23:45:08 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by mx1.freebsd.org (Postfix) with ESMTP id 354978FC18 for ; Thu, 23 Aug 2012 23:45:08 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta01.emeryville.ca.mail.comcast.net with comcast id qEbq1j05S1bwxycA1Pl7fN; Thu, 23 Aug 2012 23:45:07 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta18.emeryville.ca.mail.comcast.net with comcast id qPl61j00R4NgCEG8ePl6lZ; Thu, 23 Aug 2012 23:45:07 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7NNj4Mr025577; Thu, 23 Aug 2012 17:45:04 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Warner Losh In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 17:45:03 -0600 Message-ID: <1345765503.27688.602.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:45:08 -0000 On Thu, 2012-08-23 at 17:26 -0600, Warner Losh wrote: > > On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: > > Now we have a new type of constraint, I think of it as "granularity". > > In effect, we have a DMA system that can only do DMA in cacheline sized > > chunks. Even when the IO size -- and thus the number of "bits on the > > wire" -- is less than the cacheline size, at the end of the DMA > > operation (which includes the software-assisted coherency operations) > > the number of bytes in memory that may be modified is the size of a > > cacheline. This is because "the DMA system" is not just the engine that > > moves bytes around, it's the combination of hardware and software that > > work together to maintain cache coherency. > But this isn't new. It is an alignment requirement, which carries > with it an implicit size requirement. If you enforce the alignment, > and force all 'sub buffers' to have this alignment, you don't need the > new thing. So do you think it's safe to assume that any given dma tag that has an alignment constraint also implicitly has a buffer size constraint that the size must be a multiple of the alignment? What if we have a platform with a 32-byte cacheline / DMA granularity, and then we have a builtin device on that SoC which can only do DMA on a 64K alignment (which its tag would reflect), but the hardware can move as little as 1 byte at a time? Children of that bridge device come along and allocate little 16-byte buffers that eat 16 pages each. It doesn't seem all that far-fetched to me. -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:48:51 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76D75106564A; Thu, 23 Aug 2012 23:48:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 39D918FC17; Thu, 23 Aug 2012 23:48:50 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so2428732pbb.13 for ; Thu, 23 Aug 2012 16:48:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Mskt1FyQ/HqkFMIEUCOxaM1D7q7pn1393qcB/jAWPnU=; b=Cog3Xq8zSN5Zl7K2a0nXw/INEb20LCEzMNo7+vVAij2/aBmAPh+3U/OeBe74m0dmGu yAdtXDFPqSEqFCe8mcV593Etog3zCrMHGhF47i0iRyX9oV1gNY2rd0uKCrujMR0lghrE xoLWOYXbxIsljKOc7nRsv7VN8fykkPqlryniKOtsuUZIOnQlNS90mNOdbLPIm/EuaavE AhpgCjau9GHafS+R1SB4v3+8ZVMYaMvawTyITBPJqGICI01PtxMnkSh2codWZU5+ffbF RfElqIHdkf2EoS0iy0+zJi5plfdvMjcWRuhux+m1H+QFy7lNg/SAFMIFedzmPkv8L79C IqOw== MIME-Version: 1.0 Received: by 10.68.231.233 with SMTP id tj9mr8442260pbc.39.1345765730704; Thu, 23 Aug 2012 16:48:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Thu, 23 Aug 2012 16:48:50 -0700 (PDT) In-Reply-To: <1345765503.27688.602.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 16:48:50 -0700 X-Google-Sender-Auth: xNYhy3PgvZjcI4uwRsLgaOc5SX4 Message-ID: From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:48:51 -0000 On 23 August 2012 16:45, Ian Lepore wrote: > So do you think it's safe to assume that any given dma tag that has an > alignment constraint also implicitly has a buffer size constraint that > the size must be a multiple of the alignment? > > What if we have a platform with a 32-byte cacheline / DMA granularity, > and then we have a builtin device on that SoC which can only do DMA on a > 64K alignment (which its tag would reflect), but the hardware can move > as little as 1 byte at a time? Children of that bridge device come > along and allocate little 16-byte buffers that eat 16 pages each. It > doesn't seem all that far-fetched to me. That hardware would suck, wouldn't it? In what case though would the hardware say it can only DMA on a 64k alignment BUT move one byte at a time? Then what would the starting address be for each DMA? adrian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:51:13 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A646F1065670; Thu, 23 Aug 2012 23:51:13 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 213518FC0C; Thu, 23 Aug 2012 23:51:12 +0000 (UTC) Received: by vcbgb22 with SMTP id gb22so1903080vcb.13 for ; Thu, 23 Aug 2012 16:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; bh=JWgh66ZmryhJI/TjUNJMyoMIjSInrwXjCxbw7sDO0VU=; b=PZ8SFjrKUcp9xmS4PNcgWu/uw++fdJfnc3+SvKpzXkSwwVr50wSmz1+tRNT8abbGCF pD7n+mrr2bczBzMck/r5HVeaw2iQ8PY7TsewLVaT9uCez0hpNTwRfgygw5lg2u021j8e tKVDRgLq5mT+G83tr6qqkuMhr/6vUHFB7u1J/UHCAzT9x0KG2xC0FQ6e8xQ4VFs1Csuu hOPedDLB1aqUk7V2gTZKwnf3EQBYy9AxHZAs74HzGQGvCAC1bTyCcC+mGobafrZ7nopb Bxn2nF3M/SvOpaARk+DrRgAK1iSBsAo0uDWyhvn85Jthb90lOKSMv8An6LsuB4wpCerT c5DA== Received: by 10.220.247.137 with SMTP id mc9mr2858779vcb.52.1345765872283; Thu, 23 Aug 2012 16:51:12 -0700 (PDT) Received: from kan.dyndns.org (c-24-63-226-98.hsd1.ma.comcast.net. [24.63.226.98]) by mx.google.com with ESMTPS id s1sm4496996vdi.14.2012.08.23.16.51.10 (version=SSLv3 cipher=OTHER); Thu, 23 Aug 2012 16:51:11 -0700 (PDT) Date: Thu, 23 Aug 2012 19:51:02 -0400 From: Alexander Kabaev To: Warner Losh Message-ID: <20120823195102.2dcc1fcc@kan.dyndns.org> In-Reply-To: <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/xAKsUfCFhIS7lv_PazDak7K"; protocol="application/pgp-signature" Cc: Ian Lepore , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:51:13 -0000 --Sig_/xAKsUfCFhIS7lv_PazDak7K Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Thu, 23 Aug 2012 15:50:35 -0600 Warner Losh wrote: >=20 > On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: > > A recent innocuous change to the USB driver code caused intermittant > > errors in the umass(4) driver on ARM and MIPS platforms, and this >=20 > I think the proper solution is to segregate DMA and non-DMA parts of > structures so that you don't have both sharing a cache line. >=20 > I also wonder why we don't allocate the DMA memory for these > structures separately from the non-DMA parts. This would eliminate > the USB_CACHE_BYTES kludge (which is CPU dependent, not arch > dependent) and move the knowledge of this junk into busdma layer > where it belongs. From my understanding of the issue, this would > completely eliminate the problem forever! >=20 > Sharing a cacheline between something that is DMA aware and something > that is just begging for trouble. We're doing more work than we > need to to support this dubious feature and we'd be miles ahead if we > could not share at all. >=20 > Warner I agree with Warner that this should be addressed at busdma level. When asked for DMA buffer, it should contrain its start address size to the cache line boundary. USB is insane allocating DMA buffers inside of own transfer structure and that was stated on a more than one occasion already. Since USB code refused to budge, we came with a horrible USB_CACHE_BYTES hack as as _temporary_ workaround. --=20 Alexander Kabaev --Sig_/xAKsUfCFhIS7lv_PazDak7K Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iD8DBQFQNsHtQ6z1jMm+XZYRAqeaAJsE2dyZ0FISrcOVX3Je3YvLQ1MNZQCfcF+M bBDN5eQWXQgUWXM3/UOthNI= =mcBu -----END PGP SIGNATURE----- --Sig_/xAKsUfCFhIS7lv_PazDak7K-- From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:55:19 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C6161065672 for ; Thu, 23 Aug 2012 23:55:19 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id 193E78FC0C for ; Thu, 23 Aug 2012 23:55:18 +0000 (UTC) Received: from omta22.emeryville.ca.mail.comcast.net ([76.96.30.89]) by qmta02.emeryville.ca.mail.comcast.net with comcast id qPfC1j0011vN32cA2PvCyA; Thu, 23 Aug 2012 23:55:12 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta22.emeryville.ca.mail.comcast.net with comcast id qPvA1j00d4NgCEG8iPvBTY; Thu, 23 Aug 2012 23:55:12 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7NNt9ti025603; Thu, 23 Aug 2012 17:55:09 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 17:55:09 -0600 Message-ID: <1345766109.27688.606.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:55:19 -0000 On Thu, 2012-08-23 at 16:48 -0700, Adrian Chadd wrote: > On 23 August 2012 16:45, Ian Lepore wrote: > > > So do you think it's safe to assume that any given dma tag that has an > > alignment constraint also implicitly has a buffer size constraint that > > the size must be a multiple of the alignment? > > > > What if we have a platform with a 32-byte cacheline / DMA granularity, > > and then we have a builtin device on that SoC which can only do DMA on a > > 64K alignment (which its tag would reflect), but the hardware can move > > as little as 1 byte at a time? Children of that bridge device come > > along and allocate little 16-byte buffers that eat 16 pages each. It > > doesn't seem all that far-fetched to me. > > That hardware would suck, wouldn't it? > > In what case though would the hardware say it can only DMA on a 64k > alignment BUT move one byte at a time? Then what would the starting > address be for each DMA? > Maybe the device has a reduced number of address bits in its registers and the low-order bits always start at zero and increment internally in a wider register so that any length dma can happen, but it has to start on a 64k boundary. Maybe the address you pass it has to be a 64k boundary, but then the bytes actually end up in one-of-N slots within that 64k region, based on other parameters of the transfer. I've worked with some strange hardware over the years. -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Aug 23 23:59:17 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 458DC106566B; Thu, 23 Aug 2012 23:59:17 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0784E8FC15; Thu, 23 Aug 2012 23:59:16 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so2439273pbb.13 for ; Thu, 23 Aug 2012 16:59:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=bVpnJB73bCLefXPcIvjYhYqKlLLMR3NxqYPJzQo36bs=; b=l37Rd9vmQ/ynZvNsTSwDhK9YQVWXfbu3PrJPR6iGT/dxcAkV0v74ykjudVMxy4x9aM WQy/q8uvDFSJNXPiXtsWxCMW+oMzZoHCj+QDu153Vlr7uZs96ZrrcvurLW2Zk8jmM0Qg ynnQeJ3vNToaY8rJOte5Khe+34yMmsXk1j6N1v/1jHpK2TTD4ogFT6O81cvcKkc1J43y EFdZZjIxsDmwwXp+MgBY9UTRqFgcu16kkPXb/S01nH+a7NQqIOkS9cCsOoMTHgcyWuGe Z1ZdsSqGNOEmFWiFaxgY2u8yhwhV7czSYeJiA7uHYMWB8I0evjvDowD0FOmtmkCOrzwe oXyw== MIME-Version: 1.0 Received: by 10.68.138.169 with SMTP id qr9mr8493185pbb.27.1345766356862; Thu, 23 Aug 2012 16:59:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.36.106 with HTTP; Thu, 23 Aug 2012 16:59:16 -0700 (PDT) In-Reply-To: <1345766109.27688.606.camel@revolution.hippie.lan> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 16:59:16 -0700 X-Google-Sender-Auth: lfQnLTGTsXKt1gXoEwgg8ou0Azs Message-ID: From: Adrian Chadd To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Aug 2012 23:59:17 -0000 On 23 August 2012 16:55, Ian Lepore wrote: >> In what case though would the hardware say it can only DMA on a 64k >> alignment BUT move one byte at a time? Then what would the starting >> address be for each DMA? >> > > Maybe the device has a reduced number of address bits in its registers > and the low-order bits always start at zero and increment internally in > a wider register so that any length dma can happen, but it has to start > on a 64k boundary. > > Maybe the address you pass it has to be a 64k boundary, but then the > bytes actually end up in one-of-N slots within that 64k region, based on > other parameters of the transfer. > > I've worked with some strange hardware over the years. Right. That's pretty odd though. But now I understand where you're coming from. I still think the short term solution should be "fix the USB stack to not do that!" The longer term problem is likely to figure out what makes sense. Eg, if you're going to allow the allocator to interleave 16 byte chunks (on a 32 byte cache line platform) with some being DMA buffers and others being non-DMA buffers; or whether you enforce that the whole chunk is a DMA buffer for your hardware and you look after it, or something else.. Adrian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 24 00:51:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53698106564A for ; Fri, 24 Aug 2012 00:51:38 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from qmta15.emeryville.ca.mail.comcast.net (qmta15.emeryville.ca.mail.comcast.net [76.96.27.228]) by mx1.freebsd.org (Postfix) with ESMTP id 2F5CD8FC19 for ; Fri, 24 Aug 2012 00:51:37 +0000 (UTC) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta15.emeryville.ca.mail.comcast.net with comcast id qPWX1j0020FhH24AFQrXkH; Fri, 24 Aug 2012 00:51:31 +0000 Received: from damnhippie.dyndns.org ([24.8.232.202]) by omta08.emeryville.ca.mail.comcast.net with comcast id qQrV1j00l4NgCEG8UQrWx7; Fri, 24 Aug 2012 00:51:30 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q7O0pSKT025681; Thu, 23 Aug 2012 18:51:28 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Thu, 23 Aug 2012 18:51:28 -0600 Message-ID: <1345769488.27688.625.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2012 00:51:38 -0000 On Thu, 2012-08-23 at 16:48 -0700, Adrian Chadd wrote: > On 23 August 2012 16:45, Ian Lepore wrote: > > > So do you think it's safe to assume that any given dma tag that has an > > alignment constraint also implicitly has a buffer size constraint that > > the size must be a multiple of the alignment? > > > > What if we have a platform with a 32-byte cacheline / DMA granularity, > > and then we have a builtin device on that SoC which can only do DMA on a > > 64K alignment (which its tag would reflect), but the hardware can move > > as little as 1 byte at a time? Children of that bridge device come > > along and allocate little 16-byte buffers that eat 16 pages each. It > > doesn't seem all that far-fetched to me. > > That hardware would suck, wouldn't it? > Thinking about this some more, I think that at least for now we don't have to communicate a new constraint to bus_dma_tag_create(), nor do we need to assume that a size constraint is the same as an alignment constraint. The size constraint is machine dependant in nature, and the busdma implementation code is also MD, and thus should have some MD way of knowing about this constraint for itself without being told by callers. -- Ian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 24 03:56:25 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95AE31065673 for ; Fri, 24 Aug 2012 03:56:25 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 155718FC12 for ; Fri, 24 Aug 2012 03:56:25 +0000 (UTC) Received: by ialo14 with SMTP id o14so3229655ial.13 for ; Thu, 23 Aug 2012 20:56:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=A8ZRIPSNaFjBDKLw6gxuyEvrrxkJ3KqudBpGidIaZTY=; b=DaKuHjKnENU8HDYehpzcZ2D5+uOIpQr4raKzwzqzxVVMc+vf+2SJtLrVaKwhTS0Cbh QJ7y1TOqav35Ws2ols8U5g5EU4/BUJWWIKqEF7MHCts+7lgrGGjsXDeq2Wk27TV2J0iL RR5Az7Ptl9AzsVFt4MNslFWk5G3cSdnNPMt+VseS0czAdec/Y4IHZ5yyy3bJXtA6vEM6 qZKZINewQnx8XG0meYnxlibIrgM6IVzk34VWbtIqUysop1BR3IPNClV6xqhn6yYmTgqp EQwJNy4w9JjTFTxazVnhni5IacnSr7b9pImuvHvQsu+EuXrVFTfsvZuk1Q29XJb6mQwU RNPw== Received: by 10.50.34.131 with SMTP id z3mr649492igi.45.1345780584053; Thu, 23 Aug 2012 20:56:24 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id va9sm1181375igb.17.2012.08.23.20.56.22 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 23 Aug 2012 20:56:22 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1345765503.27688.602.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 21:56:21 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQnUp2PRFHAK7H8/V9pMr8N7GYTCzXFc1943HVRYL59rigzfz+iogqnSdz9D++ZVGnTvSn+X Cc: freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2012 03:56:25 -0000 On Aug 23, 2012, at 5:45 PM, Ian Lepore wrote: > On Thu, 2012-08-23 at 17:26 -0600, Warner Losh wrote: >>> On Aug 23, 2012, at 3:28 PM, Ian Lepore wrote: >>> Now we have a new type of constraint, I think of it as = "granularity". >>> In effect, we have a DMA system that can only do DMA in cacheline = sized >>> chunks. Even when the IO size -- and thus the number of "bits on = the >>> wire" -- is less than the cacheline size, at the end of the DMA >>> operation (which includes the software-assisted coherency = operations) >>> the number of bytes in memory that may be modified is the size of a >>> cacheline. This is because "the DMA system" is not just the engine = that >>> moves bytes around, it's the combination of hardware and software = that >>> work together to maintain cache coherency. >> But this isn't new. It is an alignment requirement, which carries >> with it an implicit size requirement. If you enforce the alignment, >> and force all 'sub buffers' to have this alignment, you don't need = the >> new thing.=20 >=20 > So do you think it's safe to assume that any given dma tag that has an > alignment constraint also implicitly has a buffer size constraint that > the size must be a multiple of the alignment? Yes. If something must be aligned to N bits, chances are it doesn't = decode the lower N bits which implies a size constraint. > What if we have a platform with a 32-byte cacheline / DMA granularity, > and then we have a builtin device on that SoC which can only do DMA on = a > 64K alignment (which its tag would reflect), but the hardware can move > as little as 1 byte at a time? Children of that bridge device come > along and allocate little 16-byte buffers that eat 16 pages each. It > doesn't seem all that far-fetched to me. This would be a very odd hardware. DMA aligned to 64k that can only = move one byte seems far fetched. How useful would such a design be? = How would you do scatter gather on such a design? But this isn't what I'm saying. If the cache line size is 32, then for = DMA we only ever give out chunks of 32 or larger. In that case, the = split cache line situation you gave as an example can't happen.= From owner-freebsd-arch@FreeBSD.ORG Fri Aug 24 04:00:48 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 063BD106566B for ; Fri, 24 Aug 2012 04:00:48 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6419F8FC0A for ; Fri, 24 Aug 2012 04:00:47 +0000 (UTC) Received: by ialo14 with SMTP id o14so3236393ial.13 for ; Thu, 23 Aug 2012 21:00:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=/fumm9ehnwMxdlhILXylmJSdzHQbk9R61mCz3E6uRRA=; b=IuMNz62qvmpHzoG5MX56WeR7iDQFUKw3beKKwcVD86mvENRsVZ9OqPjVquc1BPaEHz oLF5MpV1m45zlspURZrcdXFWswfZ6NolnLe6ml3RHwggYgTgxrwcTlIJM3thuUfFJoXd S1MyAz1hLgIXEJV8tLemsZ9MvSx6rNfC1v7TdRfDNk1QFBHyXX7r3bx/vVVpE0f57iHu KlxeZKtElDjIuM0GPGXIENYMwzoodqs5Y0XICrSdZ4cCrnlyCeEmJbwZPyybPcBK2w4c 8vDxnAphlMEPjbJt1y6IoHsX2A20Rx/vZLVTIt/ov3SACphryE90CuK1Way/rDSj8efy HOeA== Received: by 10.50.47.161 with SMTP id e1mr702151ign.11.1345780846750; Thu, 23 Aug 2012 21:00:46 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id ua2sm3449366igb.7.2012.08.23.21.00.45 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 23 Aug 2012 21:00:46 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Thu, 23 Aug 2012 22:00:44 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345766109.27688.606.camel@revolution.hippie.lan> To: Adrian Chadd X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQl++j6qcE7r4S03QjtCBkezO84OTxqdotfsKoys7bPdKIhAc+WjZ/dpH89hF3CIyaTHeNZt Cc: Ian Lepore , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2012 04:00:48 -0000 On Aug 23, 2012, at 5:59 PM, Adrian Chadd wrote: > On 23 August 2012 16:55, Ian Lepore = wrote: >=20 >>> In what case though would the hardware say it can only DMA on a 64k >>> alignment BUT move one byte at a time? Then what would the starting >>> address be for each DMA? >>>=20 >>=20 >> Maybe the device has a reduced number of address bits in its = registers >> and the low-order bits always start at zero and increment internally = in >> a wider register so that any length dma can happen, but it has to = start >> on a 64k boundary. >>=20 >> Maybe the address you pass it has to be a 64k boundary, but then the >> bytes actually end up in one-of-N slots within that 64k region, based = on >> other parameters of the transfer. >>=20 >> I've worked with some strange hardware over the years. >=20 > Right. That's pretty odd though. But now I understand where you're = coming from. >=20 > I still think the short term solution should be "fix the USB stack to > not do that!" >=20 > The longer term problem is likely to figure out what makes sense. Eg, > if you're going to allow the allocator to interleave 16 byte chunks > (on a 32 byte cache line platform) with some being DMA buffers and > others being non-DMA buffers; or whether you enforce that the whole > chunk is a DMA buffer for your hardware and you look after it, or > something else.. The bottom line is that you can't mix things like that when cache lines = are involved. The current code that tries is doomed to failure. Doomed. = You just can't control all flushes, as Ian's missive demonstrates, and = trying to accommodate code that does this I don't think can possibly = work. All the interrupt masking, copying in and out, etc I fear is = doomed to utter and abject failure. =20 Warner= From owner-freebsd-arch@FreeBSD.ORG Fri Aug 24 04:03:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B00D1106566C for ; Fri, 24 Aug 2012 04:03:23 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 692CE8FC0A for ; Fri, 24 Aug 2012 04:03:23 +0000 (UTC) Received: by ialo14 with SMTP id o14so3240743ial.13 for ; Thu, 23 Aug 2012 21:03:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=eI5+aVAQMNRlLMXxkUR0Gm4Y8aCwg0o1YoPQvX3yxX8=; b=LtoChv97V1W3vunaZJXqw7AYRgG3j6+0SXhrBLyQLaKG/PXB5Srj5pOfpI3FgROlnd J4FO6t0OJ8/zCLwmXT284ryXOIXsrZEObSsLpIW0AOV47bCOnH/I2M6rPMBdo9U3aGXR WSPct5DFgW1VPSWmvR7aXZ6SMVDnLTRbYO+Up+8U5YK7F+JenALvSdYtiINjZ0+HQ1jy VIWZTyUPiM1D904hM2r3Efxgf+/B/9L96qeDbZklMlwng9VioKKjsQDFMEMTBUYa25iM b0DILD9nvMkMfVBolSJPRhMtX3MN+TWCbEkzI1fbG752BrS182VV1dQKZDoYlPDUwqbE TV7g== Received: by 10.50.213.106 with SMTP id nr10mr617077igc.58.1345781002439; Thu, 23 Aug 2012 21:03:22 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id ey10sm3397801igb.17.2012.08.23.21.03.20 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 23 Aug 2012 21:03:21 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1345769488.27688.625.camel@revolution.hippie.lan> Date: Thu, 23 Aug 2012 22:03:19 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <6639450F-11BB-4D01-8D0C-CD66B427EF08@bsdimp.com> References: <1345757300.27688.535.camel@revolution.hippie.lan> <3A08EB08-2BBF-4B0F-97F2-A3264754C4B7@bsdimp.com> <1345763393.27688.578.camel@revolution.hippie.lan> <1345765503.27688.602.camel@revolution.hippie.lan> <1345769488.27688.625.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQnfh04Q8JgoB+Xi2YaOfugQjHOta/bD8p7AVx+Wph9c07EJtWy1j5EnzTpb/2Q9HgTjDwtP Cc: Adrian Chadd , freebsd-arm@freebsd.org, freebsd-mips@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Aug 2012 04:03:23 -0000 On Aug 23, 2012, at 6:51 PM, Ian Lepore wrote: > On Thu, 2012-08-23 at 16:48 -0700, Adrian Chadd wrote: >> On 23 August 2012 16:45, Ian Lepore = wrote: >>=20 >>> So do you think it's safe to assume that any given dma tag that has = an >>> alignment constraint also implicitly has a buffer size constraint = that >>> the size must be a multiple of the alignment? >>>=20 >>> What if we have a platform with a 32-byte cacheline / DMA = granularity, >>> and then we have a builtin device on that SoC which can only do DMA = on a >>> 64K alignment (which its tag would reflect), but the hardware can = move >>> as little as 1 byte at a time? Children of that bridge device come >>> along and allocate little 16-byte buffers that eat 16 pages each. = It >>> doesn't seem all that far-fetched to me. >>=20 >> That hardware would suck, wouldn't it? >>=20 >=20 > Thinking about this some more, I think that at least for now we don't > have to communicate a new constraint to bus_dma_tag_create(), nor do = we > need to assume that a size constraint is the same as an alignment > constraint. The size constraint is machine dependant in nature, and = the > busdma implementation code is also MD, and thus should have some MD = way > of knowing about this constraint for itself without being told by > callers. And also allow for different cache line sizes for different CPUs within = the same MACHINE/MACHINE_ARCH, which is common in MIPS land, at least. Warner