From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 02:56:24 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47F99E3B; Sun, 28 Oct 2012 02:56:24 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id D0F458FC08; Sun, 28 Oct 2012 02:56:23 +0000 (UTC) Received: from [192.168.1.47] (unknown [176.222.238.90]) by csmtp2.one.com (Postfix) with ESMTPA id 4D822300D805; Sun, 28 Oct 2012 02:56:16 +0000 (UTC) Content-Type: multipart/mixed; boundary="Apple-Mail=_2C9DFA11-07B5-49DB-92E9-FCFAD9FF2ADB" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: How about finally replacing GNATS? From: Erik Cederstrand In-Reply-To: <8639118y6q.fsf@ds4.des.no> Date: Sun, 28 Oct 2012 03:56:15 +0100 Message-Id: <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <8639118y6q.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1499) X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Eitan Adler , freebsd-arch@freebsd.org, giffunip@tutopia.com, Julien Laffaye X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 02:56:24 -0000 --Apple-Mail=_2C9DFA11-07B5-49DB-92E9-FCFAD9FF2ADB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Den 26/10/2012 kl. 09.55 skrev Dag-Erling Sm=F8rgrav : >=20 >> But maybe I could write a Python script to process emails received at >> FreeBSD-gnats-submit@freebsd.org and shepherd them into Bugzilla? >=20 > Yes. Attached is a proof-of-concept script that takes a send-pr email and = pushes it into Bugzilla. I have successfully created bugs in a test = Bugzilla installation with a stock send-pr on FreeBSD, including adding = attachments to the bug. It relies on the Bugzilla XML-RPC API (the JSON = one was too buggy). It needs to be adjusted to the actual FreeBSD Bugzilla setup (severity = state names etc.) and how we want to map send-pr fields to Bugzilla = fields, but at least it's a start. To make it listen on a given email address, stick it into = /etc/mail/aliases like this: freebsd-gnats-submit: "|/path/to/send_pr_to_bugzilla.py" Thanks, Erik --Apple-Mail=_2C9DFA11-07B5-49DB-92E9-FCFAD9FF2ADB Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_2C9DFA11-07B5-49DB-92E9-FCFAD9FF2ADB-- From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 05:25:57 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F3EDDD4A for ; Sun, 28 Oct 2012 05:25:56 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id 6CF648FC0A for ; Sun, 28 Oct 2012 05:25:55 +0000 (UTC) Received: by mail-wi0-f178.google.com with SMTP id hr7so1152125wib.13 for ; Sat, 27 Oct 2012 22:25:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8S9OTRXPZch2QiyxHoJnmmFT+tjy4tehlAZQ/5USTjk=; b=jv5wtJgoDP5EY2PE4ez8hSVwqvEW2WKPWFnGVmlkAm/w9qmJZLI2zL/1WUmOPJ1q5P 9EK1++etCtTwaqpyc7ycZqNJicilZSps5uJz569o44DvSN3gEdLr19Nlg7g+lISp15cA w+k2PTLITRoc99oYOyWMFdRRLItHZLRBsoa20= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=8S9OTRXPZch2QiyxHoJnmmFT+tjy4tehlAZQ/5USTjk=; b=ZRhWthx1jkcQ2X1w740fCrhavixYs6x8ksGFMQhRUDfc6BPOsF+spuaCGFn1CWlQFT kRvyLTckzXuFdj3VnXqeQk4XEnd5X0xCYWT8aNJhH3vGS8LujkTUIPt6yfu97oUmMZm6 aOUG+zP9zHDYwQ4TKCUb2qwbzXLhgm0GUkz75mRpJAz49UwdhVZtaHeWZczcZ2643U6m BCAvZ6bLJ6YEsxDoM1crJyDrMLjMP8eh9+dfzD0z7ahLJJeEodTYuwofINIlq7eb1NKl owVuyql1H6vapps4Z095qz9KqTM4M+WO/dvHnuC2VhL+AToAXWIOPB8CLh2GKNpnAWl3 SLDw== MIME-Version: 1.0 Received: by 10.180.91.71 with SMTP id cc7mr10528477wib.2.1351401955062; Sat, 27 Oct 2012 22:25:55 -0700 (PDT) Received: by 10.194.29.2 with HTTP; Sat, 27 Oct 2012 22:25:54 -0700 (PDT) In-Reply-To: References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> Date: Sat, 27 Oct 2012 22:25:54 -0700 Message-ID: Subject: Re: How about finally replacing GNATS? From: Peter Wemm To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmCVNC0lmWzGWKBHWq06TvNNWOviK9hB9y5XP79xp+zdQx8HxQNRuWtNbzArBD59iIb7ym6 Cc: Erik Cederstrand , Julien Laffaye , giffunip@tutopia.com, freebsd-arch@freebsd.org, Dag-Erling Sm?rgrav X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 05:25:57 -0000 On Thu, Oct 25, 2012 at 12:42 PM, Eitan Adler wrote: > On 25 October 2012 15:31, Gary Palmer wrote: >>> Is this really a problem? Casual users could just use the Bugzilla interface, and experienced users spitting out bug reports should be able to install a port. > > This is also a good point. The point of doing bugs better is to establish and maintain a relationship with the submitter. The unidirectional send-pr.sh interface is awful for that. A transition plan should include: * converting the gnats database * capturing legacy submissions arriving from existing send-pr installations. Hopefully these will die out over time. * hunting down and removing send-pr.sh with extreme prejudice. (possibly replacing it with a stub that gives a pointer to bugzilla) On the exceedingly remote chance that somebody genuinely can't/won't access an HTTP link in 2012+ there's always the option of having them email one of the lists. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 14:07:13 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CF7CDE49; Sun, 28 Oct 2012 14:07:13 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 21C578FC16; Sun, 28 Oct 2012 14:07:12 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id jf20so1733310bkc.13 for ; Sun, 28 Oct 2012 07:07:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=bbeRhWgLI2iXV0kFTnF0t0YIH4FXNaJmGPEFduOCYsE=; b=Q3A5aoxm00okrry9EXK+bzSvI+5gyO8XNIhbvYVCnLV1nCmbrA9vr+DYoSrZxMOtEV yofsL/SQ85CxQaZGzEEToXMrNrn68+LA0w+S8obFctOBoN4C7WdB7uGE3+vxupuSzYuS bYqlLZyFBC08adPHzY5qsrfRGM5MAdlOcX0p/lo2xN9YEDpWX3MR8CUahabI7v+76LAD 8G2u6K4zNZC6pESVCTsbVXElkZ+BSlcuUDG3N+kVi6pnYsoy+IC7V9yKh0nBbxqS5hzF 0jdQNgxwY4g8/TUCL4PNIaEU9vf5XSYMnHObk6t9aKJttN7NNV+Bms/hjWWeZtS1g2hs Om/w== Received: by 10.204.4.200 with SMTP id 8mr8807145bks.81.1351433231667; Sun, 28 Oct 2012 07:07:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.50.197 with HTTP; Sun, 28 Oct 2012 07:06:41 -0700 (PDT) In-Reply-To: <20121027211032.C161058094@chaos.jnpr.net> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <127FA63D-8EEE-4616-AE1E-C39469DDCC6A@xcllnt.net> <20121025211522.GA32636@dragon.NUXI.org> <3F52B7C9-A7B7-4E0E-87D0-1E67FE5D0BA7@xcllnt.net> <20121025221244.GG3808@ithaqua.etoilebsd.net> <20121026181152.GC44331@dragon.NUXI.org> <20121026204910.E1FFA58094@chaos.jnpr.net> <20121026233225.54FB858094@chaos.jnpr.net> <20121027211032.C161058094@chaos.jnpr.net> From: Chris Rees Date: Sun, 28 Oct 2012 14:06:41 +0000 Message-ID: Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program To: "Simon J. Gerraty" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 14:07:13 -0000 On 27 October 2012 22:10, Simon J. Gerraty wrote: > > On Sat, 27 Oct 2012 19:53:56 +0100, Chris Rees writes: >>I'm saying that it's unacceptable to expect people to change their >>systems just to make the ports tree work after we have broken it on a >>supposedly supported version. > > But there's no suggestion of that. > The ports tree would take care of itself. > > The comment about fixing makefiles refered to the concern about things > outside of base/ports. >From your comment, I now understand that we aren't having the same conversation :) Please answer me these to check we're on the same page: Are we planning to replace /usr/bin/make with bmake in the near future? If yes, what changes are we going to make to the ports tree to ensure that -CURRENT can still use it? Chris From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 19:11:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 442A9387; Sun, 28 Oct 2012 19:11:54 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og107.obsmtp.com (exprod7og107.obsmtp.com [64.18.2.167]) by mx1.freebsd.org (Postfix) with ESMTP id 9AE948FC0A; Sun, 28 Oct 2012 19:11:52 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob107.postini.com ([64.18.6.12]) with SMTP ID DSNKUI2DcRV6wkt+HimBJwB4h6ujsU9wHua7@postini.com; Sun, 28 Oct 2012 12:11:53 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Sun, 28 Oct 2012 12:11:45 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q9SJBjh59089; Sun, 28 Oct 2012 12:11:45 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id C10F058094; Sun, 28 Oct 2012 12:11:44 -0700 (PDT) To: Chris Rees Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program In-Reply-To: References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <127FA63D-8EEE-4616-AE1E-C39469DDCC6A@xcllnt.net> <20121025211522.GA32636@dragon.NUXI.org> <3F52B7C9-A7B7-4E0E-87D0-1E67FE5D0BA7@xcllnt.net> <20121025221244.GG3808@ithaqua.etoilebsd.net> <20121026181152.GC44331@dragon.NUXI.org> <20121026204910.E1FFA58094@chaos.jnpr.net> <20121026233225.54FB858094@chaos.jnpr.net> <20 Comments: In-reply-to: Chris Rees message dated "Sun, 28 Oct 2012 14:06:41 -0000." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Sun, 28 Oct 2012 12:11:44 -0700 Message-ID: <20121028191144.C10F058094@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 19:11:54 -0000 On Sun, 28 Oct 2012 14:06:41 +0000, Chris Rees writes: >Are we planning to replace /usr/bin/make with bmake in the near future? That was what I heard, but any such move is dependent on dealing with ports. The ~sjg/ports2bmake.tar.gz on freefall is the plan I came up with after the above "requirement" was introduced at last BSDCan. >If yes, what changes are we going to make to the ports tree to ensure >that -CURRENT can still use it? If you mean -current (aka head); the plan is to convert ports to bmake syntax wrt to the 2 conflicting modifiers. At my last test there are just under 300 makefiles in ports that use the old modifiers. Now for < head (ie. /usr/bin/make is an old version), the above ports tree detects that bmake is not being used, and invokes a shell script (bmake-sh) to do what was asked. That script will look for bmake and if necessary build/install it. To do that, it creates a temp copy of Mk/*.mk converted back to the old syntax so that the old make can build and install bmake, and then the old system is on par with -current. That's what I meant by "ports will take care of itself". The main gap btw in the above, is if a user who does not have privs to install bmake, is the only person trying to do something with ports. The above plan needs to be approved by portmgr, and obviouslty a test run of building all ports is needed (possibly more than one). Does that help? --sjg From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 19:20:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 83C67A00; Sun, 28 Oct 2012 19:20:36 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp3.one.com (csmtp3.one.com [91.198.169.23]) by mx1.freebsd.org (Postfix) with ESMTP id 3599A8FC14; Sun, 28 Oct 2012 19:20:35 +0000 (UTC) Received: from [192.168.1.47] (unknown [176.222.238.90]) by csmtp3.one.com (Postfix) with ESMTPA id 6261E2416164; Sun, 28 Oct 2012 19:20:33 +0000 (UTC) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: How about finally replacing GNATS? From: Erik Cederstrand In-Reply-To: <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> Date: Sun, 28 Oct 2012 20:20:35 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <1852E6A9-9866-4E97-9DD4-7AFA8A5FBD45@cederstrand.dk> References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <8639118y6q.fsf@ds4.des.no> <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1499) Cc: Julien Laffaye , Eitan Adler , giffunip@tutopia.com, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 19:20:36 -0000 Den 28/10/2012 kl. 03.56 skrev Erik Cederstrand : >=20 > Attached is a proof-of-concept script that takes a send-pr email and = pushes it into Bugzilla. Sorry, the script was stripped from the email. Here's a link: = https://github.com/ecederstrand/send-pr Erik= From owner-freebsd-arch@FreeBSD.ORG Sun Oct 28 20:01:02 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C9F63E39; Sun, 28 Oct 2012 20:01:02 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 142228FC1C; Sun, 28 Oct 2012 20:01:01 +0000 (UTC) Received: by mail-bk0-f54.google.com with SMTP id jf20so1779850bkc.13 for ; Sun, 28 Oct 2012 13:01:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=V+pGt6YVdmerQPUdA7zfO3wCdQBie6i31EIidHe0/yI=; b=T6pI3TKTZGZBA+I9NnB86OIIQ65KvVHrG8yvf4PceNBphnMZi05DEe0pMl1/NZeqZH PBfb0uKSXQ+CvJilKHGFXZJ7qYUyLqyEp+0IKsKjp9ajAQ+c/VzFewPD6w0YdrrLPOex i/2zwKovxe8rSp0IZr8nFf9c1mubCWc8ncrurAMTu0buLccOa4ppDtBNBQVG81yyXdBg GBfLzMLD4xvMzYfIlFRgpL43Tj2MOFhQmZtDnBsSB8OZB7PR+sH2DgaWxSZihTHiBotR MEp2uOasYO6JJ6dTQFxyewXjepWuR5W7oRx3pawLAZ5CEACi0AcJjtHDQLrdG/yiPGIR tKcw== Received: by 10.204.128.201 with SMTP id l9mr8512554bks.66.1351454460876; Sun, 28 Oct 2012 13:01:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.50.197 with HTTP; Sun, 28 Oct 2012 13:00:30 -0700 (PDT) In-Reply-To: <20121028191144.C10F058094@chaos.jnpr.net> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <127FA63D-8EEE-4616-AE1E-C39469DDCC6A@xcllnt.net> <20121025211522.GA32636@dragon.NUXI.org> <3F52B7C9-A7B7-4E0E-87D0-1E67FE5D0BA7@xcllnt.net> <20121025221244.GG3808@ithaqua.etoilebsd.net> <20121026181152.GC44331@dragon.NUXI.org> <20121026204910.E1FFA58094@chaos.jnpr.net> <20121026233225.54FB858094@chaos.jnpr.net> <20121028191144.C10F058094@chaos.jnpr.net> From: Chris Rees Date: Sun, 28 Oct 2012 20:00:30 +0000 Message-ID: Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program To: "Simon J. Gerraty" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Oct 2012 20:01:03 -0000 On 28 October 2012 19:11, Simon J. Gerraty wrote: > > On Sun, 28 Oct 2012 14:06:41 +0000, Chris Rees writes: >>Are we planning to replace /usr/bin/make with bmake in the near future? > > That was what I heard, but any such move is dependent on dealing with > ports. The ~sjg/ports2bmake.tar.gz on freefall is the plan I came up > with after the above "requirement" was introduced at last BSDCan. > >>If yes, what changes are we going to make to the ports tree to ensure >>that -CURRENT can still use it? > > If you mean -current (aka head); the plan is to convert ports to bmake > syntax wrt to the 2 conflicting modifiers. At my last test there are > just under 300 makefiles in ports that use the old modifiers. > > Now for < head (ie. /usr/bin/make is an old version), the above ports > tree detects that bmake is not being used, and invokes a shell script > (bmake-sh) to do what was asked. > > That script will look for bmake and if necessary build/install it. > To do that, it creates a temp copy of Mk/*.mk converted back to the old > syntax so that the old make can build and install bmake, and then the > old system is on par with -current. > > That's what I meant by "ports will take care of itself". > The main gap btw in the above, is if a user who does not have privs to > install bmake, is the only person trying to do something with ports. > > The above plan needs to be approved by portmgr, and obviouslty a test > run of building all ports is needed (possibly more than one). > > Does that help? Certainly, thank you. I didn't find it clear when inspecting the tarball (obviously I hadn't read the README clearly enough). I'm going to have to echo John's non-obvious comment however, and I think it would be very helpful to have a clear public writeup, perhaps Q&A style to allay any other such fears. Chris From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 03:36:15 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B72A1C30 for ; Mon, 29 Oct 2012 03:36:15 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 720DC8FC15 for ; Mon, 29 Oct 2012 03:36:15 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so4129699pbb.13 for ; Sun, 28 Oct 2012 20:36:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=nDPKEcRofpInRRgfyPz8XLtMUY73hTPJbGbjS0XscFQ=; b=lSuATHdY3q2mkA7v86kxYrnuQeyi/wv/oLRZ2Cu5OXpBGLvs1k/RVr2k31KidG7pEE d6Y1fF/COAgA5Lw++fTnE2wL9aRNsIlHycPJbuSMxm085V7zq3I5oxosk/7PtW+keptY vVTyuNdOxLd5nMogUac9dZ/XdB6zIEuiUbbwI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=nDPKEcRofpInRRgfyPz8XLtMUY73hTPJbGbjS0XscFQ=; b=F2luEJ3dPxN9oPTz8xgeubyFCFKB+Iyf3nqrNyskIK7NbP5EmRNPObBNpG62nhJbbk vf4ox6HjTsrX+awWLyouHEWaAH1wEc7ldadU8RZgkKpNs+FIgM3k1UXc5En+AJ0QFtwM fuSGpCXQUgqWmJsIizVjt5YKQSk1mFwROdpV1cZ/i3I4ekP24o1SBR9A+Q9LoAKcmdqE S2z1KCvGvBZmWMAL6j4JSwm48/E3cm4i4Q5CrRlktmX7Y7NF8yDNupyemF0SrV1mdqm4 xef7WjXifzjxBW/QbNhNMHRaE8H3FLR7xuqqxFr/QBcYT7fGeOOvAVgNyZLoAucMRWyr FfCA== Received: by 10.68.212.6 with SMTP id ng6mr88193469pbc.57.1351481774750; Sun, 28 Oct 2012 20:36:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.161.163 with HTTP; Sun, 28 Oct 2012 20:35:44 -0700 (PDT) In-Reply-To: <1852E6A9-9866-4E97-9DD4-7AFA8A5FBD45@cederstrand.dk> References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <8639118y6q.fsf@ds4.des.no> <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> <1852E6A9-9866-4E97-9DD4-7AFA8A5FBD45@cederstrand.dk> From: Eitan Adler Date: Sun, 28 Oct 2012 23:35:44 -0400 Message-ID: Subject: Re: How about finally replacing GNATS? To: Erik Cederstrand , bugmeister@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmM7KGmKDDRSbE7funxbA2GEsTNbthPdr5SBmdvVvvIREZUmudRhX2hD0SBtvEZRTt/FF1K Cc: Julien Laffaye , =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= , giffunip@tutopia.com, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 03:36:15 -0000 On 28 October 2012 15:20, Erik Cederstrand wrote: > Den 28/10/2012 kl. 03.56 skrev Erik Cederstrand : >> >> Attached is a proof-of-concept script that takes a send-pr email and pushes it into Bugzilla. > > Sorry, the script was stripped from the email. Here's a link: https://github.com/ecederstrand/send-pr This is awesome and I've added it to my bookmarks to remember to review later. The next step is to make the converter from GNATS -> Bugzilla work incrementally in order to allow running a Read-Only Bugzilla with a GNATS master. The goal is to do a staged conversion. -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 03:43:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9F73BD78 for ; Mon, 29 Oct 2012 03:43:23 +0000 (UTC) (envelope-from moonlightakkiy@yahoo.ca) Received: from nm29-vm6.bullet.mail.gq1.yahoo.com (nm29-vm6.bullet.mail.gq1.yahoo.com [98.136.216.181]) by mx1.freebsd.org (Postfix) with ESMTP id 53A6A8FC0A for ; Mon, 29 Oct 2012 03:43:23 +0000 (UTC) Received: from [98.137.12.188] by nm29.bullet.mail.gq1.yahoo.com with NNFMP; 29 Oct 2012 03:43:16 -0000 Received: from [208.71.42.208] by tm9.bullet.mail.gq1.yahoo.com with NNFMP; 29 Oct 2012 03:43:16 -0000 Received: from [127.0.0.1] by smtp219.mail.gq1.yahoo.com with NNFMP; 29 Oct 2012 03:43:16 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.ca; s=s1024; t=1351482196; bh=ifpcIiW55wLwBqyTqcAYkIvvIRYtPZAF7e12E/Tof6M=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Received:MIME-Version:Received:Received:Date:Message-ID:Subject:From:To:Cc:Content-Type; b=w5kIi6s1FskAfPsFGJJNLH/fhxtmC+/enk/qABEmeZtymoQUvczLexvjAZhpNEfnrDD7VpC6Nzm7rQKgyuuZg5Rmp1+yv3VVEHo10QeUGqRvKQkp5h6D9LUjPOPDyN4UNC2rUR1rFZXJ3dCdvAOJrWTeoD5QVkWS/BM9rTy7jqo= X-Yahoo-Newman-Id: 538426.35644.bm@smtp219.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: aNQbV10VM1nE8rtzqacKdq3yhdzyPmDpIUNLtROR3ElaVvH vYfKiAfctFBAfZcdrqXbdrzGT89gGtw9amossiuBPyd7DqrVmyHP4r1jNdBM apORDQuCzQFOHXTyxQ6mqkFH_aXPiyin5H3If7o0wdVXhuPBMS5exSdqMvc4 9QThc1Mwgrf0gQ62c1sPAYKVVL1FOhBhOCBR3oz2H4qkbPvpPv0d7HmY.DI1 xnws2FUaTI.fM6Dsu8UaTPi1MRSf2t.hJ82R.GJDHPB..H1LHZ23Hjmo0Vkm wsZ6C_drna8jjUIGkQHuEhEUJ5eKrYzZYuBqXOzwWOrBzaBnJMmO0ZGOv2BQ grPhM8kyssQyZynxAyBCJF8VxLOxnTRC8h.B_jq.8bSpZtfIZnseh38cEJOw 1wfXF3RcIB.gCbMqIGrJ2ZKxVgjihP1QUclYWuWT1tUtWiKGtZrVhdfpqCd. Zw0v9V_9FJ7LVJ4rzJe5LWMOpHzAZDkgklULDI6G9IbtxAboHXgLHO_RAm6Q HNtVhj11HE5Jw X-Yahoo-SMTP: Xr6qjFWswBAEmd20sAvB4Q3keqXvXsIH9TjJ Received: from mail-vb0-f54.google.com (moonlightakkiy@209.85.212.54 with plain) by smtp219.mail.gq1.yahoo.com with SMTP; 28 Oct 2012 20:43:16 -0700 PDT Received: by mail-vb0-f54.google.com with SMTP id l1so1074477vba.13 for ; Sun, 28 Oct 2012 20:43:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.33.229 with SMTP id u5mr256765vei.0.1351482195306; Sun, 28 Oct 2012 20:43:15 -0700 (PDT) Received: by 10.58.182.72 with HTTP; Sun, 28 Oct 2012 20:43:15 -0700 (PDT) Date: Sun, 28 Oct 2012 21:43:15 -0600 Message-ID: Subject: Re: request for help: 'fixing' the 802.11 TX path From: PseudoCylon To: Adrian Chadd , freebsd-wireless@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 03:43:23 -0000 > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 27 Oct 2012 16:18:11 -0700 > From: Adrian Chadd > To: freebsd-wireless@freebsd.org > Cc: FreeBSD Net , freebsd-arch@freebsd.org > Subject: request for help: 'fixing' the 802.11 TX path > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi all, > > I'd like to try and sort out the last remaining niggles in the 802.11 > code - this email is focusing on the TX path. > > The TX path has a few problems: > > * there's a normal versus raw TX path (for raw frames, management > frames, etc) - the raw path doesn't necessarily queue frames into the > raw TX queue, so it's a kind of "side path" to the driver. This causes > frame ordering problems with things like sequence number allocation. > * there's limited locking in the TX path, primarily because you can't > _really_ fine grain lock the TX path. Since you have multiple TX > thread contexts all sending into the same driver queue and that queue > has to share incrementing sequence number, aggregate status and CCMP > encryption replay counters, you _have_ to either use some long-held > mutexes to enforce this _or_ throw all sending into a TX thread and > run that. > * raw TX requires some extra state to be glued (the bpf_params info); > I'd like to glue that into mbufs as an option, so the driver can use > those instead of interpreting their own 802.11 header contents. > > And the one I'd like to discuss here: > > * Fragment transmission is totally broken and causes mbufs to be just > leaked. The problem with sending fragments (at least for the ath > drivr) is the packet duration calcluation requires the _next_ fragment > to be available. > > Now, this is a design hold-over from the previous, pre-vap scheme. The > driver netif would be handed a raw mbuf frame, which it would then > pass through the net80211 encapsulation code and that would > potentially generate 802.11 fragments. The rest of the driver TX path > would then see that it was handed a fragment list and TX those. > Fragments were chained together using m_nextpkt, like a normal mbuf > list. > > This doesn't happen any longer. The net80211 vap gets the raw frame, > which then sends that to the driver netif after doing the vap and > 802.11 state / encapsulation work. But since all the wifi drivers use > if_start() right now, m_nextpkt gets blanked on both encap and decap.. > and thus things leak. > > Now I can't really see a way around this, without doing dirty hacks > with mbuf attributes to link the fragments together. The only clean > way I can see is to force all wifi drivers to use if_transmit(), and > then have if_transmit() interpret a chain of frames as "the fragment > list." Cannot we just add custom hand off function to ieee80211_start()? ieee80211_start() { encap; IF_LOCK(parent->if_snd); do_seqnum_stuff; /* We can also queue mgt packets to the same queue */ for (m0 = m; m0->m_nextpkt != NULL; m0 = m0->m_nextpkt); ifq_tail = m0; IF_UNLOCK(); taskqueue_enqueue(taskqueue, parent->if_new_tx_function); /* or wake(); */ } Because of IF_LOCK, queued packets and seq num should be in order. Then, the driver only need to dequeue and Tx. The drivers can tell if the packet is a fragmented packet or a standalone packet from ieee80211 header or we can add a new member to bpf_params. AK From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 03:53:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 63DC4A45; Mon, 29 Oct 2012 03:53:11 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 269D18FC0C; Mon, 29 Oct 2012 03:53:11 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so4137872pbb.13 for ; Sun, 28 Oct 2012 20:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=kgzlGGvA9hUShv3MWfZgPHASiV2bZeqcuGNaPvjI8Sc=; b=Gg0cBSRV2WzQLy2ZCzpKW2z7QGMlMXyJTYbJkshe+W5RqDiN9gBdtMyNUT4xO857+M ecRuxL8csB0bo8vnGQ8xuJx4+fmsaJJ4H56QYrDzaSovh791XKLCgKrhXD+zxPfuh0PX blHIdHOQZF2CuUHMk3Fg2409T4G/2dpN/4Sa6HnWfAQgyeT3Ne4cwvNAJWoTlucOtN/4 npTY75BsObQydTbxC435k0YOxFwzfTPmxeyBlKhcWrz91Q8ntnwaYLXO0QYxSU7UjU4t LOubpxujwjEaww7eJAQqxggn6eLH3Qx/5gznO3iAbF4r6ul+E21plOgP30+rVcj93g60 PsMQ== MIME-Version: 1.0 Received: by 10.66.74.65 with SMTP id r1mr79954465pav.75.1351482790930; Sun, 28 Oct 2012 20:53:10 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.146.233 with HTTP; Sun, 28 Oct 2012 20:53:10 -0700 (PDT) In-Reply-To: References: Date: Sun, 28 Oct 2012 20:53:10 -0700 X-Google-Sender-Auth: ZEzdqL8YeED0RsLvQe31kI_Bp4I Message-ID: Subject: Re: request for help: 'fixing' the 802.11 TX path From: Adrian Chadd To: PseudoCylon Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , freebsd-wireless@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 03:53:11 -0000 On 28 October 2012 20:43, PseudoCylon wrote: > Cannot we just add custom hand off function to ieee80211_start()? Yes. That's the general idea. But what I don't want to do is have it just wake up the driver TX taskqueue - well, unless we have to. That means we'll have two context switches for each frame being transmitted and that as a concept just sucks. See my (very recent) email to -wireless - I broke TCP throughput quite substantially by moving ath(4) TX into the taskqueue. I thought the problem was _just_ going to be how overlapping, direct dispatch TX could be preempted by the RX tasklet and TX completion, but there's obviously more going on. I'm going to experiment some more with it before I go down this path. I don't want to do the linux thing of "just grab a txrx spinlock in mac80211 before calling the driver", serialising everything that way. That feels plain ugly. But then, a lot of our network drivers do much the same thing - they grab big, long-held locks and they either implement their own lockless queue in front of that lock (eg, what the intel/chelsio code does) or they just ignore it entirely and leave it up to the scheduler to wake up threads once the contending lock is unlocked. I have all kinds of concerns about the behaviour of if_bridge and other if_transmit() enabled friends when sending to one of these badly locked drivers, however that's going to have to wait. Well, maybe I'll poke the network people at meetbsd this weekend and see what they think. Adrian From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 09:47:20 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2992A443 for ; Mon, 29 Oct 2012 09:47:20 +0000 (UTC) (envelope-from oppermann@networx.ch) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 791B78FC17 for ; Mon, 29 Oct 2012 09:47:18 +0000 (UTC) Received: (qmail 95762 invoked from network); 29 Oct 2012 11:24:15 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 29 Oct 2012 11:24:15 -0000 Message-ID: <508E50A3.1030008@networx.ch> Date: Mon, 29 Oct 2012 10:47:15 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: request for help: 'fixing' the 802.11 TX path References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , PseudoCylon , freebsd-wireless@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 09:47:20 -0000 On 29.10.2012 04:53, Adrian Chadd wrote: > On 28 October 2012 20:43, PseudoCylon wrote: > >> Cannot we just add custom hand off function to ieee80211_start()? > > Yes. That's the general idea. But what I don't want to do is have it > just wake up the driver TX taskqueue - well, unless we have to. > > That means we'll have two context switches for each frame being > transmitted and that as a concept just sucks. > > See my (very recent) email to -wireless - I broke TCP throughput quite > substantially by moving ath(4) TX into the taskqueue. I thought the > problem was _just_ going to be how overlapping, direct dispatch TX > could be preempted by the RX tasklet and TX completion, but there's > obviously more going on. I can't believe that TCP is getting broken by just introducing some additional delay in the TX path. That can't add more than 300ms, can it? There must be something else going on. Most likely either severe packet loss (the m_nextpkt leak you mentioned earlier) or severe packet re-ordering. So don't rule out the TX taskqueue concept quite yet. -- Andre From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 17:38:14 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 29E0FC39 for ; Mon, 29 Oct 2012 17:38:14 +0000 (UTC) (envelope-from moonlightakkiy@yahoo.ca) Received: from nm16-vm3.bullet.mail.ne1.yahoo.com (nm16-vm3.bullet.mail.ne1.yahoo.com [98.138.91.146]) by mx1.freebsd.org (Postfix) with ESMTP id C814B8FC12 for ; Mon, 29 Oct 2012 17:38:13 +0000 (UTC) Received: from [98.138.90.55] by nm16.bullet.mail.ne1.yahoo.com with NNFMP; 29 Oct 2012 17:38:07 -0000 Received: from [98.138.226.31] by tm8.bullet.mail.ne1.yahoo.com with NNFMP; 29 Oct 2012 17:38:07 -0000 Received: from [127.0.0.1] by smtp202.mail.ne1.yahoo.com with NNFMP; 29 Oct 2012 17:38:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.ca; s=s1024; t=1351532287; bh=LM9cqYzZn55ZOV7/b3pl/8d+o4VoguqXOvn7kj4rCAI=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Received:MIME-Version:Received:Received:In-Reply-To:References:Date:Message-ID:Subject:From:To:Cc:Content-Type; b=TF7Xu/QO6X63h9Z4ySNSgR1dCzfqxezXHmMf+foO2o3vSVEU0/mh9WdpdpZw/L2Ak5SjxpiDXSYigYb1c2IkXQ96yFFqtx1KOJj8AzE0sFcBn7VY059sPko691wV6nasXGAAMhAcjzQAx/T/1MlcvHjMTF4JM8/QYRB1NFRIaI0= X-Yahoo-Newman-Id: 653555.32931.bm@smtp202.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 8zBtlvMVM1ny58J_AnS7cI97H1NarIBQwK6o4iTzjokaJSP ZaJHGjXBt7xXF3K7X02xwvmi38VLgOtcr3NpG7wK6ymgcgKa5I2rnwyTiVAc nUdS4xBa_eWVAcv7n8_x0z9HrBcLBRBIHJvcw8iF3KD8ssduhCQSkcKlfZH9 GFnsiOWw3_KbiGtBJHnmzJ8rtcVl9HXiJduBvPNFmrc15rM_x.cw5vDreuj. vBH_FMV5Tu7s38lLnx2tLYvbQv9_x2vX1sKVBhdcap_rJaR2u7XtdLwNGGgE OiYrd1kXcbre8hUbpKQ8oCAnrfp_QXqxCCTwa5XMUpfTwAXO5zG.6Q_rIL06 Tg8yTOTP3U2WD_OqUfHjzjK_CxlPsp_FWrW5Pg3Hev62ArLZtihtWqbZ17fE nWMZRoCNLlC5FKukcUxCskSmlNFytNAdPvq.qhXayPHHALNuke69iaWLTYeU W4YoGY2oU1TUikTVmnFA- X-Yahoo-SMTP: Xr6qjFWswBAEmd20sAvB4Q3keqXvXsIH9TjJ Received: from mail-vb0-f54.google.com (moonlightakkiy@209.85.212.54 with plain) by smtp202.mail.ne1.yahoo.com with SMTP; 29 Oct 2012 10:38:07 -0700 PDT Received: by mail-vb0-f54.google.com with SMTP id l1so1981853vba.13 for ; Mon, 29 Oct 2012 10:38:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.66.36 with SMTP id c4mr39497747vdt.6.1351532286228; Mon, 29 Oct 2012 10:38:06 -0700 (PDT) Received: by 10.58.182.72 with HTTP; Mon, 29 Oct 2012 10:38:06 -0700 (PDT) In-Reply-To: <508E50A3.1030008@networx.ch> References: <508E50A3.1030008@networx.ch> Date: Mon, 29 Oct 2012 11:38:06 -0600 Message-ID: Subject: Re: request for help: 'fixing' the 802.11 TX path From: PseudoCylon To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , Adrian Chadd , freebsd-wireless@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 17:38:14 -0000 On Mon, Oct 29, 2012 at 3:47 AM, Andre Oppermann wrote: > On 29.10.2012 04:53, Adrian Chadd wrote: >> >> On 28 October 2012 20:43, PseudoCylon wrote: >> >>> Cannot we just add custom hand off function to ieee80211_start()? >> >> >> Yes. That's the general idea. But what I don't want to do is have it >> just wake up the driver TX taskqueue - well, unless we have to. >> >> That means we'll have two context switches for each frame being >> transmitted and that as a concept just sucks. Plan B vap->iv_ifp->if_transmit = ieee80211_transmit; ieee80211_transmit() /* new */ { /* Alternatively, we can make a list of param and attach mbuf to it. */ HANDOFF(parent->if_snd, m); ieee80211_new_tx(vap, m); } /* enque packet, but keep working on the same mbuf */ ieee80211_new_tx(vap, m) { encap; if (fragment) insert_fragmented_packet_to_queue; /* don't forget about a fragmented packet */ for (; m->m_nextpkt != NULL; m = m->m_nextpkt) parent->if_new_tx(vap, m); } /* keep working on the same mbuf */ driver_new_tx(vap, m) { do_descriptor_stuff; m->m_flags |= ALL_SET; /* * If, for instance, processing of queue #5 packet finished before queue #1, * #5 packet will stay in queue until all of preceding packets get processed. */ if (parent->if_sc->sc_tx == NOT_RUNNING && ifq_head->m_flags & ALL_SET) driver_pass2hw(parent); } /* finally, process mbuf from the head of queue */ driver_pass2hw() { /* only one thread to dequeue */ if (atomic_compset(&sc->sc_tx, NOT_RUNNING, RUNNING) == 0) return; for (;;) { DEQUEUE(ifq, m); if (!(m->m_flags & ALL_SET)) { PREPEND(); break; } /* * want to do seq stuff somewhere in ieee80211_*(), * but I guess this is the only place could do. */ do_seqnum_stuff; /* simply put a packet onto dma-able memory area */ pass2hw; } sc->sc_tx = NOT_RUNNING; } No additional context switching, no long-held lock, but first queue first tx. AK >> >> See my (very recent) email to -wireless - I broke TCP throughput quite >> substantially by moving ath(4) TX into the taskqueue. I thought the >> problem was _just_ going to be how overlapping, direct dispatch TX >> could be preempted by the RX tasklet and TX completion, but there's >> obviously more going on. > > > I can't believe that TCP is getting broken by just introducing some > additional delay in the TX path. That can't add more than 300ms, > can it? There must be something else going on. Most likely either > severe packet loss (the m_nextpkt leak you mentioned earlier) or > severe packet re-ordering. > > So don't rule out the TX taskqueue concept quite yet. > > -- > Andre > From owner-freebsd-arch@FreeBSD.ORG Mon Oct 29 17:52:56 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 92F6038B; Mon, 29 Oct 2012 17:52:56 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 301E68FC16; Mon, 29 Oct 2012 17:52:55 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wc20so6299209obb.13 for ; Mon, 29 Oct 2012 10:52:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=H6UIu1viDS7/ULG8ZXlWQYGpRnJK290lXLYbkhIhJJw=; b=pgxAVRoi8mMswUhDDitkFBC8cJMJgCBkiTooDANVOc68XzFNvclBgSdSeOpxno76zH oMnjV43biBi1XyydD9HefofuufDcrM6hV5M69tym0HwjxTl5bvTx9w6IeSSgQ+r7uRrH 2gcZ13vYa9hXeFn9ZfE041NNed15redRKFncMFYTeZZM4/gfr0St4Awb6IL8HnUcRzWa ECJe0YYHdwJGt8mAfgHYGBAspHKMc6FS109gj/4bC4HHjE6qVRKGjK3FQbdb1k8H/NxN 44OWerHwkz7xsWOPmI7jvPcCJiJrGsz8ksP90Zmac/hHdgRT8GG56xX/2FkXIZ5StCEL hqig== MIME-Version: 1.0 Received: by 10.182.69.50 with SMTP id b18mr25300096obu.75.1351533175595; Mon, 29 Oct 2012 10:52:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.76.27.65 with HTTP; Mon, 29 Oct 2012 10:52:55 -0700 (PDT) In-Reply-To: References: <508E50A3.1030008@networx.ch> Date: Mon, 29 Oct 2012 10:52:55 -0700 X-Google-Sender-Auth: XduIE-ahB3kfGazWrAdsRRvCLU0 Message-ID: Subject: Re: request for help: 'fixing' the 802.11 TX path From: Adrian Chadd To: PseudoCylon Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , freebsd-wireless@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 17:52:56 -0000 The problem with this is that fragments need to be handed as an entire list to the driver. ath(4) looks ahead to the next fragment, calculates the rate, and then adds that to the NAV of the previous frame. So yeah what I'm thinking about for now is something like the following: * say that wifi drivers have to move to if_transmit(); * "abuse" if_transmit() semantics to assume a m_nextpkt chain of mbufs is a fragment list for a single frame; * for now, figure out why the hell the ath(4) TX taskqueue setup is providing crappy throughput and fix _that_; * then migrate the net80211 TX to a taskqueue; * .. and then just undo the ath(4) TX taskqueue since now TX is serialised by if_transmit() inside the net80211 TX taskqueue. There are other things to do, mostly surrounding fixing up the power save queue handling to repopulate the queue packet-by-packet, rather than doing if_snd gymnastics. Once that's done, we can look at ways to implement serialisation besides just using a taskqueue - eg, using the running/not-running flag idea you've suggested. The big problem here is that once TX completion occurs, drivers kick off new TX. Now, if all TX is triggered by a frame TX, the TX will stall until the next frame is attempted. What we'd have to do is tell the drivers how to 'kick' the net80211 stack TX again. Adrian From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 08:29:29 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CF38A56F for ; Tue, 30 Oct 2012 08:29:29 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id 3BB928FC15 for ; Tue, 30 Oct 2012 08:29:28 +0000 (UTC) Received: from server.rulingia.com (c220-239-241-202.belrs5.nsw.optusnet.com.au [220.239.241.202]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q9U8T8Uu016936 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 30 Oct 2012 19:29:09 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q9U8T30O018059 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 30 Oct 2012 19:29:03 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q9U8T0TK018058; Tue, 30 Oct 2012 19:29:00 +1100 (EST) (envelope-from peter) Date: Tue, 30 Oct 2012 19:29:00 +1100 From: Peter Jeremy To: Peter Wemm Subject: Re: How about finally replacing GNATS? Message-ID: <20121030082900.GH3309@server.rulingia.com> References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bp/iNruPH9dso1Pn" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Eitan Adler , Dag-Erling Sm?rgrav , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 08:29:29 -0000 --bp/iNruPH9dso1Pn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Oct-27 22:25:54 -0700, Peter Wemm wrote: >On the exceedingly remote chance that somebody genuinely can't/won't >access an HTTP link in 2012+ there's always the option of having them >email one of the lists. Just because a sysadmin has HTTP access doesn't mean that the machine with the issue has access. There needs to be a simple way to collect the PR data on one system and transfer it to another machine where it can be submitted. With send-pr, you can copy the PR file (or just copy it off the console). --=20 Peter Jeremy --bp/iNruPH9dso1Pn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCPj8wACgkQ/opHv/APuIc0SQCePlmCwZN3YQgLTtZDEyBIrqae NasAoIniTZ9IkeEti7hwP+0egcjjlg6B =HIwR -----END PGP SIGNATURE----- --bp/iNruPH9dso1Pn-- From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 10:06:43 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9638492A for ; Tue, 30 Oct 2012 10:06:43 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 4C2F68FC1C for ; Tue, 30 Oct 2012 10:06:42 +0000 (UTC) Received: from [192.168.1.18] (unknown [217.157.7.221]) by csmtp2.one.com (Postfix) with ESMTPA id 79EF13043024; Tue, 30 Oct 2012 10:06:36 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: How about finally replacing GNATS? From: Erik Cederstrand In-Reply-To: <20121030082900.GH3309@server.rulingia.com> Date: Tue, 30 Oct 2012 11:06:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> To: Peter Jeremy X-Mailer: Apple Mail (2.1499) Cc: Eitan Adler , freebsd-arch@freebsd.org, Dag-Erling Sm?rgrav X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 10:06:43 -0000 Den 30/10/2012 kl. 09.29 skrev Peter Jeremy : > On 2012-Oct-27 22:25:54 -0700, Peter Wemm wrote: >> On the exceedingly remote chance that somebody genuinely can't/won't >> access an HTTP link in 2012+ there's always the option of having them >> email one of the lists. >=20 > Just because a sysadmin has HTTP access doesn't mean that the machine > with the issue has access. There needs to be a simple way to collect > the PR data on one system and transfer it to another machine where it > can be submitted. With send-pr, you can copy the PR file (or just > copy it off the console). Apart from running uname -a, what does send-pr offer on a machine = without HTTP access compared to just filling out the send-pr form on a = machine that does have HTTP access? Erik= From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 15:51:29 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3C6E413 for ; Tue, 30 Oct 2012 15:51:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9A0BC8FC12 for ; Tue, 30 Oct 2012 15:51:29 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so305348pbb.13 for ; Tue, 30 Oct 2012 08:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=a//XJxMR8YqtyHh4h+kzC44ymCmbhTpkygyUZQqLsso=; b=z8hXGKxYLLU/axYXvr1icBrWNmR6ku/fhV3j0EbL97UpEr27ZJ6W/rwrkDKYr6Yezj NBMN+LA/7sNEU4gK4UVVA6/Z6AsflYnkn48zkH2GvJcaihbbPrC4zUeIRVDpdh0aXlvD 4kAGsRlIpoD8szeB8laHhyoTb+KTC62yHFRwVT0hqZB/LcBmeU6RrvJBa1hoDvEpaFny 8oVb/u0CEPWMIl/l13vHY8r575h14edBDzugjzYTb6GaJRbiN3MHS2klIi7gIjNEj7SM EXupvuvs60+2DwfgbPpx9ypWx38s576xq46jQJaqXnc8kmqYb758us0gY1f0k7qF40vK wz1g== MIME-Version: 1.0 Received: by 10.66.80.133 with SMTP id r5mr94118022pax.24.1351612288310; Tue, 30 Oct 2012 08:51:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.146.233 with HTTP; Tue, 30 Oct 2012 08:51:28 -0700 (PDT) In-Reply-To: <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> Date: Tue, 30 Oct 2012 08:51:28 -0700 X-Google-Sender-Auth: igPclLKFwjrkW5lz4rZ-d_xcSQM Message-ID: Subject: Re: How about finally replacing GNATS? From: Adrian Chadd To: Erik Cederstrand Content-Type: text/plain; charset=ISO-8859-1 Cc: Eitan Adler , Dag-Erling Sm?rgrav , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 15:51:29 -0000 On 30 October 2012 03:06, Erik Cederstrand wrote: > Apart from running uname -a, what does send-pr offer on a machine without HTTP access compared to just filling out the send-pr form on a machine that does have HTTP access? Being able to do it offline, and then having it send when you plug into the network :-) You realise SMTP, like NNTP/usenet, is really good at "partially offline" modes? Adrian From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 16:45:16 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AA8BB725; Tue, 30 Oct 2012 16:45:16 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp3.one.com (csmtp3.one.com [91.198.169.23]) by mx1.freebsd.org (Postfix) with ESMTP id 5727A8FC14; Tue, 30 Oct 2012 16:45:15 +0000 (UTC) Received: from [192.168.1.50] (unknown [176.222.238.90]) by csmtp3.one.com (Postfix) with ESMTPA id 57588240E4EF; Tue, 30 Oct 2012 16:45:14 +0000 (UTC) References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <038C004A-6177-472D-A328-90830E04F640@cederstrand.dk> X-Mailer: iPhone Mail (10A403) From: Erik Cederstrand Subject: Re: How about finally replacing GNATS? Date: Tue, 30 Oct 2012 17:45:12 +0100 To: Adrian Chadd Cc: Eitan Adler , Dag-Erling Sm?rgrav , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 16:45:16 -0000 Den 30/10/2012 kl. 16.51 skrev Adrian Chadd : > On 30 October 2012 03:06, Erik Cederstrand wrote: >=20 >> Apart from running uname -a, what does send-pr offer on a machine without= HTTP access compared to just filling out the send-pr form on a machine that= does have HTTP access? >=20 > Being able to do it offline, and then having it send when you plug > into the network :-) >=20 > You realise SMTP, like NNTP/usenet, is really good at "partially offline" m= odes? Yes. I thought we were talking about machines with permanently firewalled ou= tgoing port 80. I'm not suggesting send-pr should go. I even provided a script to keep using= it with Bugzilla. I'm just curious what's so great about it in 2012. Offlin= e is a good argument, and HTTP sucks at offline. Erik= From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 17:28:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4D375C3C for ; Tue, 30 Oct 2012 17:28:11 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id BB86F8FC0A for ; Tue, 30 Oct 2012 17:28:10 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id 16so368206wgi.31 for ; Tue, 30 Oct 2012 10:28:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3MJgF/2mfkxOMrzoA6or3Z7QVStdv8d+Y0MRc+FBTug=; b=bzC8FaIK5RBy47wI7coQSk98CXtJ9IWf/t41HIohGZZnUhHtwmnhj0nij2CUoBbsk1 AAYBni0Izo9sthZE7lKbLPMMSsWyiiGLaJXPgvtNbtvaQGF+KaocrnzUzkCVjpSUUAEq v8TQlndGlOEiHenSBwA4kWkBJm82hxPvft8FA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=3MJgF/2mfkxOMrzoA6or3Z7QVStdv8d+Y0MRc+FBTug=; b=eflgGpPoiQPe12hW3+hoXg0qewi+gLAPYG9je/Q5gkkFFlRKERpAPKKpgygg0T5ISy FwTVFvc0rewTECPk2vo3SE5bvTv/C74lGKSOMSoS1Y9oLJ6tHdsgtPVsQ77/GkdJli0D Ldzysol/jq7Ui5gIZAEC5EEXxfUIz23ViXgNksf0f2MwfVZhSQ/W1a9I5KdTYveJ3vOT QTIll4k1bgUVAVxT65ZlYiO769ou2HS74JptBVHMn1BdO5XjrKWO8xJD6xm74EOT6Rw3 gBD0ttQUyR6zSNjw7pID90fexcYYyiejP7HRVz/Krj94awqNCRMS8JTBSUt5ULoCmo8T 5FDw== MIME-Version: 1.0 Received: by 10.216.74.13 with SMTP id w13mr19074151wed.101.1351618089571; Tue, 30 Oct 2012 10:28:09 -0700 (PDT) Received: by 10.194.62.130 with HTTP; Tue, 30 Oct 2012 10:28:09 -0700 (PDT) In-Reply-To: <20121030082900.GH3309@server.rulingia.com> References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> Date: Tue, 30 Oct 2012 10:28:09 -0700 Message-ID: Subject: Re: How about finally replacing GNATS? From: Peter Wemm To: Peter Jeremy Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmiDmMjiRVSPUQUMBeaQ9mEaUdtb1C17ymOmnxiCcqaqcGsqGvUDLKgr9AH5TfmG9wmLeYy Cc: Eitan Adler , Dag-Erling Sm?rgrav , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 17:28:11 -0000 On Tue, Oct 30, 2012 at 1:29 AM, Peter Jeremy wrote: > On 2012-Oct-27 22:25:54 -0700, Peter Wemm wrote: >>On the exceedingly remote chance that somebody genuinely can't/won't >>access an HTTP link in 2012+ there's always the option of having them >>email one of the lists. > > Just because a sysadmin has HTTP access doesn't mean that the machine > with the issue has access. There needs to be a simple way to collect > the PR data on one system and transfer it to another machine where it > can be submitted. With send-pr, you can copy the PR file (or just > copy it off the console). You can also email the same data to yourself to cut/paste into a bugzilla ticket. This really shouldn't be a stumbling block. Really. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 17:39:20 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 17382E6A for ; Tue, 30 Oct 2012 17:39:20 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8713E8FC0A for ; Tue, 30 Oct 2012 17:39:19 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id x43so309219wey.13 for ; Tue, 30 Oct 2012 10:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=II6+cosgfpbnqFXsyNgMyW706MC8QrLVtPCQmxSiGp4=; b=jh5geZ2YsAkZ5kgkxXiuQnIoAd03c50g68HrYJgZSWx957cRo+xTBr0MngrRD/cSSb jMWUfIviyJoy8+oyPKHceb4HTWsLOa8VRaxkRjIbGVNPhsLKOigK+OyZOMV4QBi3Y3Vd eacDtkcC0o0bUSdUzgzlqFDjug3Mor4zXbn88= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=II6+cosgfpbnqFXsyNgMyW706MC8QrLVtPCQmxSiGp4=; b=oqKAoiLQDyzpLj7uptkssm+Ur4wyBjiDF4KUwhYp3OfRdsfP0WoNGs4YLGkJ1vbSKN zCtS40CJCIOxSY3yNRA2HcizmoCzw8farIrSJ0HMH5k5pjW24JwlaoinLz6Dtn6M1FKT 0ZuKAsp+Qb6+EAbAlcavdnsKzmt2dbpYhqeEywLeLHYc+BBE1UQ8y/D9IC2ONtDM6/yY LJ0Qe/mLqhDM3uqEofpM92iuA314kW1VyhqfmuxsPuTl17koDsw78MCc/uT+PB2cblXB xAMs+qWrqzCZXaUGp8uoe/rt8ZGA6+Uv+bMHH98H+8TSgZ9mVH6TZuu5GQ9sE/cU01gZ rmHQ== MIME-Version: 1.0 Received: by 10.216.202.97 with SMTP id c75mr14300304weo.211.1351618758372; Tue, 30 Oct 2012 10:39:18 -0700 (PDT) Received: by 10.194.62.130 with HTTP; Tue, 30 Oct 2012 10:39:18 -0700 (PDT) In-Reply-To: <038C004A-6177-472D-A328-90830E04F640@cederstrand.dk> References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> <038C004A-6177-472D-A328-90830E04F640@cederstrand.dk> Date: Tue, 30 Oct 2012 10:39:18 -0700 Message-ID: Subject: Re: How about finally replacing GNATS? From: Peter Wemm To: Erik Cederstrand Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmB9da5KVnP7URGhrxbKRUd+V4xVsEIQWccgiyMEZe2rfAlcSlty4VPuLH+6msSJzxIVPY7 Cc: Eitan Adler , Adrian Chadd , "freebsd-arch@freebsd.org" , Dag-Erling Sm?rgrav X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 17:39:20 -0000 On Tue, Oct 30, 2012 at 9:45 AM, Erik Cederstrand wrote: [..] > I'm not suggesting send-pr should go. I even provided a script to keep using it with Bugzilla. I'm just curious what's so great about it in 2012. Offline is a good argument, and HTTP sucks at offline. But I am. Once we switch to bugzilla or (whatever the flavor of the day is), we should remove send-pr from top-of-tree on all the active branches and replace it with clear instructions for how to get to bugzilla (and how to email yourself information to cut/paste into a bugzilla ticket if we're really worried that people don't know how to do that). Yes, we'll be stuck with the old installations of send-pr, which we'll have to handle. Mark my words.. once the majority of submissions come in via bugzilla, the old send-pr submissions will feel like a thorn in our sides and we'll be wishing they'll die out faster. Incidentally, did you know you can still subscribe/unsubscribe to freebsd.org mailing lists using the majordomo interface? I think. And no, I'm not going to reverse engineer that one. That is dying with this round of updates. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Tue Oct 30 17:46:59 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8D8B2143 for ; Tue, 30 Oct 2012 17:46:59 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id ED9758FC14 for ; Tue, 30 Oct 2012 17:46:58 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id b5so545616lbd.13 for ; Tue, 30 Oct 2012 10:46:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=AncKlqkW+q7Ir/Qd0g8sVxyra8pXpuz2dZOGi1DHVCw=; b=elvp+iQNP5secrOAxO31UQn0uYgklSa0kHF8wUy2SDRW5zWzJ54C/Gxo6lctujn+4D Qx3UiHXNdxvCeA0A7quJ74RGXuKi7JOUztqb4ybCJYjFLBuo0WhR7p+m49e5lxlfvVpi hoHdFiuRaKTENmvh8AiqRLjWFyMz0jzL65aXw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=AncKlqkW+q7Ir/Qd0g8sVxyra8pXpuz2dZOGi1DHVCw=; b=Al9DFUiiYu+tE7LK54BHB+v1WlA1cn1HFpsAak3C0uf5HJgpzE6/kPVipzIRV+pfQ6 LbgzqdU7+djum/A63OewNvgTHzK1yWiZcQZWDqEshLReXhK8dd41AVPjW/yQKuRKnyLK Qdws5kAPC6pf+X6UL4qmp3a1/JBxqIg5S88no3HWvDafGuzaMy/+frab+XwEkQDIzhoa QWVHX5pLdgHnq2qtKfXOtb2GYnJeKgV3v4ImFhapYHsaYgbo4YsF4Hxs5KCwAjbOgQ25 F6xxOVfX9PKS4raJeVAYWl34sWmUUzJQK5Id1XqiLWYX1+3UEQj0hvCQ/o5em5qjJXaA mzFg== Received: by 10.112.48.74 with SMTP id j10mr13672878lbn.94.1351619217446; Tue, 30 Oct 2012 10:46:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.162.71 with HTTP; Tue, 30 Oct 2012 10:46:27 -0700 (PDT) In-Reply-To: References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> <038C004A-6177-472D-A328-90830E04F640@cederstrand.dk> From: Eitan Adler Date: Tue, 30 Oct 2012 13:46:27 -0400 Message-ID: Subject: Re: How about finally replacing GNATS? To: Peter Wemm Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQk6u+BDQEYCvz7Ic5fDboKni8Zb7KYSQhu3cI0H93Yoerpo8rC3ZFHjRY3FinG9JjIpwaoJ Cc: Dag-Erling Sm?rgrav , Adrian Chadd , Erik Cederstrand , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 17:46:59 -0000 On 30 October 2012 13:39, Peter Wemm wrote: > On Tue, Oct 30, 2012 at 9:45 AM, Erik Cederstrand wrote: > [..] >> I'm not suggesting send-pr should go. I even provided a script to keep using it with Bugzilla. I'm just curious what's so great about it in 2012. Offline is a good argument, and HTTP sucks at offline. > > But I am. ... There are good reasons to keep send-pr and good reasons to get rid of it. We are not yet at the stage where it even makes sense to discuss this. -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Wed Oct 31 00:16:30 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 990F35E5; Wed, 31 Oct 2012 00:16:30 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5150D8FC08; Wed, 31 Oct 2012 00:16:30 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so594919pad.13 for ; Tue, 30 Oct 2012 17:16:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:x-mailer:from:subject:date :to; bh=zJLdEgXT4QrcCDlPAiW4R0+NdXO1swV5sEKGnDwPjRY=; b=lPDmJ6Uq215AwiCdTZf/IH8sFMvnscjO3otNz21fC90SSqnAQdcTz7EnY/DI30zf+m W1MZrN62CpHzXHQfEQj8oPWWkI9efUwXbAfyAwuR3JcdnoeZ16KRGssl1bcsnpiFhccb fs3XKvydIUP4BIf6sQgeOGZspdTSlqiYpLHzYVkXhawaIdgwVjp6VQDQgxcY8Gmi3rzw E5hXT25zHkgmecck22OY+L/XHgC8fY3ERkRfrLfb0cToL0xx7GDHjZH6gYJBRufCgSmi vjyIYPerR+UeYitawaMvUSLY2UEAlwA/Ya1HHqgkVR9kQ6mX5YIxJeIdEFJS573Sse4h tykQ== Received: by 10.66.88.197 with SMTP id bi5mr96444267pab.58.1351642589829; Tue, 30 Oct 2012 17:16:29 -0700 (PDT) Received: from [10.142.190.62] (mobile-166-147-081-075.mycingular.net. [166.147.81.75]) by mx.google.com with ESMTPS id wf8sm1286287pbc.65.2012.10.30.17.16.26 (version=SSLv3 cipher=OTHER); Tue, 30 Oct 2012 17:16:28 -0700 (PDT) References: <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <20121025193115.GD40357@in-addr.com> <20121030082900.GH3309@server.rulingia.com> <35AB4227-2053-4678-BE71-576B01E48FE1@cederstrand.dk> <038C004A-6177-472D-A328-90830E04F640@cederstrand.dk> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (10A403) From: Garrett Cooper Subject: Re: How about finally replacing GNATS? Date: Tue, 30 Oct 2012 17:16:26 -0700 To: Eitan Adler Cc: Dag-Erling Sm?rgrav , Adrian Chadd , Erik Cederstrand , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 00:16:30 -0000 On Oct 30, 2012, at 10:46 AM, Eitan Adler wrote: > On 30 October 2012 13:39, Peter Wemm wrote: >> On Tue, Oct 30, 2012 at 9:45 AM, Erik Cederstrand w= rote: >> [..] >>> I'm not suggesting send-pr should go. I even provided a script to keep u= sing it with Bugzilla. I'm just curious what's so great about it in 2012. Of= fline is a good argument, and HTTP sucks at offline. >>=20 >> But I am. >=20 > ... >=20 >=20 > There are good reasons to keep send-pr and good reasons to get rid of > it. We are not yet at the stage where it even makes sense to discuss > it. +1 about it being too soon to make official policy decisions. I think creating a catch-all to convert the PR request to a bugzilla bug as o= thers have suggested would be an awesome idea, so that way the only thing th= at changes is the submission link. If nothing more, send-pr can continue to function for ushering bugs into bug= zilla. -Garrett= From owner-freebsd-arch@FreeBSD.ORG Wed Oct 31 06:42:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1E6026BA; Wed, 31 Oct 2012 06:42:36 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D08408FC14; Wed, 31 Oct 2012 06:42:35 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so806301pbb.13 for ; Tue, 30 Oct 2012 23:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=dt5l5MclRcSnB1EKVJHhUdxaDN4rTy4U2FGmJPwu4bU=; b=Ai2MBcLt5LtPhc34L/2bxr7AdJb8p+wmKho/cRGL+/hkp19EC703QNiK1xxJMGRhNK xSA5A5lo4cQUnl/MBo0nF8CpdHtowhB4+fkcLL2SnmLWw2l2pOwmrdjLIunUTwYcnnKC YAR4QdTbCdjTn1KbZXHdayaF6DPee9jgrQnIq3WFuytAV0fC+58M+KoOJ811mkCaeLbs pXrrRj86Ki/2YtP2o88VV6rsbk83PDuySSTe/yhTS2IB4y1R/wD4dOIw03OHR3whasaD mLkVt+oraRPWZGNVIIyVJ/ldPyOuRDxhDwK8JFlQf6X2OaLlmfVXFnIhlXictDTJROQf Gl0Q== MIME-Version: 1.0 Received: by 10.66.73.230 with SMTP id o6mr99340786pav.45.1351665755328; Tue, 30 Oct 2012 23:42:35 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.124.130 with HTTP; Tue, 30 Oct 2012 23:42:35 -0700 (PDT) In-Reply-To: References: <508E50A3.1030008@networx.ch> Date: Tue, 30 Oct 2012 23:42:35 -0700 X-Google-Sender-Auth: RHA8otk43RPWLQ4-E6groky9SQo Message-ID: Subject: Re: request for help: 'fixing' the 802.11 TX path From: Adrian Chadd To: PseudoCylon Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , freebsd-wireless@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 06:42:36 -0000 So I cheaped out for now and just wrapped the ath TX path in a new TX only lock. I'm open to other suggestions about how to make the "queue everything through a taskqueue" work, but unfortunately it seems that I'm defeated by the inner workings of the network stack locking and how that plays with the scheduler. I even tried experimenting with a second taskqueue just for TX but it still suffered from much reduced performance. The annoying thing? Changing the eventtimer to enable periodic+idletick improved performance as well as flipping machdep.idle=spin. I thought that stuff was fixed but alas. So now this is done, I may create a per-VAP TX lock in net80211 in order to serialise raw and normal net80211 TX; that will fix a lot of the the serialisation and state issues that creep up. Then it's off to if_transmit() land. Thanks everyone, Adrian From owner-freebsd-arch@FreeBSD.ORG Wed Oct 31 08:15:00 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C5580A88; Wed, 31 Oct 2012 08:15:00 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp3.one.com (csmtp3.one.com [91.198.169.23]) by mx1.freebsd.org (Postfix) with ESMTP id 781E78FC14; Wed, 31 Oct 2012 08:15:00 +0000 (UTC) Received: from [192.168.1.18] (unknown [217.157.7.221]) by csmtp3.one.com (Postfix) with ESMTPA id 2EC18241124E; Wed, 31 Oct 2012 08:14:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: How about finally replacing GNATS? From: Erik Cederstrand In-Reply-To: Date: Wed, 31 Oct 2012 09:14:55 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <8639118y6q.fsf@ds4.des.no> <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> <1852E6A9-9866-4E97-9DD4-7AFA8A5FBD45@cederstrand.dk> To: Eitan Adler X-Mailer: Apple Mail (2.1499) Cc: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , freebsd-arch@freebsd.org, giffunip@tutopia.com, bugmeister@freebsd.org, Julien Laffaye X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 08:15:00 -0000 Den 29/10/2012 kl. 04.35 skrev Eitan Adler : > On 28 October 2012 15:20, Erik Cederstrand = wrote: >> Den 28/10/2012 kl. 03.56 skrev Erik Cederstrand = : >>>=20 >>> Attached is a proof-of-concept script that takes a send-pr email and = pushes it into Bugzilla. >>=20 >> Sorry, the script was stripped from the email. Here's a link: = https://github.com/ecederstrand/send-pr What's the official way to mark a commit as referencing a PR? I see PR: nnnnnnn PR: category/nnnnnnnn in the commit template. I assume it is implicit that the PR is closed = with the commit. Are there any official ways to reference a PR within commit log = messages, such that a PR mentioned in a commit log should be updated = with a comment referencing said commit log? Then there's replies by email to a bug reports, which are sent to = bug-followup@freebsd.org. I assume they should just be attached as a = comment to the Bugzilla bug mentioned in the subject. Anything else? Erik= From owner-freebsd-arch@FreeBSD.ORG Wed Oct 31 08:38:57 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2881D1F9; Wed, 31 Oct 2012 08:38:57 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C78938FC0A; Wed, 31 Oct 2012 08:38:56 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 286B36E0C; Wed, 31 Oct 2012 09:38:55 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id D249F9992; Wed, 31 Oct 2012 09:38:54 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Erik Cederstrand Subject: Re: How about finally replacing GNATS? References: <1314468512.87152.YahooMailClassic@web113511.mail.gq1.yahoo.com> <4E593932.8060303@FreeBSD.org> <86sj928vks.fsf@ds4.des.no> <4FB39F66-0434-43F1-9705-3C81CDF2A25F@cederstrand.dk> <86ehkm8s5v.fsf@ds4.des.no> <0EF669C7-0ED4-4BD4-9170-C49A7CA91603@cederstrand.dk> <8639118y6q.fsf@ds4.des.no> <6838B465-F010-4C86-AF3C-CB96B3782F28@cederstrand.dk> <1852E6A9-9866-4E97-9DD4-7AFA8A5FBD45@cederstrand.dk> Date: Wed, 31 Oct 2012 09:38:54 +0100 In-Reply-To: (Erik Cederstrand's message of "Wed, 31 Oct 2012 09:14:55 +0100") Message-ID: <86mwz3jash.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , freebsd-arch@freebsd.org, giffunip@tutopia.com, bugmeister@freebsd.org, Julien Laffaye X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 08:38:57 -0000 Erik Cederstrand writes: > What's the official way to mark a commit as referencing a PR? I see > > PR: nnnnnnn > PR: category/nnnnnnnn > > in the commit template. Both work. > I assume it is implicit that the PR is closed with the commit. Not at all. > Are there any official ways to reference a PR within commit log > messages, You listed them above. > Then there's replies by email to a bug reports, which are sent to > bug-followup@freebsd.org. I assume they should just be attached as a > comment to the Bugzilla bug mentioned in the subject. Yes. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Nov 1 00:50:54 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7BB6350E; Thu, 1 Nov 2012 00:50:54 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0BABD8FC08; Thu, 1 Nov 2012 00:50:53 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id k19so1767454qcs.13 for ; Wed, 31 Oct 2012 17:50:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qpC1FseMkdGcMxIuQ2Ru4MizxDkIAzrWTdh4te3w94s=; b=ZtZKi48ili9vPvXAnEoBDnLWM8/dfM9nIss0Bn8c+TDuG7a7AKgSXVisPpUW6uj5HA mxe2WlpnNQoK0TlTMjREDMWjnBdtA0qcc0vpeAwkn2jzDHuvWeESrDP6nbvP46l5r98w 7rSzEwYdGFnvrkTCtRniCZQux1WWrxpxdQdIHIVz57tr3YSbRm5rFL8YraMOGN+U4reC JRPUOPtCTe4Uza5/JqluSpckXJ8deFzPtIj80vXWLFb+3/KwMQS4FuqbGphcQoL7FLRa nU9TruEj3Oj8hLvgqM3yrS7zlWtj04kuM8fNa5PonKTIt1fTik1wecIV4KNmtS5r/OeV 4MMg== MIME-Version: 1.0 Received: by 10.224.198.136 with SMTP id eo8mr2857397qab.63.1351731052681; Wed, 31 Oct 2012 17:50:52 -0700 (PDT) Received: by 10.49.35.37 with HTTP; Wed, 31 Oct 2012 17:50:52 -0700 (PDT) In-Reply-To: References: <201210250918.00602.jhb@freebsd.org> <5089690A.8070503@networx.ch> <201210251732.31631.jhb@freebsd.org> Date: Wed, 31 Oct 2012 17:50:52 -0700 Message-ID: Subject: Re: CACHE_LINE_SIZE on x86 From: Jim Harris To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2012 00:50:54 -0000 On Thu, Oct 25, 2012 at 2:40 PM, Jim Harris wrote: > On Thu, Oct 25, 2012 at 2:32 PM, John Baldwin wrote: > > > > It would be good to know though if there are performance benefits from > > avoiding sharing across paired lines in this manner. Even if it has > > its own MOESI state, there might still be negative effects from sharing > > the pair. > > On 2S, I do see further benefits by using 128 byte padding instead of > 64. On 1S, I see no difference. I've been meaning to turn off > prefetching on my system to see if it has any effect in the 2S case - > I can give that a shot tomorrow. > > So tomorrow turned into next week, but I have some data finally. I've updated to HEAD from today, including all of the mtx_padalign changes. I tested 64 v. 128 byte alignment on 2S amd64 (SNB Xeon). My BIOS also has a knob to disable the adjacent line prefetching (MLC spatial prefetcher), so I ran both 64b and 128b against this specific prefetcher both enabled and disabled. MLC prefetcher enabled: 3-6% performance improvement, 1-5% decrease in CPU utilization by using 128b padding instead of 64b. MLC prefetcher disabled: performance and CPU utilization differences are in the noise - anywhere from -0.2% to +0.5%. The performanc here matches extremely closely (within 1%) with 128b padding and the MLC prefetcher enabled. I think it's safe to say that the 128b pad/alignment is worth keeping for multi-socket x86, and is most certainly due to the MLC spatial prefetcher. I still see no measurable differences with 64b v. 128b padding on 1S, but that's only testing with my benchmark. Thanks, -Jim From owner-freebsd-arch@FreeBSD.ORG Thu Nov 1 14:45:00 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E91A0AA8 for ; Thu, 1 Nov 2012 14:45:00 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 2DDCC8FC19 for ; Thu, 1 Nov 2012 14:45:00 +0000 (UTC) Received: (qmail 79469 invoked from network); 1 Nov 2012 16:21:21 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 1 Nov 2012 16:21:21 -0000 Message-ID: <50928AE5.4010107@freebsd.org> Date: Thu, 01 Nov 2012 15:44:53 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Jim Harris Subject: Re: CACHE_LINE_SIZE on x86 References: <201210250918.00602.jhb@freebsd.org> <5089690A.8070503@networx.ch> <201210251732.31631.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2012 14:45:01 -0000 On 01.11.2012 01:50, Jim Harris wrote: > > > On Thu, Oct 25, 2012 at 2:40 PM, Jim Harris > wrote: > > On Thu, Oct 25, 2012 at 2:32 PM, John Baldwin > wrote: > > > > It would be good to know though if there are performance benefits from > > avoiding sharing across paired lines in this manner. Even if it has > > its own MOESI state, there might still be negative effects from sharing > > the pair. > > On 2S, I do see further benefits by using 128 byte padding instead of > 64. On 1S, I see no difference. I've been meaning to turn off > prefetching on my system to see if it has any effect in the 2S case - > I can give that a shot tomorrow. > > > So tomorrow turned into next week, but I have some data finally. > > I've updated to HEAD from today, including all of the mtx_padalign changes. I tested 64 v. 128 byte > alignment on 2S amd64 (SNB Xeon). My BIOS also has a knob to disable the adjacent line prefetching > (MLC spatial prefetcher), so I ran both 64b and 128b against this specific prefetcher both enabled > and disabled. > > MLC prefetcher enabled: 3-6% performance improvement, 1-5% decrease in CPU utilization by using 128b > padding instead of 64b. Just to be sure. The numbers you show are just for the one location you've converted to the new padded mutex and a particular test case? -- Andre > MLC prefetcher disabled: performance and CPU utilization differences are in the noise - anywhere > from -0.2% to +0.5%. The performanc here matches extremely closely (within 1%) with 128b padding > and the MLC prefetcher enabled. > > I think it's safe to say that the 128b pad/alignment is worth keeping for multi-socket x86, and is > most certainly due to the MLC spatial prefetcher. > > I still see no measurable differences with 64b v. 128b padding on 1S, but that's only testing with > my benchmark. > > Thanks, > > -Jim > From owner-freebsd-arch@FreeBSD.ORG Thu Nov 1 18:36:15 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 92240DD6; Thu, 1 Nov 2012 18:36:15 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 016578FC08; Thu, 1 Nov 2012 18:36:14 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so3891795vba.13 for ; Thu, 01 Nov 2012 11:36:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=fA2mkPhEPFClBZ6DyzvaxAVbf3nQlVPJqv0WaBiGHyA=; b=Fv1nHpSabKszh/JIDwrr4g4Kvp+e1/NV8M1exm0iRfu1Z+DkfoECe8udQ5r0WOUxj0 0JW1zw3bCfF3JVJnxKxdcKz1AzxUeuCCwcA8ZPl6UnBrwvHwVLqPiojxILT3n7SpJ1T9 Q0Zv36w31Zz7HNwTsled+K0M4fG+BBigVfs5kJEwiv/qIjzuua7aCVkTgi2yVErpsbCl xP+k1wxjXdVzAnyjL4G+Gta8T+2QW4Dz5j4Cb5jqIlVSMY1OuDDIxfQiTjjmtcHjx2DK rxK9SH5k/ppFiGr5g4noG0UlKZ+Xm/+E3Ed9dSmnyg6DY0H7NN1j0Ne9bIAfygmkR2Nz ozoA== MIME-Version: 1.0 Received: by 10.220.226.67 with SMTP id iv3mr23829385vcb.57.1351794974098; Thu, 01 Nov 2012 11:36:14 -0700 (PDT) Received: by 10.58.225.2 with HTTP; Thu, 1 Nov 2012 11:36:13 -0700 (PDT) In-Reply-To: <50928AE5.4010107@freebsd.org> References: <201210250918.00602.jhb@freebsd.org> <5089690A.8070503@networx.ch> <201210251732.31631.jhb@freebsd.org> <50928AE5.4010107@freebsd.org> Date: Thu, 1 Nov 2012 11:36:13 -0700 Message-ID: Subject: Re: CACHE_LINE_SIZE on x86 From: Jim Harris To: Andre Oppermann Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Attilio Rao , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2012 18:36:15 -0000 On Thu, Nov 1, 2012 at 7:44 AM, Andre Oppermann wrote: > On 01.11.2012 01:50, Jim Harris wrote: > >> >> >> On Thu, Oct 25, 2012 at 2:40 PM, Jim Harris > jim.harris@gmail.com>> wrote: >> >> >> On Thu, Oct 25, 2012 at 2:32 PM, John Baldwin > jhb@freebsd.org>> wrote: >> > >> > It would be good to know though if there are performance benefits >> from >> > avoiding sharing across paired lines in this manner. Even if it >> has >> > its own MOESI state, there might still be negative effects from >> sharing >> > the pair. >> >> On 2S, I do see further benefits by using 128 byte padding instead of >> 64. On 1S, I see no difference. I've been meaning to turn off >> prefetching on my system to see if it has any effect in the 2S case - >> I can give that a shot tomorrow. >> >> >> So tomorrow turned into next week, but I have some data finally. >> >> I've updated to HEAD from today, including all of the mtx_padalign >> changes. I tested 64 v. 128 byte >> alignment on 2S amd64 (SNB Xeon). My BIOS also has a knob to disable the >> adjacent line prefetching >> (MLC spatial prefetcher), so I ran both 64b and 128b against this >> specific prefetcher both enabled >> and disabled. >> >> MLC prefetcher enabled: 3-6% performance improvement, 1-5% decrease in >> CPU utilization by using 128b >> padding instead of 64b. >> > > Just to be sure. The numbers you show are just for the one location you've > converted to the new padded mutex and a particular test case? > There are two locations actually - the struct tdq lock in the ULE scheduler, and the callout_cpu lock in kern_timeout.c. And yes, I've been only running a custom benchmark I developed here to help to try to uncover some of these areas of spinlock contention. It was originally used for NVMe driver performance testing, but has been helpful in uncovering some other issues outside of the NVMe driver itself (such as these contended spinlocks). It spawns a large number of kernel threads, each of which submits an I/O and then sleeps until it is woken by the interrupt thread when the I/O completes. It stresses the scheduler and also callout since I start and stop a timer for each I/O. I think the only thing proves is that there is benefit to having x86 CACHE_LINE_SIZE still set to 128. Thanks, -Jim From owner-freebsd-arch@FreeBSD.ORG Thu Nov 1 22:31:51 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14D8C5B3 for ; Thu, 1 Nov 2012 22:31:51 +0000 (UTC) (envelope-from 2ohohoho@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D95978FC14 for ; Thu, 1 Nov 2012 22:31:50 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so2182477pbb.13 for ; Thu, 01 Nov 2012 15:31:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=iOETCc7ItI/uJ9BFJ+/3eZOUKzaRAG0HG4nsl1i5rVY=; b=G6QDURacaeKo6Rbso7wQnlZKKc0YCAI/32/l7UsxE2ADpEso9qzao2vd0yXFsS0xN8 O/21nWsU6VrIUfg9LOyJq/mbgJexYA5+SiLmTE8RLypY3Bl2QuuHHP+JH23P9f5JEZ8h dbf3J+ipx+sWNGn2+UQpRFHBdLS0p9GOl+b8ZowHMWIIjYkd+TRFhiayv4C9rMzAztt3 /q4+NpJ4YGYJQ8glqBhk2g8BQ/kXp2LnqRoY6YrXdUWCUM+78rErBWTneBe8Red1qKWK awQXzeulmh0YPXk1YHAZ97wfRdALOcXwS+klclF3jbeE27CUYZbQt0uWFCnXxt12vCPq ei9g== MIME-Version: 1.0 Received: by 10.68.200.231 with SMTP id jv7mr125895010pbc.140.1351809109671; Thu, 01 Nov 2012 15:31:49 -0700 (PDT) Received: by 10.66.88.195 with HTTP; Thu, 1 Nov 2012 15:31:49 -0700 (PDT) Date: Fri, 2 Nov 2012 02:31:49 +0400 Message-ID: Subject: Request for help: how do teach module building about kernel options? From: li cheng <2ohohoho@gmail.com> To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2012 22:31:51 -0000 Hi, I need a bit of a hand with this. I'd like to be able to make the wlan and ath modules aware of kernel configuration options. For example, a kernel configuration with IEEE80211_SUPPORT_TDMA won't build a wlan module that'll run successfully, as wlan/Makefile doesn't know to suck in ieee80211_tdma.c . I could just wrap the whole file up in an #ifdef, but I'd like to try and instead only build / link that object in if it's needed. Similarly, the ath module currently builds everything, regardless of what options are currently enabled in the kernel configuration file. So it'll always build ar5210, ar5211, ar5212, ar5416, ar9001, ar9002 support, along with ath_rate_sample. Instead, I'd like to be able to specify which HAL objects to link in, much like how you can do this with "device ath_rfX" for the RF backends and "device ath_arX" for the chipset support. For the integrated SoC stuff, it'd be nice to only build a HAL which supports the relevant hardware, rather than having to suck it _all_ in. So, what kinds of evil ways can people dream up to achieve this? :) --=20 =F3 =D5=D7=C1=D6=C5=CE=C9=C5=CD =ED=C9=C8=C1=C9=CC =F0=CF=C9=D3=CB =D4=CF=D7=C1=D2=CF=D7 From owner-freebsd-arch@FreeBSD.ORG Fri Nov 2 02:47:44 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 41D75FC3 for ; Fri, 2 Nov 2012 02:47:44 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A3ECA8FC14 for ; Fri, 2 Nov 2012 02:47:43 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so2904315lag.13 for ; Thu, 01 Nov 2012 19:47:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=mOm91W9G4DSsRIQrzkymMMV0W8bTk+latT8BrtL8dC0=; b=rIcE306+GQrjFj0YpXjqWO/vXvpcbsBW7CzHZsxvXyT5LyuEu4miZfT9qp6dvYyxf9 OekhPDEca0TqU8/xHnWfhG/+wu+boDaOvZcwl3OK4NHv8qhnpaw3uq4CTgwpvGlLBoGL fLHlhr+KNqlfEN8NPS/WnHh6WbhR6pD5Y3+aw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=mOm91W9G4DSsRIQrzkymMMV0W8bTk+latT8BrtL8dC0=; b=fEDKvNLgiVBd0puNJZ3sF1IDied783FdrImLRdCSs9+9sdQxSivbwCv/salTZBVdna Em3HodpTApR29SZ6j/WhA59K+BPAugiyiLHlRDkq2Pbgq4Lw7RA+VSNs6KRoljAEs/Gg DF5Biou4f3jj/7CXPXnJs4J4dzg4dSqukPAPhgy/gLc+4gjclXbBGv8SPz8ilGmQn0pa XwXhAW5PTAw9g++xXJmHwEwsPft4CeBkJalTtwJMbiwnpTew4wggtks8nWgfNJk/k+52 176QltaiO5lUcMowH+ibZlvP6RMI094ax4hQDry2qAnHPjY6do7IuLYpY164NPG4yan6 smHg== Received: by 10.152.109.145 with SMTP id hs17mr354715lab.5.1351824462328; Thu, 01 Nov 2012 19:47:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.112.162.71 with HTTP; Thu, 1 Nov 2012 19:47:12 -0700 (PDT) In-Reply-To: References: <201210250918.00602.jhb@freebsd.org> <5089690A.8070503@networx.ch> <201210251732.31631.jhb@freebsd.org> <50928AE5.4010107@freebsd.org> From: Eitan Adler Date: Thu, 1 Nov 2012 22:47:12 -0400 Message-ID: Subject: Re: CACHE_LINE_SIZE on x86 To: Jim Harris Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmQSppRcdpqHrYQ9p8x06ACzib8DLmgoKXaG4unQbsloMQFPnHCtRNy9TwwXM7lfGxLszUA Cc: Attilio Rao , Andre Oppermann , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Nov 2012 02:47:44 -0000 On 1 November 2012 14:36, Jim Harris wrote: > On Thu, Nov 1, 2012 at 7:44 AM, Andre Oppermann wrote: > >> On 01.11.2012 01:50, Jim Harris wrote: >> >>> >>> >>> On Thu, Oct 25, 2012 at 2:40 PM, Jim Harris >> jim.harris@gmail.com>> wrote: >>> >>> >>> On Thu, Oct 25, 2012 at 2:32 PM, John Baldwin >> jhb@freebsd.org>> wrote: >>> > >>> > It would be good to know though if there are performance benefits >>> from >>> > avoiding sharing across paired lines in this manner. Even if it >>> has >>> > its own MOESI state, there might still be negative effects from >>> sharing >>> > the pair. >>> >>> On 2S, I do see further benefits by using 128 byte padding instead of >>> 64. On 1S, I see no difference. I've been meaning to turn off >>> prefetching on my system to see if it has any effect in the 2S case - >>> I can give that a shot tomorrow. >>> >>> >>> So tomorrow turned into next week, but I have some data finally. >>> >>> I've updated to HEAD from today, including all of the mtx_padalign >>> changes. I tested 64 v. 128 byte >>> alignment on 2S amd64 (SNB Xeon). My BIOS also has a knob to disable the >>> adjacent line prefetching >>> (MLC spatial prefetcher), so I ran both 64b and 128b against this >>> specific prefetcher both enabled >>> and disabled. >>> >>> MLC prefetcher enabled: 3-6% performance improvement, 1-5% decrease in >>> CPU utilization by using 128b >>> padding instead of 64b. >>> >> >> Just to be sure. The numbers you show are just for the one location you've >> converted to the new padded mutex and a particular test case? >> > > There are two locations actually - the struct tdq lock in the ULE > scheduler, and the callout_cpu lock in kern_timeout.c. > > And yes, I've been only running a custom benchmark I developed here to help > to try to uncover some of these areas of spinlock contention. It was > originally used for NVMe driver performance testing, but has been helpful > in uncovering some other issues outside of the NVMe driver itself (such as > these contended spinlocks). It spawns a large number of kernel threads, > each of which submits an I/O and then sleeps until it is woken by the > interrupt thread when the I/O completes. It stresses the scheduler and > also callout since I start and stop a timer for each I/O. > > I think the only thing proves is that there is benefit to having x86 > CACHE_LINE_SIZE still set to 128. Does this benchmark simulate reality or does padding the locks only help on this specific benchmark? -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Fri Nov 2 14:23:52 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D37B6476 for ; Fri, 2 Nov 2012 14:23:52 +0000 (UTC) (envelope-from andrey@zonov.org) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4104E8FC0C for ; Fri, 2 Nov 2012 14:23:50 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so3330782lag.13 for ; Fri, 02 Nov 2012 07:23:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type:x-gm-message-state; bh=cBPW3l3qR5hFtqWVIfZnAMYOOfsxoAUF6fgDHAZF958=; b=KWECH0rMTXE5twXTUYV7lFzbz7O5vWWViN+YwliStGilkmupFh2I5IwFnSpyvh2uz7 Uaj3Zs9qV1CWq10Zd0NDwyki3uAP2a/HG3ne1B5UyIN6kVO/oANJVhDa5B8DaoxW/1S6 9fAOCMyLl6NguibjkwJHAUyVrIMSIomoXub/RmVmfeXxRV6MRHtvFxia5opUUnSIgmaZ 77A9A/X8nYngQHHlKm48A8TnYuyOAMGspy98Ng2vOqvhA5GoMFHboSey34ejpnJBFIEn Twy+9UdRGgtV+ymjHfuGX325swNI1I8gEZDDujSTkI7LhvVKDMwbyKLjT7GwTfqKPJdL jiLg== Received: by 10.152.104.240 with SMTP id gh16mr1785045lab.56.1351866229769; Fri, 02 Nov 2012 07:23:49 -0700 (PDT) Received: from dhcp170-82-red.yandex.net ([2a02:6b8:0:401:c4d7:2f90:5546:d62]) by mx.google.com with ESMTPS id i7sm3258047lbg.13.2012.11.02.07.23.48 (version=SSLv3 cipher=OTHER); Fri, 02 Nov 2012 07:23:49 -0700 (PDT) Sender: Andrey Zonov Message-ID: <5093D771.5040606@FreeBSD.org> Date: Fri, 02 Nov 2012 18:23:45 +0400 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:16.0) Gecko/20121026 Thunderbird/16.0.2 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Subject: unprivileged mlock(2) X-Enigmail-Version: 1.4.5 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig62197D7EF708088F7AB36448" X-Gm-Message-State: ALoCoQmM3CkObrKthVTaPtAa7yq9+X81ZkSIi7fNlasRvmvvwxYoQbzUscX3szR6EyyKYCsnVX6V X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Nov 2012 14:23:52 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig62197D7EF708088F7AB36448 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, I'm ready to commit these four patches [1]. I'm going to do this tomorrow if there won't be any objections. I want to make clear changes in login.conf about memorylocked limits. When the system is starting up, init(8) uses daemon login class to run /etc/rc. Some daemons such as amd(8) and watchdogd(8) use mlockall(2) and may deny that call because of limits, that is why I set memorylocked limit to 64Mb for daemon class. But the problem will be still there if you are using `sudo' or `su -m' to restart services. In that case your login class (and limits) would be applied and memorylocked limit may cause failure of mlockall(2) call (like vmemoryuse limit may fail malloc(3)). [1] http://people.freebsd.org/~zont/patches/mlock/ --=20 Andrey Zonov --------------enig62197D7EF708088F7AB36448 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJQk9dzAAoJEBWLemxX/CvTw3MH/jxGuPqX7sRZmaTkzNF4IgAl jSS3HvxwWxCOO+q3xHjdpfLeWLgl917fBdMLlI7a+ztCY6YRzfK6Oxdg4ntIULyb EpKmXXPSoWsK5aV1JrXFb9DktHDNgnBFDlufa5Pe7U7RixfpwrpWxQ11l8oceRL6 hR8xj/F0BH4lz1xR7/qQPkM54kx9SUvGN7QQydZQv9sUBsUom+VJWK7fdltrHXaR Va2AAVANqfjFloP3K7T04pL9IbHk4SCue0Ru+ruvtVTrF8DDmM4YN+8Ez0kiU0Cr SztmfioErIDt+OEw1WTGN4W9o7ERfHxEnPlESKJZ7DAKR5O8gzGzR+nOdTkxQEg= =YVZS -----END PGP SIGNATURE----- --------------enig62197D7EF708088F7AB36448-- From owner-freebsd-arch@FreeBSD.ORG Fri Nov 2 14:43:25 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 976CE28F for ; Fri, 2 Nov 2012 14:43:25 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id E32E58FC15 for ; Fri, 2 Nov 2012 14:43:24 +0000 (UTC) Received: (qmail 85841 invoked from network); 2 Nov 2012 16:19:34 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 2 Nov 2012 16:19:34 -0000 Message-ID: <5093DC04.2010902@freebsd.org> Date: Fri, 02 Nov 2012 15:43:16 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Andre Oppermann Subject: Re: svn commit: r242473 - user/andre/tcp_workqueue/sys/dev/ixgbe References: <201211021343.qA2DhHHr031786@svn.freebsd.org> In-Reply-To: <201211021343.qA2DhHHr031786@svn.freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: jfv@FreeBSD.org, freebsd-net@freebsd.org, yongari@FreeBSD.org, davidch@FreeBSD.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Nov 2012 14:43:25 -0000 On 02.11.2012 14:43, Andre Oppermann wrote: > Author: andre > Date: Fri Nov 2 13:43:17 2012 > New Revision: 242473 > URL: http://svn.freebsd.org/changeset/base/242473 > > Log: > Merge ixgbe_tx_ctx_setup() and ixgbe_tso_setup() together into > ixgbe_offload_setup() as they have a large overlap in packet > inspection and variables. > > Add UDP fragmentation offload functionality as well. > > Also the driver can assume that the necessary headers for the > activated offload functions are contiguously present in the > first mbuf. This assumption is not formalized yet but significantly > simplify the driver code. The upper layers of the stack adhere > to this assumption as well. We leave enough leading space in the > mbufs to prepend all registered (max_linkhdr) encapsulations. > There may be misbehaving packet transformations that have to be > fixed. Assertion functions have to be provided as well. The new formal assumptions at the stack/driver boundary will be: CSUM_IP: lower and IP header including IP options contiguous in first mbuf ip_sum is zero CSUM_TCP: lower and TCP header including TCP options contiguous in first mbuf pseudo header checksum is valid and present in th_sum CSUM_IP is implied, but should be flagged as well CSUM_UDP: lower and UDP header contiguous in first mbuf pseudo header checksum is valid and present in uh_sum CSUM_IP is implied, but should be flagged as well CSUM_SCTP: lower and SCTP header contiguous in first mbuf CSUM_IP is implied, but should be flagged as well CSUM_TSO: lower and TCP header including TCP options contiguous in first mbuf pseudo header checksum is valid and present in th_sum CSUM_IP and CSUM_TCP is implied, but should be flagged as well no TSO flag when no TCP segmentation is required (m_pkthdr.len <= mss + tcpiphdr) no IP options present As I said the standard upper layers already behave like this. The step now is to formalize and assert/enforce this. Enforcement is done by returning an error for a misbehaving packet w/o trying to fix it up. If you have any comments, please speak up. -- Andre From owner-freebsd-arch@FreeBSD.ORG Fri Nov 2 14:49:10 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1B256550; Fri, 2 Nov 2012 14:49:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2EBB58FC18; Fri, 2 Nov 2012 14:49:08 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA03816; Fri, 02 Nov 2012 16:49:07 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <5093DD46.1040701@FreeBSD.org> Date: Fri, 02 Nov 2012 16:48:38 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:16.0) Gecko/20121029 Thunderbird/16.0.2 MIME-Version: 1.0 To: Andrey Zonov Subject: Re: unprivileged mlock(2) References: <5093D771.5040606@FreeBSD.org> In-Reply-To: <5093D771.5040606@FreeBSD.org> X-Enigmail-Version: 1.4.5 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig22C2777229BA3776F749C7E8" Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Nov 2012 14:49:10 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig22C2777229BA3776F749C7E8 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable on 02/11/2012 16:23 Andrey Zonov said the following: > Hi, >=20 > I'm ready to commit these four patches [1]. I'm going to do this > tomorrow if there won't be any objections. >=20 > I want to make clear changes in login.conf about memorylocked limits. > When the system is starting up, init(8) uses daemon login class to run > /etc/rc. Some daemons such as amd(8) and watchdogd(8) use mlockall(2) > and may deny that call because of limits, that is why I set memorylocke= d > limit to 64Mb for daemon class. >=20 > But the problem will be still there if you are using `sudo' or `su -m' > to restart services. In that case your login class (and limits) would > be applied and memorylocked limit may cause failure of mlockall(2) call= > (like vmemoryuse limit may fail malloc(3)). >=20 > [1] http://people.freebsd.org/~zont/patches/mlock/ >=20 I would start with memorylocked=3D64k or 128k for the default class. I can't justify these numbers excepts that they are nice :-) and 8MB "fee= ls" like too much. Anything greater than zero would already be an improvement and typical unprivileged applications that want wired memory usually request a single= page. So several pages should be enough. Those who would need more can always bump the limit in the local config. P.S. Not sure if it would be better to set memorylocked=3Dunlimited for t= he daemon class too. I think that in the default configuration it's only super-use= r who can use that class. --=20 Andriy Gapon --------------enig22C2777229BA3776F749C7E8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQEcBAEBAgAGBQJQk91iAAoJEHSlLSemUf4vOJMH/jIJcR6EFSAJvpCT8l5uf/eE g0CpmcRy5lbXFxUYFTviHH15S4yj2yG4B22Dn3kbaMm4tTHNbfia5Rbg8nBmopJ6 MvBHd8l/nYa2hC/JowvwjeZz8BwsNjkYkR6B1RzHcxC5GFcP9M99Ax8KbVSc2Z8z TNzJVgZWQQvC0n8/HrRpR+6PZY6XEn9gUFQFCJNJu9A5WQsTIiDqqopX/skpxgzv Z3ipe4dtLTTnsBjmgeJSuxWeRQDVWJG4dF0vvR2ZlYtzmJz67OYL8RSPypeZAC4S sJ373nx7+3/Ekt6tVGTYXXGwpWqvphIsLaQXtk3FprCy/75RTi4c4bdfbbTkThc= =LphE -----END PGP SIGNATURE----- --------------enig22C2777229BA3776F749C7E8-- From owner-freebsd-arch@FreeBSD.ORG Fri Nov 2 18:20:04 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 059DD9D3 for ; Fri, 2 Nov 2012 18:20:04 +0000 (UTC) (envelope-from 30w6UUAwHDMosxu2Avqs9A7uHMJ.s42v7uur8t-q7sxv7uur8t.47w@photos-server.bounces.google.com) Received: from mail-pb0-f74.google.com (mail-pb0-f74.google.com [209.85.160.74]) by mx1.freebsd.org (Postfix) with ESMTP id D4EAC8FC17 for ; Fri, 2 Nov 2012 18:20:03 +0000 (UTC) Received: by mail-pb0-f74.google.com with SMTP id rr4so265092pbb.1 for ; Fri, 02 Nov 2012 11:20:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.75.196 with SMTP id e4mt1776361paw.15.1351880403512; Fri, 02 Nov 2012 11:20:03 -0700 (PDT) Message-ID: Date: Fri, 02 Nov 2012 18:20:03 +0000 Subject: SalesSBR chinachem shared photos with you From: SalesSBR chinachem To: freebsd-arch@freebsd.org Content-Type: multipart/mixed; boundary=f46d042f9ddc3b1de604cd872fa9 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: SalesSBR chinachem List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Nov 2012 18:20:04 -0000 --f46d042f9ddc3b1de604cd872fa9 Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Hello Sir/Mam: Happy to contact you ! We would like to introudce our company.Our company is a large scale pertrochemical enterprise with synthetic rubber as the main product. We could supply SBR 1502 and 1712. It is manufactured by SINOPEC YANGZI. The following is the SBR quotation : * SBR 1502: USD 2500.00/TON FOB China port SBR 1712: USD 2350.00/TON FOB China port * packing : 35kg ppbag * quantity: 21ton/20'container * price validity: within 3 days . * payment terms : TT in advance or LC at sight . * delivery time: within 20 days . Any other question, contact me freely ! best wishes Adelle Liang sales manage rXuzhou Yizhengyuan Chemical Technology Co., Ltd email:chemade@163.com --f46d042f9ddc3b1de604cd872fa9--