From owner-freebsd-hackers@freebsd.org Sun Feb 14 06:33:53 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E06C2AA1EF0 for ; Sun, 14 Feb 2016 06:33:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94F351C59 for ; Sun, 14 Feb 2016 06:33:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qg0-x22b.google.com with SMTP id b67so91437320qgb.1 for ; Sat, 13 Feb 2016 22:33:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2oGWc7AJdraxIlkgbnO0MunnYNA7cVvU0j038dZXXSU=; b=CzLPOCGQPKpwQZ421T6E6iccd1uprnJQKO74VrioGlhOIM2GNsnRbLceM7FbMN8uPJ 1VTO0Wi8lC0TThukZDlJZc649qm3140L9gPCo0dX8pWuXZ77BkqGBhJtx5ztn+PA3KVc G4/h+GSzxOtYdkTdRVjdFmH3VdbxCsROU4zckyNbAXZikCzc6tAK0Z6nfVHtCdQ+cRpK o26uPzhKymLqNpRZY3E36daMYIAQJvTBfF62brV58emhSDFsJqTlPD+VmjcD4HBwPQTU OuOmv442cVtjJseKrh3lqv/fFYSMj6Y3XzalZCmw4lCwr5ASulQIULBJKXV81IyX1Oyz iZNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=2oGWc7AJdraxIlkgbnO0MunnYNA7cVvU0j038dZXXSU=; b=DrXnShaRYXSQE9eUT9tCaU4/hXf0P11qQg2wpKtVWVheg2Gi+RoN2hBY3pZwfjMNIa 47HIOuTQV5rKxswx3kLotcCiMGTWb19493VEDLk6gr8Xv79DioHzcFhBfcTZUabdZ4cd g1aIOK7JtDoZ0632Q1u0707DUiuk0HjY1D+xtV0QNaa+5caiM9DH6e8F/PEqd0qsmBWK Y5KGyFBYNKGk7zpxr3LIItsDQXxuEr9RTSsHWde8G2WuJJI3r2qEICjApRy2q5nOFfTV yUdGRe9G2a/cmd29CbdIwj25D1YBbPEb6pAhcjncVv5IXvq1ZlEhDp5y0s65qDQIJHF9 qg2g== X-Gm-Message-State: AG10YOSuLt8EKLYdGYiNGO+VzNMjreDj3+ZnOTd2wJkSEBUpi+F0NWJ2mHsQxeN1mV7ECpRjWxdZo/5bNVHtGA== MIME-Version: 1.0 X-Received: by 10.140.43.180 with SMTP id e49mr12408377qga.50.1455431632730; Sat, 13 Feb 2016 22:33:52 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.30.166 with HTTP; Sat, 13 Feb 2016 22:33:52 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com> <53120EE8.1080600@bakulin.de> <5688F015.4090002@bakulin.de> <20160213035806.4403283.12124.2928@gmail.com> Date: Sat, 13 Feb 2016 23:33:52 -0700 X-Google-Sender-Auth: KSjFZ3rwBFFvi9Y9ZYRqGsWEuvU Message-ID: Subject: Re: MMC/SDIO stack under CAM From: Warner Losh To: Russell Haley Cc: Adrian Chadd , "freebsd-hackers@freebsd.org" , Alexander Motin , Ilya Bakulin , "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2016 06:33:54 -0000 On Fri, Feb 12, 2016 at 11:48 PM, Russell Haley wrote: > On Fri, Feb 12, 2016 at 8:16 PM, Adrian Chadd > wrote: > > On 12 February 2016 at 19:58, Russell Haley > wrote: > >> Hi Ilya, so does that mean I can take a linux driver for an SDIO wifi > card and build it using a reference to your library and everything should > "just work"? > > > > nope. there's a lot more to it than that. But it's a good start - > > getting the driver up and doing IO to the card is a big step. > > > > > > > > -a > > > Okay, thanks. But if I include his CAM driver and then use the patch > below for IMX6 on my Hummingboard, I would be able to test the driver? > Will a simple kldstat be enough to verify I am using his driver or is > there something more direct in dtrace? > "camcontrol devlist" would let you know for sure. Warner > Thanks, > > Russ > > >> Thanks, > >> Russ > >> > >> Sent from my BlackBerry 10 smartphone on the Koodo network. > >> Original Message > >> From: Ilya Bakulin > >> Sent: Thursday, February 11, 2016 10:43 AM > >> To: Lundberg, Johannes > >> Cc: Adrian Chadd; freebsd-hackers@freebsd.org; Alexander Motin; > freebsd-arm@freebsd.org > >> Subject: Re: MMC/SDIO stack under CAM > >> > >> Hi Johannes, > >> > >> My work doesn't include writing drivers for SDHCI controllers. But if > the controller on your new boards is supported by FreeBSD, then you can > really test the new stack! Especially if the controller driver for your > board is based on dev/sdhci, adapting it to work with the new stack is > trivial. For example, iMX6 SDHCI needed only a couple of lines: > https://github.com/kibab/freebsd/commit/df6d8d534740aa3633979da0a9d0ca00b= 60db0e9 > >> > >> Please let me know when you get the new boards and we will figure out > what we need. > >> > >> On February 11, 2016 3:17:22 AM GMT+01:00, "Lundberg, Johannes" < > johannes@brilliantservice.co.jp> wrote: > >>>Hi Ilya > >>> > >>>This is great! > >>> > >>>I've got a Tronsmart ARA X5 and just purchased a few UP > >>> > >>>boards > >>>and it would be really nice if I could utilize the onboard eMMC. These > >>>are > >>>all Intel Cherrytrail platforms. > >>> > >>>Please let me know if there's anything (testing?) I can do to speed up > >>>the > >>>process. > >>> > >>> > >>> > >>>-- > >>>Name: Johannes Lundberg > >>>Position: Mirama project leader > >>>Phone: +1-408-636-2161 > >>>Skype: brilliantjohannes > >>>Online: LinkedIn > >>>Facebook > >>> Reddit > >>> Twitter > >>> GitHub > >>> > >>>GitLab > >>>Company: Mirama Brilliantservice US > >>> Brilliantservice JP > >>> > >>> > >>>On Sun, Jan 3, 2016 at 1:55 AM, Ilya Bakulin wrote: > >>> > >>>> So, more than one year has passed, and I'd like to resurrect this > >>>work > >>>> and move forward. > >>>> > >>>> I have uploaded a new diff and created a completely new revision to > >>>> track the development: https://reviews.freebsd.org/D4761 > >>>> > >>>> What it is able to do now: > >>>> > >>>> * Read/write on SD/SDHC/MMC cards! > >>>> * Detect SDIO cards and create devices that correspond to SDIO > >>>functions > >>>> > >>>> This all works only on BeagleBone currently, because some changes > >>>need > >>>> to be done in each SDHCI-compliant driver to make it interact with > >>>CAM. > >>>> I have purchased a Wandboard Quad that has an integrated SDIO WiFi > >>>chip, > >>>> so I hope to tweak its SDHCI driver as well. > >>>> > >>>> I haven't profiled the stack because: > >>>> * Now we have only SD/MMC cards that are slow anyway; > >>>> * I don't know how to do it in FreeBSD :-) > >>>> > >>>> Please review this diff and tell what you think! > >>>> > >>>> On 01/03/14 18:05, Adrian Chadd wrote: > >>>> > On 1 March 2014 08:46, Ilya Bakulin wrote: > >>>> >> Hi Adrian, > >>>> >> > >>>> >> On 24.02.14, 16:59, Adrian Chadd wrote: > >>>> >>> hi, > >>>> >>> > >>>> >>> Let me just reiterate some .. well, experience doing this stuff > >>>at QCA. > >>>> >>> > >>>> >>> You really, absolutely don't want too much overhead in the > >>>MMC/SDIO > >>>> >>> path between whatever is issuing things and the network driver. > >>>> >>> > >>>> >>> There was significant performance work done at QCA on a local > >>>MMC/SDIO > >>>> >>> driver and bus to get extremely low latency and CPU utilisation > >>>when > >>>> >>> pushing around small transactions. The current CAM locking model > >>>is > >>>> >>> not geared towards getting to high transaction rates. > >>>> >> So here you mean some work done on Linux MMC/SDIO stack by QCA > >>>> >> which made it far better than current Linux MMC stack in terms of > >>>> >> high SDIO I/O rates? > >>>> > Yup. The stock MMC stack/driver in Linux wasn't "fast" enough at > >>>small > >>>> > transactions to sustain the wifi speeds customers required. > >>>> > > >>>> >>> You may think this is a very architecturally pretty solution and > >>>it > >>>> >>> indeed may be. But if it doesn't perform as well as the existing > >>>local > >>>> >>> hacks that vendors have done, no company deploying this hardware > >>>is > >>>> >>> going to want to use it. They'll end up realising there's this > >>>massive > >>>> >>> CAM storage layer in between and either have to sit down to rip > >>>it up > >>>> >>> and replace it with something lightweight, or they'll say "screw > >>>it" > >>>> >>> and go back to the vendor supplied hacked up Linux solution. > >>>> >> I think that if the "architecturally pretty solution" behaves > >>>worse than > >>>> >> some ugly hacks, then it may be not so pretty or the architecture > >>>is > >>>> >> just broken > >>>> >> by design. > >>>> >> > >>>> >>> So I highly recommend you profile things - and profile things > >>>with > >>>> >>> lots of small transactions. If the CAM overhead is more than a > >>>tiny, > >>>> >>> tiny fraction of CPU at 25,000 pps, your solution won't scale. > >>>:-) > >>>> >> I don't really know what to compare with. For MMC/SD cards it is > >>>pretty > >>>> >> obvious, but then these cards will be likely the bottleneck, not > >>>the > >>>> stack. > >>>> >> And the only goal would be to not make the stack slower than it i= s > >>>now. > >>>> >> But, as ATA devices are much faster than MMC/SD, I don't think > >>>this will > >>>> >> be a problem. > >>>> >> > >>>> >> For SDIO things are different. But we don't have any drivers > >>>(yet), > >>>> except > >>>> >> mv_sdiowl that I'm writing, to test on. So I have to bring the > >>>SDIO > >>>> >> stack on CAM, > >>>> >> than bring mv_sdiowl to the state when it can actually transmit > >>>the > >>>> >> data, and then > >>>> >> compare performance with the vendor-supplied Linux driver. > >>>> >> We'll see then if there is a room for improvement... > >>>> > That sounds like a plan. > >>>> > > >>>> > Just note that although storage looks like it's doing much more > >>>> > throughput, the IO size also matters. As I said above, it's not > >>>> > uncommon to have > 1000 receive frames a second on 802.11n; and > >>>that > >>>> > can peak much higher than that. That's not the kind of IO rate you > >>>see > >>>> > on SD cards. :-) > >>>> > > >>>> > > >>>> > > >>>> > -a > >>>> > _______________________________________________ > >>>> > freebsd-hackers@freebsd.org mailing list > >>>> > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >>>> > To unsubscribe, send any mail to " > >>>> freebsd-hackers-unsubscribe@freebsd.org" > >>>> > > >>>> > >>>> > >>>> -- > >>>> Regards, > >>>> Ilya Bakulin > >>>> > >>>> > >>>> > >>> > >>>-- > >>>=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-= =3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- > >>>=E7=A7=98=E5=AF=86=E4=BF=9D=E6=8C=81=E3=81=AB=E3=81=A4=E3=81=84=E3=81= =A6=EF=BC=9A=E3=81=93=E3=81=AE=E9=9B=BB=E5=AD=90=E3=83=A1=E3=83=BC=E3=83=AB= =E3=81=AF=E3=80=81=E5=90=8D=E5=AE=9B=E4=BA=BA=E3=81=AB=E9=80=81=E4=BF=A1=E3= =81=97=E3=81=9F=E3=82=82=E3=81=AE=E3=81=A7=E3=81=82=E3=82=8A=E3=80=81=E7=A7= =98=E5=8C=BF=E7=89=B9=E6=A8=A9=E3=81=AE=E5=AF=BE=E8=B1=A1=E3=81=A8=E3=81=AA= =E3=82=8B=E6=83=85=E5=A0=B1=E3=82=92=E5=90=AB=E3=82=93=E3=81=A7=E3=81=84=E3= =81=BE=E3=81=99=E3=80=82 > >>>=E3=82=82=E3=81=97=E3=80=81=E5=90=8D=E5=AE=9B=E4=BA=BA=E4=BB=A5=E5=A4= =96=E3=81=AE=E6=96=B9=E3=81=8C=E5=8F=97=E4=BF=A1=E3=81=95=E3=82=8C=E3=81=9F= =E5=A0=B4=E5=90=88=E3=80=81=E3=81=93=E3=81=AE=E3=83=A1=E3=83=BC=E3=83=AB=E3= =81=AE=E7=A0=B4=E6=A3=84=E3=80=81=E3=81=8A=E3=82=88=E3=81=B3=E3=81=93=E3=81= =AE=E3=83=A1=E3=83=BC=E3=83=AB=E3=81=AB=E9=96=A2=E3=81=99=E3=82=8B=E4=B8=80= =E5=88=87=E3=81=AE=E9=96=8B=E7=A4=BA=E3=80=81 > >>>=E8=A4=87=E5=86=99=E3=80=81=E9=85=8D=E5=B8=83=E3=80=81=E3=81=9D=E3=81= =AE=E4=BB=96=E3=81=AE=E5=88=A9=E7=94=A8=E3=80=81=E3=81=BE=E3=81=9F=E3=81=AF= =E8=A8=98=E8=BC=89=E5=86=85=E5=AE=B9=E3=81=AB=E5=9F=BA=E3=81=A5=E3=81=8F=E3= =81=84=E3=81=8B=E3=81=AA=E3=82=8B=E8=A1=8C=E5=8B=95=E3=82=82=E3=81=95=E3=82= =8C=E3=81=AA=E3=81=84=E3=82=88=E3=81=86=E3=81=8A=E9=A1=98=E3=81=84=E7=94=B3= =E3=81=97=E4=B8=8A=E3=81=92=E3=81=BE=E3=81=99=E3=80=82 > >>>--- > >>>CONFIDENTIALITY NOTE: The information in this email is confidential > >>>and intended solely for the addressee. > >>>Disclosure, copying, distribution or any other action of use of this > >>>email by person other than intended recipient, is prohibited. > >>>If you are not the intended recipient and have received this email in > >>>error, please destroy the original message. > >> > >> -- > >> =D0=9F=D1=80=D0=BE=D1=81=D1=82=D0=B8=D1=82=D0=B5 =D0=B7=D0=B0 =D0=BA= =D1=80=D0=B0=D1=82=D0=BA=D0=BE=D1=81=D1=82=D1=8C, =D1=81=D0=BE=D0=B7=D0=B4= =D0=B0=D0=BD=D0=BE =D0=B2 K-9 Mail. > >> _______________________________________________ > >> freebsd-arm@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-arm > >> To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sun Feb 14 14:55:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FD0BAA71F9 for ; Sun, 14 Feb 2016 14:55:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 31C101F2B for ; Sun, 14 Feb 2016 14:55:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: by mailman.ysv.freebsd.org (Postfix) id 305F0AA71F8; Sun, 14 Feb 2016 14:55:34 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FFA2AA71F6 for ; Sun, 14 Feb 2016 14:55:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2B2F1F2A for ; Sun, 14 Feb 2016 14:55:33 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from chamsa.cs.huji.ac.il ([132.65.80.19]) by kabab.cs.huji.ac.il with esmtp id 1aUy58-0007jB-OT for hackers@freebsd.org; Sun, 14 Feb 2016 16:55:26 +0200 From: Daniel Braniss Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: autofs problem Message-Id: <39973B1A-BBB9-4CB7-B352-A42C14B61E0F@cs.huji.ac.il> Date: Sun, 14 Feb 2016 16:55:26 +0200 To: "hackers@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Feb 2016 14:55:34 -0000 Hi, I=E2=80=99m converting our (very) old am-utils maps to use the new = autofs, so I wrote a python script that queries the am-utils map, and = =E2=80=98generates=E2=80=99 one that autofs likes. so autofs_master has: =E2=80=A6 /cs auto_cs auto_cs is a python script and is executable. at the moment I=E2=80=99m stuck with what am-utils type=3Dlink so I=E2=80=99m using -fstype=3Dnullfs :/cs/some-path, but before this = can work I need to trigger autofs with the /cs/some-path, so before the script returns I = do a os.stat('/cs/some-path') but this returns an error, mainly because it is not caught by autofs! doing the same in the command line works, and later the nullfs too, = (i=E2=80=99m afraid of what the auto unmount will do later, but that=E2=80=99s another = issue) is this by design? or is there some better way? thanks, danny From owner-freebsd-hackers@freebsd.org Mon Feb 15 10:13:47 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 692B5AA9E05; Mon, 15 Feb 2016 10:13:47 +0000 (UTC) (envelope-from ilya@bakulin.de) Received: from olymp.kibab.com (olymp6.kibab.com [IPv6:2a01:4f8:160:84c1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 307B91E5D; Mon, 15 Feb 2016 10:13:44 +0000 (UTC) (envelope-from ilya@bakulin.de) DKIM-Filter: OpenDKIM Filter v2.10.3 olymp.kibab.com B8A1E4E674 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bakulin.de; s=default; t=1455531218; bh=EkZ+uNda3Zo1jeoLin0+zjYAR9uYfmEZF1fSpCDPhes=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=Q+/pN2KyupnHRbZaSlREPfiZ3bsLTvKFd/6qK2PS04cDNncLJnSK0IHlaBmLslAL0 7kYlELcxpaQBzyqn5a1GhX+LvnQ8+04V1WqBCS+wIEsBKVTYoZSx0BRGxq3DBuKb1i BeofuUiOZPU1c0ISZ4rh2dRF1rsH3t9AM7e7YMX4= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 15 Feb 2016 11:13:38 +0100 From: Ilya Bakulin To: Stanislav Sedov Cc: "Lundberg, Johannes" , Adrian Chadd , freebsd-hackers@freebsd.org, Alexander Motin , freebsd-arm@freebsd.org Subject: Re: MMC/SDIO stack under CAM Organization: Deglitch Networks In-Reply-To: <6942A46B-110B-4E1F-9DA1-F965009E8E92@FreeBSD.org> References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com> <53120EE8.1080600@bakulin.de> <5688F015.4090002@bakulin.de> <6942A46B-110B-4E1F-9DA1-F965009E8E92@FreeBSD.org> Message-ID: <38dd08fc2a5930d58b09e9bd3cb6d3e7@bakulin.de> X-Sender: ilya@bakulin.de X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2016 10:13:47 -0000 On 2016-02-11 19:54, Stanislav Sedov wrote: >> On Feb 11, 2016, at 10:47 AM, Ilya Bakulin wrote: >> >> I'll use an excellent opportunity to post a small status update about >> my work :-) >> * SDHC controller on Wandboard now works with the new stack; >> * SDIO block read now works! >> * camcontrol userland app is extended to support "mmcsdcmd" command >> that allows to send MMC commands from userland apps directly to the >> card via pass(4) device -- now we can write WLAN driver in userland >> :-D > > Great news, userspace drivers are the best!:) > > So what are the remaining pieces that prevent this work from hitting > the HEAD? > > -- > Stanislav Sedov > ST4096-RIPE Hi Stas, As I'm not a committer, someone needs to review my code and assist in intergration into -HEAD :-) Currently nobody was able to do a review because of -ENOTIME. The only feature that is missing in the new stack (from my PoV) is working with high-speed cards -- I just haven't implemented switching to high-speed mode yet. Although now it's possible to send required commands to the card and then switch controller speed -- all using camcontrol mmcsdcmd :-). Do you know anyone not on CC line who is able to help me with this? Or maybe you could even find some time yourself? -- Mit freundlichen Grüßen, Ilya Bakulin From owner-freebsd-hackers@freebsd.org Mon Feb 15 14:22:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BE1FAA9C22; Mon, 15 Feb 2016 14:22:35 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x22f.google.com (mail-pf0-x22f.google.com [IPv6:2607:f8b0:400e:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56C381064; Mon, 15 Feb 2016 14:22:35 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x22f.google.com with SMTP id x65so88236240pfb.1; Mon, 15 Feb 2016 06:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=qWW/ITP4JUDkR1KG3J2vjLP0/rWcIwHeOJcf8vg78pU=; b=fshUNJl6SIHTLKnuejRKHES4FzYvYaFZGpRTCvP3oYclWrZqR623PTBim5KtDicfsT B3QjWADFYV5pFZFFTgN7fiWLggsS7odOcjdpyeD1rJAhkKemjmBX615I/jpfh4anrlwI z1WPepKe+iTtqEozJeKn2HpdoNDuv3LR0eDgvdBMusDFBZURYsUN7ZBACWqivX77VZ9Q WSYK06utQ+KsPk/txceMVyVBdsfc30UC4P6dlyYl/Si20jRAoL62xL0qcmODOLeyDTuO RZ8TSNRPc9Ie5a++Gtq31/jjm6WKl1o4FmjSLynlAZGK+nmXV/E0j9gOpgA5KD8dR5qj Wbhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=qWW/ITP4JUDkR1KG3J2vjLP0/rWcIwHeOJcf8vg78pU=; b=YrTeN4GvY+cVGFVk5tL8sITkXK9J1Y3QOZ38nxbEh4MsArC6LE4ou+LJYf+xhBUQzj 3Y8KRuPFROEEDOaDtnxqufL/B8JqWLtOSzC7QGUizCBfu3dCMz2Sy9dtXfjrIqE6Rofm 772ItUj18e8Gc2W9OFdJ/cHxlF6wcLV3nrpv7COJ7RTxRlg8qN+1bwgRfgt3ZjSBGbdq NVr/rebRVbekVsbXVqFWXaiypky2djp85ck8ePgtdqtVr67DMf4cZJIKbBBriTBrBJXk WhufiubaXHvpkaxFAIy5ieGi5PTdwgDExEymr7GeLnSQACqWC7oA1ThOpDlvIgo5zcTe Exaw== X-Gm-Message-State: AG10YOTaWmoYE13nXGyfY2LESO0fIL+HAnQKUNm7sBgkhe49cS3V2b9BxmJ3AZcB7Fdb0w== X-Received: by 10.98.18.207 with SMTP id 76mr23597305pfs.53.1455546154697; Mon, 15 Feb 2016 06:22:34 -0800 (PST) Received: from [192.168.20.11] (c-24-16-212-205.hsd1.wa.comcast.net. [24.16.212.205]) by smtp.gmail.com with ESMTPSA id fl9sm39091076pab.30.2016.02.15.06.22.32 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 15 Feb 2016 06:22:32 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: MMC/SDIO stack under CAM From: NGie Cooper X-Mailer: iPhone Mail (13D15) In-Reply-To: <38dd08fc2a5930d58b09e9bd3cb6d3e7@bakulin.de> Date: Mon, 15 Feb 2016 06:22:31 -0800 Cc: Stanislav Sedov , Adrian Chadd , freebsd-hackers@freebsd.org, Alexander Motin , freebsd-arm@freebsd.org, "Lundberg, Johannes" Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com> <53120EE8.1080600@bakulin.de> <5688F015.4090002@bakulin.de> <6942A46B-110B-4E1F-9DA1-F965009E8E92@FreeBSD.org> <38dd08fc2a5930d58b09e9bd3cb6d3e7@bakulin.de> To: Ilya Bakulin X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2016 14:22:35 -0000 > On Feb 15, 2016, at 02:13, Ilya Bakulin wrote: >=20 > On 2016-02-11 19:54, Stanislav Sedov wrote: >>> On Feb 11, 2016, at 10:47 AM, Ilya Bakulin wrote: >>> I'll use an excellent opportunity to post a small status update about my= work :-) >>> * SDHC controller on Wandboard now works with the new stack; >>> * SDIO block read now works! >>> * camcontrol userland app is extended to support "mmcsdcmd" command that= allows to send MMC commands from userland apps directly to the card via pas= s(4) device -- now we can write WLAN driver in userland :-D >> Great news, userspace drivers are the best!:) >> So what are the remaining pieces that prevent this work from hitting the H= EAD? >> -- >> Stanislav Sedov >> ST4096-RIPE >=20 > Hi Stas, >=20 > As I'm not a committer, someone needs to review my code and assist in inte= rgration into -HEAD :-) > Currently nobody was able to do a review because of -ENOTIME. > The only feature that is missing in the new stack (from my PoV) is working= with high-speed cards -- I just haven't implemented switching to high-speed= mode yet. Although now it's possible to send required commands to the card a= nd then switch controller speed -- all using camcontrol mmcsdcmd :-). >=20 > Do you know anyone not on CC line who is able to help me with this? Or may= be you could even find some time yourself? Hi Ilya, Could you please post the patch to phrabricator and CC the interested pa= rties/me? Thanks! -Ngie= From owner-freebsd-hackers@freebsd.org Mon Feb 15 21:42:09 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EEE6AA8917 for ; Mon, 15 Feb 2016 21:42:09 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2E733982 for ; Mon, 15 Feb 2016 21:42:09 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 2D49AAA8916; Mon, 15 Feb 2016 21:42:09 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1479BAA8915 for ; Mon, 15 Feb 2016 21:42:09 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A127897D for ; Mon, 15 Feb 2016 21:42:08 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id g62so126271543wme.0 for ; Mon, 15 Feb 2016 13:42:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=dgyGnMm6KUVC3pGaWu2fMB5u7Ndyx/HxtihdeeKoh8Y=; b=iwLgozqmfm9WYBHPYePVh3qQIZDqNfC/CX/cb1WNIce4QJABaRgHZIy/qza1uNc1gV xe1h+Ykmd/Q+gpBp2O1MtqpcYrfpWPOStsVAqADWJ97+vkdNROtubUg5UVRrBtl1PHaa /AqKG3vJOykmqpBQfW9sbZXjsVnKJu1ksZvZq5YugjJNkuuxquaHij3MvARoCu7vk0ZO R4DhmuKp93wNp4jY7s8XUqL4hrwzexIssqOqXNlmAr9zn6Q7kQy7eynxVn6YuQNEwhJQ dwa39LspwIjiyIZG/Y2Bd9FmgOCFyhtUzHoKOY9mV55nJxzcdUdTcoAcRnZqoeqbY38J aYKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=dgyGnMm6KUVC3pGaWu2fMB5u7Ndyx/HxtihdeeKoh8Y=; b=lMttEUu0Zwlya24LMh6BQKuBIYRNYLJrgjnrLmq4lBG/Tu5lYp/nN7/fDQDn/Th3XL QwOxyBUFLhAKWI0MRK7BFc7JUE76DuGzplLED3FU06csiflGMiP2XDKBRRcM0RwEK4PK VD16exIsbf5c3djor3MA6xQ55DYXNd2sUJsNxhJWKzqXkU2VTzyc9RMtUjykflB2yG2g GvxBpvBed6vYUxPF+XLaZZDUjbITyBSgosmOclDLJO/eNttkdKScsprXwQvyDE4ozV4J TUYZVzo9H/HHRMZ6VS73Xx7gObur7/1uL+YROZ7wu+VbMXvEz4qGrZtGfw37jIU15na0 I5tg== X-Gm-Message-State: AG10YOQRcFVJPFiKAAbdmJmIgazlx94QaqEvDMTLNdSv91RfSiPBefLe0mlw+qoNPIeXJw== X-Received: by 10.194.47.133 with SMTP id d5mr20002707wjn.87.1455572526608; Mon, 15 Feb 2016 13:42:06 -0800 (PST) Received: from brick.home (49.44.broadband2.iol.cz. [83.208.44.49]) by smtp.gmail.com with ESMTPSA id gt7sm27236453wjc.1.2016.02.15.13.42.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 15 Feb 2016 13:42:05 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Mon, 15 Feb 2016 22:41:59 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Daniel Braniss Cc: "hackers@freebsd.org" Subject: Re: autofs problem Message-ID: <20160215214159.GA6160@brick.home> References: <39973B1A-BBB9-4CB7-B352-A42C14B61E0F@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <39973B1A-BBB9-4CB7-B352-A42C14B61E0F@cs.huji.ac.il> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Feb 2016 21:42:09 -0000 On 0214T1655, Daniel Braniss wrote: > Hi, > I’m converting our (very) old am-utils maps to use the new autofs, > so I wrote a python script that queries the am-utils map, and ‘generates’ > one that autofs likes. > so autofs_master has: > … > /cs auto_cs > > auto_cs is a python script and is executable. > > at the moment I’m stuck with what am-utils type=link > so I’m using -fstype=nullfs :/cs/some-path, but before this can work I need > to trigger autofs with the /cs/some-path, so before the script returns I do a > os.stat('/cs/some-path') > but this returns an error, mainly because it is not caught by autofs! > doing the same in the command line works, and later the nullfs too, (i’m afraid > of what the auto unmount will do later, but that’s another issue) > > is this by design? or is there some better way? It's by design - basically, autofs(4) has code to explicitly disable triggering by anything running with the same SID (session id) as the automountd(8). This is to prevent some obvious (and some less than obvious and quite hard to debug) deadlocks. Thus, any script ran by automountd(8) works as if the autofs mounts "weren't there". You might be able to work around this by somehow calling setsid() from the Python script before triggering. From owner-freebsd-hackers@freebsd.org Tue Feb 16 05:33:55 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 626DBAAAFF5 for ; Tue, 16 Feb 2016 05:33:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B9791E35 for ; Tue, 16 Feb 2016 05:33:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qg0-x230.google.com with SMTP id b67so125851681qgb.1 for ; Mon, 15 Feb 2016 21:33:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pnTu3IPUPG/MeiNv0qodOU5RG96ZVRjbPVQlzD4cfMg=; b=Pnj3BtSgg7CEA+314+fFddXHFV21e1TnM+NUQ4La+288Eo2LmW3J3tKKof/KO76m9m 5EmIufpOuYfKml+t+qQkJ/4IwoHf7R0lm30mKhLHVAnyTpxg51wTENAy9XsBpStyscao HHI+Mi+oLox5TixgTjxpfrtfkX/hNq8B7YwtTGETvXqXwd4mm6M7rCiv4yEX/1oxdF7W 8F2xuaRIc0OOr8iI+5i0l45Ee3bS+eWG08QwqltlCdDFqBNClwZsXZm7V59GcGE8frXu loZrLQfz3NOnBgvE50nHWZ2McRVd84QCQDz7etsHsXpt2j5BQCEIdd9o5I/GAsrISnhQ 2XKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=pnTu3IPUPG/MeiNv0qodOU5RG96ZVRjbPVQlzD4cfMg=; b=Xn9wpleXtf6D6QPru5tk6jLaWeHCAFosqsTtxNijHNjf9Yf7PAAxz0wq2XQRas6Ab9 WBnBjV4OKBDQ/6vHgqBLOtkoILu7rYWTHP/4MPQDIV5q0hJdzrunJAmT0BLiq6XUXI3d 3nT8QBRJVYgl24T42eoEEOAkbyXTmrOaiscoWuL0GHbwytekpWaszOrEso3RQz02VRDj 48OB6XvC6gANM50r5nibJ87NC6s5eIew1Um81S7ZPt1J31UBVI9KFYCmH+jkHB2V0sCu BhegmvSzbWDOFDDfhZY5oaB6rogoIuif2zFetbSH4uGcMjeC7DMD9KpN9qh8nysTMFhu joRw== X-Gm-Message-State: AG10YORA/qBXg1D00BPSC1VYothCgJF03qGDPAkq1wINJ22JpUI1T+z8grB+AkD5MLc+VzUvMWQ+OivMNdWt/w== MIME-Version: 1.0 X-Received: by 10.140.221.136 with SMTP id r130mr1799330qhb.94.1455600834155; Mon, 15 Feb 2016 21:33:54 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.30.166 with HTTP; Mon, 15 Feb 2016 21:33:54 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <20140216111153.GA74858@olymp.kibab.com> <5C2CF572-360D-4CA0-81C7-18A5C455AED5@bsdimp.com> <20140224142642.GA32538@olymp.kibab.com> <53120EE8.1080600@bakulin.de> <5688F015.4090002@bakulin.de> <6942A46B-110B-4E1F-9DA1-F965009E8E92@FreeBSD.org> <38dd08fc2a5930d58b09e9bd3cb6d3e7@bakulin.de> Date: Mon, 15 Feb 2016 22:33:54 -0700 X-Google-Sender-Auth: bP0r9qGRzgXt8RGBouhRZF4B_TM Message-ID: Subject: Re: MMC/SDIO stack under CAM From: Warner Losh To: NGie Cooper Cc: Ilya Bakulin , Adrian Chadd , "freebsd-hackers@freebsd.org" , Alexander Motin , "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2016 05:33:55 -0000 On Mon, Feb 15, 2016 at 7:22 AM, NGie Cooper wrote: > > > On Feb 15, 2016, at 02:13, Ilya Bakulin wrote: > > > > On 2016-02-11 19:54, Stanislav Sedov wrote: > >>> On Feb 11, 2016, at 10:47 AM, Ilya Bakulin wrote: > >>> I'll use an excellent opportunity to post a small status update about > my work :-) > >>> * SDHC controller on Wandboard now works with the new stack; > >>> * SDIO block read now works! > >>> * camcontrol userland app is extended to support "mmcsdcmd" command > that allows to send MMC commands from userland apps directly to the card > via pass(4) device -- now we can write WLAN driver in userland :-D > >> Great news, userspace drivers are the best!:) > >> So what are the remaining pieces that prevent this work from hitting > the HEAD? > >> -- > >> Stanislav Sedov > >> ST4096-RIPE > > > > Hi Stas, > > > > As I'm not a committer, someone needs to review my code and assist in > intergration into -HEAD :-) > > Currently nobody was able to do a review because of -ENOTIME. > > The only feature that is missing in the new stack (from my PoV) is > working with high-speed cards -- I just haven't implemented switching to > high-speed mode yet. Although now it's possible to send required commands > to the card and then switch controller speed -- all using camcontrol > mmcsdcmd :-). > > > > Do you know anyone not on CC line who is able to help me with this? Or > maybe you could even find some time yourself? > > Hi Ilya, > Could you please post the patch to phrabricator and CC the interested > parties/me? > It's been up on phab for a while. There's been some comments on it. There's some things wrong still that I've been meaning to get bcak to Ilya on. When it is ready, I plan on committing this. It goes hand in hand with the nvme CAM stuff I've been working on. Anybody can take a look at it: https://reviews.freebsd.org/D4761 Warner From owner-freebsd-hackers@freebsd.org Tue Feb 16 10:01:14 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5004AA9FF6 for ; Tue, 16 Feb 2016 10:01:14 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A997315E3 for ; Tue, 16 Feb 2016 10:01:14 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: by mailman.ysv.freebsd.org (Postfix) id A9119AA9FF5; Tue, 16 Feb 2016 10:01:14 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A89ADAA9FF4 for ; Tue, 16 Feb 2016 10:01:14 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1EEE815E1; Tue, 16 Feb 2016 10:01:14 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from chamsa.cs.huji.ac.il ([132.65.80.19]) by kabab.cs.huji.ac.il with esmtp id 1aVcRH-000DqX-HB; Tue, 16 Feb 2016 12:00:59 +0200 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: autofs problem From: Daniel Braniss In-Reply-To: <20160215214159.GA6160@brick.home> Date: Tue, 16 Feb 2016 12:00:59 +0200 Cc: "hackers@freebsd.org" Message-Id: <6FFA90F6-7870-40B6-8BDE-281B899B62D7@cs.huji.ac.il> References: <39973B1A-BBB9-4CB7-B352-A42C14B61E0F@cs.huji.ac.il> <20160215214159.GA6160@brick.home> To: =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= X-Mailer: Apple Mail (2.2104) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Feb 2016 10:01:15 -0000 > On 15 Feb 2016, at 23:41, Edward Tomasz Napiera=C5=82a = wrote: >=20 > On 0214T1655, Daniel Braniss wrote: >> Hi, >> I=E2=80=99m converting our (very) old am-utils maps to use the new = autofs, >> so I wrote a python script that queries the am-utils map, and = =E2=80=98generates=E2=80=99 >> one that autofs likes. >> so autofs_master has: >> =E2=80=A6 >> /cs auto_cs >>=20 >> auto_cs is a python script and is executable. >>=20 >> at the moment I=E2=80=99m stuck with what am-utils type=3Dlink >> so I=E2=80=99m using -fstype=3Dnullfs :/cs/some-path, but before this = can work I need >> to trigger autofs with the /cs/some-path, so before the script = returns I do a >> os.stat('/cs/some-path') >> but this returns an error, mainly because it is not caught by autofs! >> doing the same in the command line works, and later the nullfs too, = (i=E2=80=99m afraid >> of what the auto unmount will do later, but that=E2=80=99s another = issue) >>=20 >> is this by design? or is there some better way? >=20 > It's by design - basically, autofs(4) has code to explicitly disable = triggering > by anything running with the same SID (session id) as the = automountd(8). This > is to prevent some obvious (and some less than obvious and quite hard = to debug) > deadlocks. Thus, any script ran by automountd(8) works as if the = autofs mounts > "weren't there". >=20 > You might be able to work around this by somehow calling setsid() from > the Python script before triggering. bingo!, that did it thanks, danny From owner-freebsd-hackers@freebsd.org Wed Feb 17 14:28:43 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 647FBAAA0C0 for ; Wed, 17 Feb 2016 14:28:43 +0000 (UTC) (envelope-from f0andrey@gmail.com) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2480B1D8A for ; Wed, 17 Feb 2016 14:28:43 +0000 (UTC) (envelope-from f0andrey@gmail.com) Received: by mail-qg0-x22a.google.com with SMTP id y9so12812410qgd.3 for ; Wed, 17 Feb 2016 06:28:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Ztf0vtqHQ+UHzU1m7BEITMXFE1v50/Uz07S1vtRiNrw=; b=d0deqGhCxm0qgVcJJ6Oq9h8mJ9LajFQULfoRUZNr9R0kKDOcC2qyD9B3czJWsLapVB 834ZrOjgxLd0Z5e23fp5i1N8Y9qHQL7hvmE1rwTrtHWAxt3Jv70W2G7+d75B3UGbmxCD UFj8jsOCKBp40Mu7yUSQISrgWqbrYuwI/vpWQ9qpDPQLgqNiNOng64Dqq0PG6ffGyzRl po6j9m+vNBoYbU8JnhOBpoBS3Y9HHfqhNVFuM0MVu6TgWHke2EvHhnzuKz1pcaaPHKOd UwDdh5rhtroWu0Ho9xS7G1OTyzN9DoM9Fz88qsAsjp3HPUcMAkWr/lg4Ifz5qlQbZakO 2Hew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=Ztf0vtqHQ+UHzU1m7BEITMXFE1v50/Uz07S1vtRiNrw=; b=Cgf0sEKRzW2OAAXdPWjqlZXLTZrZLs6beqry67m4sIY8eZu6HE6nXnNmdIwVpuU0nU 2kMOJ2uZC3f6V2SmGHGsTEwyNEQ3RINBtSGQoJnjGccdLk5tlJ3DaZ+NKpeOpbcHpdET B3V89PA95/RZPmJnPVTYAKo5XB0RGxZcHgyu7zn1HlJlIfwTDdcTbnezm70lRTgcGtLm VYKN+3PVS7VMRxu335n7rdxVRi5j9Nkbwts3qmLF420nPM7uL71iK9YE5Fs6F7HbQfBv Lja0aMjt+Sw6x1AsLaLmCLuzoXdilsZPeR22SecvE7w1P2XLCfZq05FWiNTJ2OBjVgjr LACQ== X-Gm-Message-State: AG10YOR+M953gEAJ1GPsS/Rq/4sqLX9/5M8ZUzov/zTv2XU/Sdcindj84aUl39eszr7W+UHJKoPpFrHxhiGH+Q== MIME-Version: 1.0 X-Received: by 10.140.160.130 with SMTP id g124mr2351410qhg.88.1455719322266; Wed, 17 Feb 2016 06:28:42 -0800 (PST) Received: by 10.55.106.5 with HTTP; Wed, 17 Feb 2016 06:28:42 -0800 (PST) Date: Wed, 17 Feb 2016 17:28:42 +0300 Message-ID: Subject: =?UTF-8?Q?FreeBSD_and_Mayhem_=E2=80=93_a_hidden_threat_for_=2Anix_web_?= =?UTF-8?Q?servers?= From: Andrey Fesenko To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2016 14:28:43 -0000 Hello, There is a vulnerability https://www.virusbulletin.com/virusbulletin/2014/07/mayhem-hidden-threat-nix-web-servers?utm_content=buffercd266&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer Is known methods lock and protect it from the FreeBSD? From owner-freebsd-hackers@freebsd.org Wed Feb 17 15:55:28 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A939AAA9D1 for ; Wed, 17 Feb 2016 15:55:28 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) by mx1.freebsd.org (Postfix) with ESMTP id F0EB81D42 for ; Wed, 17 Feb 2016 15:55:27 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 086BDDABE for ; Wed, 17 Feb 2016 15:55:21 +0000 (UTC) Subject: =?UTF-8?Q?Re:_FreeBSD_and_Mayhem_=e2=80=93_a_hidden_threat_for_*nix?= =?UTF-8?Q?_web_servers?= To: freebsd-hackers@freebsd.org References: From: Allan Jude X-Enigmail-Draft-Status: N1110 Message-ID: <56C497EC.20704@freebsd.org> Date: Wed, 17 Feb 2016 10:55:24 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WWNu3KV5pQSiH15MEF0fslKVcUHgfnOuS" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Feb 2016 15:55:28 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WWNu3KV5pQSiH15MEF0fslKVcUHgfnOuS Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2016-02-17 09:28, Andrey Fesenko wrote: > Hello, > There is a vulnerability > https://www.virusbulletin.com/virusbulletin/2014/07/mayhem-hidden-threa= t-nix-web-servers?utm_content=3Dbuffercd266&utm_medium=3Dsocial&utm_sourc= e=3Dtwitter.com&utm_campaign=3Dbuffer > Is known methods lock and protect it from the FreeBSD? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 Note that that post is from 2014. Make sure you are not running a vulnerable version of popular PHP software like Wordpress, Drupal, or Joomla. If possible, keep the directories where the PHP scripts are run locked down with permissions, or better yet, a separate ZFS dataset with the readonly property turned on. Mount the /tmp directory (and possible the PHP directories) noexec, so that scripts and binaries drops by attempts to exploit your web apps will not run. As far as general advice: use jails to contain your webserver, and ZFS snapshots to be able to 'undo' anything that does happen. --=20 Allan Jude --WWNu3KV5pQSiH15MEF0fslKVcUHgfnOuS Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJWxJfvAAoJEBmVNT4SmAt+8gwQAKNgGZFqz3FgoPwDxjmHbt1c OzTmydXE67ipTMFzbiJJsAHGo0zp02uZXyhhxGL5KY4LiL8XQgSkqgIEfOKVR1Eo mQ88y1g7fyRIE/y4RRK8+Ka7a0FS/q165yvHj8fo9p+yz3OhZsiiK5aTiFDpiI4q 5ZDp5BEVwWWgis44rdwD7oIPwHfEgmN037xJhj/XqGvty9PWbR3+rTNnsJo8qLNN lUDCBz2q78rMa8Ig01gIW+Ikl582lRtUgzlAGwODFVNDwRLB7tH/GgccEmPAZnax P+qzSFSO6YjTpjepyDR7QtOBA20eAPW41hGEICv+sQGNOshFI5XdtMgfbynJYDS7 58cR7K/NZGeu59amru+DEK+RoNEnQ4T7QF8e1W+n6GOBbO8N5QziimatnImaysyk aio7T7OSCIDbymR6Wzw6ghOk3bLo1c/5c3TY92MRMjSZeAOPIzt6oeIm6GghyuCk ozV+VlJ2REi/7LZvazqX8eXh+C0aavg2kmqpWqBstiTSv9ciykxNRFmECwnP19dP cxo8hiy4dnkUtkOe2DK3tX8XZZmMFhrqZfivSmFplKBHFEVohA/jx3q3dwolRj0Z 2F1Phil4TlY4K6ypS9RcjrnSP6vvrqz3SbJzD7hT/KX8eWNbcEj1WXN2xErXCbnq twlYkgz7Z7nhVqzCGS/p =fNrY -----END PGP SIGNATURE----- --WWNu3KV5pQSiH15MEF0fslKVcUHgfnOuS-- From owner-freebsd-hackers@freebsd.org Sat Feb 20 00:13:05 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44E0BAAEB30 for ; Sat, 20 Feb 2016 00:13:05 +0000 (UTC) (envelope-from luislupe@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD86E9BA for ; Sat, 20 Feb 2016 00:13:04 +0000 (UTC) (envelope-from luislupe@gmx.com) Received: from localhost ([95.95.94.35]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0MQQzk-1aPF9Y0pdk-00TpdJ for ; Sat, 20 Feb 2016 01:12:56 +0100 Date: Sat, 20 Feb 2016 00:12:54 +0000 From: "Luis P. Mendes" To: freebsd-hackers@freebsd.org Subject: kbdmux disabled - error -> keyboard is not useable Message-ID: <20160220001254.GD73442@hp.tbl> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-Provags-ID: V03:K0:SmJQnxiMHpyc+7hxFchPmxUlenlj1OyzMRbX+ppRQcEpW8U1C8F YMUhKiZv439HsWxso4LVNucFlpWlptF4qSheqy13v1brPqA6Srl+pS6PV8lZTdTFsIFpson 3VPLs+1EX91kyOGU3MdNqZ5NvXnO1N1XB+75GuRqMRjWzTuFRGW0MmZBr4ItNx9C8dVxVI0 VtVNWlAqLZ2ms6FaRS4+A== X-UI-Out-Filterresults: notjunk:1;V01:K0:iegtsDeN+o0=:LpsTLKKKXW7LHHkOy+eaId yxrBWYgpqaMLC+S0jtbAWtREL0wRJOq/JC30oup4VtO5p9sW4tFBLcUXatvWwkXKWZCfQA2md d2hZqk5pF1cfhoqCk3IYtslfK7NoLOjTnazH0HfDBNhk3T0zckw4fAmo30GyRxNPagG08LyKf WUaJaiUi08/OF4AdRZgA5JzcCSCvjCTezowarkjIcMgSwroGAwP078o9JjsQ6MqKppkRNR4HM s5xSkVrFXQAb14OxpReFInnhwjKEz7YlNeECTyGWPpLp58W221dk6QE9V6CwemV0ffQbwY8bB yUb8YjbQyMRMheiY/5sjpU9XRH1kSGYEwLOLbK9POhx+9Xvx1xKc57CIojJbB3z6ZMX0fZ10P 649uerTwDf88LU99+XtTw61teNjxIiGoZ9ZnAVpRlI1sHb9bSsCRqgMgBLk2iWMjQEHqB4lMw +jUWS7bqXh+gtzonv/opQrOHalFMAhcQjLz85YKzoaZSwxqPDqrDZm0379CIGGqQe6LL+VSqT TLoW7Nsg9x9kr0HFkXDqid30h+icqqjYZ3jDxeR8MSgWTaZHIK2iNJsWL3PkiwHNGaAXH29mo j0L7wJCNbeh1tWdqMajKAkX7s8Na1b8xMcB5gprnr1GTpRCOIhQ5XK4okcpsF2J9K33R8MbsK Dn8HdUvelZrXyi6lH82NpIrdJIqu2IBmTanfArN56i8XYomArScGFq1FMKJOsvwLD/IBOFbUc 64ZnePJWo5aGb/sAslGM5J7ARwaLVqGztsuV7RJ4MfMDKTE3HAJS/R3Tsgk= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 00:13:05 -0000 Hi, -------------------------- My FreeBSD version is: # uname -a FreeBSD leao 10.3-BETA1 FreeBSD 10.3-BETA1 #0 r295321: Fri Feb 5 17:02:27 WET 2016 I'm using stock kernel. -------------------------- I'm setting up a multiseat workstation, where two users, with two monitors, two keyboards, two mice and two X sessions can be connected to the same computer at the same time. For this to happen, one of the things that is necessary to do is to disable kbdmux in order for the two keyboards to be considered independent. In the past, I've accomplished this with syscon, which had other problems. But now with vt, when I disable kbdmux either on /boot/device.hints or in /boot/loader.conf with: hint.kbdmux.0.disabled="1" No keys are recognized by the OS, although /var/log/messages seems to know about the existence of the USB keyboard, although it shows an error. The steps I've taken: 1. Have an USB keyboard attached to the motherboard. The keyboard is a Logitech K120 with no multimedia keys, just a 105 key with PT layout. The keyboard works fine when kbdmux is not disabled, and works fine in every other computer/OS. No AT keyboard used. 2. In /boot/loader.conf, have these lines added: ukbd_load="YES" hint.kbdmux.0.disabled="1" hint.atkbd.0.disabled="1" hint.atkbdc.0.disabled="1" I tried it with both *atkbd* lines commented in and out and the result is the same. 3. In /etc/rc.conf kbdcontrol -k /dev/ukbd0 < /dev/console 4. After commenting the two atk lines in /boot/loader.conf, what I got from: # ll /dev/*kbd* crw------- 1 root wheel 0x34 18 Fev 14:34 /dev/atkbd0 lrwxr-xr-x 1 root wheel 6 18 Fev 14:34 /dev/kbd0@ -> atkbd0 lrwxr-xr-x 1 root wheel 5 18 Fev 14:34 /dev/kbd1@ -> ukbd0 crw------- 1 root wheel 0x81 18 Fev 14:34 /dev/ukbd0 5. In /var/log/messages, there are these lines: # egrep -i -e 'kbd|keyb' messages Feb 18 14:34:58 leao kernel: module_register_init: MOD_LOAD (kbdmux, 0xffffffff805d4070, 0) error 6 Feb 18 14:34:58 leao kernel: atkbdc0: at port 0x60,0x64 on isa0 Feb 18 14:34:58 leao kernel: atkbd0: irq 1 on atkbdc0 Feb 18 14:34:58 leao kernel: kbd0 at atkbd0 Feb 18 14:34:58 leao kernel: atkbd0: [GIANT-LOCKED] Feb 18 14:34:58 leao kernel: ukbd0: on usbus0 Feb 18 14:34:58 leao kernel: kbd1 at ukbd0 Feb 18 14:34:58 leao kernel: uhid0: on usbus0 It seems there's an error 6 regarding kbdmux... 6. # dmesg | grep kbd module_register_init: MOD_LOAD (kbdmux, 0xffffffff805d4070, 0) error 6 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ukbd0: on usbus0 kbd1 at ukbd0 This is the only thing keeping me from using my workstation the way I need. I hope there's some fix to this. How to correct this problem? Is this a bug? -- Luis Mendes From owner-freebsd-hackers@freebsd.org Sat Feb 20 01:27:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72B9CAADEC2 for ; Sat, 20 Feb 2016 01:27:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 50BB7CEF for ; Sat, 20 Feb 2016 01:27:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: by mailman.ysv.freebsd.org (Postfix) id 51123AADEC1; Sat, 20 Feb 2016 01:27:35 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36CE2AADEBE for ; Sat, 20 Feb 2016 01:27:35 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CC2A1CE7; Sat, 20 Feb 2016 01:27:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:6IT1OxCp13kltn8Bc3t5UyQJP3N1i/DPJgcQr6AfoPdwSP79p8bcNUDSrc9gkEXOFd2CrakU1KyL6uu7BiQp2tWojjMrSNR0TRgLiMEbzUQLIfWuLgnFFsPsdDEwB89YVVVorDmROElRH9viNRWJ+iXhpQAbFhi3DwdpPOO9QteU1JTokb3rsMaMKyxzxxODIppKZC2sqgvQssREyaBDEY0WjiXzn31TZu5NznlpL1/A1zz158O34YIxu38I46FppIZ8VvDZeKIjUbVeEDUge0o44Mr2rh7dBV+M4WAAU2Ycnx5gDA3M7RW8VZD05HjUrO14jRObNs6+aLk/WjCv6u8/UhrhgyQDOjsR7WbYl8F0lKIdqxv39E83+JLdfIzAbKk2RajaZ95PHWc= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQAQCjwMdW/61jaINehAxtBro+AQ2BaBcMhSAQOgKBeRQBAQEBAQEBAWMngi2CFQEBBAEBASArIAsFCwIBCA4KAgINGQICIQYBCSYCBAgHBAEcBIcZSwMSDq0GiXYNhGABAQEBAQEBAwEBAQEBAQEBGHuFF4F1gkaCOoFcAQEFgxiBOgWOHohphVeFIHSDT0qDeYhUhwWHQQIeAQFChAIeLgEEAodCNH0BAQE X-IronPort-AV: E=Sophos;i="5.22,473,1449550800"; d="scan'208";a="268701670" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 19 Feb 2016 20:27:05 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id E218515F5A1; Fri, 19 Feb 2016 20:27:05 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ukqb9uaH5zbT; Fri, 19 Feb 2016 20:27:02 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id CD99215F5B4; Fri, 19 Feb 2016 20:27:02 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id BtHNVagX2aUR; Fri, 19 Feb 2016 20:27:02 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A44DE15F5A1; Fri, 19 Feb 2016 20:27:02 -0500 (EST) Date: Fri, 19 Feb 2016 20:27:02 -0500 (EST) From: Rick Macklem To: Konstantin Belousov Cc: hackers@freebsd.org, Pedro Giffuni Message-ID: <87169538.4319034.1455931622630.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20151116151513.GF5854@kib.kiev.ua> References: <9BC3EFA2-945F-4C86-89F6-778873B58469@cs.huji.ac.il> <20151115152635.GB5854@kib.kiev.ua> <3AEC67FD-2E67-4EF9-9D46-818ABF3D8118@cs.huji.ac.il> <661673285.88370232.1447682409478.JavaMail.zimbra@uoguelph.ca> <20151116151513.GF5854@kib.kiev.ua> Subject: Re: kqueue of a nfs mounted file not working MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF44 (Win)/8.0.9_GA_6191) Thread-Topic: kqueue of a nfs mounted file not working Thread-Index: g4OqLGbRl2EjAuTtsQSE2bYBY3zo9w== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 01:27:35 -0000 Kostik wrote: > On Mon, Nov 16, 2015 at 09:00:09AM -0500, Rick Macklem wrote: > > Daniel Braniss wrote: > > > > > > > On 15 Nov 2015, at 17:26, Konstantin Belousov > > > > wrote: > > > > > > > > On Sun, Nov 15, 2015 at 11:22:55AM +0200, Daniel Braniss wrote: > > > >> HI, > > > >> I???m writing a program to monitor a file using kqueue(2), if the file > > > >> is > > > >> local > > > >> all is OK, but if the file is via a nfs mounted fs, it only works > > > >> once. > > > >> stat shows the file growing, but kevent is not triggered. > > > > > > > > Does file grow due to local changes on the nfs client, or some other > > > > client changes the file, while your client tries to get kevent > > > > notifications ? > > > > > > it gets updated by a host which has the file as local, so yes, it gets > > > updated > > > by another client/host. > > > > > Hmm, I am not surprised that this doesn't work. The only indication to the > > client that the file has changed on the server is a change in the file's > > attributes when they're acquired (via a Getattr RPC or similar) from the > > server. > > > > There is a vfs operation called VFS_SYSCTL(). This isn't implemented on > > the current NFS client. It was implemented on the old one, but only for > > NFS locking events and I didn't understand what needed to be done, so I > > didn't do it. > You probably mean VOP_KQFILTER, not VFS_SYSCTL(). BTW, the later is > only used by nfs and I do not quite see why its functionality not subsumed > by the mount options. > > WRT VOP_KQFILTER, the default implementation is adequate. What is missed > for NFS is the knote activation when the code notes that the cached > metadata is invalidated by server. > > > Kostik, do you know if there is a VFS_SYSCTL() call done when the kevent > > stuff is probing for a file size change? (Or does it not probe and events > > get triggered via the write syscall or ???) I took a quick look at the > > kevent > > stuff, but admit I got lost and couldn't figure out what triggered events > > being logged? > > > > Also, is the event for "file growing" or "file changed"? > > If it is the latter, all the NFS client can do is look for a change in > > the file's modify time and this is often at a resolution of 1sec., which > > implies that a change within the same second as the previous one may not > > be noticed. (NFSv4 has a Change attribute that is always guaranteed to > > change, but that is only NFSv4.) Also, you see metadata changes as well > > as data changes, at least for the NFSv4 attribute. > > Please look at the sys/kern/vfs_subr.c lines 4296-4419. There is a bunch > of the post-VOP hooks which are executed after the filesystem VOP method > was executed. You would see the complete list of the notifications which > are sent, and the way they are sent, by calling VFS_KNOTE{_LOCKED}(vp, > NOTE_XXX). Similar calls should be spread over the nfs client code > when the attribute cache entries are deleted, possibly comparing old > and new values to select proper notification. > > But you are of course right that client cannot fully implement the > notifications without the server notifying it, so whatever efforts > are done for NFSv3, there are definitely will be cases which cannot > work. I do not know NFSv4 enough to make similar statement, but I think > something would prevent the complete implementation, e.g. for rename. > > This makes me wonder should we need to add the calls to VFS_KNOTE() into > the nfs client, at all. It is similar to lockd/nolockd options, in > that the client might be not interested in the server or other clients > updates, only in the local notifications. This plus the fact that the > feature cannot have complete implementation, raises the question. > I took a quick look at this (thanks to pfg@ for pointing it out to me) and it appears that DragonFlyBSD does it local to the client (like nolockd). (I haven't looked at the actual patch, so I don't know how easy it is to port into the FreeBSD client, but I suspect it isn't too hard to do.) http://gitweb.dragonflybsd.org/dragonfly.git/commit/05c073d67e9920d45c46eed73b9390229a50ec8a So,does anyone think doing this is a good idea (kevent local to the client only) or not? Thanks in advance for your comments, rick ps: Since Daniel Braniss found a workaround for his case, I had put this on the back burner, but if others think doing it for "client only" (ie not seeing changes done by other clients), I'll see if the patch can be ported. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sat Feb 20 12:42:47 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65775AAE0AA for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4D66C9BB for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 4C8D7AAE0A9; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 323C9AAE0A8 for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CF2879BA; Sat, 20 Feb 2016 12:42:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u1KCgW8t051461 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 20 Feb 2016 14:42:32 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u1KCgW8t051461 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u1KCgWio051460; Sat, 20 Feb 2016 14:42:32 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Feb 2016 14:42:32 +0200 From: Konstantin Belousov To: Rick Macklem Cc: hackers@freebsd.org, Pedro Giffuni Subject: Re: kqueue of a nfs mounted file not working Message-ID: <20160220124232.GB91220@kib.kiev.ua> References: <9BC3EFA2-945F-4C86-89F6-778873B58469@cs.huji.ac.il> <20151115152635.GB5854@kib.kiev.ua> <3AEC67FD-2E67-4EF9-9D46-818ABF3D8118@cs.huji.ac.il> <661673285.88370232.1447682409478.JavaMail.zimbra@uoguelph.ca> <20151116151513.GF5854@kib.kiev.ua> <87169538.4319034.1455931622630.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87169538.4319034.1455931622630.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 12:42:47 -0000 On Fri, Feb 19, 2016 at 08:27:02PM -0500, Rick Macklem wrote: > Kostik wrote: > > On Mon, Nov 16, 2015 at 09:00:09AM -0500, Rick Macklem wrote: > > > Daniel Braniss wrote: > > > > > > > > > On 15 Nov 2015, at 17:26, Konstantin Belousov > > > > > wrote: > > > > > > > > > > On Sun, Nov 15, 2015 at 11:22:55AM +0200, Daniel Braniss wrote: > > > > >> HI, > > > > >> I???m writing a program to monitor a file using kqueue(2), if the file > > > > >> is > > > > >> local > > > > >> all is OK, but if the file is via a nfs mounted fs, it only works > > > > >> once. > > > > >> stat shows the file growing, but kevent is not triggered. > > > > > > > > > > Does file grow due to local changes on the nfs client, or some other > > > > > client changes the file, while your client tries to get kevent > > > > > notifications ? > > > > > > > > it gets updated by a host which has the file as local, so yes, it gets > > > > updated > > > > by another client/host. > > > > > > > Hmm, I am not surprised that this doesn't work. The only indication to the > > > client that the file has changed on the server is a change in the file's > > > attributes when they're acquired (via a Getattr RPC or similar) from the > > > server. > > > > > > There is a vfs operation called VFS_SYSCTL(). This isn't implemented on > > > the current NFS client. It was implemented on the old one, but only for > > > NFS locking events and I didn't understand what needed to be done, so I > > > didn't do it. > > You probably mean VOP_KQFILTER, not VFS_SYSCTL(). BTW, the later is > > only used by nfs and I do not quite see why its functionality not subsumed > > by the mount options. > > > > WRT VOP_KQFILTER, the default implementation is adequate. What is missed > > for NFS is the knote activation when the code notes that the cached > > metadata is invalidated by server. > > > > > Kostik, do you know if there is a VFS_SYSCTL() call done when the kevent > > > stuff is probing for a file size change? (Or does it not probe and events > > > get triggered via the write syscall or ???) I took a quick look at the > > > kevent > > > stuff, but admit I got lost and couldn't figure out what triggered events > > > being logged? > > > > > > Also, is the event for "file growing" or "file changed"? > > > If it is the latter, all the NFS client can do is look for a change in > > > the file's modify time and this is often at a resolution of 1sec., which > > > implies that a change within the same second as the previous one may not > > > be noticed. (NFSv4 has a Change attribute that is always guaranteed to > > > change, but that is only NFSv4.) Also, you see metadata changes as well > > > as data changes, at least for the NFSv4 attribute. > > > > Please look at the sys/kern/vfs_subr.c lines 4296-4419. There is a bunch > > of the post-VOP hooks which are executed after the filesystem VOP method > > was executed. You would see the complete list of the notifications which > > are sent, and the way they are sent, by calling VFS_KNOTE{_LOCKED}(vp, > > NOTE_XXX). Similar calls should be spread over the nfs client code > > when the attribute cache entries are deleted, possibly comparing old > > and new values to select proper notification. > > > > But you are of course right that client cannot fully implement the > > notifications without the server notifying it, so whatever efforts > > are done for NFSv3, there are definitely will be cases which cannot > > work. I do not know NFSv4 enough to make similar statement, but I think > > something would prevent the complete implementation, e.g. for rename. > > > > This makes me wonder should we need to add the calls to VFS_KNOTE() into > > the nfs client, at all. It is similar to lockd/nolockd options, in > > that the client might be not interested in the server or other clients > > updates, only in the local notifications. This plus the fact that the > > feature cannot have complete implementation, raises the question. > > > I took a quick look at this (thanks to pfg@ for pointing it out to me) > and it appears that DragonFlyBSD does it local to the client (like nolockd). > (I haven't looked at the actual patch, so I don't know how easy it is to port > into the FreeBSD client, but I suspect it isn't too hard to do.) > > http://gitweb.dragonflybsd.org/dragonfly.git/commit/05c073d67e9920d45c46eed73b9390229a50ec8a > > So,does anyone think doing this is a good idea (kevent local to the client only) > or not? Client-local kevents should already work on NFS as-is, by the way of VFS plumbs the VOPs and activates events on file operations. E.g. tail -f and -F work on NFS as is. What was asked for, from my memory, is the feature where client noticing server changes to the file, also caused kevent triggering. This is why I talked about NFS node invalidation triggering kevents. > > Thanks in advance for your comments, rick > ps: Since Daniel Braniss found a workaround for his case, I had put this on the > back burner, but if others think doing it for "client only" (ie not seeing > changes done by other clients), I'll see if the patch can be ported. > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > >