From owner-freebsd-hackers@freebsd.org Tue Jun 28 02:21:31 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15F98B84E94 for ; Tue, 28 Jun 2016 02:21:31 +0000 (UTC) (envelope-from yonas@fizk.net) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE38D2154 for ; Tue, 28 Jun 2016 02:21:30 +0000 (UTC) (envelope-from yonas@fizk.net) Received: by mail-it0-x236.google.com with SMTP id f6so4626522ith.0 for ; Mon, 27 Jun 2016 19:21:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fizk.net; s=google; h=to:from:subject:message-id:date:user-agent:mime-version; bh=w/+znUr0hiD3HtsJVC6KTctaZsXiaibkZYnC0RvXJko=; b=XZFNefitKFGKVYGEJ7qiz4fBFad6N50dvO/guJXq1TMafe02XN2M+2+U1rfa+eHFMs Oj3VIxBIDRO6ag6lZCpy0IYfwIaqo9C9KV/sjeaP+NPZbYMvyomr7vh4G0wjZ9R399KF XjAIbhj7KGIGTpTo8sK0R0aXgeCASWP8YnM/w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version; bh=w/+znUr0hiD3HtsJVC6KTctaZsXiaibkZYnC0RvXJko=; b=chikn6f8fMCyTTDJEV4hq+4M8yQPHjZsk1BlkuRdO4Pu6qJHS5xj4dAJS/V2r5FPAK FdME3mows0uUw7xcN/17+v+7MceBL3LC/jPjU7hIkoFE/gkY/+asUntOsF3ECMo5mOpn 5fJbTvkMdz8O0Xgo++Mtgy7dzlhL7iXR2QCeqFYqyuqNI6tJw0fI2nOCiHFfaFXhYZBp 1sPHYh9Sep+I4xBat161mxj81S2ybLXvEjy0L3dlW8esJdtlRK/tHIMUXLJTXcM/qPY3 AGludn0A6W9QFYl+mQMviTivjjsdY3aA9UzdwGpHpBdDCf01umj68cBcI+VDoOLDGNjf pN3w== X-Gm-Message-State: ALyK8tLayKxLJIjb4fkU0RqI0aveSwtkzY+8O5/yrk9T84whTlHpvcsxRa4NIyMDvbPWKg== X-Received: by 10.36.184.130 with SMTP id m124mr12135228ite.95.1467080490019; Mon, 27 Jun 2016 19:21:30 -0700 (PDT) Received: from [192.168.2.200] (CPEf0f2494a5cf3-CMf0f2494a5cf0.cpe.net.cable.rogers.com. [99.236.80.4]) by smtp.gmail.com with ESMTPSA id o133sm5844473itg.5.2016.06.27.19.21.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Jun 2016 19:21:29 -0700 (PDT) To: freebsd-hackers@freebsd.org From: Yonas Yanfa Subject: Enabling debugging symbols Message-ID: <0870e8e1-c425-35f6-e51b-3f90ad2923a5@fizk.net> Date: Mon, 27 Jun 2016 22:21:28 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 02:21:31 -0000 Hi, How can I recompile security/sudo and /usr/bin/su with debugging symbols enabled? I tried: env DEBUG_FLAGS=3D'-O0 -ggdb3' make DEBUG_FLAGS=3D'-O0 -ggdb3' but that didn't work. --=20 Yonas Yanfa In Love With Open Source Drupal :: GitHub :: Mozilla :: iPhone fizk.net | yonas@fizk.net From owner-freebsd-hackers@freebsd.org Tue Jun 28 13:05:08 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B869FB85020; Tue, 28 Jun 2016 13:05:08 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A5362D4E; Tue, 28 Jun 2016 13:05:08 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-236-103.lns20.per1.internode.on.net [121.45.236.103]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u5SD4pkp024895 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 28 Jun 2016 06:04:54 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: The small installations network filesystem and users. To: =?UTF-8?Q?Gerrit_K=c3=bchn?= , Daniel Eischen References: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> <20160621075631.38c2eeaa7c224aa22ea9be4d@aei.mpg.de> Cc: freebsd-fs , FreeBSD Hackers From: Julian Elischer Message-ID: <761f82d3-ebe9-2cba-9499-049dafbc98df@freebsd.org> Date: Tue, 28 Jun 2016 21:04:45 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160621075631.38c2eeaa7c224aa22ea9be4d@aei.mpg.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 13:05:08 -0000 On 21/06/2016 1:56 PM, Gerrit Kühn wrote: > On Mon, 20 Jun 2016 22:00:31 -0400 (EDT) Daniel Eischen > wrote about Re: The small installations network > filesystem and users.: > > DE> We should support LDAP client out of the box, in base. What > DE> sucks now is that we need 3 packages (plus their dependencies) > DE> and multiple config files for ldap: > DE> > DE> pam_ldap > DE> nss_ldap > DE> openldap-client > > I only have to install/config ldap-clients every now and then, but I would > also strongly favour a more "integrated" setup (if that requires having it > in base is a different question, though). A few weeks ago I used > nss-pam-ldapd instead of pam_ldap and nss_ldap for the first time, and it > appeared to work with a bit less of a hassle for me (otoh, I don't do any > funky things here, I just need a replacement for what we did with NIS > something like 20 years ago). +1 I just had to reinstall certs for my server. which means copying certs to several places (in a default config) sendmail and syrus ad openssl (base) all look in different places. you COULD make them all look in the same place but that requires undersanding what is going on and not just cribbing the config file off the net somewhere. I think ports and pkg are fine, but we need to have some more thought put into how they all go together. > > > cu > Gerrit > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Tue Jun 28 15:23:33 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC989B85AED; Tue, 28 Jun 2016 15:23:33 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B02B2EE0; Tue, 28 Jun 2016 15:23:32 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.14.9) with ESMTPS id u5SFNTB1024866 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 28 Jun 2016 17:23:29 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id u5SFNRX6008891; Tue, 28 Jun 2016 17:23:27 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id u5SFNM4q008888; Tue, 28 Jun 2016 17:23:22 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Tue, 28 Jun 2016 17:23:22 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Chris Watson cc: Zaphod Beeblebrox , freebsd-fs , FreeBSD Hackers Subject: Re: The small installations network filesystem and users. In-Reply-To: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> Message-ID: References: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Tue, 28 Jun 2016 17:23:30 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 15:23:33 -0000 On Mon, 20 Jun 2016, Chris Watson wrote: > I'm glad you brought this up. I wanted to but I've heard it before on the lists and realize that there is this disconnect between the developers doing the actual work to implement these things and the end users. > > I have always been very grateful to all the developers who over the years, and I've been a FreeBSD consumer since the late 90s? And attended my first usenix/freenix conf in Monterrey in 2001?, have done some really hard work on many many things in FreeBSD. For zero pay. But the thing that has always bothered me about a lot of it is, it's just to complex to use for most end users. Not all. But people want to get work done. Sifting through .conf files, googling howtos, spending more time configuring it than installing it has always been an issue. Developers in general do not think like an end user. And this leads to non developers just going "screw it I'll just get it running on Linux with my GUI installer." Which is why FreeNas is so popular. It's taken a lot, not all, but a lot of the pain and time consuming nature of learning all the ins and outs of a NAS appliance from the equation. FreeBSD have very well done man pages. I simply read them. From owner-freebsd-hackers@freebsd.org Tue Jun 28 15:24:23 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D0D8B85BA4; Tue, 28 Jun 2016 15:24:23 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 116652070; Tue, 28 Jun 2016 15:24:22 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.14.9) with ESMTPS id u5SFMe0R024841 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 28 Jun 2016 17:22:41 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id u5SFMcRj008885; Tue, 28 Jun 2016 17:22:38 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id u5SFMXSO008882; Tue, 28 Jun 2016 17:22:33 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Tue, 28 Jun 2016 17:22:33 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Zaphod Beeblebrox cc: freebsd-fs , FreeBSD Hackers Subject: Re: The small installations network filesystem and users. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Tue, 28 Jun 2016 17:22:41 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 15:24:23 -0000 > Correct me if I'm wrong, but amidst discussions of pNFS (among other > things) I thought I should bring up something someone said to me: and that > is (to quote him) "using NFS is too hard, I always fail." Never had problems with NFS. And i'm using in regularly. From owner-freebsd-hackers@freebsd.org Tue Jun 28 16:11:45 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B28ADB82E9C for ; Tue, 28 Jun 2016 16:11:45 +0000 (UTC) (envelope-from emorrasg@yahoo.es) Received: from nm38.bullet.mail.ir2.yahoo.com (nm38.bullet.mail.ir2.yahoo.com [212.82.96.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 123DF2123 for ; Tue, 28 Jun 2016 16:11:44 +0000 (UTC) (envelope-from emorrasg@yahoo.es) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.es; s=s2048; t=1467130296; bh=BhS4VaHyTCA5Tk20SZ4Z57/Gqj+pZnmSuKrhQZNqAIw=; h=Date:From:To:Subject:In-Reply-To:References:From:Subject; b=divC8s22iafYN1eUKakPnAGRFfpgQxIPxazupM04qNfca05nLOQ64FXer4aQIXHcUuxXMTMLCUSC75pf6jhp2BZomysCrNcjVx3AdrrEyjUdJ1BoBG/GIUz8KNY/l757bXwBOWHg1A9VrYUH2OqjcvfRBMSIuTUH/OgUUti2JX7Wpy917PjQISVgybmi/2fRBI6vVVpH0lCIeY9IahJpfvY+uLON6xzpRTx80rw0KFxQIDlU8XRfEeEpBHGK1iPaF4dtSEYZlZryBLS/Mk8NncYlQ7oz8W4R0lrGqJW96GAMdIM5PmxarCtl5WUhvgvwakV7Uj/9lK1x2tm/jcio8A== Received: from [212.82.98.59] by nm38.bullet.mail.ir2.yahoo.com with NNFMP; 28 Jun 2016 16:11:36 -0000 Received: from [46.228.39.105] by tm12.bullet.mail.ir2.yahoo.com with NNFMP; 28 Jun 2016 16:11:35 -0000 Received: from [127.0.0.1] by smtp142.mail.ir2.yahoo.com with NNFMP; 28 Jun 2016 16:11:35 -0000 X-Yahoo-Newman-Id: 980215.66211.bm@smtp142.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: Zlyrj5EVM1nEdprW.mYsSK2ykamyfChQrrisyZWLhdJyA6h mKoE.DoHO0bJhdmreXQ.77xGDkwh2_jBEYBKGSSLbWj.McQ1FSUUItmhW3Z6 AOyOtjsRTlDZar.Ep.TQi0DuStcfijJ9Jfc6y_QcDza84HtnhiB1q6grlomi KrNUB.TuK8C_YQqoOI_7qKIacJ71K7FrmBHLthwPk_wJSbGw819eYPL0TwW2 Fp2ZUGAf0FwhOY0rtMdgnkeA1OuLnQbTyZ13rgDTvowKM1wgvh0CNeGA6XKP Hk3_ReTbHA2DEvyqsSmNhFstIr2WzwSi7SPXq6cfU_dYcXwTXTMnHSNYjeNQ IWn.eqSoaTMFWzLFUFI5eo_Wz8CflcGSNyzgVWhJP3kItcLK_Jp173iYdHUF XmRfZyb3K7Hdlrd.1Mv2txZBdxDls50aOYYGjWYjrodNlx7txoyJhST2.uJb qyd5z0FDzXfe71SRfgPJjEp9Xag9hoXtbLmsOy.4RZElKvaRQsFWhQf_qeOS wIey9OQXHmHqf0UpBzGpKHblf X-Yahoo-SMTP: mX392iiswBAeJNdO_s.EW62LZDJR Date: Tue, 28 Jun 2016 18:11:40 +0200 From: Eduardo Morras To: freebsd-hackers@freebsd.org Subject: Re: The small installations network filesystem and users. Message-Id: <20160628181140.933d144cd5d830275e4be6c3@yahoo.es> In-Reply-To: <761f82d3-ebe9-2cba-9499-049dafbc98df@freebsd.org> References: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> <20160621075631.38c2eeaa7c224aa22ea9be4d@aei.mpg.de> <761f82d3-ebe9-2cba-9499-049dafbc98df@freebsd.org> X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.29; amd64-portbld-freebsd10.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 16:11:45 -0000 On Tue, 28 Jun 2016 21:04:45 +0800 Julian Elischer wrote: > On 21/06/2016 1:56 PM, Gerrit K=FChn wrote: > > On Mon, 20 Jun 2016 22:00:31 -0400 (EDT) Daniel Eischen > > wrote about Re: The small installations > > network filesystem and users.: > > > > DE> We should support LDAP client out of the box, in base. What > > DE> sucks now is that we need 3 packages (plus their dependencies) > > DE> and multiple config files for ldap: > > DE> > > DE> pam_ldap > > DE> nss_ldap > > DE> openldap-client > > > > I only have to install/config ldap-clients every now and then, but > > I would also strongly favour a more "integrated" setup (if that > > requires having it in base is a different question, though). A few > > weeks ago I used nss-pam-ldapd instead of pam_ldap and nss_ldap for > > the first time, and it appeared to work with a bit less of a hassle > > for me (otoh, I don't do any funky things here, I just need a > > replacement for what we did with NIS something like 20 years ago). >=20 > +1 > I just had to reinstall certs for my server. which means copying=20 > certs to several places (in a default config) > sendmail and syrus ad openssl (base) all look in different places. > you COULD make them all look in the same place > but that requires undersanding what is going on and not just cribbing=20 > the config file off the net somewhere. I use dhcpd to pass that configuration. On system startup, dhclient asks to dhcpd server the ip, time-ntp, dns, and configuration for its current job (pkgs/ports to install, apache conf, postgres conf, certs, etc.= .) depending on it's intended current use. I followed an old paper from EuroBSDCon,... this http://2004.eurobsdcon.org/uploads/media/EBSD04_slides_41.pdf to do the setup. Easier and faster (at least for me) than ldap and related for server setup. For user management, don't know, I don't have jelly users, only daemons. >=20 > I think ports and pkg are fine, but we need to have some more thought=20 > put into how they all go together. --- --- Eduardo Morras From owner-freebsd-hackers@freebsd.org Wed Jun 29 19:40:26 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C02FB86E5D for ; Wed, 29 Jun 2016 19:40:26 +0000 (UTC) (envelope-from shettyrn22@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D15B22F58 for ; Wed, 29 Jun 2016 19:40:25 +0000 (UTC) (envelope-from shettyrn22@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id 187so293168wmz.1 for ; Wed, 29 Jun 2016 12:40:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=guOYPu3oDHUHdGIJo0+aXPJLdvH0OhBZVo5WaTQ+I6M=; b=rpSb5M/K70MuBxCHCxGR6wUJhH6FpYXjm9lqQod2ZoMs0b3iqpIXEsrY5KLJ71/iBg Z/ls6xoKuxeK2x319FipjqpxMYIs93wDm0N7e59+z0mRdgUgLt+xCOb1biElhbtqvZSp wFv2GjpBT8ewNezSluBTRaJ442risgmHUXKGU4i5c9HXF2Nd2f7xPeyq+w+eKjKF4bhX JVjfHfiOBdfvtNasHjvCqU1e9GRZ7+xReieP5JBDVHi/Tok07aRaePLcII0eClRxQxQy s9XYnTRH6KWWnYGoMxeN6ueWWmJpmxbcRe8YK0yPpB6FXa5O5Zc3hlgpHHKZNjOz6Vqf 5DTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=guOYPu3oDHUHdGIJo0+aXPJLdvH0OhBZVo5WaTQ+I6M=; b=hPtdr8dFLHX/muIXduDFR7Kdm/7n5v2fgk3+cqy6sFbGscBuAs/AylMHqDjfot8l8t j+j5ooFNiYvAD98fA4ZLNIqtadPB6alrhcV90hVoOxX0DTVq07Y642u5EgCqosA391ez A4ETHKxqB5dPQj5Xb+Tw5FWopwuuahQJZk21jvz8qlk9xyh4S7yh7UsHcItwjX0yTDxi AHt8U7qR1FJ04tA8fPPU4alt6gjZ02DwIVhYBf+H62kqHUfQemfJEJ4J/qXedbMd9mj8 wz9O+/h50mbz5YkurM/xRyvKUbk5Oi0seFHy251nOaF1mcqoLCnbKo6m+vJFt6v2fWsj DBXg== X-Gm-Message-State: ALyK8tJp/A2zQqL3kQepgLIjwx4zBm1VPDPabWXn+YHqfOZExEBDxPzVw9C0SSSWEbngpNTBWT+Ijz6WEeweVw== X-Received: by 10.28.17.132 with SMTP id 126mr23556680wmr.90.1467229223132; Wed, 29 Jun 2016 12:40:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.88.129 with HTTP; Wed, 29 Jun 2016 12:39:43 -0700 (PDT) From: Rajneesh Shetty Date: Thu, 30 Jun 2016 01:09:43 +0530 Message-ID: Subject: Microsoft Paint & NXP Semiconductors To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2016 19:40:26 -0000 *To whom it may concern* *I found your group when one of your online subscribers had a problem in the BSD Group. Although I did not react to his protests, I would like you to know, that I am sympathetic to his concerns. I am however unable to locate his email ID.* *The purpose of this email is to let you know that I would like to assist your GROUP with two software programs written by myself with the assistance of my friends from University between 1989-1995. The "i squared C", bus architecture is a proprietary architecture to NXP Semiconductors, an offshoot of Phillips, Holland. We wrote a non-volatile program for DC Loads and a Microsoft solution called "PC Paint=E2=80=9D. * *Trust your clients & team mates have not been disappointed by our efforts. I have no hesitation in being grateful to Microsoft Corporation, Intel Corporation and AMD (Radeon) after all these years of their incessant efforts to keep these technologies alive. * *Any suggestions you may have with regards to contributing to efforts already made would be appreciated here.* *Regards,* *Rajneesh * Tel : (+91) 8711 9653 19 http://www.i2c-bus.org/i2c-bus/ http://www.telos.info/download/i2c-studioframework/ Weblog : www.rns-thoughts.blogspot.com On 29 June 2016 at 17:30, wrote: > Send freebsd-hackers mailing list submissions to > freebsd-hackers@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > or, via email, send a message with subject or body 'help' to > freebsd-hackers-request@freebsd.org > > You can reach the person managing the list at > freebsd-hackers-owner@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-hackers digest..." > > > Today's Topics: > > 1. Re: The small installations network filesystem and users. > (Julian Elischer) > 2. Re: The small installations network filesystem and users. > (Wojciech Puchar) > 3. Re: The small installations network filesystem and users. > (Wojciech Puchar) > 4. Re: The small installations network filesystem and users. > (Eduardo Morras) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 28 Jun 2016 21:04:45 +0800 > From: Julian Elischer > To: Gerrit K?hn , Daniel Eischen > > Cc: freebsd-fs , FreeBSD Hackers > > Subject: Re: The small installations network filesystem and users. > Message-ID: <761f82d3-ebe9-2cba-9499-049dafbc98df@freebsd.org> > Content-Type: text/plain; charset=3Dwindows-1252; format=3Dflowed > > On 21/06/2016 1:56 PM, Gerrit K?hn wrote: > > On Mon, 20 Jun 2016 22:00:31 -0400 (EDT) Daniel Eischen > > wrote about Re: The small installations network > > filesystem and users.: > > > > DE> We should support LDAP client out of the box, in base. What > > DE> sucks now is that we need 3 packages (plus their dependencies) > > DE> and multiple config files for ldap: > > DE> > > DE> pam_ldap > > DE> nss_ldap > > DE> openldap-client > > > > I only have to install/config ldap-clients every now and then, but I > would > > also strongly favour a more "integrated" setup (if that requires having > it > > in base is a different question, though). A few weeks ago I used > > nss-pam-ldapd instead of pam_ldap and nss_ldap for the first time, and = it > > appeared to work with a bit less of a hassle for me (otoh, I don't do a= ny > > funky things here, I just need a replacement for what we did with NIS > > something like 20 years ago). > > +1 > I just had to reinstall certs for my server. which means copying > certs to several places (in a default config) > sendmail and syrus ad openssl (base) all look in different places. you > COULD make them all look in the same place > but that requires undersanding what is going on and not just cribbing > the config file off the net somewhere. > > I think ports and pkg are fine, but we need to have some more thought > put into how they all go together. > > > > > > > cu > > Gerrit > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > > > ------------------------------ > > Message: 2 > Date: Tue, 28 Jun 2016 17:23:22 +0200 (CEST) > From: Wojciech Puchar > To: Chris Watson > Cc: Zaphod Beeblebrox , freebsd-fs > , FreeBSD Hackers > > Subject: Re: The small installations network filesystem and users. > Message-ID: > Content-Type: text/plain; charset=3DUS-ASCII; format=3Dflowed > > > > On Mon, 20 Jun 2016, Chris Watson wrote: > > > I'm glad you brought this up. I wanted to but I've heard it before on > the lists and realize that there is this disconnect between the developer= s > doing the actual work to implement these things and the end users. > > > > I have always been very grateful to all the developers who over the > years, and I've been a FreeBSD consumer since the late 90s? And attended = my > first usenix/freenix conf in Monterrey in 2001?, have done some really ha= rd > work on many many things in FreeBSD. For zero pay. But the thing that has > always bothered me about a lot of it is, it's just to complex to use for > most end users. Not all. But people want to get work done. Sifting throug= h > .conf files, googling howtos, spending more time configuring it than > installing it has always been an issue. Developers in general do not thin= k > like an end user. And this leads to non developers just going "screw it > I'll just get it running on Linux with my GUI installer." Which is why > FreeNas is so popular. It's taken a lot, not all, but a lot of the pain a= nd > time consuming nature of learning all the ins and outs of a NAS appliance > from the equation. > > FreeBSD have very well done man pages. I simply read them. > > > ------------------------------ > > Message: 3 > Date: Tue, 28 Jun 2016 17:22:33 +0200 (CEST) > From: Wojciech Puchar > To: Zaphod Beeblebrox > Cc: freebsd-fs , FreeBSD Hackers > > Subject: Re: The small installations network filesystem and users. > Message-ID: > Content-Type: text/plain; charset=3DUS-ASCII; format=3Dflowed > > > Correct me if I'm wrong, but amidst discussions of pNFS (among other > > things) I thought I should bring up something someone said to me: and > that > > is (to quote him) "using NFS is too hard, I always fail." > > Never had problems with NFS. And i'm using in regularly. > > > > ------------------------------ > > Message: 4 > Date: Tue, 28 Jun 2016 18:11:40 +0200 > From: Eduardo Morras > To: freebsd-hackers@freebsd.org > Subject: Re: The small installations network filesystem and users. > Message-ID: <20160628181140.933d144cd5d830275e4be6c3@yahoo.es> > Content-Type: text/plain; charset=3DISO-8859-1 > > On Tue, 28 Jun 2016 21:04:45 +0800 > Julian Elischer wrote: > > > On 21/06/2016 1:56 PM, Gerrit K?hn wrote: > > > On Mon, 20 Jun 2016 22:00:31 -0400 (EDT) Daniel Eischen > > > wrote about Re: The small installations > > > network filesystem and users.: > > > > > > DE> We should support LDAP client out of the box, in base. What > > > DE> sucks now is that we need 3 packages (plus their dependencies) > > > DE> and multiple config files for ldap: > > > DE> > > > DE> pam_ldap > > > DE> nss_ldap > > > DE> openldap-client > > > > > > I only have to install/config ldap-clients every now and then, but > > > I would also strongly favour a more "integrated" setup (if that > > > requires having it in base is a different question, though). A few > > > weeks ago I used nss-pam-ldapd instead of pam_ldap and nss_ldap for > > > the first time, and it appeared to work with a bit less of a hassle > > > for me (otoh, I don't do any funky things here, I just need a > > > replacement for what we did with NIS something like 20 years ago). > > > > +1 > > I just had to reinstall certs for my server. which means copying > > certs to several places (in a default config) > > sendmail and syrus ad openssl (base) all look in different places. > > you COULD make them all look in the same place > > but that requires undersanding what is going on and not just cribbing > > the config file off the net somewhere. > > I use dhcpd to pass that configuration. On system startup, dhclient > asks to dhcpd server the ip, time-ntp, dns, and configuration for its > current job (pkgs/ports to install, apache conf, postgres conf, certs, > etc..) > depending on it's intended current use. I followed an old paper from > EuroBSDCon,... this > http://2004.eurobsdcon.org/uploads/media/EBSD04_slides_41.pdf to do the > setup. Easier and faster (at least for me) than ldap and related for > server setup. For user management, don't know, I don't have jelly > users, only daemons. > > > > > I think ports and pkg are fine, but we need to have some more thought > > put into how they all go together. > > > > --- --- > Eduardo Morras > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > > ------------------------------ > > End of freebsd-hackers Digest, Vol 689, Issue 2 > *********************************************** > From owner-freebsd-hackers@freebsd.org Thu Jun 30 04:00:10 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DCDFB8755F; Thu, 30 Jun 2016 04:00:10 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 12FF82882; Thu, 30 Jun 2016 04:00:09 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 49EC0346DED6; Wed, 29 Jun 2016 21:00:09 -0700 (PDT) To: FreeBSD Hackers , python@freebsd.org From: Alfred Perlstein Subject: bootloader in python. Organization: FreeBSD Message-ID: <6d6e6c7d-6d0d-3316-972f-529cfe216a42@freebsd.org> Date: Wed, 29 Jun 2016 21:00:12 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 04:00:10 -0000 Josh Triplett started out with "the punchline" for his PyCon 2015 talk on porting Python to run without an operating system: he and his Intel colleagues got the interpreter to run in the GRUB boot loader for either BIOS or EFI systems. But that didn't spoil the rest of the talk by any means. He had plenty of interesting things to say and a number of eye-opening demos to show as well. https://lwn.net/Articles/641244/ https://lwn.net/SubscriberLink/692638/2ebf68539c678a33/ Enjoy! -Alfred From owner-freebsd-hackers@freebsd.org Thu Jun 30 04:06:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E52C4B878D0 for ; Thu, 30 Jun 2016 04:06:34 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B837B2E8C for ; Thu, 30 Jun 2016 04:06:34 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: by mail-pa0-x22a.google.com with SMTP id hl6so24016155pac.2 for ; Wed, 29 Jun 2016 21:06:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version :content-transfer-encoding; bh=Uu73p51+jpTA3TritXQi/VMilDXJqNMrnzwRMVm9nUs=; b=vymyVdiEtTSJ3uI59tredPsq9Si9SNLVSfGEqpSyyNp6XPUI9aOaOfG/OfrTLtUxoT JJtjlCzuUdXYQ9MMT5Jhys9ihAUKE1+aFeyaVoYBWM7iOT5JslKyPk4Ye44QBFeAJBgA tn6h2HOh5I3sruAJXiakEcmfTTpi0z3VI3lBXQgfLbiuXBP+SSyOZa1UPu9wU0HBdWlm K9ROdqfzuvScKjUOTvyTSFDy7VFow+8ERQLiri7sJ2RpirlU8c0DTeGwP8CCZy45aBIa 3yh/xCM+evdVuaNEBptryAsWyaEfa0dRPH0EhLgISMyQ77ZumfmBsXTZd0EHRne3s39j e78w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-transfer-encoding; bh=Uu73p51+jpTA3TritXQi/VMilDXJqNMrnzwRMVm9nUs=; b=acSv7BTIg8YR/Z0vKD6rHIhTTM715YztpL6+q6MiVvdQJO7p3yTOZmH9q8pnpO5jHu 2orTJlJgjr6XyKSBubBW0SqKP+PDCFhlrqWb9xERCtdrf1ByPWzL8Xgu3yH6Q70HQpUh vk2AZA5aenn5XOCaRH01UCs6QKNRGaNL1fnZ8LMa70I5WBFSGt+3BhQEwbvN0zN1ETXG a3MKmNKDig5pfaUH0In5za5bW/eusZKmp2B6Lj7kCPDY9vkkWMLKCV5xtWzm7MnTDrkY iFBj0/B5k5szX55uFJsQ7xHKO2efiIbKrvzYVgINlvORxoSB0jjR06C7zDpJcJv0r92h W1kg== X-Gm-Message-State: ALyK8tLnzL+fCG1pt4uBhMqpOdje7fp2c4bXLB2tOeyp6u9FzDGjuKKsLLvReUxYa4oQHA== X-Received: by 10.66.189.225 with SMTP id gl1mr18026557pac.158.1467259593800; Wed, 29 Jun 2016 21:06:33 -0700 (PDT) Received: from splash.akips.com (CPE-120-146-191-2.static.qld.bigpond.net.au. [120.146.191.2]) by smtp.gmail.com with ESMTPSA id v62sm1366627pfv.50.2016.06.29.21.06.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Jun 2016 21:06:33 -0700 (PDT) Date: Thu, 30 Jun 2016 14:06:25 +1000 From: Paul Koch To: freebsd-hackers@freebsd.org Subject: ZFS ARC and mmap/page cache coherency question Message-ID: <20160630140625.3b4aece3@splash.akips.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 04:06:35 -0000 Posted this to -stable on the 15th June, but no feedback... We are trying to understand a performance issue when syncing large mmap'ed files on ZFS. Example test box setup: FreeBSD 10.3-p5 Intel i7-5820K 3.30GHz with 64G RAM 6 * 2 Tbyte Seagate ST2000DM001-1ER164 in a ZFS stripe Read performance of a sequentially written large file on the pool is typically around 950Mbytes/sec using dd. Our software mmap's some large database files using MAP_NOSYNC, and we call fsync() every 10 minutes when we know the file system is mostly idle. In our test setup, the database files are 1.1G, 2G, 1.4G, 12G, 4.7G and ~20 small files (under 10M). All of the memory pages in the mmap'ed files are updated every minute with new values, so the entire mmap'ed file needs to be synced to disk, not just fragments. When the 10 minute fsync() occurs, gstat typically shows very little disk reads and very high write speeds, which is what we expect. But, every 80 minutes we process the data in the large mmap'ed files and store it in highly compressed blocks of a ~300G file using pread/pwrite (i.e. not mmap'ed). After that, the performance of the next fsync() of the mmap'ed files falls off a cliff. We are assuming it is because the ARC has thrown away the cached data of the mmap'ed files. gstat shows lots of read/write contention and lots of things tend to stall waiting for disk. Is this just a lack of ZFS ARC and page cache coherency ?? Is there a way to prime the ARC with the mmap'ed files again before we call fsync() ? We've tried cat and read() on the mmap'ed files but doesn't seem to touch the disk at all and the fsync() performance is still poor, so it looks like the ARC is not being filled. msync() doesn't seem to be much different. mincore() stats show the mmap'ed data is entirely incore and referenced. Paul. From owner-freebsd-hackers@freebsd.org Thu Jun 30 06:14:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEDC5B876E0 for ; Thu, 30 Jun 2016 06:14:52 +0000 (UTC) (envelope-from andrewbates09@gmail.com) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B07C252F for ; Thu, 30 Jun 2016 06:14:52 +0000 (UTC) (envelope-from andrewbates09@gmail.com) Received: by mail-qk0-x231.google.com with SMTP id a125so128459221qkc.2 for ; Wed, 29 Jun 2016 23:14:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=QGmhrXJ1jmKaZKqfJtWycwTRHjwcL1rvUWToSgF7sEU=; b=ly+SznhbJyMFh/Uc08okaID+cV9gmRpSngddXL+G6ZhZmNrR+MuHZK+SCjJLAIKVfY MWjINOzlzWu0MB/zflH5RHc1ApeiPpf5gnP8xNMx/eL9xkn1ItnPZS5jODPj2bKYeH+n T2G3b5DY0Ilm97WBsUDVursB9hHND5wiM798+qnCds3JFDpvWlnsoifkBu8vcjlx5oxV DaTeokJ7Kpr4P2FeI69/3E7+05EWxiYy5+MNxdWFjiv2aga4VhXMK5Jnn3Zeo2j2CaJo uaSyqqoy6n4XLOLigDZiBFs7/yRdR9ApQhQTSMm7bWu25bFsmVgz562GJGhA5IbuLQDF g4gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=QGmhrXJ1jmKaZKqfJtWycwTRHjwcL1rvUWToSgF7sEU=; b=Rx+UTHoFyKlQvTylueyTr2+gfIeHt4SKzV7obp+zOhuO+jG/f3e2iO7nfkUYG/Ya37 WxY6JhxBEE2w1swbJShAxkrQa5Rny3uh0VZLbxyCsGHSZK+lQniQbqlmO5bVZDE4JOdX nG2t8z8MLMIIvCQnZCNstJAiqS8AWNE8oaqn65MXE1+HURKX1pCTMEM8pXtuIyFzvehZ PZ3+rFB9zzbCxlJlPHOo5oeW11Tpqogj4rlzPFWSPj4/Fz0uXh9lSXhqYqkLfjDHaqxY syBmdC7E4twLyfzsmG0q8OVwvb2FBdtYQ2rtnZ8ykuofWVn9+90aSdxM15Waf9Bsdh3Q Izzg== X-Gm-Message-State: ALyK8tLCE/4rlvFR8kQRecEcpzKgCVJz05mgvLwSxLUlZuajXTK0ecBzTAti8Q2aB4AM98maaknrjWufKenRng== X-Received: by 10.55.195.75 with SMTP id a72mr17123264qkj.4.1467267291650; Wed, 29 Jun 2016 23:14:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.233.222.4 with HTTP; Wed, 29 Jun 2016 23:14:51 -0700 (PDT) In-Reply-To: <20160630140625.3b4aece3@splash.akips.com> References: <20160630140625.3b4aece3@splash.akips.com> From: Andrew Bates Date: Wed, 29 Jun 2016 23:14:51 -0700 Message-ID: Subject: Re: ZFS ARC and mmap/page cache coherency question To: Paul Koch Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 06:14:52 -0000 Heya Paul, How is your ZFS configured ( zfs get all tank0 )? These certainly aren't absolute, law, or perfect - but if you haven't yet, I suggest you take a peek at the following: * http://open-zfs.org/wiki/Performance_tuning * https://www.joyent.com/blog/bruning-questions-zfs-record-size * http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide On Wed, Jun 29, 2016 at 9:06 PM, Paul Koch wrote: > > Posted this to -stable on the 15th June, but no feedback... > > We are trying to understand a performance issue when syncing large mmap'ed > files on ZFS. > > Example test box setup: > FreeBSD 10.3-p5 > Intel i7-5820K 3.30GHz with 64G RAM > 6 * 2 Tbyte Seagate ST2000DM001-1ER164 in a ZFS stripe > > Read performance of a sequentially written large file on the pool is > typically around 950Mbytes/sec using dd. > > Our software mmap's some large database files using MAP_NOSYNC, and we call > fsync() every 10 minutes when we know the file system is mostly idle. In > our test setup, the database files are 1.1G, 2G, 1.4G, 12G, 4.7G and ~20 > small files (under 10M). All of the memory pages in the mmap'ed files are > updated every minute with new values, so the entire mmap'ed file needs to > be > synced to disk, not just fragments. > > When the 10 minute fsync() occurs, gstat typically shows very little disk > reads and very high write speeds, which is what we expect. But, every 80 > minutes we process the data in the large mmap'ed files and store it in > highly > compressed blocks of a ~300G file using pread/pwrite (i.e. not mmap'ed). > After that, the performance of the next fsync() of the mmap'ed files falls > off a cliff. We are assuming it is because the ARC has thrown away the > cached data of the mmap'ed files. gstat shows lots of read/write > contention > and lots of things tend to stall waiting for disk. > > Is this just a lack of ZFS ARC and page cache coherency ?? > > Is there a way to prime the ARC with the mmap'ed files again before we call > fsync() ? > > We've tried cat and read() on the mmap'ed files but doesn't seem to touch > the > disk at all and the fsync() performance is still poor, so it looks like the > ARC is not being filled. msync() doesn't seem to be much different. > mincore() stats show the mmap'ed data is entirely incore and referenced. > > Paul. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- V/Respectfully, Andrew M Bates From owner-freebsd-hackers@freebsd.org Thu Jun 30 06:50:23 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EABC9B87F86 for ; Thu, 30 Jun 2016 06:50:23 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99318253E for ; Thu, 30 Jun 2016 06:50:23 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1bIVna-00039A-Ix for freebsd-hackers@freebsd.org; Thu, 30 Jun 2016 08:50:06 +0200 Received: from ip184-189-249-34.sb.sd.cox.net ([184.189.249.34]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Jun 2016 08:50:06 +0200 Received: from julian by ip184-189-249-34.sb.sd.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 30 Jun 2016 08:50:06 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Julian Hsiao Subject: ggatel(8) extension for binding multiple files Date: Wed, 29 Jun 2016 23:34:58 -0700 Lines: 795 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ip184-189-249-34.sb.sd.cox.net User-Agent: Unison/2.1.10 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 06:50:24 -0000 Hi, I've been working on extending ggatel(8) to support binding multiple files to one device, similar to how sparse bundles work on OS X. You could simulate the functionality with ggatel(8) / md(4) + gconcat(8), but this scales poorly when you have 10^5 files. I've got a working prototype that passes some simple tests I wrote. I also tried using it as backing store for UFS and ZFS, copied /usr/{src,obj} over, and ran make build world (with ZFS followed by zpool scrub). To use it, instead of passing a file to ggatel(8) as the last argument, pass a directory with files numerically named in hex (i.e. 0, 1, 2 ..., 9, a, b, ..., e, f, 10, 11, ...). So, I think it's a good time to gauge the community's interest in this feature and whether it'd be possible to merge it to trunk. Known issues: - I reserve an address space the size of the device, so on 32-bit systems you can't create devices bigger than a few GiB. - Similarly, the minimum sector size is most likely 4 kiB. - At most 100 files are mapped simultaneously, and the eviction algorithm is random replacement. - I use alloca(3) instead of malloc(3) in map_bundle() because using the latter causes incorrect behavior somehow. It's probably buffer overruns and / or UBs somewhere in my code. - Both ggatel(8) and md(4) implement BIO_DELETE by zeroing the requested range. However, this interacts poorly with ZFS since it BIO_DELETEs the entire device at pool creation. I know there is a ZFS sysctl to disable the behavior, but I think the device should just not advertise support if it had to fake it anyway, so I didn't implement it. There are definitely many more issues than ones listed above, so any feedback are welcome. Thanks. Julian Hsiao Index: sbin/ggate/ggatel/Makefile =================================================================== diff --git a/stable/10/sbin/ggate/ggatel/Makefile b/stable/10/sbin/ggate/ggatel/Makefile --- a/stable/10/sbin/ggate/ggatel/Makefile (revision 301921) +++ b/stable/10/sbin/ggate/ggatel/Makefile (working copy) @@ -4,7 +4,7 @@ PROG= ggatel MAN= ggatel.8 -SRCS= ggatel.c ggate.c +SRCS= ggatel.c ggatel2.c ggate.c CFLAGS+= -DLIBGEOM CFLAGS+= -I${.CURDIR}/../shared Index: sbin/ggate/ggatel/ggatel.c =================================================================== diff --git a/stable/10/sbin/ggate/ggatel/ggatel.c b/stable/10/sbin/ggate/ggatel/ggatel.c --- a/stable/10/sbin/ggate/ggatel/ggatel.c (revision 301921) +++ b/stable/10/sbin/ggate/ggatel/ggatel.c (working copy) @@ -46,6 +46,8 @@ #include #include "ggate.h" +int check_divs(const char *const, unsigned int *const, size_t *const, size_t *const); +void g_gatel_serve_bundle(const int, const size_t, const size_t, const size_t, const int); static enum { UNSET, CREATE, DESTROY, LIST, RESCUE } action = UNSET; @@ -168,17 +170,39 @@ g_gatel_create(void) { struct g_gate_ctl_create ggioc; - int fd; + int fd, isdir = -1; + size_t div_size, num_divs; fd = open(path, g_gate_openflags(flags) | O_DIRECT | O_FSYNC); - if (fd == -1) - err(EXIT_FAILURE, "Cannot open %s", path); + if (fd == -1) { + if (errno == EISDIR) { + isdir = 1; + } else { + err(EXIT_FAILURE, "Cannot open %s", path); + } + } else { + struct stat sb; + if (fstat(fd, &sb) == -1) { + err(EXIT_FAILURE, "stat(%s) failed", path); + } + isdir = S_ISDIR(sb.st_mode); + } + assert(isdir != -1); + memset(&ggioc, 0, sizeof(ggioc)); ggioc.gctl_version = G_GATE_VERSION; ggioc.gctl_unit = unit; - ggioc.gctl_mediasize = g_gate_mediasize(fd); - if (sectorsize == 0) - sectorsize = g_gate_sectorsize(fd); + if (isdir) { + if (fd != -1 && close(fd) == -1) { + err(EXIT_FAILURE, "close(%s) failed", path); + } + fd = check_divs(path, §orsize, &div_size, &num_divs); + ggioc.gctl_mediasize = (off_t) div_size * num_divs; + } else { + ggioc.gctl_mediasize = g_gate_mediasize(fd); + if (sectorsize == 0) + sectorsize = g_gate_sectorsize(fd); + } ggioc.gctl_sectorsize = sectorsize; ggioc.gctl_timeout = timeout; ggioc.gctl_flags = flags; @@ -188,7 +212,11 @@ if (unit == -1) printf("%s%u\n", G_GATE_PROVIDER_NAME, ggioc.gctl_unit); unit = ggioc.gctl_unit; - g_gatel_serve(fd); + if (isdir) { + g_gatel_serve_bundle(fd, sectorsize, div_size, num_divs, unit); + } else { + g_gatel_serve(fd); + } } static void Index: sbin/ggate/ggatel/ggatel2.c =================================================================== diff --git a/stable/10/sbin/ggate/ggatel/ggatel2.c b/stable/10/sbin/ggate/ggatel/ggatel2.c new file mode 10644 --- /dev/null (nonexistent) +++ b/stable/10/sbin/ggate/ggatel/ggatel2.c (working copy) @@ -0,0 +1,638 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include "ggate.h" + +/* ======== */ + +//#pragma clang diagnostic push +//#pragma clang diagnostic error "-Weverything" + +/* ======== */ + +int check_divs(const char *const, unsigned int *const, size_t *const, size_t *const); +void g_gatel_serve_bundle(const int, const unsigned int, const size_t, const size_t, const int); + +/* ======== */ + +static void +g_gate_verbose_log(const int v, const int p, const char *const m, ...) +{ + if (g_gate_verbose >= v) { + va_list ap; + va_start(ap, m); + g_gate_vlog(p, m, ap); + va_end(ap); + } +} + +#ifdef NDEBUG +__attribute__((unused)) +#endif +static inline bool +mul_overflow(const size_t a, const size_t b) +{ + return(a != 0 && (a * b) / a != b); +} + +static unsigned int +MINDIV_SIZE(void) +{ + static unsigned int ps; + if (ps == 0) { + const long ps2 = sysconf(_SC_PAGESIZE); + static_assert(sizeof(size_t) >= sizeof(long), ""); + assert(ps2 > 0); + assert(ps2 <= UINT_MAX); + ps = (unsigned int) ps2; + } + return(ps); +} + +static size_t +DIV_NAME_BUFSIZE(void) +{ + static size_t dnbs; + if (dnbs == 0) { + dnbs = (size_t) (ceil(log(SIZE_MAX) / log(16))); + ++dnbs; + dnbs *= sizeof(char); + + } + return(dnbs); +} + +/* ======== */ + +static void +numtohexstr(const size_t num, char *const buf, const size_t buflen) +{ +#ifdef NDEBUG + __attribute__((unused)) +#endif + const int r = snprintf(buf, buflen, "%zx", num); + assert(r > 0); + assert((unsigned int) r < buflen); +} + +int +check_divs(const char *const bundle, unsigned int *const blk_size, + size_t *const div_size, size_t *const num_divs) +{ + assert(blk_size != NULL); + assert(div_size != NULL); + assert(num_divs != NULL); + + int dfd; + if ((dfd = open(bundle, O_RDONLY | O_DIRECTORY | O_CLOEXEC)) == -1) { + err(5, "open(%s) failed", bundle); + } + char *buf = malloc(DIV_NAME_BUFSIZE()); + if (buf == NULL) { + err(5, "malloc(DIV_NAME_BUFSIZE) failed"); + } + + for (*num_divs = 0; *num_divs < SIZE_MAX; ++(*num_divs)) { + int fd; + struct stat sb = { .st_dev = 0 }; + + numtohexstr(*num_divs, buf, DIV_NAME_BUFSIZE()); + if ((fd = openat(dfd, buf, O_RDONLY | O_CLOEXEC)) == -1) { + if (errno == ENOENT) { + break; + } + err(5, "open(%s/%s) failed", bundle, buf); + } + if (fstat(fd, &sb) == -1) { + err(5, "fstat(%s/%s) failed", bundle, buf); + } + + if (S_ISCHR(sb.st_mode)) { + if (ioctl(fd, DIOCGMEDIASIZE, &sb.st_size) == -1) { + err(5, "ioctl(%s/%s, DIOCGMEDIASIZE) failed", bundle, buf); + } + + unsigned int bs; + if (ioctl(fd, DIOCGSECTORSIZE, &bs) == -1) { + err(5, "ioctl(%s/%s, DIOCGSECTORSIZE) failed", bundle, buf); + } + if (*blk_size == 0) { + *blk_size = bs; + } else if (*blk_size != bs) { + errx(5, "sector size of %s/%s (%u bytes) is not the same as " + "requested size or that of other divs (%u bytes).", + bundle, buf, bs, *blk_size); + } + } else if (!S_ISREG(sb.st_mode)) { + errx(5, "%s/%s must be a file or character device.", bundle, buf); + } + + if (close(fd) == -1) { + err(5, "close(%s/%s) failed", bundle, buf); + } + + assert(sb.st_size > 0); + static_assert(sizeof(size_t) >= sizeof(sb.st_size), ""); + const size_t st_size = (size_t) sb.st_size; + + if (st_size < MINDIV_SIZE()) { + errx(5, "size of %s/%s is less than %u bytes.", + bundle, buf, MINDIV_SIZE()); + } + if (st_size % MINDIV_SIZE() != 0) { + errx(5, "size of %s/%s is not a multiple of %u.", + bundle, buf, MINDIV_SIZE()); + } + + if (*num_divs == 0) { + *div_size = st_size; + } else if (st_size != *div_size) { + errx(5, "%s/%s is not the same size as other divs (%zu bytes).", + bundle, buf, *div_size); + } + } + + if (*num_divs == 0) { + errx(5, "No divs found in %s.", bundle); + } + + *blk_size = (*blk_size == 0) ? MINDIV_SIZE() : *blk_size; + if (*blk_size < MINDIV_SIZE()) { + errx(5, "sector size must be at least %u bytes.", MINDIV_SIZE()); + } + if (*blk_size % MINDIV_SIZE() != 0) { + errx(5, "sector size must be a multiple of %u bytes.", MINDIV_SIZE()); + } + if (*blk_size > *div_size) { + errx(5, "sector size cannot be greater than div size (%zu bytes).", + *div_size); + } + + struct g_gate_ctl_create ggcc = { .gctl_mediasize = 0 }; + static_assert(sizeof(long) == sizeof(ggcc.gctl_mediasize), ""); + assert(!mul_overflow(*div_size, *num_divs)); + assert(*div_size * *num_divs <= LONG_MAX); + static_assert(sizeof(unsigned int) == sizeof(ggcc.gctl_sectorsize), ""); + g_gate_verbose_log(1, LOG_DEBUG, "blk_size = %u", *blk_size); + g_gate_verbose_log(1, LOG_DEBUG, "div_size = %zu", *div_size); + g_gate_verbose_log(1, LOG_DEBUG, "num_divs = %zu", *num_divs); + + free(buf); + return(dfd); +} + +static void +map_fd(void *const addr, const int fd, +#ifdef NDEBUG + __attribute__((unused)) +#endif + const size_t div_size) +{ + assert(div_size != 0); + + struct stat sb; + if (fstat(fd, &sb) == -1) { + err(1, "fstat() failed"); + } + + assert(sb.st_size > 0); + static_assert(sizeof(size_t) >= sizeof(sb.st_size), ""); + const size_t st_size = (size_t) sb.st_size; + assert(st_size == div_size); + assert(st_size % MINDIV_SIZE() == 0); + + void *m; + const int prot = PROT_READ | PROT_WRITE; + const int flags = MAP_SHARED | MAP_FIXED | MAP_NOCORE/* | MAP_NOSYNC*/; + if ((m = mmap(addr, st_size, prot, flags, fd, 0)) == MAP_FAILED) { + err(1, "mmap() failed"); + } +} + +static void +map_bundle(const uintptr_t base, const uintptr_t addr, const size_t div_size, + const int bundlefd) +{ + assert(base % MINDIV_SIZE() == 0); + assert(addr % MINDIV_SIZE() == 0); + assert(addr >= base); + assert((addr - base) % MINDIV_SIZE() == 0); + + int divfd; + // FIXME +/* + char *div = malloc(DIV_NAME_BUFSIZE()); + assert(div != NULL); + */ + char *div = alloca(DIV_NAME_BUFSIZE()); + + numtohexstr((addr - base) / div_size, div, DIV_NAME_BUFSIZE()); + + g_gate_verbose_log(3, LOG_DEBUG, + "-> [ 0x%09" PRIxPTR ", 0x%09" PRIxPTR " ): 0x%lx; %s", + addr, addr + div_size, div_size, div); + + if ((divfd = openat(bundlefd, div, O_RDWR | O_CLOEXEC)) == -1) { + err(6, "open(%s) failed", div); + } + map_fd((void *) addr, divfd, div_size); + if (close(divfd) == -1) { + err(6, "close(%s) failed", div); + } + +/* free(div); */ +} + +static void * +resv_vaddr(void *const addr, const size_t len) +{ + void *p; + const int prot = PROT_NONE; + const int flags = MAP_PRIVATE | MAP_ANON | + ((addr == NULL) ? 0 : MAP_FIXED); + if ((p = mmap(addr, len, prot, flags, -1, 0)) == MAP_FAILED) { + err(2, "mmap() failed"); + } + + return(p); +} + +/* ======== */ + +static sigjmp_buf memfault_env; +static siginfo_t memfault_info; + +__attribute__((noreturn)) static void +memfault_hdl(int sig, siginfo_t *info, __attribute__((unused)) void *uap) +{ + memfault_info = *info; + siglongjmp(memfault_env, sig); +} + +static void +install_memfault_hdl() +{ + struct sigaction a = { + .sa_sigaction = memfault_hdl, + .sa_flags = SA_SIGINFO + }; + if (sigaction(SIGSEGV, &a, NULL) == -1) { + err(3, "sigaction(SIGSEGV) failed"); + } + struct sigaction b = { + .sa_sigaction = memfault_hdl, + .sa_flags = SA_SIGINFO + }; + if (sigaction(SIGBUS, &b, NULL) == -1) { + err(3, "sigaction(SIGBUS) failed"); + } + + // Use uncatchable signal as memfault_hdl installed flag + memfault_info.si_signo = SIGKILL; +} + +static void +check_expected_memfault(const void *const as, const void *const ae) +{ + if (memfault_info.si_signo == SIGKILL) { + errx(4, "memfault_info not initialized!"); + } else if (memfault_info.si_signo != SIGSEGV && + memfault_info.si_signo != SIGBUS) { + errx(4, "unexpected %s", strsignal(memfault_info.si_signo)); + } else if ( + !((as <= memfault_info.si_addr) && (memfault_info.si_addr < ae)) + ) { + errx(4, "unexpected address %p", memfault_info.si_addr); + } +} + +/* ======== */ + +static void +msync2(const void *const a, const size_t n, const int f) +{ + const uintptr_t b = (uintptr_t) a; + const size_t m = b % MINDIV_SIZE(); + const uintptr_t c = b - m; + const size_t nn = n + m; + + g_gate_verbose_log(3, LOG_DEBUG, + " msync(0x%09" PRIxPTR ", 0x%08lx)", c, nn); + if (nn > 0 && msync((void *) c, nn, f) == -1) { + err(8, "msync() failed"); + } +} + +__attribute__((unused)) static void * +memcpy_msync(void *const d, const void *const s, const size_t n) +{ + const uintptr_t dd = (uintptr_t) d; + g_gate_verbose_log(3, LOG_DEBUG, + "memcpy(0x%09" PRIxPTR ", ..., 0x%08lx)", dd, n); + + memcpy(d, s, n); + msync2(d, n, MS_SYNC); + return(d); +} + +/* ======== */ + +#pragma clang diagnostic push +#pragma clang diagnostic ignored "-Wpadded" +struct bundle_spec { + size_t resv; + uintptr_t as; + uintptr_t ae; + + int bundlefd; + size_t div_size; + size_t num_divs; + unsigned int blk_size; +}; +#pragma clang diagnostic pop + +#define mapped_addrs_size 100 +static void *mapped_addrs[mapped_addrs_size]; + +static void +do_read(const struct bundle_spec *const bspec, + struct g_gate_ctl_io *const ggio) +{ + static_assert(sizeof(ggio->gctl_length) <= sizeof(size_t), ""); + static_assert(sizeof(ggio->gctl_offset) <= sizeof(uintptr_t), ""); + assert(ggio->gctl_length >= 0); + assert(ggio->gctl_offset >= 0); + + assert(memfault_info.si_signo == SIGKILL); + + assert(!mul_overflow(bspec->div_size, mapped_addrs_size)); + const size_t max_len = bspec->div_size * mapped_addrs_size; + if ((size_t) ggio->gctl_length > max_len) { + ggio->gctl_error = ENOMEM; + return; + } + + const uintptr_t a = bspec->as + (uintptr_t) ggio->gctl_offset; + assert(a >= bspec->as); + assert(a < bspec->ae); + const uintptr_t b = a + (uintptr_t) ggio->gctl_length; + assert(b <= bspec->ae); + const size_t m = ((size_t) ggio->gctl_offset) % bspec->div_size; + + for (;;) { + volatile uintptr_t c = (volatile uintptr_t) NULL; + if (sigsetjmp(memfault_env, 1) == 0) { + for (c = a - m; c < b; c += bspec->div_size) { + static_assert(CHAR_BIT == 8, ""); + volatile unsigned char *d1 = (volatile unsigned char *) c; + __attribute__((unused)) volatile unsigned char d2 = *d1; + } + break; + } else { + check_expected_memfault((void *) bspec->as, (void *) bspec->ae); + const uintptr_t si_addr = (uintptr_t) memfault_info.si_addr; + memfault_info.si_signo = SIGKILL; + + assert((void *) c != NULL); + assert(si_addr == c); + + // UB?? + assert(bspec->as <= si_addr); + const size_t n = (si_addr - bspec->as) % bspec->div_size; + const uintptr_t d = si_addr - n; + assert(d <= si_addr); + assert(si_addr < d + 2 * bspec->div_size); + + assert(d >= bspec->as); + assert(d < bspec->ae); + assert((d - bspec->as) % bspec->div_size == 0); + + // Too lazy to do LRU + assert(mapped_addrs_size <= UINT_MAX); + const size_t i = arc4random_uniform( + (unsigned int) mapped_addrs_size); + if (mapped_addrs[i] != NULL) { + resv_vaddr(mapped_addrs[i], bspec->div_size); + + const uintptr_t mai = (uintptr_t) mapped_addrs[i]; + g_gate_verbose_log(3, LOG_DEBUG, + "<- [ 0x%09" PRIxPTR ", 0x%09" PRIxPTR " ): 0x%lx", + mai, mai + bspec->div_size, bspec->div_size); + } + mapped_addrs[i] = (void *) d; + assert(d % bspec->blk_size == 0); + map_bundle(bspec->as, d, bspec->div_size, bspec->bundlefd); + } + } + + g_gate_verbose_log(2, LOG_DEBUG, "do_read(0x%lx, 0x%lx)", + ggio->gctl_offset, ggio->gctl_length); + ggio->gctl_data = (void *) a; + ggio->gctl_error = 0; +} + +static void +do_write(const struct bundle_spec *const bspec, + struct g_gate_ctl_io *const ggio) +{ + static_assert(sizeof(ggio->gctl_length) <= sizeof(size_t), ""); + static_assert(sizeof(ggio->gctl_offset) <= sizeof(uintptr_t), ""); + assert(ggio->gctl_length >= 0); + assert(ggio->gctl_offset >= 0); + + assert(memfault_info.si_signo == SIGKILL); + assert(((size_t) ggio->gctl_length) % bspec->blk_size == 0); + + uintptr_t d = bspec->as + (uintptr_t) ggio->gctl_offset; + assert(d >= bspec->as); + assert(d < bspec->ae); + uintptr_t s = (uintptr_t) ggio->gctl_data; + size_t len = (size_t) ggio->gctl_length; + + for (;;) { + if (sigsetjmp(memfault_env, 1) == 0) { + g_gate_verbose_log(3, LOG_DEBUG, + "memcpy(0x%09" PRIxPTR ", ..., 0x%08lx)", d, len); + /*for (size_t i = 0; i < len; i += bspec->blk_size) { + memcpy((void *) (d + i), (void *) (s + i), bspec->blk_size); + }*/ + memcpy((void *) d, (void *) s, len); + break; + } else { + check_expected_memfault((void *) bspec->as, (void *) bspec->ae); + const uintptr_t si_addr = (uintptr_t) memfault_info.si_addr; + memfault_info.si_signo = SIGKILL; + + // UB?? + assert(bspec->as <= si_addr); + const size_t m = (si_addr - bspec->as) % bspec->div_size; + const uintptr_t a = si_addr - m; + assert(a <= si_addr); + assert(si_addr < a + 2 * bspec->div_size); + + // Yup, still lazy... + assert(mapped_addrs_size <= UINT_MAX); + const size_t i = arc4random_uniform( + (unsigned int) mapped_addrs_size); + if (mapped_addrs[i] != NULL) { + resv_vaddr(mapped_addrs[i], bspec->div_size); + + const uintptr_t mai = (uintptr_t) mapped_addrs[i]; + g_gate_verbose_log(3, LOG_DEBUG, + "<- [ 0x%09" PRIxPTR ", 0x%09" PRIxPTR " ): 0x%lx", + mai, mai + bspec->div_size, bspec->div_size); + } + mapped_addrs[i] = (void *) a; + assert(a % bspec->blk_size == 0); + map_bundle(bspec->as, a, bspec->div_size, bspec->bundlefd); + + // More UB?? + assert(d <= si_addr); + assert(len >= si_addr - d); + len = len - (si_addr - d); + s = s + (si_addr - d); + d = si_addr; + } + } + + g_gate_verbose_log(2, LOG_DEBUG, "do_write(0x%lx, 0x%lx)", + ggio->gctl_offset, ggio->gctl_length); + ggio->gctl_error = 0; +} + +static void +do_flush(const struct bundle_spec *const bspec, + __attribute__((unused)) const struct g_gate_ctl_io *const ggio) +{ + if (g_gate_verbose >= 4) { + for (size_t i = 0; i < mapped_addrs_size; ++i) { + const void *ma = mapped_addrs[i]; + if (ma != NULL) { + g_gate_verbose_log(4, LOG_DEBUG, + "0x%09" PRIxPTR, (uintptr_t) ma); + } + } + } + g_gate_verbose_log(2, LOG_DEBUG, "do_flush()"); + msync2((void *) bspec->as, bspec->resv, MS_SYNC); +} + +__attribute__((noreturn)) void +g_gatel_serve_bundle(const int dfd_, const unsigned int ss_, const size_t ds_, + const size_t nd_, const int unit) +{ + freopen("/dev/null", "r", stdin); + if (g_gate_verbose == 0) { + if (daemon(0, 0) == -1) { + g_gate_destroy(unit, 1); + err(EXIT_FAILURE, "Cannot daemonize"); + } + freopen("/dev/null", "w", stdout); + freopen("/dev/null", "w", stderr); + } + g_gate_verbose_log(1, LOG_DEBUG, "Worker created: %u.", getpid()); + + assert(dfd_ != -1); + assert(!mul_overflow(ds_, nd_)); + struct bundle_spec bspec = { + .resv = ds_ * nd_, + .bundlefd = dfd_, + .div_size = ds_, + .num_divs = nd_, + .blk_size = ss_, + }; + bspec.as = (uintptr_t) (resv_vaddr(NULL, bspec.resv)); + bspec.ae = bspec.as + bspec.resv; + + g_gate_verbose_log(1, LOG_DEBUG, + "[ 0x%09" PRIxPTR ", 0x%09" PRIxPTR " ): 0x%lx", + bspec.as, bspec.ae, bspec.resv); + + assert(bspec.blk_size > 0); + void *ggio_data_buf; + if ((ggio_data_buf = malloc(bspec.blk_size)) == NULL) { + err(EXIT_FAILURE, "malloc() failed"); + } + + install_memfault_hdl(); + + for (;;) { + struct g_gate_ctl_io ggio = { + .gctl_version = G_GATE_VERSION, + .gctl_unit = unit, + .gctl_data = (void *) ggio_data_buf, + .gctl_length = 262144 + }; + + g_gate_ioctl(G_GATE_CMD_START, &ggio); + + switch (ggio.gctl_error) { + case 0: + break; + case ECANCELED: + g_gate_close_device(); + if (close(bspec.bundlefd) == -1) { + err(EXIT_FAILURE, "close() failed"); + } + do_flush(&bspec, &ggio); + free(ggio_data_buf); + g_gate_verbose_log(1, LOG_DEBUG, "Finished."); + exit(EXIT_SUCCESS); + case ENOMEM: + assert(ggio.gctl_cmd == BIO_WRITE); + assert(ggio.gctl_length > 0); + static_assert(sizeof(ggio.gctl_length) <= sizeof(size_t), ""); + if (realloc(ggio_data_buf, (size_t) ggio.gctl_length) == NULL) { + err(EXIT_FAILURE, "realloc() failed"); + } + continue; + case ENXIO: + default: + g_gate_xlog("ioctl(/dev/%s): %s.", G_GATE_CTL_NAME, + strerror(ggio.gctl_error)); + } + + switch(ggio.gctl_cmd) { + case BIO_READ: + do_read(&bspec, &ggio); + break; + case BIO_WRITE: + do_write(&bspec, &ggio); + break; + case BIO_FLUSH: + do_flush(&bspec, &ggio); + break; + default: + g_gate_verbose_log(1, LOG_DEBUG, "unsupported: %d", ggio.gctl_cmd); + ggio.gctl_error = EOPNOTSUPP; + break; + } + + g_gate_ioctl(G_GATE_CMD_DONE, &ggio); + g_gate_verbose_log(3, LOG_DEBUG, "========"); + } +} Property changes on: stable/10/sbin/ggate/ggatel/ggatel2.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property From owner-freebsd-hackers@freebsd.org Thu Jun 30 14:00:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 659E6B86432 for ; Thu, 30 Jun 2016 14:00:52 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de [80.67.31.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D1202116 for ; Thu, 30 Jun 2016 14:00:51 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.169.69] (helo=fabiankeil.de) by smtprelay05.ispgateway.de with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.84) (envelope-from ) id 1bIcWH-0006aD-H1 for freebsd-hackers@freebsd.org; Thu, 30 Jun 2016 16:00:41 +0200 Date: Thu, 30 Jun 2016 16:00:40 +0200 From: Fabian Keil To: freebsd-hackers@freebsd.org Subject: Re: ggatel(8) extension for binding multiple files Message-ID: <20160630160040.279fc306@fabiankeil.de> In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/LuxbYHEM76xLGs6Q6PdZ1HH"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 14:00:52 -0000 --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Julian Hsiao wrote: > I've been working on extending ggatel(8) to support binding multiple files > to one device, similar to how sparse bundles work on OS X. You could > simulate the functionality with ggatel(8) / md(4) + gconcat(8), but this > scales poorly when you have 10^5 files. I've got a working prototype that > passes some simple tests I wrote. I also tried using it as backing store > for UFS and ZFS, copied /usr/{src,obj} over, and ran make build world (wi= th > ZFS followed by zpool scrub). >=20 > To use it, instead of passing a file to ggatel(8) as the last argument, > pass a directory with files numerically named in hex (i.e. 0, 1, 2 ..., 9, > a, b, ..., e, f, 10, 11, ...). >=20 > So, I think it's a good time to gauge the community's interest in this > feature and whether it'd be possible to merge it to trunk. In what situation do you think this is preferable to simply using a single sparse file or zvol as backing store? Even using gvirstor sounds like less maintenance hassle to me. > Known issues: [...] > - Both ggatel(8) and md(4) implement BIO_DELETE by zeroing the requested > range. However, this interacts poorly with ZFS since it BIO_DELETEs > the entire device at pool creation. I know there is a ZFS sysctl to > disable the behavior, but I think the device should just not advertise > support if it had to fake it anyway, so I didn't implement it. In ElectroBSD, ggated writes zeros for BIO_DELETEs[0] and as far as I'm concerned this interacts rather well with ZFS as ZFS will convert the zeros back to BIO_DELETEs (if any compression is enabled) and thus the intended effect is achieved. I suspect that the reason why it may perform poorly with ggatel is that IIRC ggatel does not use a request queue and worker threads and thus performs poorly in general (compared to ggatec/ggated). Fabian [0] Patch 042 in: https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-r295083-0c7f4d6.= diff (The patch set is out of date, but the ggate[cd]-related patches should sti= ll apply.) --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAld1JggACgkQBYqIVf93VJ2AsQCcDsQbrwAiLzaYrbVfjCOQ/Ulq gH0An2Y2+JaCmMLfkPXivVtfI0wo6bb1 =HZon -----END PGP SIGNATURE----- --Sig_/LuxbYHEM76xLGs6Q6PdZ1HH-- From owner-freebsd-hackers@freebsd.org Fri Jul 1 01:08:20 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D148B8855F for ; Fri, 1 Jul 2016 01:08:20 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53AC62743 for ; Fri, 1 Jul 2016 01:08:19 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1bImwJ-00074g-L4 for freebsd-hackers@freebsd.org; Fri, 01 Jul 2016 03:08:15 +0200 Received: from ip184-189-249-34.sb.sd.cox.net ([184.189.249.34]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 01 Jul 2016 03:08:15 +0200 Received: from julian by ip184-189-249-34.sb.sd.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 01 Jul 2016 03:08:15 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Julian Hsiao Subject: Re: ggatel(8) extension for binding multiple files Date: Fri, 1 Jul 2016 01:08:06 +0000 (UTC) Lines: 51 Message-ID: References: <20160630160040.279fc306@fabiankeil.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 184.189.249.34 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:48.0) Gecko/20100101 Firefox/48.0) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 01:08:20 -0000 Fabian Keil fabiankeil.de> writes: > In what situation do you think this is preferable to simply using > a single sparse file or zvol as backing store? You pretty much said it: when you can't use ZFS. More specifically, when your backup destination isn't on ZFS, so you have to use something like rsync for backups. Even with a sparse file, rsync would still have to scan the entire file, whereas with multiple files it could skip over unmodified files quickly based on mtime. Perhaps rsync supports lseek(SEEK_HOLE), but other tools might not. > Even using gvirstor sounds like less maintenance hassle to me. Almost none of the geom classes will accept a plain file as a provider, including gvirstor, so we're back to GEOM-ifying with ggatel or md first. AFAIK, only ggatel, md, ctl, and zfs would accept plain files in lieu of character devices. Finally, gvirstor has some deficiencies per its man page, so I'd rather not use it. >> Known issues: >> [...] >> - Both ggatel(8) and md(4) implement BIO_DELETE by zeroing the >> requested range. However, this interacts poorly with ZFS since it >> BIO_DELETEs the entire device at pool creation. I know there is a >> ZFS sysctl to disable the behavior, but I think the device should >> just not advertise support if it had to fake it anyway, so I didn't >> implement it. > > In ElectroBSD, ggated writes zeros for BIO_DELETEs[0] and as far as I'm > concerned this interacts rather well with ZFS as ZFS will convert the > zeros back to BIO_DELETEs (if any compression is enabled) and thus the > intended effect is achieved. > > I suspect that the reason why it may perform poorly with ggatel is that > IIRC ggatel does not use a request queue and worker threads and thus > performs poorly in general (compared to ggatec/ggated). Sure, the interaction could work out favorably, but BIO_DELETE is still being simulated, and it just so happens that the simulated behavior maps to the underlying FS's logic for issuing BIO_DELETE. According to the comments in the patch you linked, this also only happens with compression enabled (which admittedly is probably the common case). In any case, changing my code to conform is pretty simple, and I can add an option to control the behavior. Julian Hsiao From owner-freebsd-hackers@freebsd.org Fri Jul 1 01:32:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE3EFB88B80 for ; Fri, 1 Jul 2016 01:32:52 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BDD5C22E2 for ; Fri, 1 Jul 2016 01:32:52 +0000 (UTC) (envelope-from paul.koch137@gmail.com) Received: by mail-pf0-x22a.google.com with SMTP id i123so34798317pfg.0 for ; Thu, 30 Jun 2016 18:32:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cUAXGR7Ks4CxvX49INaFiYYhK4XmRlHEPzb5kaqgI8g=; b=kCYeqEq8O0M7AEHEziuOJcqfPH2g9aA2vkkmNx+HK0NbnZ6P3mafRilR6PGCOzE7+v Ofs29fRES3t0pzpdKtq6KN/tDDeOHDUhRgvNL6KgWhDJuGAHYZCgRuFz0/NEvppyrHjm qF4WvjzMLH0dQWNZQ/lmW2ExV2Q1uI4KJhogDL4SUobcBLbTlUib+3OWn3ibQNEPJ2b5 pQbcItslIj/AdiLzJSGBNTeH4p68ZIs+cmxSAdovkaXEiLFk3dyo0RvscM7zUM1ve5mt BoJ5XW3M5DgZwj1bJCWbF/onqH9GkunS+bTeL+GYOJrkJnA7JxpZnW8XcY7Qg4Aq4CRl RdqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cUAXGR7Ks4CxvX49INaFiYYhK4XmRlHEPzb5kaqgI8g=; b=fIrrI4Ru90dP618OA1uw6GQf7CrE7izvL0ISNeKZ8iz7WpR+NCpP9rQ+vS9xbHVien WLeFYX4kAiDoufB38PjAM0LDiClcQmqia3WioirgfAbpJ8H8q22TE+r2oPrDBUVmx1MK pNp8b4tdP0iMpEtiuaKRYviwY+iM88biIYYBe6Y/iO6pnVjMdvdHWfy2kdaryUeB0mTc b7VM12hhW7iFrm7vvL+I6ioTdHZSHBwQeuh1xEHdIccJouOVMnqKc7o4qw6xYSO5FbtF iiwbjp8SEsE3SJ3uQI6q9HHkp/OSJqt69dhjjjUsdmxWOVeT1Z9QRMWJDErbOEwaBcIn ROzA== X-Gm-Message-State: ALyK8tJYUgQaR1kMKVhZeYJmeGYf6w6ExXbThkOyqkFsHd02QyDq4LDVgtxPjteTgKga/Q== X-Received: by 10.98.131.206 with SMTP id h197mr26468664pfe.124.1467336772251; Thu, 30 Jun 2016 18:32:52 -0700 (PDT) Received: from splash.akips.com (CPE-120-146-191-2.static.qld.bigpond.net.au. [120.146.191.2]) by smtp.gmail.com with ESMTPSA id k74sm647175pfk.78.2016.06.30.18.32.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Jun 2016 18:32:51 -0700 (PDT) Date: Fri, 1 Jul 2016 11:32:43 +1000 From: Paul Koch To: Andrew Bates Cc: "freebsd-hackers@freebsd.org" Subject: Re: ZFS ARC and mmap/page cache coherency question Message-ID: <20160701113243.307739cc@splash.akips.com> In-Reply-To: References: <20160630140625.3b4aece3@splash.akips.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.2) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 01:32:53 -0000 Hi Andrew, further info below... > Heya Paul, > > How is your ZFS configured ( zfs get all tank0 )? > > These certainly aren't absolute, law, or perfect - but if you haven't yet, > I suggest you take a peek at the following: > > * http://open-zfs.org/wiki/Performance_tuning > * https://www.joyent.com/blog/bruning-questions-zfs-record-size > * http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide > > On Wed, Jun 29, 2016 at 9:06 PM, Paul Koch wrote: > > > > > Posted this to -stable on the 15th June, but no feedback... > > > > We are trying to understand a performance issue when syncing large mmap'ed > > files on ZFS. > > > > Example test box setup: > > FreeBSD 10.3-p5 > > Intel i7-5820K 3.30GHz with 64G RAM > > 6 * 2 Tbyte Seagate ST2000DM001-1ER164 in a ZFS stripe > > > > Read performance of a sequentially written large file on the pool is > > typically around 950Mbytes/sec using dd. > > > > Our software mmap's some large database files using MAP_NOSYNC, and we > > call fsync() every 10 minutes when we know the file system is mostly > > idle. In our test setup, the database files are 1.1G, 2G, 1.4G, 12G, > > 4.7G and ~20 small files (under 10M). All of the memory pages in the > > mmap'ed files are updated every minute with new values, so the entire > > mmap'ed file needs to be > > synced to disk, not just fragments. > > > > When the 10 minute fsync() occurs, gstat typically shows very little disk > > reads and very high write speeds, which is what we expect. But, every 80 > > minutes we process the data in the large mmap'ed files and store it in > > highly > > compressed blocks of a ~300G file using pread/pwrite (i.e. not mmap'ed). > > After that, the performance of the next fsync() of the mmap'ed files falls > > off a cliff. We are assuming it is because the ARC has thrown away the > > cached data of the mmap'ed files. gstat shows lots of read/write > > contention > > and lots of things tend to stall waiting for disk. > > > > Is this just a lack of ZFS ARC and page cache coherency ?? > > > > Is there a way to prime the ARC with the mmap'ed files again before we > > call fsync() ? > > > > We've tried cat and read() on the mmap'ed files but doesn't seem to touch > > the > > disk at all and the fsync() performance is still poor, so it looks like > > the ARC is not being filled. msync() doesn't seem to be much different. > > mincore() stats show the mmap'ed data is entirely incore and referenced. > > > > Paul. Here is our zfs get all akips NAME PROPERTY VALUE SOURCE akips type filesystem - akips creation Sat Apr 9 7:29 2016 - akips used 835G - akips available 9.70T - akips referenced 96K - akips compressratio 1.00x - akips mounted no - akips quota none default akips reservation none default akips recordsize 128K default akips mountpoint none local akips sharenfs off default akips checksum on default akips compression off default akips atime off local akips devices on default akips exec on default akips setuid on default akips readonly off default akips jailed off default akips snapdir hidden default akips aclmode discard default akips aclinherit restricted default akips canmount on default akips xattr on default akips copies 1 default akips version 5 - akips utf8only off - akips normalization none - akips casesensitivity sensitive - akips vscan off default akips nbmand off default akips sharesmb off default akips refquota none default akips refreservation none default akips primarycache all default akips secondarycache all default akips usedbysnapshots 0 - akips usedbydataset 96K - akips usedbychildren 835G - akips usedbyrefreservation 0 - akips logbias latency default akips dedup off default akips mlslabel - akips sync standard default akips refcompressratio 1.00x - akips written 96K - akips logicalused 834G - akips logicalreferenced 9.50K - akips volmode default default akips filesystem_limit none default akips snapshot_limit none default akips filesystem_count none default akips snapshot_count none default akips redundant_metadata all default The problem appears to be similar to what is described here: http://zfs-discuss.opensolaris.narkive.com/tgP1NV9l/remedies-for-suboptimal-mmap-performance-on-zfs So basically our problem is, we mmap a large file, which gets double cached in both ZFS ARC and page cache. We update every page in the mmap'ed data, then flush it our every 10 minutes when we know the disks are mostly idle. Performance is great.... unless the ZFS ARC no longer has the doubled cached mmap'ed data, which means it has to go to disk and cause heaps of read/write contention, then performance falls off a cliff. What we want to know is if there is no ARC/page cache coherency, how do we prime the ARC cache again with the same data so we can get good write performance on the fsync(). Paul. From owner-freebsd-hackers@freebsd.org Fri Jul 1 03:41:45 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 792A2B86625 for ; Fri, 1 Jul 2016 03:41:45 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from nm16.bullet.mail.bf1.yahoo.com (nm16.bullet.mail.bf1.yahoo.com [98.139.212.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3760F260A for ; Fri, 1 Jul 2016 03:41:45 +0000 (UTC) (envelope-from pfg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1467344098; bh=Lit9//W9cCccesdKnkmIyZXDrqpWnLpzU6fzLqN5LNU=; h=To:From:Subject:Date:From:Subject; b=baxh3mUCMUCIGKsilPBW4N/ugCukfZJIcVHVxqoBfB0Cd8leSHONOWy/7nphHHP3lEjPFeXRhymxgrLL0NDcWBeFzOhN0ggyHhawsF/U8Ysk6sTq95bDuRI02okkQ7hVPDwDhUAn3uGWpfFDyTxdpoYftudPmjT2znOyOdUpFSoPRXehL/l/vULa/DbbXMp44MfEuDs4cJSemfhr/2p6FfoOvmURmPiKNUb18iLvhMCa8xTT14KuQfWnEZVtfjfBqqg6ZOQazsF86GDjc01biNRkiXozB0i9O/KHMrU/f8pLhn5MmHTdQUATZJ2OCgiPd6H8bpBWMuqb9YdCVgvuoA== Received: from [66.196.81.170] by nm16.bullet.mail.bf1.yahoo.com with NNFMP; 01 Jul 2016 03:34:58 -0000 Received: from [68.142.230.73] by tm16.bullet.mail.bf1.yahoo.com with NNFMP; 01 Jul 2016 03:34:58 -0000 Received: from [127.0.0.1] by smtp230.mail.bf1.yahoo.com with NNFMP; 01 Jul 2016 03:34:58 -0000 X-Yahoo-Newman-Id: 349199.86132.bm@smtp230.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: QnFmQLEVM1nZSHq46gn5R2thwAM_gRlW1FvZwBfT4VjWk3T 540N.wg1aZkx.KFX1W98dQCf.5aVsjIqtmroUa6.Z8G.8GRI4RFO0UlndAji mk0gIDgixaQH1sMS1MaTbYsg2gCAtzEHp4M.9CjJfnnIk1mVsdUvvYHtrQeN wPewWOmUyJ_rtSQ6x.y8ACZl_c.DYhwDZpcsRnNYJSzyRpxJZIgDc8FJhFw1 xUK9BMUd3drAQx1MikWf5_QBe_avQkJwyfSSgw1RRlWNSVV.lUG5EbRCggls nWtJl3ClVin6IIz.z4zIFgHNyckozeszDP.D2ghSQ_s28DltlMUUMUz4fZm7 UATVWsGthpi22EIRck_7ogeLA8u._aO3CvruHtQOzuzPAlJ7pN3MDEfFS455 Recg2xEAxaZBP4QPHUEpaa.9E68iRpZDOVEkN6WTNi9r3PZCUGYPuLks1EKC jNMfx6Ly.Y268haxPWk5S11Cyj0wOUr.tRQVvJpr1ckVUzvuJquziLirX.uw rtFJdEKjAMQ35.XYGvDsDtHdZjSXYpatQcwh0kI7SsAgPgw-- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf To: Alfred Perlstein , FreeBSD Hackers , python@freebsd.org From: Pedro Giffuni Subject: Re: bootloader in python. Message-ID: <6b329a2f-8734-73d8-f83e-ee81fe6bbf5b@FreeBSD.org> Date: Thu, 30 Jun 2016 22:34:58 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 03:41:45 -0000 > Josh Triplett started out with "the punchline" for his PyCon 2015 talk > on porting Python to run without an operating system: he and his Intel > colleagues got the interpreter to run in the GRUB boot loader for either > BIOS or EFI systems. But that didn't spoil the rest of the talk by any > means. He had plenty of interesting things to say and a number of > eye-opening demos to show as well. > > https://lwn.net/Articles/641244/ > > https://lwn.net/SubscriberLink/692638/2ebf68539c678a33/ > > > Enjoy! > > -Alfred Hmm ... Not completely unrelated, there was a Lua GSoC but I guess we could also have a 64k python as an alternative to forth in the bootloader as well: http://www.tinypy.org/ Cheers, Pedro. From owner-freebsd-hackers@freebsd.org Fri Jul 1 06:43:15 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDA19B8D200 for ; Fri, 1 Jul 2016 06:43:15 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from c.mail.sonic.net (c.mail.sonic.net [64.142.111.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AD8102FC5 for ; Fri, 1 Jul 2016 06:43:15 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from comporellon.tachypleus.net (75-101-50-44.static.sonic.net [75.101.50.44]) (authenticated bits=0) by c.mail.sonic.net (8.15.1/8.15.1) with ESMTPSA id u616hDru011676 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Thu, 30 Jun 2016 23:43:13 -0700 To: "freebsd-hackers@freebsd.org" From: Nathan Whitehorn Subject: Review request: sparse CPU ID maps Message-ID: <57761101.3030101@freebsd.org> Date: Thu, 30 Jun 2016 23:43:13 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Sonic-CAuth: UmFuZG9tSVZrCPE4NuFGbfScBqjTtPOSI2oGxsQxaxxK7xSzOhuBwgId/x1La5+YlSZlroCsaYFVWsYpGW2858LqEcIQA++Uy50d6xYfu/A= X-Sonic-ID: C;gIxaFVc/5hG5PZtMTlz00w== M;NGScFVc/5hG5PZtMTlz00w== X-Spam-Flag: No X-Sonic-Spam-Details: 0.0/5.0 by cerberusd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 06:43:15 -0000 I have been working on fixing PR 210106 (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210106) and have run into the fact that several pieces of the kernel, notably parts of subr_taskqueue.c, require that CPU IDs be dense in the range [0, mp_ncpus), which the kernel does not guarantee, for example in the case of CPUs with hyperthreading in which the threading is disabled. This is leading to hangs in late boot in -CURRENT. I've prepared the following patch, which fixes PR 210106, but I would like a few more eyeballs on it before committing it. It fixes most of the bogus uses of mp_ncpus in the kernel, but not all: doing grep -R '< mp_ncpus' /sys | wc -l gives 52 remaining instances of loops in [0, mp_ncpus) or [1, mp_ncpus), most or all of which should instead be CPU_FOREACH(), but none of which I feel comfortable changing (36 are in ARM code for hardware I don't have access to). The patch lives here: http://people.freebsd.org/~nwhitehorn/sparse_cpu_ids.diff -Nathan From owner-freebsd-hackers@freebsd.org Sat Jul 2 14:49:01 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64A52B8F934 for ; Sat, 2 Jul 2016 14:49:01 +0000 (UTC) (envelope-from mmatalka@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E996E2AD3 for ; Sat, 2 Jul 2016 14:49:00 +0000 (UTC) (envelope-from mmatalka@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id a66so63074296wme.0 for ; Sat, 02 Jul 2016 07:49:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:user-agent:mime-version; bh=xS5OuNvS/wU/t+PPJbw/Sl6TOhCr8qugXcPKB/lfIeE=; b=kN4qJHoCpELJvgXVdgaTbWZPK5qfug6iC3B/tdncjCfLQzfAlLBrjBqyJiOz40jsKm YsAEA6n9BkczSlGVe2QwXrsnO0aaM0H1wScLfDbF4FCvJh5ipfCBAehOMATDoSdFBF5y 1zidEdG1qBc4d3U5lMuIBRnpU8KiXg8mqWQ/2By/LErWOTrtjAsVDP/dAURNdpXDMr+W EgOtfvl3ftXkGu1a5y3fy4PU78RkhDtOG6cEdDTZEZsKCwJcyg5ZdMzHynXD8Sq1eJBQ inFZYptNnsrl0NZ3DSrl8at92aSGwikMq7JsdCdUyYy4nRhf4QDpY7ljMt3wrH/AAhdi rUBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version; bh=xS5OuNvS/wU/t+PPJbw/Sl6TOhCr8qugXcPKB/lfIeE=; b=kjqbby7sc9M9ELC8qLrThTBctfgYMQPWhOwsIrcgJAmVN71NNLAACE7nx6G7cjOh4q x0sDZMOAumPrWaCw/Vhj01QgsqlR4bZGHaYdacPDLB8kKtX1nWxnwChMj0JBoiI6CAIk hKZkwcDKjFccm5SwTT3gzboviHwk0760zvyzHX1bZpK8AsL+nzMmvEIcpTZQG+3i98Kf P8/pUGFvHsjv7S2mZbVElRb0FNcw83t0RhGeSvlzOXCE/03gqJz1Bmkscwy4q95tyq/4 /77ruLxG3QQu/6rFMo5DsDTJ5m1KFdtE0dXhqCCznRFIbwvTzxKb9u08HY9R1FDzkqpG WQNQ== X-Gm-Message-State: ALyK8tIoe9peKyPZH7FC0g+hoBQ9n6Bgc2fM1XDhN5h2wMK01QbUPH+kPzPT494lpd30Cg== X-Received: by 10.28.193.14 with SMTP id r14mr3395181wmf.94.1467470939024; Sat, 02 Jul 2016 07:48:59 -0700 (PDT) Received: from localhost ([37.153.108.22]) by smtp.gmail.com with ESMTPSA id kc8sm1528049wjb.0.2016.07.02.07.48.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 02 Jul 2016 07:48:57 -0700 (PDT) From: Malcolm Matalka To: freebsd-hackers@freebsd.org Subject: ASUS ZenBook UX305CA Trackpad Date: Sat, 02 Jul 2016 14:48:57 +0000 Message-ID: <86d1mwyu1y.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 14:49:01 -0000 Hello, I recently purchased the above mentioned laptop and the trackpad does not work. When booting in verbose mode it says: psm0: unable to allocate IRQ. I've tracked this down in the source code (psm.c) and would like to figure out how to fix this. I see a similar issue came up for a Dell a bit back[0] but I don't understand the solution. I understand how to dump acpi and what not, but I don't actually understand what it's doing. I'd like to help make this laptop work great under FreeBSD, can anyone point me in the right direction or offer some advice on how to address hardware just not working as expected? And in particular, how to develop a fix that can be brought into a release to make it work for everyone rather than doing acpi hacker? I know how to program, but I've spent all my time in user space so I'm really lost on how to attack the kernel and especially hardware. Thanks! /Malcolm From owner-freebsd-hackers@freebsd.org Sat Jul 2 17:31:04 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54129B8FC7B for ; Sat, 2 Jul 2016 17:31:04 +0000 (UTC) (envelope-from outro.pessoa@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D875629EB; Sat, 2 Jul 2016 17:31:03 +0000 (UTC) (envelope-from outro.pessoa@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id r201so66062419wme.1; Sat, 02 Jul 2016 10:31:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7uvQsamNXBQm32CjS4GtLwRiJieUrEQCylSlJGfF544=; b=e/Cy1lV9I9H6Tc9ov5/PS5G6gNHs1AiQZTnAlJtljKqR0frukQNa/r37pjGeH7L9q/ YYKx9eY+FDPsNBwkeruWbbVq4JzBcyjmdM57+he4228LDxpMefEtlUPcR33tvO0BoxCS rY1qtZ5o24xh0hF6YRf4HmwjOUkh/toKQM3YXU9x/XqMETvFn9o4g8C5DDBhxmnpOgql Rj7BmpUcd44Nhuq4uDIbUQYVwj02ZsH905+G7qVbI8N5AjqGHGlQsL8Xc694/TdX/+Xz lleWfs7085hn3luIZcpZowr+yO+lGpPj3xXcI7GHAe2pEPZMgBpQEfzHgOCDuwT+TFRB RDFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7uvQsamNXBQm32CjS4GtLwRiJieUrEQCylSlJGfF544=; b=OfCN06nozxZO21HujhRJFfZBQiSqkpY/Zixeo7IwdnUa3fZeF+83rEohbaFm7LNAb0 iu0B2HuPi0qrY5xTye+XgS3VfMJ/ff7R/kGeB5/w33zb36iqBUsiF6YFMykLf+2/uZNQ /K+8qZzacEx4bXdacT9NmGScf1RAim+MGYqRWbsWOq88/qinWBH/4VMTFjtGcF6cVflD NLMSbGSIPAQ2NMdqR93r1ImHQUn/BJa37Y96I7TLaeBn31x9BqrTOIHfAINpbTbUZTlY qE3O8xM0ogOz6d31nHDTd1ASqEWZMHRHiByXkq2SS+GWwM26Q9l8pPy1bP2CrF6kQJyD 5CAQ== X-Gm-Message-State: ALyK8tKiYCA9eHAW5CprNJCCDhivtjAyD5OJpMGEY+pU0d5/yzo5ME66qWSceWRI+VTdcU4UvKNFp2d8yof66g== X-Received: by 10.28.164.68 with SMTP id n65mr3780833wme.12.1467480661879; Sat, 02 Jul 2016 10:31:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.217.148 with HTTP; Sat, 2 Jul 2016 10:31:01 -0700 (PDT) In-Reply-To: <57761101.3030101@freebsd.org> References: <57761101.3030101@freebsd.org> From: outro pessoa Date: Sat, 2 Jul 2016 13:31:01 -0400 Message-ID: Subject: Re: Review request: sparse CPU ID maps To: Nathan Whitehorn Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 17:31:04 -0000 Nathan, What type of ARM hardware do you need? On Fri, Jul 1, 2016 at 2:43 AM, Nathan Whitehorn wrote: > I have been working on fixing PR 210106 ( > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210106) and have run > into the fact that several pieces of the kernel, notably parts of > subr_taskqueue.c, require that CPU IDs be dense in the range [0, mp_ncpus), > which the kernel does not guarantee, for example in the case of CPUs with > hyperthreading in which the threading is disabled. This is leading to hangs > in late boot in -CURRENT. > > I've prepared the following patch, which fixes PR 210106, but I would like > a few more eyeballs on it before committing it. It fixes most of the bogus > uses of mp_ncpus in the kernel, but not all: doing grep -R '< mp_ncpus' > /sys | wc -l gives 52 remaining instances of loops in [0, mp_ncpus) or [1, > mp_ncpus), most or all of which should instead be CPU_FOREACH(), but none > of which I feel comfortable changing (36 are in ARM code for hardware I > don't have access to). > > The patch lives here: > http://people.freebsd.org/~nwhitehorn/sparse_cpu_ids.diff > -Nathan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sat Jul 2 18:26:56 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC812B8F8BE for ; Sat, 2 Jul 2016 18:26:56 +0000 (UTC) (envelope-from cedric.blancher@gmail.com) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B088A2C5A for ; Sat, 2 Jul 2016 18:26:56 +0000 (UTC) (envelope-from cedric.blancher@gmail.com) Received: by mail-pa0-x22c.google.com with SMTP id b13so47673492pat.0 for ; Sat, 02 Jul 2016 11:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7JKjZdN1Y2GGYPLsk5/JXaDenJExyG7jnNQnwrvO/Ng=; b=F6dWHo8goma7ca3Z0hdFjkEd1gDDHue0EuQFm9RN2Go9ce0zFrVtGWaF/EjtH7LPuf eVFt2vvEXJ7G/NvG88wiW2LAK7p0m/3mJJr4Mq4X8611CVfYMlNTKthXfITEpXpyTn20 gaYrlHmf0dg2LBSK9ozI3e45SjSJ/kKVMrTM2Kl2nnkAB/+MXwU9vbypdJNWKuHw9dEG LxpgxvKRfKrqPONXLuD/hyG7iXum+/JEh7wcpCsLA1fDszw/h2memkFPgtBmpH230BU1 5SfvaZnDuMcaadvbkzFjQumkw5IEVlirkUjWoiBYIzKv09ZiIkpNfWRKgQ0c5/l1QWgS osAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7JKjZdN1Y2GGYPLsk5/JXaDenJExyG7jnNQnwrvO/Ng=; b=Va+4Dk6R1Nk1kr16V56OxpgGDcBATfwzNY5VWJri9Jz0s9c71dG/0oJsghrPTNugYQ j5DFEpAEOY4ZgJbs5EjOivCz44NROgj2re/gg8xXIb9OiyDy3mY/PYK8MZULyAX23MsI uf8tArimFb+8se/be6UF88sx1v5gcPgB123JO7JMd7lTDSDZ84MG0VW2THldLDMrCGAz mtg3Ybr6PEjNtXWDAgkiL5MWXtEjqsTJ//psgSYP7ZmQA0X+erhT+tC/RcYfHxBDYg8g xgjxss1iwsyizE4FDOg5PtK+8qQ7HK6fKDZpc/B1iT1K0/sj5hqPiXMmF4l3FQR5Om25 vBEg== X-Gm-Message-State: ALyK8tIoHPb8eQA1W0zKDpbCEfozaWVN2lGJn5i4X97mH1G0qrFkVmxbd2haZ1rD8EalVvyZ7ejDM//hG3AtQQ== X-Received: by 10.66.52.11 with SMTP id p11mr8081710pao.155.1467484016160; Sat, 02 Jul 2016 11:26:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.173.8 with HTTP; Sat, 2 Jul 2016 11:26:55 -0700 (PDT) In-Reply-To: <20160630140625.3b4aece3@splash.akips.com> References: <20160630140625.3b4aece3@splash.akips.com> From: Cedric Blancher Date: Sat, 2 Jul 2016 20:26:55 +0200 Message-ID: Subject: Re: ZFS ARC and mmap/page cache coherency question To: Paul Koch Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 18:26:57 -0000 Short story: ZFS was tacked on the kernel and was never properly integrated into the VM page management, which leads to DRAMATIC poor performance for anything which uses mmap() for write IO. This was solved in Oracle Solaris with the great VM allocator rewrite which landed after Opensolaris was made closed source again. Without a complete rewrite of the VM system this problem is unsolvable. Ced On 30 June 2016 at 06:06, Paul Koch wrote: > > Posted this to -stable on the 15th June, but no feedback... > > We are trying to understand a performance issue when syncing large mmap'ed > files on ZFS. > > Example test box setup: > FreeBSD 10.3-p5 > Intel i7-5820K 3.30GHz with 64G RAM > 6 * 2 Tbyte Seagate ST2000DM001-1ER164 in a ZFS stripe > > Read performance of a sequentially written large file on the pool is > typically around 950Mbytes/sec using dd. > > Our software mmap's some large database files using MAP_NOSYNC, and we call > fsync() every 10 minutes when we know the file system is mostly idle. In > our test setup, the database files are 1.1G, 2G, 1.4G, 12G, 4.7G and ~20 > small files (under 10M). All of the memory pages in the mmap'ed files are > updated every minute with new values, so the entire mmap'ed file needs to be > synced to disk, not just fragments. > > When the 10 minute fsync() occurs, gstat typically shows very little disk > reads and very high write speeds, which is what we expect. But, every 80 > minutes we process the data in the large mmap'ed files and store it in highly > compressed blocks of a ~300G file using pread/pwrite (i.e. not mmap'ed). > After that, the performance of the next fsync() of the mmap'ed files falls > off a cliff. We are assuming it is because the ARC has thrown away the > cached data of the mmap'ed files. gstat shows lots of read/write contention > and lots of things tend to stall waiting for disk. > > Is this just a lack of ZFS ARC and page cache coherency ?? > > Is there a way to prime the ARC with the mmap'ed files again before we call > fsync() ? > > We've tried cat and read() on the mmap'ed files but doesn't seem to touch the > disk at all and the fsync() performance is still poor, so it looks like the > ARC is not being filled. msync() doesn't seem to be much different. > mincore() stats show the mmap'ed data is entirely incore and referenced. > > Paul. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Cedric Blancher Institute Pasteur From owner-freebsd-hackers@freebsd.org Sat Jul 2 18:31:24 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6179EB8FA81; Sat, 2 Jul 2016 18:31:24 +0000 (UTC) (envelope-from dcrosstech@gmail.com) Received: from mail-yw0-x241.google.com (mail-yw0-x241.google.com [IPv6:2607:f8b0:4002:c05::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 232A02FA5; Sat, 2 Jul 2016 18:31:24 +0000 (UTC) (envelope-from dcrosstech@gmail.com) Received: by mail-yw0-x241.google.com with SMTP id f75so1674076ywb.3; Sat, 02 Jul 2016 11:31:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=elkJZWGeMHtK533zurGlXYZV7HdKI6pS4Cy3ww05f6I=; b=kxRhP0sZbcBPgMm51A//ciu6ZeAaGwqIVe/x3i153xRw876qiIUYvNng9Z66liK7Xh bfHXLxKLJffOSs6aLmWAa5AgYK+TxfMzCJKbysj95smBXWRaeEgIuLJTqBjoEI/guUDt rHeqF2zWjXBPIYzsOg+p1RPAckhEo2xbLssRAFu7bETnD1IGYvEHUE9HQ4cArKxG8jm7 yt+A1F8vImRXLmPo8IzVT3SDU1jf3TDiajAXC9lp7CqdeT9BOUpGA8W4vAAlvDdzwIKK bP2TrSYpabDgmVQi2loQAaSLUXyUTpyNJhHlL3MIbFfwtwQK3hYeTKVq/TquDLbGXy6W TJeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=elkJZWGeMHtK533zurGlXYZV7HdKI6pS4Cy3ww05f6I=; b=fXKat9QoarpTtO98M+RPMyS1mnxnaPg1K03bU7RU1NIStdVyjDt4fXsuM1RQx3zbuf T4obvi0pXMNkEcoJ6f32G6yJgivkvTBFIklAp950sXMuD447TNHZsExe2GrJSSlQJF/r DUSS0dHyc0fiZYkhVGTw4ChxsFJQZnj2cWVI48hEeuOoD/L20bAeai4zbgUkAqzI9mJH 2iKah0i352+equWFQbMZkM/+J3pqLGWiGxF9eACkggUKEk2/8BlU+lhs1IQYgzfVHzKs IP0bd+/duUsH99eSJGUf2kYYricEU7GooEHOkMx/vIPk+KX2LgbTuYG1syrq1XDki6Zh aHYw== X-Gm-Message-State: ALyK8tLBB08ru9zEPzZqs2nrdwDEfmn+TDlyi7YeOCCaTZD3qb9aP2dleLVgxoGxwKM/BvZUClCBujB5Wffm7Q== X-Received: by 10.129.80.131 with SMTP id e125mr2684540ywb.45.1467484283184; Sat, 02 Jul 2016 11:31:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.212.66 with HTTP; Sat, 2 Jul 2016 11:31:22 -0700 (PDT) From: David Cross Date: Sat, 2 Jul 2016 14:31:22 -0400 Message-ID: Subject: Reproducable panic in FFS with softupdates and no journaling (10.3-RELEASE-pLATEST) To: freebsd-stable@freebsd.org, freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sat, 02 Jul 2016 19:16:46 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 18:31:24 -0000 Ok, I have been trying to trace this down for awhile..I know quite a bit about it.. but there's a lot I don't know, or I would have a patch. I have been trying to solve this on my own, but bringing in some outside assistance will let me move on with my life. First up: The stacktrace (from a debugging kernel, with coredump #0 doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:298 #1 0xffffffff8071018a in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:486 #2 0xffffffff80710afc in vpanic ( fmt=0xffffffff80c7a325 "softdep_deallocate_dependencies: dangling deps b_ioflags: %d, b_bufsize: %ld, b_flags: %d, bo_flag: %d", ap=0xfffffe023ae5cf40) at /usr/src/sys/kern/kern_shutdown.c:889 #3 0xffffffff807108c0 in panic ( fmt=0xffffffff80c7a325 "softdep_deallocate_dependencies: dangling deps b_ioflags: %d, b_bufsize: %ld, b_flags: %d, bo_flag: %d") at /usr/src/sys/kern/kern_shutdown.c:818 #4 0xffffffff80a7c841 in softdep_deallocate_dependencies ( bp=0xfffffe01f030e148) at /usr/src/sys/ufs/ffs/ffs_softdep.c:14099 #5 0xffffffff807f793f in buf_deallocate (bp=0xfffffe01f030e148) at buf.h:428 #6 0xffffffff807f59c9 in brelse (bp=0xfffffe01f030e148) at /usr/src/sys/kern/vfs_bio.c:1599 #7 0xffffffff807f3132 in bufwrite (bp=0xfffffe01f030e148) at /usr/src/sys/kern/vfs_bio.c:1180 #8 0xffffffff80ab226a in bwrite (bp=0xfffffe01f030e148) at buf.h:395 #9 0xffffffff80aafb1b in ffs_write (ap=0xfffffe023ae5d2b8) at /usr/src/sys/ufs/ffs/ffs_vnops.c:800 #10 0xffffffff80bdf0ed in VOP_WRITE_APV (vop=0xffffffff80f15480, a=0xfffffe023ae5d2b8) at vnode_if.c:999 #11 0xffffffff80b1d02e in VOP_WRITE (vp=0xfffff80077e7a000, uio=0xfffffe023ae5d378, ioflag=8323232, cred=0xfffff80004235000) at vnode_if.h:413 #12 0xffffffff80b1ce97 in vnode_pager_generic_putpages (vp=0xfffff80077e7a000, ma=0xfffffe023ae5d660, bytecount=16384, flags=1, rtvals=0xfffffe023ae5d580) at /usr/src/sys/vm/vnode_pager.c:1138 #13 0xffffffff80805a57 in vop_stdputpages (ap=0xfffffe023ae5d478) at /usr/src/sys/kern/vfs_default.c:760 #14 0xffffffff80be201e in VOP_PUTPAGES_APV (vop=0xffffffff80f00218, a=0xfffffe023ae5d478) at vnode_if.c:2861 #15 0xffffffff80b1d7e3 in VOP_PUTPAGES (vp=0xfffff80077e7a000, m=0xfffffe023ae5d660, count=16384, sync=1, rtvals=0xfffffe023ae5d580, offset=0) at vnode_if.h:1189 #16 0xffffffff80b196f3 in vnode_pager_putpages (object=0xfffff8014a1fce00, m=0xfffffe023ae5d660, count=4, flags=1, rtvals=0xfffffe023ae5d580) at /usr/src/sys/vm/vnode_pager.c:1016 #17 0xffffffff80b0a605 in vm_pager_put_pages (object=0xfffff8014a1fce00, m=0xfffffe023ae5d660, count=4, flags=1, rtvals=0xfffffe023ae5d580) at vm_pager.h:144 #18 0xffffffff80b0a18c in vm_pageout_flush (mc=0xfffffe023ae5d660, count=4, flags=1, mreq=0, prunlen=0xfffffe023ae5d6f8, eio=0xfffffe023ae5d77c) at /usr/src/sys/vm/vm_pageout.c:533 #19 0xffffffff80afec76 in vm_object_page_collect_flush ( object=0xfffff8014a1fce00, p=0xfffff8023a882370, pagerflags=1, flags=1, clearobjflags=0xfffffe023ae5d780, eio=0xfffffe023ae5d77c) at /usr/src/sys/vm/vm_object.c:971 #20 0xffffffff80afe91e in vm_object_page_clean (object=0xfffff8014a1fce00, start=0, end=0, flags=1) at /usr/src/sys/vm/vm_object.c:897 #21 0xffffffff80afe1fa in vm_object_terminate (object=0xfffff8014a1fce00) at /usr/src/sys/vm/vm_object.c:735 #22 0xffffffff80b1a0f1 in vnode_destroy_vobject (vp=0xfffff80077e7a000) at /usr/src/sys/vm/vnode_pager.c:164 #23 0xffffffff80abb191 in ufs_prepare_reclaim (vp=0xfffff80077e7a000) at /usr/src/sys/ufs/ufs/ufs_inode.c:190 #24 0xffffffff80abb1f9 in ufs_reclaim (ap=0xfffffe023ae5d968) at /usr/src/sys/ufs/ufs/ufs_inode.c:219 #25 0xffffffff80be0ade in VOP_RECLAIM_APV (vop=0xffffffff80f15ec0, a=0xfffffe023ae5d968) at vnode_if.c:2019 #26 0xffffffff80827849 in VOP_RECLAIM (vp=0xfffff80077e7a000, td=0xfffff80008931960) at vnode_if.h:830 #27 0xffffffff808219a9 in vgonel (vp=0xfffff80077e7a000) at /usr/src/sys/kern/vfs_subr.c:2943 #28 0xffffffff808294e8 in vlrureclaim (mp=0xfffff80008b2e000) at /usr/src/sys/kern/vfs_subr.c:882 #29 0xffffffff80828ea9 in vnlru_proc () at /usr/src/sys/kern/vfs_subr.c:1000 #30 0xffffffff806b66c5 in fork_exit (callout=0xffffffff80828c50 , arg=0x0, frame=0xfffffe023ae5dc00) at /usr/src/sys/kern/kern_fork.c:1027 #31 0xffffffff80b21dce in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #32 0x0000000000000000 in ?? () This is a kernel compiled -O -g, its "almost" GENERIC; the only difference is some removed drivers, I have reproduced this on a few different kernels, including a BHYVE one so I can poke at it and not take out the main machine. The reproduction as it currently stands needs to have jails running, but I don't believe this is a jail interaction, I think its just that the process that sets up the problem happens to be running in a jail. The step is "start jail; run "find /mountpoint -xdev >/dev/null" on the filesystem, when the vnlru forces the problem vnode out the system panics. I made a few modifications to the kernel to spit out information about the buf that causes the issue, but that is it. Information about the buf in question; it has a single softdependency worklist for direct allocation: (kgdb) print *bp->b_dep->lh_first $6 = {wk_list = {le_next = 0x0, le_prev = 0xfffffe01f030e378}, wk_mp = 0xfffff80008b2e000, wk_type = 4, wk_state = 163841} The file that maps to that buffer: ls -lh MOUNTPOINT/jails/mail/var/imap/db/__db.002 -rw------- 1 cyrus cyrus 24K Jul 1 20:32 MOUNTPOINT/jails/mail/var/imap/db/__db.002 Any help is appreciated, until then I will keep banging my head against the proverbial wall on this :)