From owner-freebsd-hackers@freebsd.org Sun Mar 19 14:44:00 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DAF0D121DB; Sun, 19 Mar 2017 14:44:00 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D9CB410DE; Sun, 19 Mar 2017 14:43:59 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id D37F4B846; Sun, 19 Mar 2017 14:43:51 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 9D544468C; Sun, 19 Mar 2017 15:43:46 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Andrey Chernov Cc: Xin LI , Steven Chamberlain , kostikbel@gmail.com, "freebsd-security\@freebsd.org" , freebsd Subject: Re: arc4random weakness References: <20170313220639.GB65190@pyro.eu.org> <20170315130615.GC25448@pyro.eu.org> <5160183b-9778-59aa-6cf9-118014a588eb@freebsd.org> <8677f9d8-b326-2526-47ce-f2e18421c074@freebsd.org> Date: Sun, 19 Mar 2017 15:43:46 +0100 In-Reply-To: <8677f9d8-b326-2526-47ce-f2e18421c074@freebsd.org> (Andrey Chernov's message of "Thu, 16 Mar 2017 22:26:09 +0300") Message-ID: <861sttpbrx.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Mar 2017 14:44:00 -0000 Andrey Chernov writes: > Theo kindly explained that zeroing whole page instead of single variable > suits to his newest arc4random better, since clears two structs at once > (including ChaCha state), making some form of backward secrecy. Yes, avoiding leaking key material to child processes would be useful for more than just arc4random. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Sun Mar 19 15:12:41 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A1AAD12CBF; Sun, 19 Mar 2017 15:12:41 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id CECE8196; Sun, 19 Mar 2017 15:12:40 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 96C71B953; Sun, 19 Mar 2017 15:12:39 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id A357A46A4; Sun, 19 Mar 2017 16:12:29 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov Cc: freebsd-security@freebsd.org, Steven Chamberlain , freebsd-hackers@freebsd.org Subject: Re: arc4random weakness References: <20170313220639.GB65190@pyro.eu.org> <20170315130615.GC25448@pyro.eu.org> <5160183b-9778-59aa-6cf9-118014a588eb@freebsd.org> <86k27pz8sy.fsf@desk.des.no> <20170316131946.GN16105@kib.kiev.ua> Date: Sun, 19 Mar 2017 16:12:29 +0100 In-Reply-To: <20170316131946.GN16105@kib.kiev.ua> (Konstantin Belousov's message of "Thu, 16 Mar 2017 15:19:46 +0200") Message-ID: <86wpblnvvm.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Mar 2017 15:12:41 -0000 Konstantin Belousov writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Wouldn't it be possible to just set up the page entry but leave it > > unmapped, so that it is paged in (and zeroed if necessary) on first > > access? Thus, a process that uses arc4random() and fork()s would not > > incur a penalty until (and unless) the child uses arc4random() too. > This is how the forking code works, without any additional coding, > for the INHERIT_ZERO regions as well. Well then I don't see the problem... I just assumed from ache@'s objection that it wasn't the case. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Sun Mar 19 15:42:00 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2717D13554 for ; Sun, 19 Mar 2017 15:42:00 +0000 (UTC) (envelope-from thedarshansharma@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B3DB212D3 for ; Sun, 19 Mar 2017 15:42:00 +0000 (UTC) (envelope-from thedarshansharma@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B31BCD13553; Sun, 19 Mar 2017 15:42:00 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1245D13551 for ; Sun, 19 Mar 2017 15:42:00 +0000 (UTC) (envelope-from thedarshansharma@gmail.com) Received: from mail-qk0-x22f.google.com (mail-qk0-x22f.google.com [IPv6:2607:f8b0:400d:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A07312D2; Sun, 19 Mar 2017 15:42:00 +0000 (UTC) (envelope-from thedarshansharma@gmail.com) Received: by mail-qk0-x22f.google.com with SMTP id v127so94709877qkb.2; Sun, 19 Mar 2017 08:42:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=Op6egQnu1txoduzM1fT8Oz1T9kG98PVFxY/3Urq1O98=; b=pKal/bzu7QQ9oXgtJSqEXoXckuRuqE6Acw7hqG0F3+TXF+qpggfh8DsV37P/WKEJ+a 655Tz4I5BSI8/x4tho//Xe/0cWji6/36QCWlAwvrzZNBGxyNC89mJM24MlJray83mHet CMV5eAXDyVnyHzNFXFAqgzkfE62C6xknUaNGqJe+2y336/h2iwRSy2fSDKsg0bL1XM1E q9/Bq3eUmFF2PQcV3mjnbIs7MB4K3daMmgl7rqAEx1xCYcXEt9DPvTg+zBZXFQo/sAa0 5/lweOPu/636Xt2ZSVT9SyJ7JHJwlsKuGXcVd8+SPGdSu0hpiU5Lq0lon0Pfj6wz+wzR ta3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=Op6egQnu1txoduzM1fT8Oz1T9kG98PVFxY/3Urq1O98=; b=hIdBQHdbVl7CH3QvtreAxKTSLAsnup3STB41rwH41omgcyC2qxgSG+yBB99PxQOtp7 2Cf0bgeCy6i2tVPTPDYdlFTMqzyb5pFDvnVD0JtGqzXUWzbYEmk5a+Q6tCxO1+koy7ba 3c/dWnEjaL7Yz21kSRi13X4kD+T9Joas5e7G8OXXMqRlAbtQoY9YAdW2VNdraHnCivk3 /+w/aokjIATSbaRhm3HevFX/8wCsmJZ5Aridf6bEkQcq/iq2f04pozO0K06QtEEg/9I5 Z4kFyDOZCJHPfzC5hOF7Z37zmq7Vg8OnS/AgccaCcTnSKcfCXXQ/6kAQ0Rr08eQ/DEpp 8mRg== X-Gm-Message-State: AFeK/H0/hiluWkaonGN0wuziikLMEXvoulKxI3IYwvs9qym3cdQhLWLb4V/TFQxN9PGhBvMHOCOFF+zu3/zcfg== X-Received: by 10.233.239.201 with SMTP id d192mr21057799qkg.294.1489938119069; Sun, 19 Mar 2017 08:41:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.49.116 with HTTP; Sun, 19 Mar 2017 08:41:58 -0700 (PDT) From: Darshan Sharma Date: Sun, 19 Mar 2017 21:11:58 +0530 Message-ID: Subject: GSOC Project - "Replace mergesort implementation" To: brooks@freebsd.org Cc: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Mar 2017 15:42:00 -0000 Hello, I am Darshan Sharma from Chandigarh, India. I am in the prefinal year of my CS under graduation. I am doing it from Chandigarh College Of Engineering and Technology, Panjab University. After going through all the projects I find that I can do - "Replacement of merge sort". I had maintained GP of 8.0 out of 10.0 till now. I had done some projects with my professors in college and helped them many times. This is my first time I am applying for GSOC and I am very much motivated to do this project with FreeBSD. I have experience in C programming and doing it from past 3 years. I think I will be able to do this project before time period as I know how much crucial time for GSOC is. Till now I had seen the code of merge sort at Github and now know how to tackle it. Regards, Darshan Sharma Chandigarh College of Engineering and Technology, PU Contact - +91-8437073522 IRC nick - hitman1 From owner-freebsd-hackers@freebsd.org Sun Mar 19 21:20:53 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C719AD12E63 for ; Sun, 19 Mar 2017 21:20:53 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A7DD4100C for ; Sun, 19 Mar 2017 21:20:53 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mailman.ysv.freebsd.org (Postfix) id A734ED12E62; Sun, 19 Mar 2017 21:20:53 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6DE2D12E60 for ; Sun, 19 Mar 2017 21:20:53 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yw0-x234.google.com (mail-yw0-x234.google.com [IPv6:2607:f8b0:4002:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68A0E100A for ; Sun, 19 Mar 2017 21:20:53 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yw0-x234.google.com with SMTP id v198so79541317ywc.2 for ; Sun, 19 Mar 2017 14:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=2xNI8cKtOQr3ckPrBp1XO8Cm6Df2fBdIOSC+28ubjSw=; b=MA/J0sMMipf7SMdgba6QxMosEJBxZVhEuYtSL17Ak+e7GnqYpE6rtgRAjE7INu0BMj ltgDCAzo/MFr8Z8IsLwn7IPmIhFH0qdriWgjDX1VHYRhA+vqFDKrhJ3RmZx/DVzw2XJl 2d3E3e+V3U1bVGm5h9mFbY8FnhNtpBTisGPDb/4Nk1sQ2BzD53/iFbfMSkIqixLdjt0l G+FrnmRMyqKzKyYo1Cb0GLzFNr9UOWQCYd6EHNoIe6I3hl14HqURkLWljdjymRVe7y3W f73PO+0FnWIPkoRDuIU/9d1eFrvBcSaWrMeeairEBkDUVMMpo+mrO2ZuUfb74FjtvV+c MJ7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=2xNI8cKtOQr3ckPrBp1XO8Cm6Df2fBdIOSC+28ubjSw=; b=I8/dIChNOUrwDbDp2VN/4cKPsaKgeNJd8Cx3NEzDi9GEJFTaCdlqyoS6uFw8JDA/dr 6F7bfJV+4RxspZRDvoNXUIX3BWivsEUF7lJiWNM0Xt02NvnHuHQVAZe1M4QwqhF9On8h oF88XPso76oLe7yIn4Duya0DJbFzLC2Ef50tuymLWamnyTy16V8fQOAEARXj7w/8qi5I YWiBYedWJXrvW8gP4zESBbrr98FNa/3Vh4sGHtkeniiTUX76pDnwzy3MCDxDOHce4w+4 AsYe2sESRag+xwFn5mLLk6qn6e63/adDuTlsXybgfoLrPlneEOmxI3j4iBLRmQLJfXam 1GFg== X-Gm-Message-State: AFeK/H0QmKevtHgIwpOUGKgKuoCelt9CDugIwW1utXQmivVVwf4q5MI0gcTrvB6H9h9429TADg0r1WVFgvTgyg== X-Received: by 10.129.40.135 with SMTP id o129mr11913216ywo.236.1489958452558; Sun, 19 Mar 2017 14:20:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.51.198 with HTTP; Sun, 19 Mar 2017 14:20:22 -0700 (PDT) In-Reply-To: References: From: Ed Schouten Date: Sun, 19 Mar 2017 22:20:22 +0100 Message-ID: Subject: Re: GSOC Project - "Replace mergesort implementation" To: Darshan Sharma Cc: Brooks Davis , hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Mar 2017 21:20:53 -0000 Hi Darshan, 2017-03-19 16:41 GMT+01:00 Darshan Sharma : > After going through all the projects I find that I can do - "Replacement of merge sort". Nice! When working on this, be sure to experiment with Block Sort / Grail Sort: https://en.wikipedia.org/wiki/Block_sort https://github.com/Mrrl/GrailSort Basically it's a version of merge sort that uses an in-place list merging algorithm. This means that end up with a sorting algorithm that doesn't need to allocate any memory, but has the advantage over qsort() that it's stable and has a worst-case running time of O(n log n). That said, I don't know how expensive the in-place list merging is. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Mon Mar 20 13:12:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC432D139B8 for ; Mon, 20 Mar 2017 13:12:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 713DEF76 for ; Mon, 20 Mar 2017 13:12:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cpx6x-000FS0-On for freebsd-hackers@freebsd.org; Mon, 20 Mar 2017 16:12:35 +0300 Date: Mon, 20 Mar 2017 16:12:35 +0300 From: Slawa Olhovchenkov To: freebsd-hackers@freebsd.org Subject: jmalloc in shared memory Message-ID: <20170320131235.GB86500@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Mar 2017 13:12:38 -0000 How I can use jmalloc in shared memory? I.e. parent process do mmap w/ MAP_ANON|MAP_SHARED, create jmalloc "instance" in this memory and use jmalloc routines for memory management in this region. From owner-freebsd-hackers@freebsd.org Tue Mar 21 11:26:45 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C872CD16487; Tue, 21 Mar 2017 11:26:45 +0000 (UTC) (envelope-from julian@elischer.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A136A1A98; Tue, 21 Mar 2017 11:26:45 +0000 (UTC) (envelope-from julian@elischer.org) Received: from Julian-MBP3.local (115-166-5-28.dyn.iinet.net.au [115.166.5.28]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v2LBQc3e035713 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 21 Mar 2017 04:26:41 -0700 (PDT) (envelope-from julian@elischer.org) Subject: Re: effect of strip(1) on du(1) To: "Rodney W. Grimes" , Subbsd References: <201703030031.v230VvIl066398@pdx.rh.CN85.dnsmgr.net> Cc: Peter Jeremy , freebsd-hackers , freebsd-current Current From: Julian Elischer Message-ID: <5688704a-83fe-b5dc-fc21-03fb4a560a4b@elischer.org> Date: Tue, 21 Mar 2017 19:26:32 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <201703030031.v230VvIl066398@pdx.rh.CN85.dnsmgr.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Tue, 21 Mar 2017 11:42:00 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 11:26:45 -0000 On 3/3/17 8:31 am, Rodney W. Grimes wrote: >> On Fri, Mar 3, 2017 at 2:04 AM, Peter Jeremy wrote: >>> On 2017-Mar-02 22:29:46 +0300, Subbsd wrote: >>>> During some interval after strip call, du will show 512B for any file. >>>> If execute du(1) after strip(1) without delay, this behavior is reproduced 100%: >>> What filesystem are you using? strip(1) rewrites the target file and du(1) >>> reports the number of blocks reported by stat(2). It seems that you are >>> hitting a situation where the file metadata isn't immediately updated. >>> >>> -- >>> Peter Jeremy >> >> Got it. My filesystem is ZFS. Looks like when ZFS open and write data >> to file, we get wrong number of blocks during a small interval after >> writing. Thanks for pointing this out! > Even if that is the case file system cache effects should NOT be > visible to a userland process. This is NOT as if your running > 2 different processing beating on a file. Your test cases are > serialially syncronous shell invoked commands seperated with > && the results should be exact and predictable. > > When strip returns the operation from the userland perspecive > is completed and any and all processeses started after that > should have the view of the completed strip command. > > This IS a bug. actually it's all in how you look at it. Due to the way ZFS is doing the work and the metadata transitions, that amount of storage is actually directly attributable to that file's existence. so from that perspective the du is correct. From owner-freebsd-hackers@freebsd.org Tue Mar 21 22:57:45 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C0ECD17EE5 for ; Tue, 21 Mar 2017 22:57:45 +0000 (UTC) (envelope-from chris@sinjakli.co.uk) Received: from mail-wr0-x243.google.com (mail-wr0-x243.google.com [IPv6:2a00:1450:400c:c0c::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3CA210F4 for ; Tue, 21 Mar 2017 22:57:44 +0000 (UTC) (envelope-from chris@sinjakli.co.uk) Received: by mail-wr0-x243.google.com with SMTP id y90so713862wrb.1 for ; Tue, 21 Mar 2017 15:57:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sinjakli-co-uk.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=oLyVSepcjM9eLvq2+DjNDGrtclkkxRg7QA5nam/yVu4=; b=hb2lJUdtIVs4wwmxSkRCNSymt95CclbUgkKpTmHUCMA5wH9ZnZ7g764VILjR00z07a DmKBB90p7Cv96mwnyVGZBf9cBb5/i0aqsMSoldRK7Xw4RnRaBg71SCeaoTJA3BvShIkd 053M0DdQQ543bv5AvruksAsvU8IPeOIqb7P/9Xy3OqbBVkPmk7dFCQ049axtd7YAW+JI VnpbkO3C1k/DOOzQPhnBgBjESIh4sc6s/p6r7hFj0sRuY/30x7slELfEhgPCxfGPT6QG hR+m8i0LB2edyGdU4sz0CrnCg3kCkcV/UGwzYNjqPyB527olTLjDIP9E0hZ8GXXF5tEn GKUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=oLyVSepcjM9eLvq2+DjNDGrtclkkxRg7QA5nam/yVu4=; b=evB1NQQ++RVD9oovMthbQhOqc5+p+gcUMqeMjoT9Fa4hfdgcCtSv3nXVJSTSzM177y bOiZ1K2lZCokztW+Rxo6NuaLmWAJcCliCUUUfQIx7hpJzOY1cY6OLFs78W1a1TCAYYFJ +c73zz0LMm96+svPMt7zEBX3RMrbcBfXx41KRsP6yRHdoteWf56Ylm8Z9xUvKEYBChdS eXJmrzpNQlpLDIO2Uo6/V41zyPKYSjleB4rPQ0UMPhvuiGT2JJqPnTwnmOWvrOidSiQs K9qxfkenT4HSVoV+fR0IXfqtAPvZKNL1uACIrOxrhbufHdB0Nq+hFLoVf5bUFwVADAcl DI6w== X-Gm-Message-State: AFeK/H1wserB0P4b2iMzi7rEbWvBDH6PS9oAfP9AjahoNBiWff+lsaQ0K9WphVx9s6dbryWhk17lbKKJLSB0TQ== X-Received: by 10.223.155.17 with SMTP id b17mr31523878wrc.181.1490137062805; Tue, 21 Mar 2017 15:57:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.176.4 with HTTP; Tue, 21 Mar 2017 15:57:22 -0700 (PDT) From: Chris Sinjakli Date: Tue, 21 Mar 2017 22:57:22 +0000 Message-ID: Subject: A historical curiosity in su(1) To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 22:57:45 -0000 Hi folks, I'm doing some digging for a talk I'm working on for PGConf US[1]. The talk is one I've given before[2], and last time I gave it I left an open question right near the end. This time, I'd love to be able to answer it! As a heads-up, this is a behaviour I encountered through Ubuntu, but the code involved is present in a very similar form in FreeBSD, and probably originated here or in a shared ancestor (the trail goes cold at a 1994 import from BSD 4.4). The open question revolves around the way su(1) determines the user it is being invoked by. Specifically, it does this using a combination of the result of getuid and getlogin[3][4]. If calling getpwnam on the result of getlogin returns a passwd struct, and the uid in that struct matches the uid returned by getuid, it returns that passwd struct. If the getpwnam approach fails for either reason, it falls back to using the result of getpwuid on the result of getuid. The thing I'm curious about is why it goes to the trouble of trying to use the result of getpwnam/getlogin at all. The only time it will return something different from getpwuid/getuid is if there are two users with the same uid but different information in the rest of their passwd entry. Are there cases where you might want to set up a system this way? I've always avoided assigning the same uid to multiple users - it seems like a bad idea! Jumping back to the point I made in the talk, the result of getlogin can often be surprising. For example, a daemon restarted by a supervisor (e.g. upstart) will be associated with the user that issued the restart (i.e. getlogin would return "chris" if I restarted it, rather than something like "daemon-user"). Any daemon that calls su will run into this behaviour. If, as we do at my current employer, you store your human users in LDAP and your system users locally, that means a network round-trip every time su is called, just because a human happened to restart a daemon! For what it's worth, this isn't new behaviour. You can find extremely similar code in FreeBSD's su implementation, and it's been there since at least 1994[5], with something similar existing in the current code[6]. It seems likely to me that all of this behaviour needs to be preserved because somewhere out there, something will depend on it in a strange way. That said, if someone knows why it behaves like this, I'd be really interested to know, and to share the answer as part of my talk! Many thanks for reading this, and entertaining a historical itch. Chris [1]: http://www.pgconf.us/conferences/2017 [2]: https://www.youtube.com/watch?v=Tu-cf-Jki60 [3]: https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/src/su.c#L731 [4]: https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/libmisc/myname.c#L44 [5]: https://github.com/freebsd/freebsd/blob/f9ab90d9d6d02989a075d0f0074496d5b1045e4b/usr.bin/su/su.c#L125 [6]: https://github.com/freebsd/freebsd/blob/d8179bc9ec98d230e00546e4afa2a244e2f123ed/usr.bin/su/su.c#L255 From owner-freebsd-hackers@freebsd.org Tue Mar 21 16:29:35 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4ACBED16539 for ; Tue, 21 Mar 2017 16:29:35 +0000 (UTC) (envelope-from sparkvish1@gmail.com) Received: from mail-pf0-x22f.google.com (mail-pf0-x22f.google.com [IPv6:2607:f8b0:400e:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C1566AE for ; Tue, 21 Mar 2017 16:29:35 +0000 (UTC) (envelope-from sparkvish1@gmail.com) Received: by mail-pf0-x22f.google.com with SMTP id p189so58890090pfp.1 for ; Tue, 21 Mar 2017 09:29:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:mime-version:to:from:subject:date:importance; bh=PNepCk/tqxYype6jFMO11UbVc+ICswBRK2kgyThuDSk=; b=tpHO4ZlKg3gToDaLC5eVMOnP9LwgTt13VJYaW152Gdc0pa4X6KuMXTbt8F6KIUeMtk GSp5PkYCMjAMz9O4QcFYem6h/RzYZ+/xkgrY44LZZlkeOF7qy9lpqeozxyME5NieOfeU F0zLoyiipjjPx5d4mDi6r8hkUAjgv0Tv11Ot5M3NLg0XNP2b0o240aKNfTEYQkxqmqyx pAr+Hm/1BdFerQveJrGzT+xlh18FkFp3Lokwpu9kZUDizrlWYd+EtlH2p27YwvZrhllr b9eLrp5LPxxPRBaYnC4IFpmvk0WtKDKxg287mKbvQlYuw2bBQo4bSfgecpn4aOQqOPrB V1kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:mime-version:to:from:subject:date :importance; bh=PNepCk/tqxYype6jFMO11UbVc+ICswBRK2kgyThuDSk=; b=frruVjew82uSeujp1GpZK/Ldmv4OZHz6kmQoc8AFxNsNPNfBY47XeAKf8PxEyBBHzL GtuOCUDEM5kMIsQwKdsREn82Gakbv5VWqdER5TY21KfadXqk+bbAi2FwmBvM3nFnxtgx V3CQ2T7fNGJaLx2R8du4GyqBQH6TOEyVD1zCA7MCYPWqP68ghzibKfJQCmva7wHUiD/8 3+bWVIo0T7lidIo0/5LcBi75QNCTV8OdrcILJGGLL48ZKqvcaIaOuZW3poPrkGioDuFh WyuKsnXBYwE6Me+B/LPCEOUCs60j9DrUYPutmcOG91spYIoazZqlAVDDDdvWIXoH+6EJ /ONg== X-Gm-Message-State: AFeK/H2a64h7foJm5POD7qLnAecqZnT1TUbHPNJhU0822DH6zsnHGKrjAYQhTE4L4bTwrg== X-Received: by 10.98.87.1 with SMTP id l1mr21925566pfb.92.1490113774561; Tue, 21 Mar 2017 09:29:34 -0700 (PDT) Received: from ?IPv6:::ffff:192.168.1.8? ([117.192.162.132]) by smtp.gmail.com with ESMTPSA id v4sm40598084pfb.36.2017.03.21.09.29.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Mar 2017 09:29:34 -0700 (PDT) Message-ID: <58d154ee.0461620a.eb748.8401@mx.google.com> MIME-Version: 1.0 To: "freebsd-hackers@freebsd.org" From: Vishwas Suryanarayan Subject: [Advice Required] GSoC Proposal Date: Tue, 21 Mar 2017 21:59:41 +0530 Importance: normal X-Priority: 3 X-Mailman-Approved-At: Tue, 21 Mar 2017 23:13:16 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 16:29:35 -0000 Hello, As a pre-final year student pursuing my bachelor=E2=80=99s degree in Comput= er Science, this is my first time applying to the Google Summer of Code ini= tiative.=20 Of the organizations that were listed, I found FreeBSD as one among the mos= t interesting communities, with project ideas that seemed to align perfectl= y with my interests, namely networking and OS internals.=20 I would be privileged to be able to do my part by contributing to =C2=A0thi= s project. After going through the ideas list, I was most interested in the= network and OS related projects, as I am familiar with sockets and UNIX AP= Is. In particular, the usbdump, dummynet, and the boot environment manager were= quite fascinating.=20 I was led to understand that the students are encouraged to correspond with= the organization before submitting a proposal, and would like to enquire w= hat is expected in a proposal. Thank You, Vishwas S =C2=A0 From owner-freebsd-hackers@freebsd.org Tue Mar 21 23:14:29 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFEDED1678E for ; Tue, 21 Mar 2017 23:14:29 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F4001CE3 for ; Tue, 21 Mar 2017 23:14:29 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-wm0-f51.google.com with SMTP id v203so5250338wmg.0 for ; Tue, 21 Mar 2017 16:14:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=CYKVfsxdm+pMNh7TMZo7NYMWvoZqPlwPIL3aSUIHnpM=; b=Dq2tHhVSbp8XrpO+G06zHq2VqdJXGTp0RSU15yVf8RohGNa9rcTCZRMKh7wDanOzoO wVapZ4M6E9DMoGGi92Ep8ktyPaXevSz812qJzLFynVMloQSD2wwAxYxYz1E1zo8iu+cF FlWw4H0CCnia5ufMsvkeXTczknQY4FuciAuzpxLvFOrqUBES41ObmdJPlmx2DJFfchhG f9laRptPILy5Gyn9MgM3MV7cCAxs4wA40P1G0rW/J2AqQgqciWaVkM+RHfyx5l7AoYh1 gJjM8+SIYtdZA4rw4lr/1J69tcNBfJyM8waB52E+vWOf3SalVWttO1jgTRO8lIzeWvaS cKew== X-Gm-Message-State: AFeK/H3ubjJjxLFDN6L4qAgZznn9r4YmoD8+n4BKpHLSRwgAnO4WwvCw5nKemZM1TsZEdg== X-Received: by 10.28.140.135 with SMTP id o129mr4843262wmd.101.1490138062373; Tue, 21 Mar 2017 16:14:22 -0700 (PDT) Received: from mail-wr0-f179.google.com (mail-wr0-f179.google.com. [209.85.128.179]) by smtp.gmail.com with ESMTPSA id q29sm753950wrc.49.2017.03.21.16.14.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Mar 2017 16:14:22 -0700 (PDT) Received: by mail-wr0-f179.google.com with SMTP id g10so120863987wrg.2 for ; Tue, 21 Mar 2017 16:14:22 -0700 (PDT) X-Received: by 10.223.145.97 with SMTP id j88mr32151087wrj.178.1490138061932; Tue, 21 Mar 2017 16:14:21 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.169.4 with HTTP; Tue, 21 Mar 2017 16:14:21 -0700 (PDT) In-Reply-To: References: From: Conrad Meyer Date: Tue, 21 Mar 2017 16:14:21 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: A historical curiosity in su(1) To: Chris Sinjakli Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 23:14:30 -0000 Hi Chris, You might be interested in FreeBSD's copy of the CSRG repository.[0] Some of this logic was changed by karels@ in "r45520"[1] (of course it wasn't Subversion at the time). The previous change[2] message includes "use new getlogin()." Prior to that change, su did not use getlogin/getpwnam. The "BUGS" section of the getlogin(2) manual page says: In earlier versions of the system, getlogin() failed unless the process was associated with a login terminal. The current implementation (using setlogin()) allows getlogin to succeed even when the process has no con- trolling terminal. In earlier versions of the system, the value returned by getlogin() could not be trusted without checking the user ID. Porta- ble programs should probably still make this check. That might explain it? Best, Conrad [0]: https://svnweb.freebsd.org/csrg/usr.bin/su/su.c?view=annotate [1]: https://svnweb.freebsd.org/csrg?view=revision&revision=45520 [2]: https://svnweb.freebsd.org/csrg?view=revision&revision=45127 On Tue, Mar 21, 2017 at 3:57 PM, Chris Sinjakli wrote: > Hi folks, > > I'm doing some digging for a talk I'm working on for PGConf US[1]. > > The talk is one I've given before[2], and last time I gave it I left an open > question right near the end. This time, I'd love to be able to answer it! > > As a heads-up, this is a behaviour I encountered through Ubuntu, but the code > involved is present in a very similar form in FreeBSD, and probably originated > here or in a shared ancestor (the trail goes cold at a 1994 import from BSD > 4.4). > > The open question revolves around the way su(1) determines the user it is being > invoked by. Specifically, it does this using a combination of the result of > getuid and getlogin[3][4]. > > If calling getpwnam on the result of getlogin returns a passwd struct, and the > uid in that struct matches the uid returned by getuid, it returns that passwd > struct. > > If the getpwnam approach fails for either reason, it falls back to using the > result of getpwuid on the result of getuid. > > The thing I'm curious about is why it goes to the trouble of trying to use the > result of getpwnam/getlogin at all. The only time it will return something > different from getpwuid/getuid is if there are two users with the same uid but > different information in the rest of their passwd entry. > > Are there cases where you might want to set up a system this way? I've always > avoided assigning the same uid to multiple users - it seems like a bad idea! > > Jumping back to the point I made in the talk, the result of getlogin can often > be surprising. For example, a daemon restarted by a supervisor (e.g. upstart) > will be associated with the user that issued the restart (i.e. getlogin would > return "chris" if I restarted it, rather than something like "daemon-user"). Any > daemon that calls su will run into this behaviour. > > If, as we do at my current employer, you store your human users in LDAP and your > system users locally, that means a network round-trip every time su is called, > just because a human happened to restart a daemon! > > For what it's worth, this isn't new behaviour. You can find extremely similar > code in FreeBSD's su implementation, and it's been there since at least 1994[5], > with something similar existing in the current code[6]. > > It seems likely to me that all of this behaviour needs to be preserved because > somewhere out there, something will depend on it in a strange way. That said, if > someone knows why it behaves like this, I'd be really interested to know, and to > share the answer as part of my talk! > > Many thanks for reading this, and entertaining a historical itch. > > Chris > > > [1]: http://www.pgconf.us/conferences/2017 > > [2]: https://www.youtube.com/watch?v=Tu-cf-Jki60 > > [3]: > https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/src/su.c#L731 > > [4]: > https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/libmisc/myname.c#L44 > > [5]: > https://github.com/freebsd/freebsd/blob/f9ab90d9d6d02989a075d0f0074496d5b1045e4b/usr.bin/su/su.c#L125 > > [6]: > https://github.com/freebsd/freebsd/blob/d8179bc9ec98d230e00546e4afa2a244e2f123ed/usr.bin/su/su.c#L255 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Mar 21 23:30:14 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 606F2D16B71 for ; Tue, 21 Mar 2017 23:30:14 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E601411B3 for ; Tue, 21 Mar 2017 23:30:13 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-wm0-f51.google.com with SMTP id n11so22936449wma.1 for ; Tue, 21 Mar 2017 16:30:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=46lrGVbx8o6TTkjexp5mO9/HJJ9tD8y/2jS+Q9XH4Uk=; b=slCK1kvVhqepVax32qDMNRjhtvHvcu9MNoc/LBUkbrR4vHC50Ic5zyuft8VWoylWBL Dg9qpRlXY/enKdcO8whAjs2YwJ3lynn/X2Owt7g4UirsaWyFqCmyJv1c7k5bXDQnh3nF 8BicOpkHDWo2LUBHO0UotBWgyvqh/t2AoNZCWdc/vsyUjTu2hPbNvwa+xFabr07WB4Vi JtW4NkAj/fs2NcMNMrZ1INFWbM8T/FiUSrh78f93w6wIw0bGYqhBq8qyJONPrGq9C7xk MMwe2lAaTtpdm9oEQvtOSnl/yGhISfTCNNmRRuBSe0PidsVxTFT0NFrZL+QjP0LzVI4R CNCQ== X-Gm-Message-State: AFeK/H1wopFVyAYA0fNINETBGAMK8QSsi/03DRTnxYduAWR/1SThFN9o8MBfiR9RB9Bx7w== X-Received: by 10.28.47.15 with SMTP id v15mr4960746wmv.116.1490138606529; Tue, 21 Mar 2017 16:23:26 -0700 (PDT) Received: from mail-wr0-f175.google.com (mail-wr0-f175.google.com. [209.85.128.175]) by smtp.gmail.com with ESMTPSA id y4sm16409791wra.16.2017.03.21.16.23.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Mar 2017 16:23:26 -0700 (PDT) Received: by mail-wr0-f175.google.com with SMTP id g10so120936545wrg.2 for ; Tue, 21 Mar 2017 16:23:26 -0700 (PDT) X-Received: by 10.223.145.97 with SMTP id j88mr32169737wrj.178.1490138606284; Tue, 21 Mar 2017 16:23:26 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.169.4 with HTTP; Tue, 21 Mar 2017 16:23:25 -0700 (PDT) In-Reply-To: References: From: Conrad Meyer Date: Tue, 21 Mar 2017 16:23:25 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: A historical curiosity in su(1) To: Chris Sinjakli Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 23:30:14 -0000 On Tue, Mar 21, 2017 at 4:14 PM, Conrad Meyer wrote: > Hi Chris, > > You might be interested in FreeBSD's copy of the CSRG repository.[0] > Some of this logic was changed by karels@ in "r45520"[1] (of course it > wasn't Subversion at the time). The previous change[2] message > includes "use new getlogin()." Prior to that change, su did not use > getlogin/getpwnam. > > The "BUGS" section of the getlogin(2) manual page says: > > In earlier versions of the system, getlogin() failed unless the process > was associated with a login terminal. The current implementation (using > setlogin()) allows getlogin to succeed even when the process has no con- > trolling terminal. In earlier versions of the system, the value returned > by getlogin() could not be trusted without checking the user ID. Porta- > ble programs should probably still make this check. > > That might explain it? > > Best, > Conrad > > [0]: https://svnweb.freebsd.org/csrg/usr.bin/su/su.c?view=annotate > [1]: https://svnweb.freebsd.org/csrg?view=revision&revision=45520 > [2]: https://svnweb.freebsd.org/csrg?view=revision&revision=45127 Additionally, that getlogin(2) BUGS blurb dates to karels in 1990 as well.[3] The commit message was "good enough for now." A different time :-). [3]: https://svnweb.freebsd.org/csrg?view=revision&revision=43655 From owner-freebsd-hackers@freebsd.org Tue Mar 21 23:12:30 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06D69D16679 for ; Tue, 21 Mar 2017 23:12:30 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB7F81B89 for ; Tue, 21 Mar 2017 23:12:29 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v2LNCP3A071746; Tue, 21 Mar 2017 16:12:25 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v2LNCN7A071745; Tue, 21 Mar 2017 16:12:23 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201703212312.v2LNCN7A071745@pdx.rh.CN85.dnsmgr.net> Subject: Re: A historical curiosity in su(1) In-Reply-To: To: Chris Sinjakli Date: Tue, 21 Mar 2017 16:12:23 -0700 (PDT) CC: freebsd-hackers@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Wed, 22 Mar 2017 04:05:58 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Mar 2017 23:12:30 -0000 > Hi folks, > > I'm doing some digging for a talk I'm working on for PGConf US[1]. > > The talk is one I've given before[2], and last time I gave it I left an open > question right near the end. This time, I'd love to be able to answer it! > > As a heads-up, this is a behaviour I encountered through Ubuntu, but the code > involved is present in a very similar form in FreeBSD, and probably originated > here or in a shared ancestor (the trail goes cold at a 1994 import from BSD > 4.4). You need to go find the UCB sccs files for history prior to that. They are out there on the net, you should be able to back track this into the 80's using them. > The open question revolves around the way su(1) determines the user it is being > invoked by. Specifically, it does this using a combination of the result of > getuid and getlogin[3][4]. > > If calling getpwnam on the result of getlogin returns a passwd struct, and the > uid in that struct matches the uid returned by getuid, it returns that passwd > struct. > > If the getpwnam approach fails for either reason, it falls back to using the > result of getpwuid on the result of getuid. > > The thing I'm curious about is why it goes to the trouble of trying to use the > result of getpwnam/getlogin at all. The only time it will return something > different from getpwuid/getuid is if there are two users with the same uid but > different information in the rest of their passwd entry. > > Are there cases where you might want to set up a system this way? I've always > avoided assigning the same uid to multiple users - it seems like a bad idea! By default the system ships that way, uid 0 is root and is also toor. So yes, the case is very much already there. This is how you give 2 users the same access to the same files with 2 different passwords, not ideal, but used more often than one might think. > Jumping back to the point I made in the talk, the result of getlogin can often > be surprising. For example, a daemon restarted by a supervisor (e.g. upstart) > will be associated with the user that issued the restart (i.e. getlogin would > return "chris" if I restarted it, rather than something like "daemon-user"). Any > daemon that calls su will run into this behaviour. su - should resolve that behavior. > If, as we do at my current employer, you store your human users in LDAP and your > system users locally, that means a network round-trip every time su is called, > just because a human happened to restart a daemon! > > For what it's worth, this isn't new behaviour. You can find extremely similar > code in FreeBSD's su implementation, and it's been there since at least 1994[5], > with something similar existing in the current code[6]. > > It seems likely to me that all of this behaviour needs to be preserved because > somewhere out there, something will depend on it in a strange way. That said, if > someone knows why it behaves like this, I'd be really interested to know, and to > share the answer as part of my talk! I think the system itself depends on it because of the root/toor user accounts. > Many thanks for reading this, and entertaining a historical itch. > > Chris > > > [1]: http://www.pgconf.us/conferences/2017 > > [2]: https://www.youtube.com/watch?v=Tu-cf-Jki60 > > [3]: > https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/src/su.c#L731 > > [4]: > https://github.com/shadow-maint/shadow/blob/db57db52cfd99069c33604eb4a0fee56eb659133/libmisc/myname.c#L44 > > [5]: > https://github.com/freebsd/freebsd/blob/f9ab90d9d6d02989a075d0f0074496d5b1045e4b/usr.bin/su/su.c#L125 > > [6]: > https://github.com/freebsd/freebsd/blob/d8179bc9ec98d230e00546e4afa2a244e2f123ed/usr.bin/su/su.c#L255 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-hackers@freebsd.org Wed Mar 22 04:13:11 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CC3DD14333 for ; Wed, 22 Mar 2017 04:13:11 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17A861039; Wed, 22 Mar 2017 04:13:11 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id D91E02C76; Wed, 22 Mar 2017 04:13:09 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Wed, 22 Mar 2017 04:13:07 +0000 From: Glen Barber To: "Rodney W. Grimes" Cc: Chris Sinjakli , freebsd-hackers@freebsd.org, "Rodney W. Grimes" Subject: Re: A historical curiosity in su(1) Message-ID: <20170322041307.GB42680@FreeBSD.org> References: <201703212312.v2LNCN7A071745@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="WhfpMioaduB5tiZL" Content-Disposition: inline In-Reply-To: <201703212312.v2LNCN7A071745@pdx.rh.CN85.dnsmgr.net> X-Operating-System: FreeBSD 11.0-STABLE amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer X-Spidey-Sense: Uh oh, Peter logged in User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 04:13:11 -0000 --WhfpMioaduB5tiZL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 21, 2017 at 04:12:23PM -0700, Rodney W. Grimes wrote: > > Jumping back to the point I made in the talk, the result of getlogin ca= n often > > be surprising. For example, a daemon restarted by a supervisor (e.g. up= start) > > will be associated with the user that issued the restart (i.e. getlogin= would > > return "chris" if I restarted it, rather than something like "daemon-us= er"). Any > > daemon that calls su will run into this behaviour. >=20 > su - should resolve that behavior. >=20 For clarification with special characters, this is 'su -', which indeed does resolve the behavior described. From the su(1) manual page: su -l foo Simulate a login for user foo. su - foo Same as above. su - Simulate a login for root. Glen --WhfpMioaduB5tiZL Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCAAGBQJY0fnTAAoJEAMUWKVHj+KTQuYQAISgvJQLcgtSfcFZwOSLPuVu eTWJ7cl4VUKR3Q7c6yp6gyEg7DLa5Ba2RMLFzOxNgvTaTblnIFTGpypN4brerCdW AU4ofPOc5m95t5WHkWmNx9eAP+EvYWsTlNF/28ko8bK3beyX3sGs0PXSRrN5+J5K TZsvJpLS7+emC0sRvfXjWhBfCfh9JI4VCVaONkzcphSeTgmyJVo/BZHPYDqec1xi Vq5m9Xqsn9o5IO5DIAGWroJXHhA9Kl4R/7m1iVQiUu98m4VIhZQbe5Nk7tHNriCW HKNLwcuIysauNEq/p7EVxN8kBNN/77abS4ePCkyjBidwGF+BBv3orslydlUsVrDw OobXdygPGUyK5xA9D/9cH4aCg49SDtGZrh+1RJLyFcRgUqD4EDpLxiLqqys5eXN1 FKHPCRFw0kXtG4/Bes/SpIJ8mfmKUYEZVp7mp+TjS923tDiOuIXIvWOYBqAnAF65 e6nA9d9jQbSOMst+YfYoaMhxD/WMpVXU7ZwKAbELUEvud/HhDcIENYwRwy6GAY0K gw3/bpj1Bb8twJRnaEHAbyoBbz9gibons5WR+LqYM7k+AdV90Vev1sQM6wPVM70F I8iWlNjZjpoQSUwUnA+psx8YLJ5Od0nZoQSi7hRKva+kJfozTj9f30Xc5akZF+Cv wLEwa8oDvlLmWRm5vAYO =gRA7 -----END PGP SIGNATURE----- --WhfpMioaduB5tiZL-- From owner-freebsd-hackers@freebsd.org Wed Mar 22 09:19:48 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58B96D1573F; Wed, 22 Mar 2017 09:19:48 +0000 (UTC) (envelope-from tomek.cedro@gmail.com) Received: from mail-ot0-x244.google.com (mail-ot0-x244.google.com [IPv6:2607:f8b0:4003:c0f::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 160521F66; Wed, 22 Mar 2017 09:19:48 +0000 (UTC) (envelope-from tomek.cedro@gmail.com) Received: by mail-ot0-x244.google.com with SMTP id y88so1946913ota.1; Wed, 22 Mar 2017 02:19:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=udwVm6logtLyIqKroUrcXN3HZDV5bSjQ3Z+kWLnJZ5g=; b=Yzgl0wvm25TB84cmtZ7hkSfViFRfN4hO1TiyEED7Z3zy2xmno37DSe73w9lRg0pK+o qGKA6XWUv3OSdv6lvRRUhCmerpgVE94MFyVjhqLeG3d8xNIyzW2avoJuKdm7z9xW0Ebx 1+FMRRa3FQo5lHY3Bz6gEv+R6h7fALITkl5eEEBXGBYiX+XwstX9YG2qZj2zxLSU3NdU pNtk8NTyJ66f2aGHnMf2tQQN6yDqPZ/RqdhLtHoa55q1iRu8JyHd5+VYzFDJJdd4n9A7 DM5oDiz9pjnRFMjUoNL4tZ74r1iKC0s/HGu8mbHpRzs8dpxXOJR0L30Gcm/C0GoJY3Hn VvEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=udwVm6logtLyIqKroUrcXN3HZDV5bSjQ3Z+kWLnJZ5g=; b=HF7+Q8WALKiPP96nZ/Upd6s9PmE+Rh2Nj6nqGggm9WZFRkIVFjcBvHJqygoWHn1f6A JtF1xUmbriCcptEd03/0ANr4FtN4oiKU5Q2dLmLNBUVhJVtEtowdiVa1Y88YFGZ7YqPk vCxUrzxkMefWSIWNjq/G9BHgKfRtPZPS8ENOdcIehRrN7vBSCtBSkOjPgO4IoVCu2tgZ URGH10zrDtewNkrbH5E2B3ovQWG9gfMeHkjxPQi0TdeOaE3c2cNM1VDHm/MQF9ixzmKL uU00Quv70Gx8zxvLJySncd6lw1pZn5/Ls3z6XbbbdH3dJA9a/J4ZanjkCdR0DrRd3og6 vA7w== X-Gm-Message-State: AFeK/H3Ho7RDvMLGD82hI+n/ure+ipLbkJ2Ibqk1Wpj/QQgxckYlxi7JEhMaHk2JiSDF6qQz7u0R3kOAIqQ3Cw== X-Received: by 10.157.43.232 with SMTP id u95mr547006ota.70.1490174387448; Wed, 22 Mar 2017 02:19:47 -0700 (PDT) MIME-Version: 1.0 Sender: tomek.cedro@gmail.com Received: by 10.157.18.211 with HTTP; Wed, 22 Mar 2017 02:19:47 -0700 (PDT) Received: by 10.157.18.211 with HTTP; Wed, 22 Mar 2017 02:19:47 -0700 (PDT) In-Reply-To: References: From: Tomasz CEDRO Date: Wed, 22 Mar 2017 10:19:47 +0100 X-Google-Sender-Auth: xbwbah1jmaH9JSRs_AvVBoFj2Hk Message-ID: Subject: Re: Filtering Against Persistent Firmware Rootkits - BadUSB, HDDHack, UEFI To: grarpamp Cc: FreeBSD Questions Mailing List , freebsd-hardware@freebsd.org, freebsd-security@freebsd.org, freebsd-hackers@freebsd.org X-Mailman-Approved-At: Wed, 22 Mar 2017 11:35:35 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 09:19:48 -0000 I have created www.libswd.com and www.iCeDeROM.com for low-level access to embedded system resources, all developed on FreeBSD :-) Still no interest from investors/sponsors to support iCeDeROM so I could focus 108% on its development :-/ -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From owner-freebsd-hackers@freebsd.org Wed Mar 22 11:43:27 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72AE1D153FE for ; Wed, 22 Mar 2017 11:43:27 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:375::1:5]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 367E01F37 for ; Wed, 22 Mar 2017 11:43:27 +0000 (UTC) (envelope-from Alexander@leidinger.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1490182972; bh=S6XC4xSpsOUvEDvqKTLwZN6/Z9I48vfZtaZCk3gXQm4=; h=Date:From:To:Cc:Subject:In-Reply-To; b=bx3FEUBXB44toc8gTx01NolwtjiRNkwXyy9HGrqlAs2ZMNx5uHrHZbsWwulsK2pDr KnMu57Mxcp8gPL++z9QurotzDaQwQ2UDK/tZNvhj5FR2cWnmYkoe8qTg+A9n4A498Q RauLqhW8d1U5k3mBEtv2hIbTojr26uI+l9EYG/4aFshA3PYOjfZxvkltDWRQCm3sSn /vvwhDEPjmGeuhSUzuiAPDPvfiS2+IjRaPHea9RVJWxVsANZ/5zmHGLGGSiUiHU3P8 sYwRfKOuIn+4a9ATCvJydb5TqooWqk1qvW+osq9eUqwYPeJcNRcqMxUKg0CaPby+QS emO/BcG41wy4Q== DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1490182996; bh=S6XC4xSpsOUvEDvqKTLwZN6/Z9I48vfZtaZCk3gXQm4=; h=Date:From:To:Cc:Subject:In-Reply-To; b=jFC7ucbboY2ollIlW1X7mmBvabOv/JvW2acLMcZotm/gy2rKMG4tk4mCfr0rJPFZZ elra3XXYfzy7gP01hKGZlNty74T+41d106L3ssllPHssf0rcUOAafPHEfN1YppKBj9 mz0PvZowUNKXDsX9QJjy8Xi4yBwKycab5TvyOZ3O15h9kdHGu81y3esWGfzvT55FBd Ef9m/lc+2MmpRZVGgqKG4tcVpbL2Llbe+MyT3dKEFsa3Qa/+gghM+qBR/nEX4ut0S3 2WawdqjWwLgoLLpj18bUBx+830I9ywLWNFZtzrp0BKKSBJ1nkxiUJb/p2d9oFvrdon gsyv4Q4lkGIpQ== Date: Wed, 22 Mar 2017 12:42:51 +0100 Message-ID: <20170322124251.Horde.z8ZVs-2rs9rED8SXmAid8VT@webmail.leidinger.net> From: Alexander Leidinger To: Vishwas Suryanarayan Cc: freebsd-hackers@freebsd.org Subject: Re: [Advice Required] GSoC Proposal In-Reply-To: <58d154ee.0461620a.eb748.8401@mx.google.com> User-Agent: Horde Application Framework 5 Content-Type: multipart/signed; boundary="=_RiD-D1Tp9H_Hd7f2bvTzSQh"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 22 Mar 2017 12:12:10 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 11:43:27 -0000 This message is in MIME format and has been PGP signed. --=_RiD-D1Tp9H_Hd7f2bvTzSQh Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting Vishwas Suryanarayan (from Tue, 21 Mar=20=20 2017=2021:59:41 +0530): > I was led to understand that the students are encouraged to=20=20 >=20correspond with the organization before submitting a proposal, and=20= =20 >=20would like to enquire what is expected in a proposal. Please read the page https://www.freebsd.org/projects/summerofcode.html we have some things explained there. If you've read it already, please=20= =20 explain=20where you see need of more information or explain what is not=20= =20 clear=20to you. Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_RiD-D1Tp9H_Hd7f2bvTzSQh Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJY0mM7AAoJEKrxQhqFIICE/EMP/joZZbM5f+oOfSAm7nOby9NE LmWWrOAiDuV64MOI8oZYrqA5uIfzqGJqrWojIxZ0pO8wkAhWRpM5BnS4p+O9i6oI lLs8yq0lxCgZSLakk3XRpHykNyRALG8eqDPJrQU5R4Z2YLEkaeEnkVfFqTmrV1UP 1mHh+fIIMFJtJWEFtrr37ZKcEoOnnabr9U0tGaHWWWkpx0oYsdZPB1EwOUkiowLU KJa6pojO1SiDv9ICcObSmFZi1RGIoRm71rnpTe9EwWzg3vKqdr14/P2NCAAE+HEq y8nMqH/gb6Ts0k5CNWyMbaBDO5htk/jf+9KDjfOaqgvWl8G7t9ZKa5At86fr1Gjd F2nVQMyu31WqSmqZGzG9Ll/eg4hoafwJAomLakF5qHP7yrD0Abk8kB/0dcApZxNO P9NgkC6C97n6XKMi0Ub7Hb6FzkkMDPYfg0ihSo4m9X6bJizmsaNyzERBE7t/62YW 5ArlYbVCfZkn6xMOBphHXi9okOzr62dWjxQ2kzbFXeOqGFA9wPhy8scbxfWFSIOk I8Jc3XLGGmA6IVZg+lFuwDdL/qb+tySSQht9ZXNfCrcFyhgsKjidEHMZf6USDCFa GVCcLIZ9dbGBFxryrzLOuhydvZSKkdde3jlxAqx4afzWanu1InABZziSusG/iJmX Xv5FugjZ4LuGMpI5mwpo =dB4e -----END PGP SIGNATURE----- --=_RiD-D1Tp9H_Hd7f2bvTzSQh-- From owner-freebsd-hackers@freebsd.org Wed Mar 22 13:18:51 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CA43D1751D for ; Wed, 22 Mar 2017 13:18:51 +0000 (UTC) (envelope-from jamie@dyslexicfish.net) Received: from dyslexicfish.net (deadcat.mail.dyslexicfish.net [45.63.12.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D67FD120B for ; Wed, 22 Mar 2017 13:18:50 +0000 (UTC) (envelope-from jamie@dyslexicfish.net) Received: from dyslexicfish.net (deadcat.mail.dyslexicfish.net [45.63.12.202]) by dyslexicfish.net (8.14.5/8.14.5) with ESMTP id v2MDIbwx037112; Wed, 22 Mar 2017 13:18:37 GMT (envelope-from jamie@dyslexicfish.net) Received: (from jamie@localhost) by dyslexicfish.net (8.14.5/8.14.5/Submit) id v2MDIboh037111; Wed, 22 Mar 2017 13:18:37 GMT (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <201703221318.v2MDIboh037111@dyslexicfish.net> Date: Wed, 22 Mar 2017 13:18:37 +0000 To: freebsd-hackers@freebsd.org, chris@sinjakli.co.uk Subject: Re: A historical curiosity in su(1) References: In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (dyslexicfish.net [45.63.12.202]); Wed, 22 Mar 2017 13:18:38 +0000 (GMT) X-Mailman-Approved-At: Wed, 22 Mar 2017 14:02:27 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 13:18:51 -0000 Chris Sinjakli wrote: > The thing I'm curious about is why it goes to the trouble of trying to use the > result of getpwnam/getlogin at all. The only time it will return something > different from getpwuid/getuid is if there are two users with the same uid but > different information in the rest of their passwd entry. > > Are there cases where you might want to set up a system this way? I've always > avoided assigning the same uid to multiple users - it seems like a bad idea! I do this. I use a number of different mailboxes for different lists etc. each coresponding email address is mapped to a unique unix username (hence, a unique unix-mail user) So, a few different mailboxes in /var/mail - nothing unusual so far. These additional users are only used for the email processing, and only by me, so to avoid having to play around with file permissions etc., all of them use the same uid/gid as my main user. As long as the main user appears first in the passwd and group files, it seems to not cause any problems. Another use I once had: A non-privileged user needed both interactive and ftp access. I only allow ssh for interactive use, but he was unable to use scp/sftp Only having ssh interactive, but allowing clear-text passwords for ftp, would defeat the security purpose, so, I created one ftp-only account (blocked from access in ssh config) and one ssh-only account (blocked from ftp access with a special default shell, and also configuration in /etc/ftpusers, if I recall.) I ensured both used different passwords, but gave both the same uid/gid so that there wouldn't be permission problems with ftp vs interactive files. (Yes, I know anyone who knew the ftp password could maybe ftp some trojan in, but that was beyond the scope here.) TL: DR: One account, but different passwords and login 'shells' Cheers! Jamie. From owner-freebsd-hackers@freebsd.org Wed Mar 22 18:25:57 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86AE5D180D7 for ; Wed, 22 Mar 2017 18:25:57 +0000 (UTC) (envelope-from reichert@numachi.com) Received: from away.numachi.com (away.numachi.com [66.228.38.138]) by mx1.freebsd.org (Postfix) with SMTP id 3E0681C17 for ; Wed, 22 Mar 2017 18:25:56 +0000 (UTC) (envelope-from reichert@numachi.com) Received: (qmail 10558 invoked from network); 22 Mar 2017 18:19:14 -0000 Received: from unknown (HELO meisai.numachi.com) (71.168.69.18) by away.numachi.com with SMTP; 22 Mar 2017 18:19:14 -0000 Received: (qmail 154 invoked by uid 1001); 22 Mar 2017 18:01:44 -0000 Date: Wed, 22 Mar 2017 14:01:44 -0400 From: Brian Reichert To: Chris Sinjakli Cc: freebsd-hackers@freebsd.org Subject: Re: A historical curiosity in su(1) Message-ID: <20170322180144.GE84031@numachi.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.9i X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 18:25:57 -0000 On Tue, Mar 21, 2017 at 10:57:22PM +0000, Chris Sinjakli wrote: > Hi folks, > > I'm doing some digging for a talk I'm working on for PGConf US[1]. > > The talk is one I've given before[2], and last time I gave it I left an open > question right near the end. This time, I'd love to be able to answer it! > > As a heads-up, this is a behaviour I encountered through Ubuntu, but the code > involved is present in a very similar form in FreeBSD, and probably originated > here or in a shared ancestor (the trail goes cold at a 1994 import from BSD > 4.4). Did you dive into this repo? Description: https://www.dmst.aueb.gr/dds/pubs/conf/2015-MSR-Unix-History/html/Spi15c.html The evolution of the Unix operating system is made available as a version-control repository, covering the period from its inception in 1972 as a five thousand line kernel, to 2015 as a widely-used 26 million line system. The repository contains 659 thousand commits and 2306 merges. ... It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, Berkeley University, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. Repo itself: https://github.com/dspinellis/unix-history-repo > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Brian Reichert BSD admin/developer at large From owner-freebsd-hackers@freebsd.org Thu Mar 23 03:04:02 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B47F4D19FD6 for ; Thu, 23 Mar 2017 03:04:02 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8272D1908; Thu, 23 Mar 2017 03:04:02 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (115-166-5-28.dyn.iinet.net.au [115.166.5.28]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v2N33tH9045477 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 22 Mar 2017 20:03:59 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Warner Losh , Allan Jude References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> Cc: "freebsd-hackers@freebsd.org" , Saurav Sachidanand From: Julian Elischer Message-ID: <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> Date: Thu, 23 Mar 2017 11:03:49 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 03:04:02 -0000 On 1/3/17 4:14 pm, Julian Elischer wrote: > On 28/2/17 2:01 am, Warner Losh wrote: >> On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude >> wrote: >>> On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >>> wrote: >>>> Hello FreeBSD community, >>>> >>>> I'm >>>> Saurav Sachidanand, and I'm >>>> a CS sophomore studying in India >>>> . >>>> I have an interest in operating systems development and wish to >>>> contribute >>>> to the FreeBSD community. I'm proficient with C and have some >>>> experience in >>>> kernel programming. Hence, I'd like to propose an original >>>> project for >>>> GSoC >>>> 2017 that I feel would benefit this community. >>>> >>>> In past years, the Lua interpreter was ported to run inside the >>>> Linux >>>> and >>>> NetBSD kernel [1]. Lua was chosen because it's interpreter is very >>>> small (~240 >>>> KB) compared to that of Python or Ruby, it's MIT licensed, and is >>>> almost >>>> freestanding. A working demonstration of it is a packet filtering >>>> algorithm >>>> written entirely in kernel Lua [2]. >>>> >>>> Specifically, my proposal would be to port the following that are >>>> currently >>>> written for NetBSD: >>>> - the modified Lua VM source code with _KERNEL preprocessor >>>> directives >>>> to >>>> exclude user-space functionality like floating point, the io and os >>>> module >>>> in the standard library, etc. [3] >>>> - the kernel module device driver for /dev/lua, to which Lua scripts >>>> are >>>> fed to be executed [4], [5] >>>> - the luactl user-space program to control the Lua device and a >>>> couple >>>> of >>>> sysctl variables which serve similar purpose [6], [7] >>>> >>>> And then: >>>> - run the Lua test suite targeting whatever we support in the >>>> kernel to >>>> make sure it works [8] >>>> - and write Lua bindings to the kernel interfaces that would >>>> interest >>>> the >>>> FreeBSD community >>>> >>>> Since NetBSD and FreeBSD have similar kernel interfaces (mutexes, >>>> linked >>>> lists, device switch interface), the porting shouldn't involve >>>> too much >>>> code refactoring. Also, this would all be an experiment in that we >>>> don't >>>> fully know what the real world use cases might be, but it would >>>> attract >>>> more people to writing kernel code who otherwise wouldn't because of >>>> having >>>> to do everything in C. And it would be interesting to carry out >>>> it out >>>> in >>>> FreeBSD as well since it has a larger community than NetBSD. >>>> >>>> I humbly request anyone who is interested in this project to be my >>>> potential mentor(s) for GSoC. >>>> >>>> More slides on kernel Lua in NetBSD - [9], [10]. >>>> >>>> Thanks, >>>> Saurav >>>> >>>> [1] - http://www.netbsd.org/~lneto/dls14.pdf >>>> [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf >>>> [3] - >>>> https://github.com/jsonn/src/tree/trunk/external/mit/lua/dist/src >>>> [4] - >>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/lua >>>> [5] - >>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/luasystm >>>> >>>> [6] - >>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin/luactl >>>> [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current >>>> [8] - http://www.lua.org/tests/ >>>> [9] - >>>> https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012/kernel_mode_lua.pdf >>>> >>>> [10] - https://www.lua.org/wshop13/Cormack.pdf >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org" >>> This may be quite a nice thing to have. Another upcoming use for >>> LUA in the kernel is ZFS Channel Programs. These allow a number of >>> ZFS operations to be completed as a single atomic transaction. >>> >>> I would hope we could structure this in such a way as to not end >>> up with two copies of Lua in the kernel. >> There's also a 3/4 finished lua in the boot loader that you might be >> able to leverage as well.... > > I'd like to see that finished. While Devin has done Heroic work with > the forth in the loader, I think it's time has come. > It' be nice to have something a little less '60s. some news: Pedro Arthur who did the project just mailed me: =================== Hi, I've just finished getting the lua-bootloader code on top of HEAD, it should compile and run fine. The code can be found in [1]. [1] - https://github.com/grandao/freebsd/tree/projects/lua-bootloader =================== so it's up to date and we should look at it again > >> >> Warner >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" >> > From owner-freebsd-hackers@freebsd.org Thu Mar 23 03:41:40 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37AD7D1975B for ; Thu, 23 Mar 2017 03:41:40 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-211-178.reflexion.net [208.70.211.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE4FC15F9 for ; Thu, 23 Mar 2017 03:41:39 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 31842 invoked from network); 23 Mar 2017 03:41:32 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 23 Mar 2017 03:41:32 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.30.2) with SMTP; Wed, 22 Mar 2017 23:41:32 -0400 (EDT) Received: (qmail 28816 invoked from network); 23 Mar 2017 03:41:31 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 23 Mar 2017 03:41:31 -0000 Received: from [192.168.1.111] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 41926EC7B14 for ; Wed, 22 Mar 2017 20:41:31 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: Leak in file backed swap Message-Id: Date: Wed, 22 Mar 2017 20:41:29 -0700 To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 03:41:40 -0000 Giulio Ferro auryn at zirakzigil.org wrote on Thu Feb 2 15:05:11 UTC 2017 of problems with a file system based swap space (a swap file). FYI: See Bugzilla 206048 and its Comment #7 in particular, that quotes Konstantin Belousov from an arm list submittal. (Repeated later below.) Many folks in many environments have observed lock ups for files used as swap space, UFS files included (without any ZFS present). Comment #3 from Tom Vijlbrief reported: stress -d 2 -m 3 --vm-keep or the like will hang up. (He listed a 1 GByte RAM and 1 GByte swapfile as an example context for hanging during linking of the kernel.) On 2017-Feb-13, at 7:20 PM, Konstantin Belousov wrote on the freebsd-arm list: . . . swapfile write requires the write request to come through the filesystem write path, which might require the filesystem to allocate more memory and read some data. E.g. it is known that any ZFS write request allocates memory, and that write request on large UFS file might require allocating and reading an indirect block buffer to find the block number of the written block, if the indirect block was not yet read. As result, swapfile swapping is more prone to the trivial and unavoidable deadlocks where the pagedaemon thread, which produces free memory, needs more free memory to make a progress. Swap write on the raw partition over simple partitioning scheme directly over HBA are usually safe, while e.g. zfs over geli over umass is the worst construction. === Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Thu Mar 23 04:22:00 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 716EAD192A0 for ; Thu, 23 Mar 2017 04:22:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x244.google.com (mail-io0-x244.google.com [IPv6:2607:f8b0:4001:c06::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40F361D18 for ; Thu, 23 Mar 2017 04:22:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x244.google.com with SMTP id f103so12084051ioi.2 for ; Wed, 22 Mar 2017 21:22:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=OivUfRBp1YtL2VgSd+OSECIZCgzyMqNZo6UtgPeglaI=; b=2Jf/KMxEvnm7rikV5MmHXjiF3AZa4Gd9mFDPeTsFN40WVUAuiwh65iDKXo5PDMMhH9 BeSyUksUGuDKqtFxSsbCoKIvwjqcYvo6Ci3Yai61bueRrG0S/3dqxXbId5vegMKCDaWR 0RI0Ix0wrY+6fmQpSPf/6sYzJvfBSfxcPaboavT25lx+uzcGxahw+vwvcQt+cWJHMKs/ 8aVxHr+eryfx/UGd5xk7Y63f0pnl6yStmEAUolP8amKgfE4qvSwRRT259J+eVcLk43IM 0N3uyAoLdDwX33HMfYUdnLe6CbzJdTwW72rQ5waWWRR/pCcsw0lgj4rDIvJYujJByLAD GAdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=OivUfRBp1YtL2VgSd+OSECIZCgzyMqNZo6UtgPeglaI=; b=j6sbeKa5Lk9kxNu4MQW/MzqT7LOfWvSlGLsusGtIBPp5RuPdFKZlAsLeUfDGU8AguY szeL5m6UDsrIWhPVsyKddrHQLBeY/1yHrPSjTGTxOOfauM/2NEH02DZPEYLPMazcFhwk Kq5ou+mafL3BiFWBaTdJqQqfMbvwrEiazvxPf0vmRfrKGH94eHbmsEcVFh0/vkNkh+HP Kl0PkV39T7fr2rYyi2hc/JN3Xd0qlX0z1Vcwr16gKPi78YGJigbYHRtnXP6V7Jp6D5+j PVPUSuIIA0Yj9hy1eR/cmtTfJQ60hE3kVAv8lJ5I6bGrbzbR/T7NS14NOrtyoQo6hYdT 8hOA== X-Gm-Message-State: AFeK/H2O7k0SVIGpYf3YP+h6L6uLtWKBroQ+cnUjazYfohZZ0MgB88h6HV8TNETlh0YfbpOPkOTxKouzleityA== X-Received: by 10.107.174.220 with SMTP id n89mr905257ioo.166.1490242919131; Wed, 22 Mar 2017 21:21:59 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Wed, 22 Mar 2017 21:21:58 -0700 (PDT) X-Originating-IP: [2607:fb10:7021:1::3224] In-Reply-To: <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> From: Warner Losh Date: Wed, 22 Mar 2017 21:21:58 -0700 X-Google-Sender-Auth: Tf9Oc8llO6QLTb6JShLjGY7B9Lk Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Julian Elischer Cc: Allan Jude , "freebsd-hackers@freebsd.org" , Saurav Sachidanand Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 04:22:00 -0000 I just did exactly the same thing last night.... Warner On Wed, Mar 22, 2017 at 8:03 PM, Julian Elischer wrote: > On 1/3/17 4:14 pm, Julian Elischer wrote: >> >> On 28/2/17 2:01 am, Warner Losh wrote: >>> >>> On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude >>> wrote: >>>> >>>> On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >>>> wrote: >>>>> >>>>> Hello FreeBSD community, >>>>> >>>>> I'm >>>>> Saurav Sachidanand, and I'm >>>>> a CS sophomore studying in India >>>>> . >>>>> I have an interest in operating systems development and wish to >>>>> contribute >>>>> to the FreeBSD community. I'm proficient with C and have some >>>>> experience in >>>>> kernel programming. Hence, I'd like to propose an original project for >>>>> GSoC >>>>> 2017 that I feel would benefit this community. >>>>> >>>>> In past years, the Lua interpreter was ported to run inside the Linux >>>>> and >>>>> NetBSD kernel [1]. Lua was chosen because it's interpreter is very >>>>> small (~240 >>>>> KB) compared to that of Python or Ruby, it's MIT licensed, and is >>>>> almost >>>>> freestanding. A working demonstration of it is a packet filtering >>>>> algorithm >>>>> written entirely in kernel Lua [2]. >>>>> >>>>> Specifically, my proposal would be to port the following that are >>>>> currently >>>>> written for NetBSD: >>>>> - the modified Lua VM source code with _KERNEL preprocessor directives >>>>> to >>>>> exclude user-space functionality like floating point, the io and os >>>>> module >>>>> in the standard library, etc. [3] >>>>> - the kernel module device driver for /dev/lua, to which Lua scripts >>>>> are >>>>> fed to be executed [4], [5] >>>>> - the luactl user-space program to control the Lua device and a couple >>>>> of >>>>> sysctl variables which serve similar purpose [6], [7] >>>>> >>>>> And then: >>>>> - run the Lua test suite targeting whatever we support in the kernel to >>>>> make sure it works [8] >>>>> - and write Lua bindings to the kernel interfaces that would interest >>>>> the >>>>> FreeBSD community >>>>> >>>>> Since NetBSD and FreeBSD have similar kernel interfaces (mutexes, >>>>> linked >>>>> lists, device switch interface), the porting shouldn't involve too much >>>>> code refactoring. Also, this would all be an experiment in that we >>>>> don't >>>>> fully know what the real world use cases might be, but it would attract >>>>> more people to writing kernel code who otherwise wouldn't because of >>>>> having >>>>> to do everything in C. And it would be interesting to carry out it out >>>>> in >>>>> FreeBSD as well since it has a larger community than NetBSD. >>>>> >>>>> I humbly request anyone who is interested in this project to be my >>>>> potential mentor(s) for GSoC. >>>>> >>>>> More slides on kernel Lua in NetBSD - [9], [10]. >>>>> >>>>> Thanks, >>>>> Saurav >>>>> >>>>> [1] - http://www.netbsd.org/~lneto/dls14.pdf >>>>> [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf >>>>> [3] - https://github.com/jsonn/src/tree/trunk/external/mit/lua/dist/src >>>>> [4] - >>>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/lua >>>>> [5] - >>>>> >>>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/luasystm >>>>> [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin/luactl >>>>> [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current >>>>> [8] - http://www.lua.org/tests/ >>>>> [9] - >>>>> >>>>> https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012/kernel_mode_lua.pdf >>>>> [10] - https://www.lua.org/wshop13/Cormack.pdf >>>>> _______________________________________________ >>>>> freebsd-hackers@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> To unsubscribe, send any mail to >>>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>> >>>> This may be quite a nice thing to have. Another upcoming use for LUA in >>>> the kernel is ZFS Channel Programs. These allow a number of ZFS operations >>>> to be completed as a single atomic transaction. >>>> >>>> I would hope we could structure this in such a way as to not end up with >>>> two copies of Lua in the kernel. >>> >>> There's also a 3/4 finished lua in the boot loader that you might be >>> able to leverage as well.... >> >> >> I'd like to see that finished. While Devin has done Heroic work with the >> forth in the loader, I think it's time has come. >> It' be nice to have something a little less '60s. > > some news: Pedro Arthur who did the project just mailed me: > > =================== > > Hi, > I've just finished getting the lua-bootloader code on top of HEAD, it should > compile and run fine. > The code can be found in [1]. > > [1] - https://github.com/grandao/freebsd/tree/projects/lua-bootloader > > =================== > so it's up to date and we should look at it again > > >> >>> >>> Warner >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to >>> "freebsd-hackers-unsubscribe@freebsd.org" >>> >> > From owner-freebsd-hackers@freebsd.org Thu Mar 23 04:56:18 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD50FD19601 for ; Thu, 23 Mar 2017 04:56:18 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A351F1E22 for ; Thu, 23 Mar 2017 04:56:18 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x241.google.com with SMTP id f103so12132938ioi.2 for ; Wed, 22 Mar 2017 21:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=vA0gdSA3aQT41aaylnxhiEAEYc7ZxuB43MP5tpynna0=; b=0U2gnho6xAnKzkVDkd8rZw5sYUqK1aHVsWIX1HdMdGQPMtPMHJkBnZmDTumQbdkGfN /WzXUXkEl6A2/3ySgW2WNeHWxzDX6Ys5pA2aMBWp9qzBuvvfFGimG9rzTPYh119SBs69 p0JUyz9VlQtNPZN153CzOetFSIfmN4Uj1TUN5iUUskvD0pDZ+7rgzxF3TGQcqSe1o45Q BuBXJpe9FnAPXNJlBNySkp+rj5n7s5xC1M6kVYXsOPDCLWmHNA6ABvm9GiI0UM+ukHK3 DSCckQPyNLkO3mGLl0zZ9SBSqchfpxFbalczBs9QarqgOOQgfelm0qwjAXo3pn9aoXU9 toKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=vA0gdSA3aQT41aaylnxhiEAEYc7ZxuB43MP5tpynna0=; b=VvuEGEwha6JPcha3r0zoE9jU0G3hu4OvPuCrxLWLEi6VtP4gZq58hG7eHMh6kA/hju tlXOEFdCyCZRnfCht2EKSN1mnsem2JL9JB5jxSXcLOponk9B7qi/BGH6k/bh8/mBTZzn Is9Q34v5+EhCeKzgXPyy6FWwDZu/5/4dZ/1Sovv9bvWXpnS/1Pv+xnWo70MYQXtxuMeL /mCyFGujDs08Z98MOFkjk9Xf/QAOGK3dFBTq2BsEbiP/eqUTPtwqJ5JLZjcsmB0iT8pf Y6rYp5LudHfQuuXghU/t6Jfebr1605fP3MK46ivQGEt8FvOFsLNtTDbzc3sgB0Dnk8FG QyQg== X-Gm-Message-State: AFeK/H3RGey0J1e2hAFiWF51tzI2CdLc65wUIX4UYnrU4ev5EEHwcj0ovb6JKxxsgcrgnohedVd4ANXg/ut7xw== X-Received: by 10.107.134.94 with SMTP id i91mr927751iod.0.1490244977844; Wed, 22 Mar 2017 21:56:17 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Wed, 22 Mar 2017 21:56:17 -0700 (PDT) X-Originating-IP: [2607:fb10:7021:1::3224] In-Reply-To: <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> From: Warner Losh Date: Wed, 22 Mar 2017 21:56:17 -0700 X-Google-Sender-Auth: E2X8dpEyVhjorCinxuyZTutvu78 Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Julian Elischer Cc: Allan Jude , "freebsd-hackers@freebsd.org" , Saurav Sachidanand Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 04:56:19 -0000 On Wed, Mar 22, 2017 at 8:03 PM, Julian Elischer wrote: > On 1/3/17 4:14 pm, Julian Elischer wrote: >> >> On 28/2/17 2:01 am, Warner Losh wrote: >>> >>> On Mon, Feb 27, 2017 at 8:26 AM, Allan Jude >>> wrote: >>>> >>>> On February 27, 2017 5:28:41 AM PST, Saurav Sachidanand >>>> wrote: >>>>> >>>>> Hello FreeBSD community, >>>>> >>>>> I'm >>>>> Saurav Sachidanand, and I'm >>>>> a CS sophomore studying in India >>>>> . >>>>> I have an interest in operating systems development and wish to >>>>> contribute >>>>> to the FreeBSD community. I'm proficient with C and have some >>>>> experience in >>>>> kernel programming. Hence, I'd like to propose an original project for >>>>> GSoC >>>>> 2017 that I feel would benefit this community. >>>>> >>>>> In past years, the Lua interpreter was ported to run inside the Linux >>>>> and >>>>> NetBSD kernel [1]. Lua was chosen because it's interpreter is very >>>>> small (~240 >>>>> KB) compared to that of Python or Ruby, it's MIT licensed, and is >>>>> almost >>>>> freestanding. A working demonstration of it is a packet filtering >>>>> algorithm >>>>> written entirely in kernel Lua [2]. >>>>> >>>>> Specifically, my proposal would be to port the following that are >>>>> currently >>>>> written for NetBSD: >>>>> - the modified Lua VM source code with _KERNEL preprocessor directives >>>>> to >>>>> exclude user-space functionality like floating point, the io and os >>>>> module >>>>> in the standard library, etc. [3] >>>>> - the kernel module device driver for /dev/lua, to which Lua scripts >>>>> are >>>>> fed to be executed [4], [5] >>>>> - the luactl user-space program to control the Lua device and a couple >>>>> of >>>>> sysctl variables which serve similar purpose [6], [7] >>>>> >>>>> And then: >>>>> - run the Lua test suite targeting whatever we support in the kernel to >>>>> make sure it works [8] >>>>> - and write Lua bindings to the kernel interfaces that would interest >>>>> the >>>>> FreeBSD community >>>>> >>>>> Since NetBSD and FreeBSD have similar kernel interfaces (mutexes, >>>>> linked >>>>> lists, device switch interface), the porting shouldn't involve too much >>>>> code refactoring. Also, this would all be an experiment in that we >>>>> don't >>>>> fully know what the real world use cases might be, but it would attract >>>>> more people to writing kernel code who otherwise wouldn't because of >>>>> having >>>>> to do everything in C. And it would be interesting to carry out it out >>>>> in >>>>> FreeBSD as well since it has a larger community than NetBSD. >>>>> >>>>> I humbly request anyone who is interested in this project to be my >>>>> potential mentor(s) for GSoC. >>>>> >>>>> More slides on kernel Lua in NetBSD - [9], [10]. >>>>> >>>>> Thanks, >>>>> Saurav >>>>> >>>>> [1] - http://www.netbsd.org/~lneto/dls14.pdf >>>>> [2] - https://www.netbsd.org/~lneto/eurobsdcon14.pdf >>>>> [3] - https://github.com/jsonn/src/tree/trunk/external/mit/lua/dist/src >>>>> [4] - >>>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/lua >>>>> [5] - >>>>> >>>>> https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sys/modules/luasystm >>>>> [6] - https://github.com/IIJ-NetBSD/netbsd-src/tree/master/sbin/luactl >>>>> [7] - http://netbsd.gw.com/cgi-bin/man-cgi?lua+4+NetBSD-current >>>>> [8] - http://www.lua.org/tests/ >>>>> [9] - >>>>> >>>>> https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2012/kernel_mode_lua.pdf >>>>> [10] - https://www.lua.org/wshop13/Cormack.pdf >>>>> _______________________________________________ >>>>> freebsd-hackers@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> To unsubscribe, send any mail to >>>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>> >>>> This may be quite a nice thing to have. Another upcoming use for LUA in >>>> the kernel is ZFS Channel Programs. These allow a number of ZFS operations >>>> to be completed as a single atomic transaction. >>>> >>>> I would hope we could structure this in such a way as to not end up with >>>> two copies of Lua in the kernel. >>> >>> There's also a 3/4 finished lua in the boot loader that you might be >>> able to leverage as well.... >> >> >> I'd like to see that finished. While Devin has done Heroic work with the >> forth in the loader, I think it's time has come. >> It' be nice to have something a little less '60s. > > some news: Pedro Arthur who did the project just mailed me: > > =================== > > Hi, > I've just finished getting the lua-bootloader code on top of HEAD, it should > compile and run fine. > The code can be found in [1]. > > [1] - https://github.com/grandao/freebsd/tree/projects/lua-bootloader > > =================== > so it's up to date and we should look at it again I did a rebase based of a few minutes ago. https://github.com/bsdimp/freebsd/tree/lua-bootloader and it's rebased off of freebsd:master rather than the old lua branch so will be easier to merge. There's also a bunch of commits that seem to be missing from grandao's tree. I need to get it compiling, and I can mine grandao's if there's issues... Warner From owner-freebsd-hackers@freebsd.org Wed Mar 22 22:09:33 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B9BBD188FE for ; Wed, 22 Mar 2017 22:09:33 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-211-177.reflexion.net [208.70.211.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48E14C02 for ; Wed, 22 Mar 2017 22:09:33 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 1134 invoked from network); 22 Mar 2017 22:09:26 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 22 Mar 2017 22:09:26 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.30.2) with SMTP; Wed, 22 Mar 2017 18:09:26 -0400 (EDT) Received: (qmail 2376 invoked from network); 22 Mar 2017 22:09:26 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 22 Mar 2017 22:09:26 -0000 Received: from [192.168.1.111] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 55D83EC8974; Wed, 22 Mar 2017 15:09:25 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: fork-then-swap-out [zero RES(ident memory)] questions tied to arm64 failures (zeroed memory pages) Message-Id: <59C6BC12-1BC2-41D5-8B47-D0AD44D2CDF0@dsl-only.net> Date: Wed, 22 Mar 2017 15:09:23 -0700 Cc: freebsd-arm To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3259) X-Mailman-Approved-At: Thu, 23 Mar 2017 12:01:51 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 22:09:33 -0000 The later questions are associated with: Bugzilla 217239 and 217138 (which I now expect have a common cause) https://lists.freebsd.org/pipermail/freebsd-arm/2017-March/015867.html (and its thread) These are tied to some process memory pages being trashed (to be zero) in particular types of arm64 contexts. This is reproducible in multiple arm64 contexts. The context is head but I believe there are reports in the lists tied to 11 as well. [Unfortunately the above all very much shows a learn-as-I-go property. Also the list has a sub-exchange on my testing other devices to check for device failures that is not directly relevant here.] These are tied to problems with fork-then-swap-out-then-swap-in contexts on arm64. (Even though I've occasionally typed amd64 accidentally in places in those materials.) Memory allocations from before the fork are involved, ones not yet accessed by the child side of the fork at the time of the fork. fork sets up copy-on-write so that the child process temporarily shares pages (those it does not write), or should. But what if the parent process or both parent and child are swapped-out just shortly after the fork (so, say, top -PCwaopid shows zero for RES(ident memory)? What is the handling of, say, the child swapping back in while the parent still is swapped out? I notice that the child can have a much smaller SWAP figure than the parent so it would appear that the parent swap-out has pages that the child does not. So what if the child needs some of those pages? What should happen? (Vs. what does happen on arm64 in specific types of contexts? More below.) I ask mostly to see if I can do more to give evidence of what is going on and what to test for any proposed fix. I'm not likely to find the underlying problem(s) for arm64 directly, unlike my investigation that lead to fork-trampoline being fixed in head's -r313772 (2017-Feb-15). [ https://lists.freebsd.org/pipermail/freebsd-arm/2017-February/015656.html and its thread, including when its title changed in: https://lists.freebsd.org/pipermail/freebsd-arm/2017-February/015678.html .] Part of that unlikely-to-solve status is because the context seems to be bound to a lot of special conditions and interesting behaviors simultaneously: A) Both my original reproductions of problem reports on the lists and the only (simple) programs for reproducing the probablems involve fork-then-swap-out [zero RES(ident memory)]. Neither fork by itself nor swap-out/in by itself have been sufficient. B) jemalloc's tcache being in use (__je_tcache_maxclass == 32*1024) is part of every example of reproduction of the problem. C) allocations <= SMALL_MAXCLASS (SMALL_MAXCLASS==14*1024) get the failure (but bigger ones work, both fitting inside __je_tcache_maxclass and not). Again: every example reproduction of the problem has this status. D) FreeBSD sometimes gets into a state where /etc/malloc.conf doing tcache:false does not seem to disable tcache. (Rebooting goes back to tcache:false working after such has been observed.) [Related or independent? I've no clue.] Usually tcache:false seems to work and so avoid the failures. E) Use of POSIX_MADV_WILLNEED on the problematical allocation(s) in the child process after the fork but before the swap-outs of the child and parent prevents the failures (no read or write access to the memory from the child until after the swap-in). Doing so just in the parent process does not prevent the failures. F) Similar to (E) but read-accessing a byte or more of one or more pages from the problematical allocations from the child process after the fork but before the swap-out makes those specific pages not fail. (The others still fail, if any.) Done from the parent process instead does not avoid the failures. G) In a sequence like: su creates a sh which then runs one of my test programs that then forks off a child it can be that all of the 4 processes show the zeroed memory area like the child process does. su and sh need to have swapped-out and back in for them to get failures. su and sh die once they hit an assert that fails due to the zeroed memory page(s). The asserts involve addresses also messed up in the test program processes (parent and child). In my reading I've not been able to determine what to expect for fork-then-swap-out-and-back-in for pages that the child process had not accessed yet but which might not be around for later activity because of the parent process's own swapped-out status at the time. Note: While I usually run a non-debug kernel I've tried a debug kernel and it provided no notices of problems. I got no additional information from the attempt. [My usual KERNCONF file includes GENERIC and then disables various debug items.] The bugzilla reports have example code for showing the problems and various behaviors. The two in 217239 are probably of more interest than the first one on 217138. === Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Wed Mar 22 23:12:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 746AFD18C01; Wed, 22 Mar 2017 23:12:38 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-vk0-x244.google.com (mail-vk0-x244.google.com [IPv6:2607:f8b0:400c:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FC0317C0; Wed, 22 Mar 2017 23:12:38 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: by mail-vk0-x244.google.com with SMTP id j64so17376253vkg.0; Wed, 22 Mar 2017 16:12:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=J4yAAyNGmVYThYlbDvizeJDSZR5kVFHNArcr3ArW2dE=; b=cCNZTAZr+miW4WCw12CgeCa6gr/86/LehyriCtiSWi5fDPvzoY7rsY7xSwuJTLbuMV lQau7ooK8pEkXzyspVu19WYbrVUVvljj5uT2O4zEtZOSo0iSfCdvR68m7mTC+bEul/3z iU1TJ8S87RRhRXzQoWxYmX+vUlrS1UFAOlHtKOSyIrwXq2nTA1LJc/VH1+UA15TIbXRB k31EAr9F/ehvcyEOc0jqzPI+Vb1m/dA0jv1ClGVisXSzh72J0Axyn9218HjlQAkDcXVl M1I3lSRLPx74duJqhXGa9G/gT4NvF4wmhdMrElW7i3f3COpb/pFHl5FF0YrjoLZRVisD uWtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=J4yAAyNGmVYThYlbDvizeJDSZR5kVFHNArcr3ArW2dE=; b=steE8h4bRgqjRsrN1QxDmZIiNNMEuh7tLiu5Bd9vePKRpSJponQ2qAKYvcPmxSSipF LHx3jk39B7XZSI1cY05wDTge1DG3SQwKz/meuVIdbuX8JXLdBYtjA0ooEMga/d7LhKOK sjwdC6r5EmtWn3xKsAiJC6BLMnSe84DXX/VOvLafurPeOJonD690+3g0VbuN5ZPi5crM V0lqS5fHDouaNAMDXKTln7n1Jvuca3a0Y7mhB+ExcrPbdbqzWKvUzLcoTxxPezEaCUsN vY/l6GAUeJSt2gFq6WT7Vv+IjCnKXEh5mdOsyJo6PvxH6KSZKenTL2ApaxAttz1ehBaZ R2ZA== X-Gm-Message-State: AFeK/H1Nok8Gx8kU5gWUkkY7gwcrbXtCSvWXGOqgr5bZ/BziCIbW7dE5Iz/RNDCHi8ML20uUZjspjmsr2RxlzQ== X-Received: by 10.176.91.87 with SMTP id v23mr15252195uae.90.1490224356990; Wed, 22 Mar 2017 16:12:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.33.37 with HTTP; Wed, 22 Mar 2017 16:11:56 -0700 (PDT) In-Reply-To: References: From: grarpamp Date: Wed, 22 Mar 2017 19:11:56 -0400 Message-ID: Subject: Re: Filtering Against Persistent Firmware Rootkits - BadUSB, HDDHack, UEFI To: freebsd-security@freebsd.org Cc: freebsd-hardware@freebsd.org, freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Thu, 23 Mar 2017 12:02:09 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2017 23:12:38 -0000 > It is virtually impossible to guard against firmware rootkits because > cpu cannot prevent the card's or device's cpu from from executing that code. > This was made known by the malware embedded in disk drives' FW, and > other peripherals' FW, such as wifi and graphics, to name a couple. > It is possible for such device FW to insert malware into, > or modify, the RAM resident OS. > Apparently making OS's executable segments "non-writeable" can be gotten > around. There are two very different write directions involved... HW -> OS / SW ... Yes, as above, you're screwed. SW -> OS -> HW ... However, as before, you can add kernel filters to further help prevent software from writing the screwed firmware to your hardware in the first place. From owner-freebsd-hackers@freebsd.org Thu Mar 23 12:48:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1DD8D19BBB for ; Thu, 23 Mar 2017 12:48:49 +0000 (UTC) (envelope-from bygrandao@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2E3C80C; Thu, 23 Mar 2017 12:48:49 +0000 (UTC) (envelope-from bygrandao@gmail.com) Received: by mail-io0-x235.google.com with SMTP id l7so81513631ioe.3; Thu, 23 Mar 2017 05:48:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zGQSC2/XFjNdcLCt3oyFrVMvWEnVjLu3K41yBIQgO4Q=; b=Fi502EyiCk5SAmN8s0UzRKrrdXgruiJwCJEoP3xDbpQsIEVdFSm9j6+h9p+tpi6MBV CeD8wg/iGwh/noK7Qivb++feIlgkiHCXCOCdMhyHh296ZuUnuP0Lrvl/84PNbC3yQN4S pFI6P9IfQMdbXciP+zZUPbBuN22cJBFqxqSEZ9mS8Qp3vjS9bzmQZf1SbTZgJE8k6l35 bFpgKJnTnSRmtxm9K1S/HuVlVpnmhIUHeUFkjS6jSjH+NaVOmG9HuqFAMjg9lwplYFXU U8MirPpAO0cNeSyaMvgMkrfP7/VL5t5Jt0VljSCc/Ssc3KtS2YzuCGVnBpX+T0wvHo0I bR1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zGQSC2/XFjNdcLCt3oyFrVMvWEnVjLu3K41yBIQgO4Q=; b=XwWFgb3SvEWgVcF9ctuu4Tocwgbwma9rG9Rjt3FARFXxQraYdV7DArTxzD+ZwaPA1j ilSB2AYCe6EiCXcRSek7/BzGhN/w34cWUGkCUEIYa4A30ZS8AzcWgFqKVcgEyqq8zp4b 1dfHvupw8LIJPo60FnzSNWQvUNo9XQXfUEaMhT89JczV5YVe/u7cE9XmA/4vz0p56T7e 6dJMsIcXRCj84uSsh59cp82pQO7e2IRpylKN8mVLxePGpp88Ox/fScz01eKtzF3L6wUw +yG0TnSV7SHqCpT5m5SSM+yG7kdFntNhxutzvzRCIDxD1C+38zxlVaiaGaYZ3bDXzTc2 VCZQ== X-Gm-Message-State: AFeK/H1RYnJGm2uWicS4vB+eMFq/QkuICdpoBGaP61f5nnd2ekUCFn9C+nCErytstOCcy7fPOrPk+OwaaQlBOQ== X-Received: by 10.107.37.12 with SMTP id l12mr2211263iol.159.1490273329007; Thu, 23 Mar 2017 05:48:49 -0700 (PDT) MIME-Version: 1.0 References: <244231A2-EB18-4E58-A2B2-927F55D54950@FreeBSD.org> <9db670f6-54d1-7862-bcd2-7c80df92e724@freebsd.org> In-Reply-To: From: Pedro Arthur Date: Thu, 23 Mar 2017 12:48:38 +0000 Message-ID: Subject: Re: [GSoC 2017] Original proposal: Port kernel Lua to FreeBSD To: Warner Losh , Julian Elischer Cc: Allan Jude , Saurav Sachidanand , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2017 12:48:50 -0000 Hi, I rebased rpaulo lua-bootloader against master. The commits may diverge from the original gsoc branch but this should be the most up to date branch. From owner-freebsd-hackers@freebsd.org Fri Mar 24 09:16:20 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A83ACCA12E9 for ; Fri, 24 Mar 2017 09:16:20 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-211-181.reflexion.net [208.70.211.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D24C1CEC for ; Fri, 24 Mar 2017 09:16:19 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 978 invoked from network); 24 Mar 2017 09:18:50 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 24 Mar 2017 09:18:50 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.30.2) with SMTP; Fri, 24 Mar 2017 05:16:13 -0400 (EDT) Received: (qmail 19409 invoked from network); 24 Mar 2017 09:16:13 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 24 Mar 2017 09:16:13 -0000 Received: from [192.168.1.119] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 5A30AEC8173; Fri, 24 Mar 2017 02:16:12 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: head -r315870 (e.g.): fork-then-swap-out [zero RES(ident memory)] questions tied to arm64 failures (zeroed memory pages) From: Mark Millard In-Reply-To: <59C6BC12-1BC2-41D5-8B47-D0AD44D2CDF0@dsl-only.net> Date: Fri, 24 Mar 2017 02:16:11 -0700 Cc: freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: <4D7B13F4-142E-4294-A6E7-11CCD4C92AC5@dsl-only.net> References: <59C6BC12-1BC2-41D5-8B47-D0AD44D2CDF0@dsl-only.net> To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Mar 2017 09:16:20 -0000 On 2017-Mar-22, at 3:09 PM, Mark Millard wrote: > The later questions are associated with: > > Bugzilla 217239 and 217138 (which I now expect have a common cause) > https://lists.freebsd.org/pipermail/freebsd-arm/2017-March/015867.html > (and its thread) > > These are tied to some process memory pages being trashed (to > be zero) in particular types of arm64 contexts. This is > reproducible in multiple arm64 contexts. The context is head > but I believe there are reports in the lists tied to 11 as > well. > > [Unfortunately the above all very much shows a learn-as-I-go > property. Also the list has a sub-exchange on my testing other > devices to check for device failures that is not directly > relevant here.] > > These are tied to problems with fork-then-swap-out-then-swap-in > contexts on arm64. (Even though I've occasionally typed amd64 > accidentally in places in those materials.) Memory allocations > from before the fork are involved, ones not yet accessed by > the child side of the fork at the time of the fork. > > fork sets up copy-on-write so that the child process temporarily > shares pages (those it does not write), or should. > > But what if the parent process or both parent and child are > swapped-out just shortly after the fork (so, say, top -PCwaopid > shows zero for RES(ident memory)? What is the handling of, say, > the child swapping back in while the parent still is swapped > out? > > I notice that the child can have a much smaller SWAP figure > than the parent so it would appear that the parent swap-out > has pages that the child does not. > > So what if the child needs some of those pages? What should > happen? (Vs. what does happen on arm64 in specific types > of contexts? More below.) > > I ask mostly to see if I can do more to give evidence of > what is going on and what to test for any proposed fix. > I'm not likely to find the underlying problem(s) for arm64 > directly, unlike my investigation that lead to > fork-trampoline being fixed in head's -r313772 > (2017-Feb-15). > > [ https://lists.freebsd.org/pipermail/freebsd-arm/2017-February/015656.html > and its thread, including when its title changed in: > https://lists.freebsd.org/pipermail/freebsd-arm/2017-February/015678.html > .] > > Part of that unlikely-to-solve status is because the > context seems to be bound to a lot of special conditions > and interesting behaviors simultaneously: > > A) Both my original reproductions of problem reports on the > lists and the only (simple) programs for reproducing the > probablems involve fork-then-swap-out [zero RES(ident > memory)]. Neither fork by itself nor swap-out/in by > itself have been sufficient. > > B) jemalloc's tcache being in use (__je_tcache_maxclass == 32*1024) > is part of every example of reproduction of the problem. > > C) allocations <= SMALL_MAXCLASS (SMALL_MAXCLASS==14*1024) get > the failure (but bigger ones work, both fitting inside > __je_tcache_maxclass and not). Again: every example > reproduction of the problem has this status. > > D) FreeBSD sometimes gets into a state where /etc/malloc.conf > doing tcache:false does not seem to disable tcache. (Rebooting > goes back to tcache:false working after such has been > observed.) [Related or independent? I've no clue.] Usually > tcache:false seems to work and so avoid the failures. > > E) Use of POSIX_MADV_WILLNEED on the problematical allocation(s) > in the child process after the fork but before the swap-outs > of the child and parent prevents the failures (no read or > write access to the memory from the child until after the > swap-in). Doing so just in the parent process does not prevent > the failures. > > F) Similar to (E) but read-accessing a byte or more of one or > more pages from the problematical allocations from the child > process after the fork but before the swap-out makes those > specific pages not fail. (The others still fail, if any.) > Done from the parent process instead does not avoid the > failures. > > G) In a sequence like: su creates a sh which then runs one > of my test programs that then forks off a child it can be > that all of the 4 processes show the zeroed memory area > like the child process does. su and sh need to have > swapped-out and back in for them to get failures. su and > sh die once they hit an assert that fails due to the zeroed > memory page(s). The asserts involve addresses also messed > up in the test program processes (parent and child). > > In my reading I've not been able to determine what to expect > for fork-then-swap-out-and-back-in for pages that the child > process had not accessed yet but which might not be around > for later activity because of the parent process's own > swapped-out status at the time. > > Note: While I usually run a non-debug kernel I've tried > a debug kernel and it provided no notices of problems. I > got no additional information from the attempt. > > [My usual KERNCONF file includes GENERIC and then disables > various debug items.] > > The bugzilla reports have example code for showing the > problems and various behaviors. The two in 217239 are > probably of more interest than the first one on 217138. I just updated the pine64+ 2GB to head -r315870 and it still gets the trashed-with-zeros pages from sequences such as: allocation(s) (tcache in use with fitting <= SMALL_MAXCLASS) initialize them (to non-zero bytes) fork sleep/wait then swap-out [zero RES(ident memory)] (both parent and child in my tests) (Note the lack of access so far on the child process side.) swap-in After swap-in both the Child and Parent see the indicated allocation(s) as having only zero bytes instead of the initialization values. === Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Sat Mar 25 00:38:32 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F53DD1B90C for ; Sat, 25 Mar 2017 00:38:32 +0000 (UTC) (envelope-from crb@chrisbowman.com) Received: from mail-pg0-x234.google.com (mail-pg0-x234.google.com [IPv6:2607:f8b0:400e:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AA6CA2A for ; Sat, 25 Mar 2017 00:38:32 +0000 (UTC) (envelope-from crb@chrisbowman.com) Received: by mail-pg0-x234.google.com with SMTP id w20so2692119pgc.0 for ; Fri, 24 Mar 2017 17:38:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisbowman-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:message-id:date:to; bh=6XIlb2lBMuVPp/Z3hJRpaM4DZ+pujceVy2xtlfpYdxc=; b=Qv2AtdZaCjCFMdsZB8dXX3M8238vCvt7b0e4YdcItpX8afgPfd4EBF5TJaDJ/v1+TW mRy+nGE/s/Ebvu/WZGKvFd57XupNGag/ywM8SA5WM4NVAICrCotepFHVC9pk+olZ5oZQ HZ3z9879rkmI2Of6NvjYebCf+8Td2spKIk2k+TmsuzrlZyxaM9cM+qALz2cD7mWqrbaw dA+VUh+h4G8WnuG6jh7hpFz6DNUNA5dlKl/AEodmO54VrX4DzT22Nj7zK0E6RNwlg2ug 13cKSebrLdWv+AfzJ3AEHkV3FwXEWwM23ZFsQpjn4whqieMqVK+05dEjkDDKjfAt7kKu o0DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:to; bh=6XIlb2lBMuVPp/Z3hJRpaM4DZ+pujceVy2xtlfpYdxc=; b=a1zDa3GtH4OrseOXPj3vebJvvcm+TBp2srJzSMOd+ZX/3G401vbHr+mOgElbadnPhM rqSkdH4xtJbYC+Wq+RatkNRrCyHbmNm3ugE/J20OXGzpgcZr0XqgKymzTqtpOKhyHE7S QlIapyWEdyo/2ky7+5Q0IVxEfPBu+y1vJOTIhPHKvLFJaDX0wa9MXo+2SEH66ZtE7hbO kwnUxgBCPRPhHEfx8L4Q81CPJYbf0oQiVZ8Ned6cb97ZTIZgwaBScZpMaRgYt+oMS75l ygFdONVqe6unJdZexxWwATKyZ2FKAH+dcBvWiBxbpdvQ/DwhHD9uzawcGtbqw/u18FC8 +tpQ== X-Gm-Message-State: AFeK/H2kXkWD04GKZgc2FPicJvP5o6uK+1Hz7AMCsTXL80QZLcY7rK3a534QGkLW7PGMIQ== X-Received: by 10.99.155.17 with SMTP id r17mr11958942pgd.193.1490402311423; Fri, 24 Mar 2017 17:38:31 -0700 (PDT) Received: from ?IPv6:2601:647:4e00:bbb5:84f2:e7a1:fa51:c741? ([2601:647:4e00:bbb5:84f2:e7a1:fa51:c741]) by smtp.gmail.com with ESMTPSA id d4sm6745654pfb.104.2017.03.24.17.38.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Mar 2017 17:38:30 -0700 (PDT) From: Christopher Bowman Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: large PCIe BARs Message-Id: <21FF81F3-D290-4682-AB88-5769C52499C3@chrisbowman.com> Date: Fri, 24 Mar 2017 17:38:29 -0700 To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3259) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Mar 2017 00:38:32 -0000 I have an FPGA PCIe device and driver I designing. I think my driver is = trying to map the PCIe address space for one of the BARs and it=E2=80=99s = failing. My guess is that it=E2=80=99s failing due to the size. The = board has 2G of onboard memory that I=E2=80=99d like to present as one = large BAR. Does FreeBSD (11.0 release) allow for mapping PCIe resources = that larger? If not is there a limit? If so any one have any comments? Thanks, Christopher=20 sp6050: probing for SP605 sp6050: at device 0.0 on pci3 sp6050: attach of SP605 sp6050: 0x40000000 bytes of rid 0x10 res 3 failed (0, = 0xffffffffffffffff). sp6050: Could not map memory device_attach: sp6050 attach returned 6 From owner-freebsd-hackers@freebsd.org Sat Mar 25 01:30:59 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E95DD1C81C for ; Sat, 25 Mar 2017 01:30:59 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D9D6156C for ; Sat, 25 Mar 2017 01:30:58 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-wm0-f50.google.com with SMTP id n11so24726255wma.1 for ; Fri, 24 Mar 2017 18:30:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc:content-transfer-encoding; bh=D1coHJyxAmm9kxRGK1BlkFZdpqN65G02xP6GxYEW0cU=; b=C9duv9G0/NcDpvezP3QXq1VqbNmvr7XMpPBeji1NA25aKjgUNxai3OaJ8IJ3sbeZtc M1lmT7Ibc5RQKqtHL2wPz+2rSuNbDLNrDV9anESh/lu2WTPMPSgC1yuha+A8gqZUf7/g P5Qos5yxhtFlsngas4A3NDDYt7wauYeCab91/NBIGbrAIFEXX/HO6JXKZIpIR0Ni/rtV 7HAb/VRbZkeuveah4kTBldFb0nK4pFIA6UZhZ/5qr8yx+5svo8xsAbLw08vA+M9D+zVb Np2FWgpPKnSk2nbGKdP+fd+/AHNohrVXWCxBIyUYUrJfSFGFTAzHhUNxhiwelkGe/V3z 0leQ== X-Gm-Message-State: AFeK/H3IkahF3EhMZ6Gtmqo/ZpDx9VTrMbBGbfjjkWKq5sHDb9YW43Tp8J0pXYgF+R102g== X-Received: by 10.28.130.139 with SMTP id e133mr20594wmd.133.1490403559246; Fri, 24 Mar 2017 17:59:19 -0700 (PDT) Received: from mail-wr0-f175.google.com (mail-wr0-f175.google.com. [209.85.128.175]) by smtp.gmail.com with ESMTPSA id k139sm4464094wmg.11.2017.03.24.17.59.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Mar 2017 17:59:19 -0700 (PDT) Received: by mail-wr0-f175.google.com with SMTP id u1so3137067wra.2 for ; Fri, 24 Mar 2017 17:59:19 -0700 (PDT) X-Received: by 10.223.160.243 with SMTP id n48mr11113636wrn.198.1490403558925; Fri, 24 Mar 2017 17:59:18 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.169.4 with HTTP; Fri, 24 Mar 2017 17:59:18 -0700 (PDT) In-Reply-To: <21FF81F3-D290-4682-AB88-5769C52499C3@chrisbowman.com> References: <21FF81F3-D290-4682-AB88-5769C52499C3@chrisbowman.com> From: Conrad Meyer Date: Fri, 24 Mar 2017 17:59:18 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: large PCIe BARs To: Christopher Bowman Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Mar 2017 01:30:59 -0000 Hi Christopher, Yeah, FreeBSD can easily map BARs that large. I've personally mapped a 512GB BAR, although I don't recommend it. We regularly use 32GB BARs. This might just be a driver bug. Best, Conrad On Fri, Mar 24, 2017 at 5:38 PM, Christopher Bowman w= rote: > I have an FPGA PCIe device and driver I designing. I think my driver is = trying to map the PCIe address space for one of the BARs and it=E2=80=99s f= ailing. My guess is that it=E2=80=99s failing due to the size. The board = has 2G of onboard memory that I=E2=80=99d like to present as one large BAR.= Does FreeBSD (11.0 release) allow for mapping PCIe resources that larger?= If not is there a limit? If so any one have any comments? > Thanks, > Christopher > > sp6050: probing for SP605 > sp6050: at device 0.0 on pci3 > sp6050: attach of SP605 > sp6050: 0x40000000 bytes of rid 0x10 res 3 failed (0, 0xffffffffffffffff)= . > sp6050: Could not map memory > device_attach: sp6050 attach returned 6 > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@freebsd.org Sat Mar 25 03:29:27 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD336D0FF82; Sat, 25 Mar 2017 03:29:27 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-vk0-x233.google.com (mail-vk0-x233.google.com [IPv6:2607:f8b0:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78024159; Sat, 25 Mar 2017 03:29:27 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: by mail-vk0-x233.google.com with SMTP id r69so7559498vke.2; Fri, 24 Mar 2017 20:29:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Te0jIJKZTFGAutgQir6X+lltBWu6HsL9UcePTWXrUnk=; b=qpN7XMXtzsl763r2EuUI8fEMSCstHhJ246CgApowPgYfvYO2fSyIUmi7UyBSZ7BStQ lv6hYMEHpWq5/j/SZe9RYd7HagPuSZl3LvH+Y8ylkA1yk2BI89rsvyhvbVDO/8qsrvp4 SO3TQYh/NB7Zkf1jjZsUJ7HZNTHZc7K5/ddAhV5PhuN84LkRbRYVVPMpmLaakiMBN1Gg ByrpGMFNsatQIbbvKohLNZgqXaGx08IJIV4lokZayOgWOfY+pUcOSd9EUJuoOqbplHeZ 4FgfDY9U8XU229j1pHd8eT4hsdoFPNqouID8Lh5TuhjgqWxHLCTddoiwTM0lB9hRxkkh +r0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Te0jIJKZTFGAutgQir6X+lltBWu6HsL9UcePTWXrUnk=; b=eOaW3ktyjs2A1mBhx6nTGkNFi2gBFhCutEY6Rmu2gO3wOwPz5lLN5I8aR71kqcWPDd pI4a8TLBWs1u6QjuC+GyMog7u037jwuz0e9nsFxr92cOrchqN3TI7Tf4MmG/Nw8A7UvU JLgr9Gn7NJrkiuUXCYt+/0gR4gCipAHwqAzBZbb3ATWYeM3Y45sDDnDuKVeptlhg/hyF WJ0txqgvx7Nkhrjneu+APQwPHnZTUDSBG97CcS0hen63Jv+qQ5H9p083dQjOvBwV/vTk gg1308JZ2ytffnqFcWjUX4m80fiOoqZScPE3F4zJrmIuBvM0E97wqS4wvmEjwRUFcbK7 HFng== X-Gm-Message-State: AFeK/H3C8gUQMDgE6XOr8Qnt3JlwRNwZuSeu6ufQII2hrerVh5wGnGJ9MWvV+r7ue6fCzELyUn/g0b+a8ys0SQ== X-Received: by 10.176.83.141 with SMTP id k13mr4549508uaa.64.1490412566354; Fri, 24 Mar 2017 20:29:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.33.37 with HTTP; Fri, 24 Mar 2017 20:28:46 -0700 (PDT) In-Reply-To: References: From: grarpamp Date: Fri, 24 Mar 2017 23:28:46 -0400 Message-ID: Subject: Re: Filtering Against Persistent Firmware Rootkits - BadUSB, HDDHack, UEFI To: freebsd-security@freebsd.org Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Mar 2017 03:29:27 -0000 Over two years ago this "trojans in the firmware" was mentioned here. These attacks are real and are in the wild. They are created and used by various hats from adversary to researcher to miscreant... and ultimately can end up passing unwittingly through degrees of separation to and among you and your peers over daily sharing and other physical transactions, use of unaudited application and systems code, dual booting, parking lot attacks, computer labs, libraries, component swapping, etc. Some mitigation may be possible through kernel filtering modes... - Filter and log all known firmware / bios writing opcodes. - Filter and log all opcodes except those required for daily use, such as: read, write, erase unit, inquiry, reset, etc. - Filter and log all opcodes execpt those in some user defined rulesets. Default permit / deny, the usual schemes. In a securelevel, this may provide some resistance and extra steps of defense in depth to attacks that presume they have direct access to firmware without needing to smash the kernel further beyond root (also, root access is foolishly yet often available to users). FreeBSD should consider addressing any oppurtunities to further inhibit these attack vectors. Details via links below. (CC'd to a few lists to promote general awareness. Replies are perhaps best made only to freebsd-security@ . This post is what people were replying to but never made it.) # CAM - hdd, tape, optical, etc https://www.schneier.com/blog/archives/2014/01/iratemonk_nsa_e.html http://spritesmods.com/?art=hddhack http://s3.eurecom.fr/~zaddach/ https://www.ibr.cs.tu-bs.de/users/kurmus/ https://www.malwaretech.com/2015/04/hard-disk-firmware-hacking-part-1.html https://www.malwaretech.com/2015/06/hard-disk-firmware-rootkit-surviving.html http://web.archive.org/web/20150615181236/http://malwaretech.net/MTSBK.pdf https://securelist.com/files/2015/02/Equation_group_questions_and_answers.pdf http://arstechnica.com/security/2015/02/how-omnipotent-hackers-tied-to-the-nsa-hid-for-14-years-and-were-found-at-last/ http://web.archive.org/web/20130228090611/http://www.recover.co.il/SA-cover/SA-cover.pdf http://www.spiegel.de/media/media-35661.pdf # USB https://opensource.srlabs.de/projects/badusb https://github.com/robertfisk/USG/wiki # BIOS, UEFI http://blog.trendmicro.com/trendlabs-security-intelligence/hacking-team-uses-uefi-bios-rootkit-to-keep-rcs-9-agent-in-target-systems/ http://arstechnica.com/security/2013/10/meet-badbios-the-mysterious-mac-and-pc-malware-that-jumps-airgaps/ # CPU http://inertiawar.com/microcode/ https://wiki.archlinux.org/index.php/microcode http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3a-part-1-manual.pdf https://en.wikipedia.org/wiki/Intel_Active_Management_Technology # FreeBSD, UFS - supported https://www.schneier.com/blog/archives/2014/01/iratemonk_nsa_e.html http://leaksource.files.wordpress.com/2013/12/nsa-ant-iratemonk.jpg https://www.schneier.com/blog/archives/2014/02/swap_nsa_exploi.html http://leaksource.files.wordpress.com/2013/12/nsa-ant-swap.jpg http://leaksource.files.wordpress.com/2013/12/nsa-ant-sierramontana.jpg # various https://en.wikipedia.org/wiki/NSA_ANT_catalog https://firmwaresecurity.com/ From owner-freebsd-hackers@freebsd.org Sat Mar 25 04:31:37 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB58ED1DAAE for ; Sat, 25 Mar 2017 04:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B6271EB1 for ; Sat, 25 Mar 2017 04:31:37 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v2P4VVw1088887; Fri, 24 Mar 2017 21:31:31 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v2P4VTRa088886; Fri, 24 Mar 2017 21:31:29 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201703250431.v2P4VTRa088886@pdx.rh.CN85.dnsmgr.net> Subject: Re: large PCIe BARs In-Reply-To: <21FF81F3-D290-4682-AB88-5769C52499C3@chrisbowman.com> To: Christopher Bowman Date: Fri, 24 Mar 2017 21:31:29 -0700 (PDT) CC: freebsd-hackers@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Mailman-Approved-At: Sat, 25 Mar 2017 11:25:23 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Mar 2017 04:31:37 -0000 > I have an FPGA PCIe device and driver I designing. I think my driver is trying to map the PCIe address space for one of the BARs and it?s failing. My guess is that it?s failing due to the size. The board has 2G of onboard memory that I?d like to present as one large BAR. Does FreeBSD (11.0 release) allow for mapping PCIe resources that larger? If not is there a limit? If so any one have any comments? > Thanks, > Christopher > > sp6050: probing for SP605 > sp6050: at device 0.0 on pci3 > sp6050: attach of SP605 > sp6050: 0x40000000 bytes of rid 0x10 res 3 failed (0, 0xffffffffffffffff). > sp6050: Could not map memory > device_attach: sp6050 attach returned 6 Are you on amd64? You should be able to map that much as the other reply posted, but I do not see a uname -a here to tell what your really running, just that your on FreeBSD 11.0. Also it would help to get the 'pciconf -l -b' output to see just how the pci information is being presented. -- Rod Grimes rgrimes@freebsd.org