From owner-svn-src-head@freebsd.org Sat Apr 15 14:43:17 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FF7ED3F8F6; Sat, 15 Apr 2017 14:43:17 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 60A55F0D; Sat, 15 Apr 2017 14:43:17 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1czOuv-0001T5-I8; Sat, 15 Apr 2017 17:43:13 +0300 Date: Sat, 15 Apr 2017 17:43:13 +0300 From: Slawa Olhovchenkov To: rgrimes@freebsd.org Cc: Mark Johnston , Xin LI , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , Ngie Cooper , Alan Somers Subject: Re: svn commit: r316938 - head/sbin/savecore Message-ID: <20170415144313.GG70430@zxy.spb.ru> References: <20170415083952.GA83631@zxy.spb.ru> <201704151400.v3FE0vXk012250@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201704151400.v3FE0vXk012250@pdx.rh.CN85.dnsmgr.net> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Apr 2017 14:43:17 -0000 On Sat, Apr 15, 2017 at 07:00:57AM -0700, Rodney W. Grimes wrote: > > On Fri, Apr 14, 2017 at 03:05:25PM -0700, Mark Johnston wrote: > > > > > > And with textdumps available, the benefit > > > > of having compression is limited because we can request for minidump > > > > or full dumps only when the textdumps are not good enough for > > > > diagnosing the kernel bug. > > > > > > Sure, but in this case the compression may be vital. > > > > > > > > > > > I don't think security (e.g. leaking information because of the use of > > > > compression) is a very big concern in this context because in order > > > > for the potential attacker to read the raw material needs a > > > > compromised system (unlike an attack from the network, where someone > > > > who controls the network would have access to the raw material); the > > > > dump is usually quite large, and measuring downtime would be hard at > > > > that scale. > > > > > > Ok. > > > > > > > > > > > By the way (not meant to bikeshed) if I was to do this I'd prefer > > > > using lz4 or something that compresses faster than zlib. > > > > > > I agree, but I think the existing lz4 implementation in the kernel is > > > not so well suited to running after a panic. It seems fixable, but > > > compression speed also isn't hugely important here IMO. > > > > On production system this is downtime. > > For may case, dumped about 32GB (from 256GB RAM). This is take several > > minutes. Can compression increase this to hour? > > On productions systems the compression layer of dump may very well > be a win situation depending on choosen algorith (you want something > fairly fast, but still effective). If your rate to compress bytes > is close to the disk write bandwith you have an over all win caused > by writting less to disk. > > Someone who enjoys math should write an equation for given > compression bandwidth cb and given disk bandwidth db and > compression ratio cr what do the curves look like? > > IIRC we measure cpu/memory bandwidth in the 10'sG bytes/sec wrong. single thread cpu/memory bandwidth very different on i7 and e5 cpu (e5 less, about 6 GB/s). > range, compression should be some place under that, and > our disk bandwidth even on SSD is in the 500MB range, > even the fastest 15k rpm spinning rust is in the <200MB > range, we should be able to compress at a higher rate than > this. yes.