From owner-svn-src-head@freebsd.org Sat Apr 15 14:01:09 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38D76D3E92C; Sat, 15 Apr 2017 14:01:09 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F164C1E42; Sat, 15 Apr 2017 14:01:08 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v3FE0vnT012251; Sat, 15 Apr 2017 07:00:57 -0700 (PDT) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v3FE0vXk012250; Sat, 15 Apr 2017 07:00:57 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201704151400.v3FE0vXk012250@pdx.rh.CN85.dnsmgr.net> Subject: Re: svn commit: r316938 - head/sbin/savecore In-Reply-To: <20170415083952.GA83631@zxy.spb.ru> To: Slawa Olhovchenkov Date: Sat, 15 Apr 2017 07:00:57 -0700 (PDT) CC: Mark Johnston , Xin LI , "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , Ngie Cooper , Alan Somers Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Apr 2017 14:01:09 -0000 > On Fri, Apr 14, 2017 at 03:05:25PM -0700, Mark Johnston wrote: > > > > And with textdumps available, the benefit > > > of having compression is limited because we can request for minidump > > > or full dumps only when the textdumps are not good enough for > > > diagnosing the kernel bug. > > > > Sure, but in this case the compression may be vital. > > > > > > > > I don't think security (e.g. leaking information because of the use of > > > compression) is a very big concern in this context because in order > > > for the potential attacker to read the raw material needs a > > > compromised system (unlike an attack from the network, where someone > > > who controls the network would have access to the raw material); the > > > dump is usually quite large, and measuring downtime would be hard at > > > that scale. > > > > Ok. > > > > > > > > By the way (not meant to bikeshed) if I was to do this I'd prefer > > > using lz4 or something that compresses faster than zlib. > > > > I agree, but I think the existing lz4 implementation in the kernel is > > not so well suited to running after a panic. It seems fixable, but > > compression speed also isn't hugely important here IMO. > > On production system this is downtime. > For may case, dumped about 32GB (from 256GB RAM). This is take several > minutes. Can compression increase this to hour? On productions systems the compression layer of dump may very well be a win situation depending on choosen algorith (you want something fairly fast, but still effective). If your rate to compress bytes is close to the disk write bandwith you have an over all win caused by writting less to disk. Someone who enjoys math should write an equation for given compression bandwidth cb and given disk bandwidth db and compression ratio cr what do the curves look like? IIRC we measure cpu/memory bandwidth in the 10'sG bytes/sec range, compression should be some place under that, and our disk bandwidth even on SSD is in the 500MB range, even the fastest 15k rpm spinning rust is in the <200MB range, we should be able to compress at a higher rate than this. -- Rod Grimes rgrimes@freebsd.org