From owner-freebsd-arch@FreeBSD.ORG Tue Nov 11 06:55:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 387A781C for ; Tue, 11 Nov 2014 06:55:02 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 071A0D1E for ; Tue, 11 Nov 2014 06:55:02 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id bj1so10019100pad.17 for ; Mon, 10 Nov 2014 22:55:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=Cd9rrLZVp5oYvdlFN935iRNZWOTm5NUO9/nGIf2YXZE=; b=RBFYLpACuFxCJ6Dko+jiAzZCoetZZBAxpRgBWxPOsEbls3ewuZscsXUaCmuoZZlUCi 5mxSJo88lvDDC85uTmxMQXZoU8Qp+m8ZOn6lethmn2BH9Z8EXvOo0Y0aTERGY2yI6EM0 v5T/hWio20THNlpDslM2dchjvq03XTyp4+vuW1LCcuOnHpwSI2T+OxgkIgUkFKVCTs67 U75w+lGzTKob6WXYIX0nwQGPwRsGOyW8SrkHlH9lGtZlbZkYm4gUF7r9q+QGRFKISl3g Rj4U9tj0ziHymNb3MHnM6uOakx18KBCZJIB/JFVvkByZCChcDK2LwN4CcfIL+8nQERoS xffA== X-Received: by 10.68.113.164 with SMTP id iz4mr9886991pbb.137.1415688901599; Mon, 10 Nov 2014 22:55:01 -0800 (PST) Received: from raichu ([198.244.104.6]) by mx.google.com with ESMTPSA id fv6sm18307078pdb.83.2014.11.10.22.55.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Nov 2014 22:55:00 -0800 (PST) Sender: Mark Johnston Date: Mon, 10 Nov 2014 22:54:51 -0800 From: Mark Johnston To: freebsd-arch@freebsd.org Subject: [RFC] core dump compression Message-ID: <20141111065451.GA9757@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2014 06:55:02 -0000 Hello, It's possible to have the kernel write out userland core dumps as gzip files, provided that the COMPRESS_USER_CORES option is present in the kernel. Currently this feature doesn't work properly because of some locking changes introduced in r272535. It's easy to fix this directly by adding IO_RANGELOCKED to the flags of the vn_rdwr_inchunks() calls in kern_gzio.c, but this approach seems somewhat hacky - kern_gzio.c is ostensibly a general-purpose interface (it's enabled by "device gzio" in the kernel config), so modifying the flags is wrong. I rewrote kern_gzio.c to provide a small callback-based interface to the in-kernel zlib, and changed the userland core dump code to use it. Together with this is a patch which optionally enables the kernel to compress its own core dumps as they're written out to the dump device. The idea is to make it more likely that we'll be able to fit a core on the dump device (which is often the swap partition for the system). The patches are available at: https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_gzio.diff and https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff The first fixes userland core dump compression. It replaces the gzio device and COMPRESS_USER_CORES option with a GZIO option. When the kernel is compiled with this option, the kern.compress_user_cores sysctl determines whether userland core dumps will be compressed (defaulting to off). The second patch adds kern_dump.c, which provides a small interface for use by the MD kernel core dump code (i.e. dumpsys() and minidumpsys()). Its purpose is to hide the details of where the dump gets written on the dump device. Normally, dumpsys/minidumpsys computes the total size of the dump ahead of time, and writes it out so that the end of the dump coincides with the end of the device. With compression this isn't possible, since we can't know the final size ahead of time. The approach I've taken is to write the dump starting at the same offset an uncompressed dump would use. Once that's done, the headers are written to the beginning of the dump and the end of the device, and savecore(8) has been taught to detect compressed dumps and recover them. With the patch, kern_dump.c becomes responsible for writing the headers and for keeping track of the current offset within the dump; this lets us remove quite a bit of duplicated code from the various dumpsys/minidumpsys implementations without introducing any complexity in the !compressed case. For core dump compression, kern_dump.c uses the GZIO interface; the callback just calls dump_write() to write the compressed data to disk. With the second patch, kernel core dump compression is enabled using the kern.compress_kernel_dumps sysctl/tunable. At the moment it can only be set as a tunable, but I'll fix this soon. In practice, the compression factor that we get is between 6 and 14, with worse compression for larger cores. On the several amd64 systems that I've tested, the time required to actually write a core is increased slightly; on the other hand, a savecore(8) for a compressed dump is obviously much faster, so the boot following a crash will complete more quickly. The patches are still a bit rough, but I was hoping for some feedback on the approach and the new interfaces defined in kern_dump.c and kern_gzio.c. I've tested on a number of physical systems and VMs without any problems - further testing is very welcome, particularly on non-x86 systems. Thanks! -Mark